2024-05-21T02:58:37.691 INFO:root:teuthology version: 0.0.1.dev287+gb9e3da8 2024-05-21T02:58:37.691 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/ffsb}} --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2024-05-21_01:33:07-fs:workload-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7717852 --name pdonnell-2024-05-21_01:33:07-fs:workload-wip-lusov-quiesce-overdrive-export-distro-default-smithi -- /home/teuthworker/archive/pdonnell-2024-05-21_01:33:07-fs:workload-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7717852/orig.config.yaml 2024-05-21T02:58:37.755 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-21T02:58:37.851 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-21_01:33:07-fs:workload-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7717852 branch: wip-lusov-quiesce-overdrive-export description: fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/ffsb}} email: null first_in_suite: false job_id: '7717852' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: pdonnell-2024-05-21_01:33:07-fs:workload-wip-lusov-quiesce-overdrive-export-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiesce-overdrive-export ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 filestore flush min: 0 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd heartbeat grace: 60 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - SLOW_OPS - slow request sha1: 3b748ea08891006c1152513d25ab4a03f7c9a995 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 3b748ea08891006c1152513d25ab4a03f7c9a995 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiesce-overdrive-export sha1: 3b748ea08891006c1152513d25ab4a03f7c9a995 owner: scheduled_pdonnell@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8818 sha1: 3b748ea08891006c1152513d25ab4a03f7c9a995 sleep_before_teardown: 0 subset: 419/111111 suite: fs:workload suite_branch: wip-lusov-quiesce-overdrive-export suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_3b748ea08891006c1152513d25ab4a03f7c9a995/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 3b748ea08891006c1152513d25ab4a03f7c9a995 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi027.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNZVnv4VChF+JjNIn5hPvt+/uX/mY7E/LzVol5Fdj62lH1E5iNplEqY4CqJ2uO3oHYzvBVzt7j2k4e5ZFoeDrE0= smithi039.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBObSLJxvt8YHqI14BR01ZXJS+2mAX62lEWElecM+DQN4uOtTZUlHAD1Jm1o3oXig6T1cb3Oc9mpYRg8i+Qk5jKE= smithi165.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGTPoRc8AOkMF0RkqeVU0kl0239XnF0ItmzZXuusd8b6wVZN0u9UVXhRBOj34kUey46+7Px1Nd39hh65+Qezpcs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - quiescer: null - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a timestamp: 2024-05-21_01:33:07 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-05-21T02:58:37.851 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_3b748ea08891006c1152513d25ab4a03f7c9a995/qa; will attempt to use it 2024-05-21T02:58:37.852 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_3b748ea08891006c1152513d25ab4a03f7c9a995/qa/tasks 2024-05-21T02:58:37.852 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-21T02:58:37.854 INFO:teuthology.task.internal:Checking packages... 2024-05-21T02:58:37.880 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3b748ea08891006c1152513d25ab4a03f7c9a995' 2024-05-21T02:58:37.881 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-21T02:58:37.881 INFO:teuthology.packaging:ref: None 2024-05-21T02:58:37.881 INFO:teuthology.packaging:tag: None 2024-05-21T02:58:37.881 INFO:teuthology.packaging:branch: wip-lusov-quiesce-overdrive-export 2024-05-21T02:58:37.881 INFO:teuthology.packaging:sha1: 3b748ea08891006c1152513d25ab4a03f7c9a995 2024-05-21T02:58:37.881 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiesce-overdrive-export 2024-05-21T02:58:38.108 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3830.g3b748ea0 2024-05-21T02:58:38.110 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-21T02:58:38.117 INFO:teuthology.task.internal:no buildpackages task found 2024-05-21T02:58:38.117 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-21T02:58:38.122 INFO:teuthology.task.internal:Saving configuration 2024-05-21T02:58:38.154 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-21T02:58:38.160 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-21T02:58:38.186 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-21_01:33:07-fs:workload-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7717852', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-21 02:52:11.397082', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNZVnv4VChF+JjNIn5hPvt+/uX/mY7E/LzVol5Fdj62lH1E5iNplEqY4CqJ2uO3oHYzvBVzt7j2k4e5ZFoeDrE0='} 2024-05-21T02:58:38.212 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi039.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-21_01:33:07-fs:workload-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7717852', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-21 02:52:11.399068', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBObSLJxvt8YHqI14BR01ZXJS+2mAX62lEWElecM+DQN4uOtTZUlHAD1Jm1o3oXig6T1cb3Oc9mpYRg8i+Qk5jKE='} 2024-05-21T02:58:38.235 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi165.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-21_01:33:07-fs:workload-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7717852', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-21 02:52:11.400771', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGTPoRc8AOkMF0RkqeVU0kl0239XnF0ItmzZXuusd8b6wVZN0u9UVXhRBOj34kUey46+7Px1Nd39hh65+Qezpcs='} 2024-05-21T02:58:38.235 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-21T02:58:38.241 INFO:teuthology.task.internal:roles: ubuntu@smithi027.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-21T02:58:38.241 INFO:teuthology.task.internal:roles: ubuntu@smithi039.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-21T02:58:38.241 INFO:teuthology.task.internal:roles: ubuntu@smithi165.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-21T02:58:38.241 INFO:teuthology.run_tasks:Running task console_log... 2024-05-21T02:58:38.334 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9c0a04fac0>, signals=[15]) 2024-05-21T02:58:38.334 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-21T02:58:38.339 INFO:teuthology.task.internal:Opening connections... 2024-05-21T02:58:38.339 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi027.front.sepia.ceph.com 2024-05-21T02:58:38.340 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T02:58:38.421 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi039.front.sepia.ceph.com 2024-05-21T02:58:38.422 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T02:58:38.496 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi165.front.sepia.ceph.com 2024-05-21T02:58:38.497 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T02:58:38.574 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-21T02:58:38.580 DEBUG:teuthology.orchestra.run.smithi027:> uname -m 2024-05-21T02:58:38.598 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64 2024-05-21T02:58:38.599 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release 2024-05-21T02:58:38.657 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Stream" 2024-05-21T02:58:38.657 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="9" 2024-05-21T02:58:38.657 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos" 2024-05-21T02:58:38.657 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora" 2024-05-21T02:58:38.657 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="9" 2024-05-21T02:58:38.657 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el9" 2024-05-21T02:58:38.657 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-21T02:58:38.657 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31" 2024-05-21T02:58:38.658 INFO:teuthology.orchestra.run.smithi027.stdout:LOGO="fedora-logo-icon" 2024-05-21T02:58:38.658 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-21T02:58:38.658 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://centos.org/" 2024-05-21T02:58:38.658 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-21T02:58:38.658 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-21T02:58:38.658 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-21T02:58:38.659 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2024-05-21T02:58:38.682 DEBUG:teuthology.orchestra.run.smithi039:> uname -m 2024-05-21T02:58:38.700 INFO:teuthology.orchestra.run.smithi039.stdout:x86_64 2024-05-21T02:58:38.701 DEBUG:teuthology.orchestra.run.smithi039:> cat /etc/os-release 2024-05-21T02:58:38.759 INFO:teuthology.orchestra.run.smithi039.stdout:NAME="CentOS Stream" 2024-05-21T02:58:38.760 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION="9" 2024-05-21T02:58:38.760 INFO:teuthology.orchestra.run.smithi039.stdout:ID="centos" 2024-05-21T02:58:38.760 INFO:teuthology.orchestra.run.smithi039.stdout:ID_LIKE="rhel fedora" 2024-05-21T02:58:38.760 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_ID="9" 2024-05-21T02:58:38.760 INFO:teuthology.orchestra.run.smithi039.stdout:PLATFORM_ID="platform:el9" 2024-05-21T02:58:38.760 INFO:teuthology.orchestra.run.smithi039.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-21T02:58:38.760 INFO:teuthology.orchestra.run.smithi039.stdout:ANSI_COLOR="0;31" 2024-05-21T02:58:38.760 INFO:teuthology.orchestra.run.smithi039.stdout:LOGO="fedora-logo-icon" 2024-05-21T02:58:38.760 INFO:teuthology.orchestra.run.smithi039.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-21T02:58:38.760 INFO:teuthology.orchestra.run.smithi039.stdout:HOME_URL="https://centos.org/" 2024-05-21T02:58:38.760 INFO:teuthology.orchestra.run.smithi039.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-21T02:58:38.761 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-21T02:58:38.761 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-21T02:58:38.761 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server 2024-05-21T02:58:38.782 DEBUG:teuthology.orchestra.run.smithi165:> uname -m 2024-05-21T02:58:38.800 INFO:teuthology.orchestra.run.smithi165.stdout:x86_64 2024-05-21T02:58:38.800 DEBUG:teuthology.orchestra.run.smithi165:> cat /etc/os-release 2024-05-21T02:58:38.858 INFO:teuthology.orchestra.run.smithi165.stdout:NAME="CentOS Stream" 2024-05-21T02:58:38.858 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION="9" 2024-05-21T02:58:38.858 INFO:teuthology.orchestra.run.smithi165.stdout:ID="centos" 2024-05-21T02:58:38.858 INFO:teuthology.orchestra.run.smithi165.stdout:ID_LIKE="rhel fedora" 2024-05-21T02:58:38.858 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION_ID="9" 2024-05-21T02:58:38.859 INFO:teuthology.orchestra.run.smithi165.stdout:PLATFORM_ID="platform:el9" 2024-05-21T02:58:38.859 INFO:teuthology.orchestra.run.smithi165.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-21T02:58:38.859 INFO:teuthology.orchestra.run.smithi165.stdout:ANSI_COLOR="0;31" 2024-05-21T02:58:38.859 INFO:teuthology.orchestra.run.smithi165.stdout:LOGO="fedora-logo-icon" 2024-05-21T02:58:38.859 INFO:teuthology.orchestra.run.smithi165.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-21T02:58:38.859 INFO:teuthology.orchestra.run.smithi165.stdout:HOME_URL="https://centos.org/" 2024-05-21T02:58:38.859 INFO:teuthology.orchestra.run.smithi165.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-21T02:58:38.859 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-21T02:58:38.859 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-21T02:58:38.859 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server 2024-05-21T02:58:38.882 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-21T02:58:38.891 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-21T02:58:38.904 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-21T02:58:38.904 DEBUG:teuthology.orchestra.run.smithi027:> test '!' -e /home/ubuntu/cephtest 2024-05-21T02:58:38.906 DEBUG:teuthology.orchestra.run.smithi039:> test '!' -e /home/ubuntu/cephtest 2024-05-21T02:58:38.909 DEBUG:teuthology.orchestra.run.smithi165:> test '!' -e /home/ubuntu/cephtest 2024-05-21T02:58:38.925 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-21T02:58:38.931 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-21T02:58:38.931 DEBUG:teuthology.orchestra.run.smithi027:> test -z $(ls -A /var/lib/ceph) 2024-05-21T02:58:38.964 DEBUG:teuthology.orchestra.run.smithi039:> test -z $(ls -A /var/lib/ceph) 2024-05-21T02:58:38.967 DEBUG:teuthology.orchestra.run.smithi165:> test -z $(ls -A /var/lib/ceph) 2024-05-21T02:58:39.074 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-21T02:58:39.149 INFO:teuthology.run_tasks:Running task kernel... 2024-05-21T02:58:39.164 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-21T02:58:39.164 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-21T02:58:39.164 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-21T02:58:39.165 DEBUG:teuthology.orchestra.run.smithi027:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-21T02:58:39.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T02:58:39.182 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2024-05-21T02:58:39.241 INFO:teuthology.orchestra.run.smithi027.stdout:5.14.0-437.el9.x86_64 2024-05-21T02:58:39.241 INFO:teuthology.task.kernel:Running kernel on smithi027: 5.14.0-437.el9.x86_64 2024-05-21T02:58:39.241 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum install -y kernel 2024-05-21T02:58:42.656 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.1 MB 00:01 2024-05-21T02:58:59.606 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:15 2024-05-21T02:59:05.181 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.2 MB 00:01 2024-05-21T02:59:06.907 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-05-21T02:59:08.769 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-05-21T02:59:14.100 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-21T02:59:16.062 INFO:teuthology.orchestra.run.smithi027.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-21T02:59:16.064 INFO:teuthology.orchestra.run.smithi027.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-21T02:59:16.186 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-05-21T02:59:16.194 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-05-21T02:59:16.194 INFO:teuthology.orchestra.run.smithi027.stdout: Package Architecture Version Repository Size 2024-05-21T02:59:16.194 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-05-21T02:59:16.194 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-05-21T02:59:16.194 INFO:teuthology.orchestra.run.smithi027.stdout: kernel x86_64 5.14.0-447.el9 baseos 576 k 2024-05-21T02:59:16.194 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-05-21T02:59:16.194 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-core x86_64 5.14.0-447.el9 baseos 16 M 2024-05-21T02:59:16.194 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules x86_64 5.14.0-447.el9 baseos 34 M 2024-05-21T02:59:16.194 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules-core x86_64 5.14.0-447.el9 baseos 28 M 2024-05-21T02:59:16.195 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T02:59:16.195 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-05-21T02:59:16.195 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-05-21T02:59:16.195 INFO:teuthology.orchestra.run.smithi027.stdout:Install 4 Packages 2024-05-21T02:59:16.195 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T02:59:16.195 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 78 M 2024-05-21T02:59:16.195 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 123 M 2024-05-21T02:59:16.196 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-05-21T02:59:16.338 INFO:teuthology.orchestra.run.smithi027.stdout:[MIRROR] kernel-core-5.14.0-447.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-447.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-21T02:59:16.339 INFO:teuthology.orchestra.run.smithi027.stdout:[MIRROR] kernel-5.14.0-447.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-447.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-21T02:59:16.339 INFO:teuthology.orchestra.run.smithi027.stdout:[MIRROR] kernel-modules-5.14.0-447.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-447.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-21T02:59:17.110 INFO:teuthology.orchestra.run.smithi027.stdout:(1/4): kernel-5.14.0-447.el9.x86_64.rpm 715 kB/s | 576 kB 00:00 2024-05-21T02:59:27.626 INFO:teuthology.orchestra.run.smithi027.stdout:(2/4): kernel-modules-5.14.0-447.el9.x86_64.rpm 3.0 MB/s | 34 MB 00:11 2024-05-21T02:59:31.751 INFO:teuthology.orchestra.run.smithi027.stdout:(3/4): kernel-core-5.14.0-447.el9.x86_64.rpm 1.0 MB/s | 16 MB 00:15 2024-05-21T02:59:48.716 INFO:teuthology.orchestra.run.smithi027.stdout:(4/4): kernel-modules-core-5.14.0-447.el9.x86_6 909 kB/s | 28 MB 00:31 2024-05-21T02:59:48.716 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-05-21T02:59:48.716 INFO:teuthology.orchestra.run.smithi027.stdout:Total 2.4 MB/s | 78 MB 00:32 2024-05-21T02:59:49.429 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-05-21T02:59:49.847 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-05-21T02:59:49.848 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-05-21T02:59:50.804 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-05-21T02:59:50.804 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-05-21T02:59:53.353 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-05-21T02:59:53.994 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-modules-core-5.14.0-447.el9.x86_64 1/4 2024-05-21T02:59:54.083 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-21T02:59:56.871 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-21T02:59:56.996 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-21T03:00:02.069 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-21T03:00:02.124 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-5.14.0-447.el9.x86_64 4/4 2024-05-21T03:00:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-21T03:00:37.999 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 4/4 2024-05-21T03:00:38.039 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 4/4 2024-05-21T03:00:38.398 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-5.14.0-447.el9.x86_64 4/4 2024-05-21T03:00:38.399 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-5.14.0-447.el9.x86_64 1/4 2024-05-21T03:00:38.399 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-21T03:00:38.399 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-21T03:00:39.198 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-21T03:00:39.198 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:00:39.198 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-05-21T03:00:39.198 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-5.14.0-447.el9.x86_64 2024-05-21T03:00:39.198 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-core-5.14.0-447.el9.x86_64 2024-05-21T03:00:39.199 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules-5.14.0-447.el9.x86_64 2024-05-21T03:00:39.199 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules-core-5.14.0-447.el9.x86_64 2024-05-21T03:00:39.199 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:00:39.199 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-05-21T03:00:39.545 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q kernel | sort -rV | head -n 1 2024-05-21T03:00:39.622 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-21T03:00:39.623 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-21T03:00:39.623 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-447.el9.x86_64 2024-05-21T03:00:39.623 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-21T03:00:39.623 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-21T03:00:39.623 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-21T03:00:39.623 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi027.front.sepia.ceph.com, path=None, version=distro) 2024-05-21T03:00:39.623 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q kernel | sort -rV | head -n 1 2024-05-21T03:00:39.675 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-21T03:00:39.675 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-21T03:00:39.675 DEBUG:teuthology.orchestra.run.smithi027:> sudo rpm -qi grub2-tools 2024-05-21T03:00:39.706 INFO:teuthology.orchestra.run.smithi027.stdout:Name : grub2-tools 2024-05-21T03:00:39.706 INFO:teuthology.orchestra.run.smithi027.stdout:Epoch : 1 2024-05-21T03:00:39.707 INFO:teuthology.orchestra.run.smithi027.stdout:Version : 2.06 2024-05-21T03:00:39.707 INFO:teuthology.orchestra.run.smithi027.stdout:Release : 68.el9 2024-05-21T03:00:39.707 INFO:teuthology.orchestra.run.smithi027.stdout:Architecture: x86_64 2024-05-21T03:00:39.707 INFO:teuthology.orchestra.run.smithi027.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-21T03:00:39.707 INFO:teuthology.orchestra.run.smithi027.stdout:Group : Unspecified 2024-05-21T03:00:39.707 INFO:teuthology.orchestra.run.smithi027.stdout:Size : 8264254 2024-05-21T03:00:39.707 INFO:teuthology.orchestra.run.smithi027.stdout:License : GPLv3+ 2024-05-21T03:00:39.707 INFO:teuthology.orchestra.run.smithi027.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-21T03:00:39.707 INFO:teuthology.orchestra.run.smithi027.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-21T03:00:39.707 INFO:teuthology.orchestra.run.smithi027.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-21T03:00:39.707 INFO:teuthology.orchestra.run.smithi027.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-21T03:00:39.707 INFO:teuthology.orchestra.run.smithi027.stdout:Packager : builder@centos.org 2024-05-21T03:00:39.707 INFO:teuthology.orchestra.run.smithi027.stdout:Vendor : CentOS 2024-05-21T03:00:39.708 INFO:teuthology.orchestra.run.smithi027.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-21T03:00:39.708 INFO:teuthology.orchestra.run.smithi027.stdout:Summary : Support tools for GRUB. 2024-05-21T03:00:39.708 INFO:teuthology.orchestra.run.smithi027.stdout:Description : 2024-05-21T03:00:39.708 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:00:39.708 INFO:teuthology.orchestra.run.smithi027.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-21T03:00:39.708 INFO:teuthology.orchestra.run.smithi027.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-21T03:00:39.708 INFO:teuthology.orchestra.run.smithi027.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-21T03:00:39.708 INFO:teuthology.orchestra.run.smithi027.stdout:hardware devices. 2024-05-21T03:00:39.708 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:00:39.708 INFO:teuthology.orchestra.run.smithi027.stdout:This subpackage provides tools for support of all platforms. 2024-05-21T03:00:39.709 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-21T03:00:39.709 INFO:teuthology.task.kernel:Updating grub on smithi027 to boot 5.14.0-447.el9.x86_64 2024-05-21T03:00:39.709 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-21T03:00:40.357 INFO:teuthology.orchestra.run.smithi027.stderr:Generating grub configuration file ... 2024-05-21T03:00:41.874 INFO:teuthology.orchestra.run.smithi027.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-21T03:00:41.891 INFO:teuthology.orchestra.run.smithi027.stderr:done 2024-05-21T03:00:41.893 DEBUG:teuthology.orchestra.run.smithi027:> mktemp 2024-05-21T03:00:41.909 INFO:teuthology.orchestra.run.smithi027.stdout:/tmp/tmp.jfQO6UGzhd 2024-05-21T03:00:41.909 DEBUG:teuthology.orchestra.run.smithi027:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.jfQO6UGzhd 2024-05-21T03:00:41.980 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0666 /tmp/tmp.jfQO6UGzhd 2024-05-21T03:00:42.125 DEBUG:teuthology.orchestra.remote:smithi027:/tmp/tmp.jfQO6UGzhd is 6KB 2024-05-21T03:00:42.174 DEBUG:teuthology.orchestra.run.smithi027:> rm -fr /tmp/tmp.jfQO6UGzhd 2024-05-21T03:00:42.189 DEBUG:teuthology.orchestra.run.smithi027:> sudo /bin/ls /boot/loader/entries || true 2024-05-21T03:00:42.256 INFO:teuthology.orchestra.run.smithi027.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-21T03:00:42.256 INFO:teuthology.orchestra.run.smithi027.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-21T03:00:42.256 INFO:teuthology.orchestra.run.smithi027.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-21T03:00:42.256 INFO:teuthology.orchestra.run.smithi027.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64.conf 2024-05-21T03:00:42.257 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64 2024-05-21T03:00:42.529 DEBUG:teuthology.orchestra.run.smithi027:> sudo shutdown -r now 2024-05-21T03:01:12.532 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-21T03:01:12.532 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-05-21T03:01:12.533 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T03:01:47.167 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-05-21T03:01:56.167 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-05-21T03:01:56.169 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T03:01:59.230 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-05-21T03:02:11.231 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-05-21T03:02:11.232 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T03:02:14.302 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-05-21T03:02:29.303 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-05-21T03:02:29.304 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T03:03:29.306 DEBUG:teuthology.orchestra.remote:timed out 2024-05-21T03:03:47.307 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-05-21T03:03:47.308 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T03:03:47.559 DEBUG:teuthology.orchestra.run.smithi027:> true 2024-05-21T03:03:47.931 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-05-21T03:03:47.932 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-447.el9.x86_64"... 2024-05-21T03:03:47.932 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2024-05-21T03:03:47.950 INFO:teuthology.orchestra.run.smithi027.stdout:5.14.0-447.el9.x86_64 2024-05-21T03:03:47.951 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-447.el9.x86_64 vs 5.14.0-447.el9.x86_64 2024-05-21T03:03:47.951 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-21T03:03:47.951 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-21T03:03:48.953 DEBUG:teuthology.parallel:result is None 2024-05-21T03:03:48.953 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-21T03:03:48.969 INFO:teuthology.task.internal:Creating test directory... 2024-05-21T03:03:48.969 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-21T03:03:48.972 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-21T03:03:48.976 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-21T03:03:48.994 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-21T03:03:49.001 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-21T03:03:49.009 INFO:teuthology.task.internal:Creating archive directory... 2024-05-21T03:03:49.010 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-21T03:03:49.031 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-21T03:03:49.034 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-21T03:03:49.149 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-21T03:03:49.157 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-21T03:03:49.157 DEBUG:teuthology.orchestra.run.smithi039:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-21T03:03:49.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T03:03:49.175 DEBUG:teuthology.orchestra.run.smithi165:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-21T03:03:49.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T03:03:49.192 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-21T03:03:49.194 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-21T03:03:49.217 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-21T03:03:49.260 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T03:03:49.279 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T03:03:49.280 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T03:03:49.299 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T03:03:49.304 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T03:03:49.309 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T03:03:49.311 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-21T03:03:49.318 INFO:teuthology.task.internal:Configuring sudo... 2024-05-21T03:03:49.319 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-21T03:03:49.321 DEBUG:teuthology.orchestra.run.smithi039:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-21T03:03:49.324 DEBUG:teuthology.orchestra.run.smithi165:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-21T03:03:49.350 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-21T03:03:49.360 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-21T03:03:49.360 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-21T03:03:49.393 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-21T03:03:49.395 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-21T03:03:49.411 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T03:03:49.497 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T03:03:49.547 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T03:03:49.630 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T03:03:49.677 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-05-21T03:03:49.677 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-21T03:03:49.745 DEBUG:teuthology.orchestra.run.smithi039:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T03:03:49.793 DEBUG:teuthology.orchestra.run.smithi039:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T03:03:49.842 DEBUG:teuthology.orchestra.run.smithi039:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T03:03:49.927 DEBUG:teuthology.orchestra.run.smithi039:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T03:03:49.971 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-21T03:03:49.971 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-21T03:03:50.039 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T03:03:50.085 DEBUG:teuthology.orchestra.run.smithi165:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T03:03:50.143 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T03:03:50.187 DEBUG:teuthology.orchestra.run.smithi165:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T03:03:50.227 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-05-21T03:03:50.227 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-21T03:03:50.291 DEBUG:teuthology.orchestra.run.smithi027:> sudo service rsyslog restart 2024-05-21T03:03:50.294 DEBUG:teuthology.orchestra.run.smithi039:> sudo service rsyslog restart 2024-05-21T03:03:50.296 DEBUG:teuthology.orchestra.run.smithi165:> sudo service rsyslog restart 2024-05-21T03:03:50.333 INFO:teuthology.orchestra.run.smithi039.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T03:03:50.337 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T03:03:50.369 INFO:teuthology.orchestra.run.smithi165.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T03:03:50.810 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-21T03:03:50.818 INFO:teuthology.task.internal:Starting timer... 2024-05-21T03:03:50.818 INFO:teuthology.run_tasks:Running task pcp... 2024-05-21T03:03:50.864 INFO:teuthology.run_tasks:Running task selinux... 2024-05-21T03:03:50.871 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-21T03:03:50.872 DEBUG:teuthology.orchestra.run.smithi027:> sudo service auditd rotate 2024-05-21T03:03:50.920 INFO:teuthology.orchestra.run.smithi027.stdout:Rotating logs: 2024-05-21T03:03:50.923 DEBUG:teuthology.orchestra.run.smithi039:> sudo service auditd rotate 2024-05-21T03:03:50.982 INFO:teuthology.orchestra.run.smithi039.stdout:Rotating logs: 2024-05-21T03:03:50.983 DEBUG:teuthology.orchestra.run.smithi165:> sudo service auditd rotate 2024-05-21T03:03:51.039 INFO:teuthology.orchestra.run.smithi165.stdout:Rotating logs: 2024-05-21T03:03:51.041 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-21T03:03:51.041 DEBUG:teuthology.orchestra.run.smithi027:> /usr/sbin/getenforce 2024-05-21T03:03:51.065 INFO:teuthology.orchestra.run.smithi027.stdout:Permissive 2024-05-21T03:03:51.065 DEBUG:teuthology.orchestra.run.smithi039:> /usr/sbin/getenforce 2024-05-21T03:03:51.086 INFO:teuthology.orchestra.run.smithi039.stdout:Permissive 2024-05-21T03:03:51.087 DEBUG:teuthology.orchestra.run.smithi165:> /usr/sbin/getenforce 2024-05-21T03:03:51.106 INFO:teuthology.orchestra.run.smithi165.stdout:Permissive 2024-05-21T03:03:51.107 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi027.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi039.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi165.front.sepia.ceph.com': 'permissive'} 2024-05-21T03:03:51.107 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T03:03:51.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T03:03:51.138 DEBUG:teuthology.orchestra.run.smithi039:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T03:03:51.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T03:03:51.165 DEBUG:teuthology.orchestra.run.smithi165:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T03:03:51.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T03:03:51.190 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-21T03:03:51.191 DEBUG:teuthology.orchestra.run.smithi027:> sudo /usr/sbin/setenforce permissive 2024-05-21T03:03:51.220 DEBUG:teuthology.orchestra.run.smithi039:> sudo /usr/sbin/setenforce permissive 2024-05-21T03:03:51.247 DEBUG:teuthology.orchestra.run.smithi165:> sudo /usr/sbin/setenforce permissive 2024-05-21T03:03:51.276 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-21T03:03:51.286 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-21T03:03:51.289 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-21T03:03:51.411 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-21T03:03:51.422 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-21T03:03:51.423 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi027.front.sepia.ceph.com,smithi039.front.sepia.ceph.com,smithi165.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-21T03:09:08.133 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi027.front.sepia.ceph.com'), Remote(name='ubuntu@smithi039.front.sepia.ceph.com'), Remote(name='ubuntu@smithi165.front.sepia.ceph.com')] 2024-05-21T03:09:08.134 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-05-21T03:09:08.135 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T03:09:08.216 DEBUG:teuthology.orchestra.run.smithi027:> true 2024-05-21T03:09:08.251 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-05-21T03:09:08.252 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-05-21T03:09:08.252 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T03:09:08.326 DEBUG:teuthology.orchestra.run.smithi039:> true 2024-05-21T03:09:08.408 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-05-21T03:09:08.408 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-05-21T03:09:08.408 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T03:09:08.483 DEBUG:teuthology.orchestra.run.smithi165:> true 2024-05-21T03:09:08.562 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-05-21T03:09:08.563 INFO:teuthology.run_tasks:Running task clock... 2024-05-21T03:09:08.573 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-21T03:09:08.574 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-21T03:09:08.574 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T03:09:08.576 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-21T03:09:08.577 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T03:09:08.579 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-21T03:09:08.579 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T03:09:08.605 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-21T03:09:08.610 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-21T03:09:08.624 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-21T03:09:08.625 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-21T03:09:08.630 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-21T03:09:08.644 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-21T03:09:08.650 INFO:teuthology.orchestra.run.smithi027.stderr:sudo: ntpd: command not found 2024-05-21T03:09:08.651 INFO:teuthology.orchestra.run.smithi039.stderr:sudo: ntpd: command not found 2024-05-21T03:09:08.662 INFO:teuthology.orchestra.run.smithi027.stdout:506 Cannot talk to daemon 2024-05-21T03:09:08.663 INFO:teuthology.orchestra.run.smithi039.stdout:506 Cannot talk to daemon 2024-05-21T03:09:08.667 INFO:teuthology.orchestra.run.smithi165.stderr:sudo: ntpd: command not found 2024-05-21T03:09:08.676 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-21T03:09:08.678 INFO:teuthology.orchestra.run.smithi165.stdout:506 Cannot talk to daemon 2024-05-21T03:09:08.678 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-21T03:09:08.690 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-21T03:09:08.691 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-21T03:09:08.692 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-21T03:09:08.704 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-21T03:09:08.748 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2024-05-21T03:09:08.749 INFO:teuthology.orchestra.run.smithi039.stderr:bash: line 1: ntpq: command not found 2024-05-21T03:09:08.751 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T03:09:08.751 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2024-05-21T03:09:08.752 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T03:09:08.752 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T03:09:08.752 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T03:09:08.752 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T03:09:08.753 INFO:teuthology.orchestra.run.smithi039.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T03:09:08.753 INFO:teuthology.orchestra.run.smithi039.stdout:=============================================================================== 2024-05-21T03:09:08.753 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T03:09:08.754 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T03:09:08.754 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T03:09:08.754 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T03:09:08.761 INFO:teuthology.orchestra.run.smithi165.stderr:bash: line 1: ntpq: command not found 2024-05-21T03:09:08.764 INFO:teuthology.orchestra.run.smithi165.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T03:09:08.764 INFO:teuthology.orchestra.run.smithi165.stdout:=============================================================================== 2024-05-21T03:09:08.764 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T03:09:08.764 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T03:09:08.764 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T03:09:08.764 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T03:09:08.765 INFO:teuthology.run_tasks:Running task install... 2024-05-21T03:09:08.774 DEBUG:teuthology.task.install:project ceph 2024-05-21T03:09:08.774 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3b748ea08891006c1152513d25ab4a03f7c9a995'}} 2024-05-21T03:09:08.774 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3b748ea08891006c1152513d25ab4a03f7c9a995'} 2024-05-21T03:09:08.774 INFO:teuthology.task.install:Using flavor: default 2024-05-21T03:09:08.780 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-21T03:09:08.780 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-21T03:09:08.780 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3b748ea08891006c1152513d25ab4a03f7c9a995', 'tag': None, 'wait_for_package': False} 2024-05-21T03:09:08.781 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3b748ea08891006c1152513d25ab4a03f7c9a995 2024-05-21T03:09:08.782 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3b748ea08891006c1152513d25ab4a03f7c9a995', 'tag': None, 'wait_for_package': False} 2024-05-21T03:09:08.783 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3b748ea08891006c1152513d25ab4a03f7c9a995 2024-05-21T03:09:08.784 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3b748ea08891006c1152513d25ab4a03f7c9a995', 'tag': None, 'wait_for_package': False} 2024-05-21T03:09:08.784 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3b748ea08891006c1152513d25ab4a03f7c9a995 2024-05-21T03:09:08.928 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/flavors/default/ 2024-05-21T03:09:08.929 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3830.g3b748ea0 2024-05-21T03:09:08.995 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/flavors/default/ 2024-05-21T03:09:08.995 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3830.g3b748ea0 2024-05-21T03:09:09.010 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/flavors/default/ 2024-05-21T03:09:09.010 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3830.g3b748ea0 2024-05-21T03:09:09.044 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-21T03:09:09.044 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-21T03:09:09.044 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-21T03:09:09.070 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-21T03:09:09.070 DEBUG:teuthology.orchestra.run.smithi039:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3b748ea08891006c1152513d25ab4a03f7c9a995/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-21T03:09:09.106 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-21T03:09:09.107 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-05-21T03:09:09.107 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-21T03:09:09.120 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-21T03:09:09.120 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-05-21T03:09:09.121 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-21T03:09:09.133 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-21T03:09:09.134 DEBUG:teuthology.orchestra.run.smithi027:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3b748ea08891006c1152513d25ab4a03f7c9a995/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-21T03:09:09.136 DEBUG:teuthology.orchestra.run.smithi039:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-21T03:09:09.146 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-21T03:09:09.146 DEBUG:teuthology.orchestra.run.smithi165:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3b748ea08891006c1152513d25ab4a03f7c9a995/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-21T03:09:09.202 DEBUG:teuthology.orchestra.run.smithi027:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-21T03:09:09.211 DEBUG:teuthology.orchestra.run.smithi165:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-21T03:09:09.212 DEBUG:teuthology.orchestra.run.smithi039:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-21T03:09:09.276 INFO:teuthology.orchestra.run.smithi039.stdout:check_obsoletes = 1 2024-05-21T03:09:09.277 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum clean all 2024-05-21T03:09:09.278 DEBUG:teuthology.orchestra.run.smithi027:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-21T03:09:09.283 DEBUG:teuthology.orchestra.run.smithi165:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-21T03:09:09.344 INFO:teuthology.orchestra.run.smithi027.stdout:check_obsoletes = 1 2024-05-21T03:09:09.345 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum clean all 2024-05-21T03:09:09.346 INFO:teuthology.orchestra.run.smithi165.stdout:check_obsoletes = 1 2024-05-21T03:09:09.348 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum clean all 2024-05-21T03:09:09.510 INFO:teuthology.orchestra.run.smithi039.stdout:45 files removed 2024-05-21T03:09:09.538 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-21T03:09:09.572 INFO:teuthology.orchestra.run.smithi165.stdout:45 files removed 2024-05-21T03:09:09.586 INFO:teuthology.orchestra.run.smithi027.stdout:45 files removed 2024-05-21T03:09:09.600 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-21T03:09:09.613 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-21T03:09:10.194 INFO:teuthology.orchestra.run.smithi039.stdout:ceph packages for x86_64 220 kB/s | 84 kB 00:00 2024-05-21T03:09:10.294 INFO:teuthology.orchestra.run.smithi027.stdout:ceph packages for x86_64 214 kB/s | 84 kB 00:00 2024-05-21T03:09:10.343 INFO:teuthology.orchestra.run.smithi165.stdout:ceph packages for x86_64 174 kB/s | 84 kB 00:00 2024-05-21T03:09:10.553 INFO:teuthology.orchestra.run.smithi039.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-05-21T03:09:10.669 INFO:teuthology.orchestra.run.smithi027.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-05-21T03:09:10.777 INFO:teuthology.orchestra.run.smithi165.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-05-21T03:09:10.937 INFO:teuthology.orchestra.run.smithi039.stdout:ceph source packages 5.5 kB/s | 1.9 kB 00:00 2024-05-21T03:09:11.095 INFO:teuthology.orchestra.run.smithi027.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-05-21T03:09:11.128 INFO:teuthology.orchestra.run.smithi165.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-05-21T03:09:13.145 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - BaseOS 4.0 MB/s | 8.1 MB 00:02 2024-05-21T03:09:14.111 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - BaseOS 2.6 MB/s | 8.1 MB 00:03 2024-05-21T03:09:15.147 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-05-21T03:09:15.853 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - BaseOS 1.7 MB/s | 8.1 MB 00:04 2024-05-21T03:09:15.940 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-05-21T03:09:21.048 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - CRB 2.8 MB/s | 6.2 MB 00:02 2024-05-21T03:09:22.573 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 16 kB 00:00 2024-05-21T03:09:23.480 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - CRB 1.6 MB/s | 6.2 MB 00:03 2024-05-21T03:09:24.727 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-05-21T03:09:24.939 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - Extras packages 65 kB/s | 16 kB 00:00 2024-05-21T03:09:26.582 INFO:teuthology.orchestra.run.smithi039.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-05-21T03:09:29.983 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-05-21T03:09:31.899 INFO:teuthology.orchestra.run.smithi039.stdout:lab-extras 49 kB/s | 1.7 kB 00:00 2024-05-21T03:09:32.243 INFO:teuthology.orchestra.run.smithi027.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-21T03:09:32.244 INFO:teuthology.orchestra.run.smithi027.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-21T03:09:32.255 INFO:teuthology.orchestra.run.smithi027.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-21T03:09:32.256 INFO:teuthology.orchestra.run.smithi027.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-21T03:09:32.257 INFO:teuthology.orchestra.run.smithi027.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-21T03:09:32.258 INFO:teuthology.orchestra.run.smithi027.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-21T03:09:32.259 INFO:teuthology.orchestra.run.smithi027.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-21T03:09:32.262 INFO:teuthology.orchestra.run.smithi027.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-21T03:09:32.262 INFO:teuthology.orchestra.run.smithi027.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-21T03:09:32.328 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-05-21T03:09:32.344 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================= 2024-05-21T03:09:32.344 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-05-21T03:09:32.344 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================= 2024-05-21T03:09:32.345 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-05-21T03:09:32.345 INFO:teuthology.orchestra.run.smithi027.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-21T03:09:32.345 INFO:teuthology.orchestra.run.smithi027.stdout: ceph x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 6.6 k 2024-05-21T03:09:32.345 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 5.4 M 2024-05-21T03:09:32.345 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 891 k 2024-05-21T03:09:32.345 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 145 k 2024-05-21T03:09:32.345 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 1.5 M 2024-05-21T03:09:32.345 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 143 k 2024-05-21T03:09:32.345 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 3.5 M 2024-05-21T03:09:32.345 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 7.4 M 2024-05-21T03:09:32.345 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 49 k 2024-05-21T03:09:32.345 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 11 M 2024-05-21T03:09:32.346 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 49 M 2024-05-21T03:09:32.346 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 273 k 2024-05-21T03:09:32.346 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 764 k 2024-05-21T03:09:32.346 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-mirror x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 225 k 2024-05-21T03:09:32.346 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-shell x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 27 k 2024-05-21T03:09:32.346 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-top noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 18 k 2024-05-21T03:09:32.346 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-21T03:09:32.346 INFO:teuthology.orchestra.run.smithi027.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-21T03:09:32.346 INFO:teuthology.orchestra.run.smithi027.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-21T03:09:32.346 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 32 k 2024-05-21T03:09:32.347 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2 x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 746 k 2024-05-21T03:09:32.347 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 127 k 2024-05-21T03:09:32.347 INFO:teuthology.orchestra.run.smithi027.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-21T03:09:32.347 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-21T03:09:32.347 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-21T03:09:32.347 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-21T03:09:32.347 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-21T03:09:32.347 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-21T03:09:32.347 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 161 k 2024-05-21T03:09:32.347 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 320 k 2024-05-21T03:09:32.347 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 300 k 2024-05-21T03:09:32.347 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 99 k 2024-05-21T03:09:32.348 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 85 k 2024-05-21T03:09:32.348 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 3.1 M 2024-05-21T03:09:32.348 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 171 k 2024-05-21T03:09:32.348 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-21T03:09:32.348 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrading: 2024-05-21T03:09:32.348 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-21T03:09:32.348 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-21T03:09:32.348 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-21T03:09:32.348 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-21T03:09:32.348 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-21T03:09:32.348 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-21T03:09:32.348 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-21T03:09:32.348 INFO:teuthology.orchestra.run.smithi027.stdout: librados2 x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 3.4 M 2024-05-21T03:09:32.349 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1 x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 3.1 M 2024-05-21T03:09:32.349 INFO:teuthology.orchestra.run.smithi027.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-21T03:09:32.349 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-21T03:09:32.349 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-05-21T03:09:32.349 INFO:teuthology.orchestra.run.smithi027.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-21T03:09:32.349 INFO:teuthology.orchestra.run.smithi027.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-21T03:09:32.349 INFO:teuthology.orchestra.run.smithi027.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-21T03:09:32.349 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 21 M 2024-05-21T03:09:32.349 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 34 k 2024-05-21T03:09:32.349 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 2.4 M 2024-05-21T03:09:32.349 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 274 k 2024-05-21T03:09:32.349 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 4.7 M 2024-05-21T03:09:32.349 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 17 M 2024-05-21T03:09:32.350 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 16 k 2024-05-21T03:09:32.350 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 25 k 2024-05-21T03:09:32.350 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-21T03:09:32.350 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-21T03:09:32.350 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-21T03:09:32.350 INFO:teuthology.orchestra.run.smithi027.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-21T03:09:32.350 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-21T03:09:32.350 INFO:teuthology.orchestra.run.smithi027.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-21T03:09:32.350 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-21T03:09:32.350 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-21T03:09:32.350 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 163 k 2024-05-21T03:09:32.350 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-21T03:09:32.351 INFO:teuthology.orchestra.run.smithi027.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-21T03:09:32.351 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-21T03:09:32.351 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-21T03:09:32.351 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-21T03:09:32.351 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1 x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 501 k 2024-05-21T03:09:32.351 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-21T03:09:32.351 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2 x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 5.4 M 2024-05-21T03:09:32.351 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-21T03:09:32.351 INFO:teuthology.orchestra.run.smithi027.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-21T03:09:32.351 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-21T03:09:32.351 INFO:teuthology.orchestra.run.smithi027.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-21T03:09:32.351 INFO:teuthology.orchestra.run.smithi027.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-21T03:09:32.351 INFO:teuthology.orchestra.run.smithi027.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-21T03:09:32.352 INFO:teuthology.orchestra.run.smithi027.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-21T03:09:32.352 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-21T03:09:32.352 INFO:teuthology.orchestra.run.smithi027.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-21T03:09:32.352 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-21T03:09:32.352 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-21T03:09:32.352 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-21T03:09:32.352 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-21T03:09:32.352 INFO:teuthology.orchestra.run.smithi027.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-21T03:09:32.352 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-21T03:09:32.352 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-21T03:09:32.352 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-21T03:09:32.352 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 45 k 2024-05-21T03:09:32.352 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 136 k 2024-05-21T03:09:32.353 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-21T03:09:32.353 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-21T03:09:32.353 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-21T03:09:32.353 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-21T03:09:32.353 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-21T03:09:32.353 INFO:teuthology.orchestra.run.smithi027.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-21T03:09:32.353 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-21T03:09:32.353 INFO:teuthology.orchestra.run.smithi027.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-21T03:09:32.353 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-21T03:09:32.353 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-21T03:09:32.353 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-21T03:09:32.353 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-21T03:09:32.353 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-21T03:09:32.353 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-21T03:09:32.354 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-21T03:09:32.354 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-21T03:09:32.354 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-21T03:09:32.354 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-21T03:09:32.354 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-21T03:09:32.354 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-21T03:09:32.354 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-21T03:09:32.354 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-21T03:09:32.354 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-21T03:09:32.354 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-21T03:09:32.354 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-21T03:09:32.354 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-21T03:09:32.355 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-21T03:09:32.355 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-21T03:09:32.355 INFO:teuthology.orchestra.run.smithi027.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-21T03:09:32.355 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-21T03:09:32.355 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-21T03:09:32.355 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-21T03:09:32.355 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-21T03:09:32.355 INFO:teuthology.orchestra.run.smithi027.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-21T03:09:32.355 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-21T03:09:32.355 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-21T03:09:32.355 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-21T03:09:32.355 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-21T03:09:32.355 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-21T03:09:32.356 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-21T03:09:32.356 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-21T03:09:32.356 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-21T03:09:32.356 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-21T03:09:32.356 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-21T03:09:32.356 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-21T03:09:32.356 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-21T03:09:32.356 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-21T03:09:32.356 INFO:teuthology.orchestra.run.smithi027.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-21T03:09:32.356 INFO:teuthology.orchestra.run.smithi027.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-21T03:09:32.356 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-21T03:09:32.356 INFO:teuthology.orchestra.run.smithi027.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-21T03:09:32.356 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-21T03:09:32.357 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-21T03:09:32.357 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-21T03:09:32.357 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-21T03:09:32.357 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-21T03:09:32.357 INFO:teuthology.orchestra.run.smithi027.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-21T03:09:32.357 INFO:teuthology.orchestra.run.smithi027.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-21T03:09:32.357 INFO:teuthology.orchestra.run.smithi027.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-21T03:09:32.357 INFO:teuthology.orchestra.run.smithi027.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-21T03:09:32.357 INFO:teuthology.orchestra.run.smithi027.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-21T03:09:32.357 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-21T03:09:32.357 INFO:teuthology.orchestra.run.smithi027.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-21T03:09:32.357 INFO:teuthology.orchestra.run.smithi027.stdout:Installing weak dependencies: 2024-05-21T03:09:32.357 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-21T03:09:32.357 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:09:32.358 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-05-21T03:09:32.358 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================= 2024-05-21T03:09:32.358 INFO:teuthology.orchestra.run.smithi027.stdout:Install 148 Packages 2024-05-21T03:09:32.358 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrade 11 Packages 2024-05-21T03:09:32.358 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:09:32.358 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 227 M 2024-05-21T03:09:32.359 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-05-21T03:09:33.251 INFO:teuthology.orchestra.run.smithi027.stdout:(1/159): ceph-19.0.0-3830.g3b748ea0.el9.x86_64. 38 kB/s | 6.6 kB 00:00 2024-05-21T03:09:33.485 INFO:teuthology.orchestra.run.smithi027.stdout:(2/159): ceph-fuse-19.0.0-3830.g3b748ea0.el9.x8 3.7 MB/s | 891 kB 00:00 2024-05-21T03:09:33.560 INFO:teuthology.orchestra.run.smithi027.stdout:(3/159): ceph-immutable-object-cache-19.0.0-383 1.9 MB/s | 145 kB 00:00 2024-05-21T03:09:33.601 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:16 2024-05-21T03:09:33.960 INFO:teuthology.orchestra.run.smithi027.stdout:(4/159): ceph-mds-19.0.0-3830.g3b748ea0.el9.x86 5.9 MB/s | 2.4 MB 00:00 2024-05-21T03:09:34.101 INFO:teuthology.orchestra.run.smithi039.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-21T03:09:34.101 INFO:teuthology.orchestra.run.smithi039.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-21T03:09:34.113 INFO:teuthology.orchestra.run.smithi039.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-21T03:09:34.114 INFO:teuthology.orchestra.run.smithi039.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-21T03:09:34.115 INFO:teuthology.orchestra.run.smithi039.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-21T03:09:34.115 INFO:teuthology.orchestra.run.smithi039.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-21T03:09:34.117 INFO:teuthology.orchestra.run.smithi039.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-21T03:09:34.119 INFO:teuthology.orchestra.run.smithi039.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-21T03:09:34.120 INFO:teuthology.orchestra.run.smithi039.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-21T03:09:34.187 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-05-21T03:09:34.205 INFO:teuthology.orchestra.run.smithi039.stdout:======================================================================================= 2024-05-21T03:09:34.205 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2024-05-21T03:09:34.205 INFO:teuthology.orchestra.run.smithi039.stdout:======================================================================================= 2024-05-21T03:09:34.206 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-05-21T03:09:34.206 INFO:teuthology.orchestra.run.smithi039.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-21T03:09:34.206 INFO:teuthology.orchestra.run.smithi039.stdout: ceph x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 6.6 k 2024-05-21T03:09:34.206 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-base x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 5.4 M 2024-05-21T03:09:34.206 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-fuse x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 891 k 2024-05-21T03:09:34.206 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 145 k 2024-05-21T03:09:34.206 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 1.5 M 2024-05-21T03:09:34.206 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 143 k 2024-05-21T03:09:34.206 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 3.5 M 2024-05-21T03:09:34.206 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 7.4 M 2024-05-21T03:09:34.206 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-rook noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 49 k 2024-05-21T03:09:34.206 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-radosgw x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 11 M 2024-05-21T03:09:34.207 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-test x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 49 M 2024-05-21T03:09:34.207 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-volume noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 273 k 2024-05-21T03:09:34.207 INFO:teuthology.orchestra.run.smithi039.stdout: cephadm noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 764 k 2024-05-21T03:09:34.207 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-mirror x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 225 k 2024-05-21T03:09:34.207 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-shell x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 27 k 2024-05-21T03:09:34.207 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-top noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 18 k 2024-05-21T03:09:34.207 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-21T03:09:34.207 INFO:teuthology.orchestra.run.smithi039.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-21T03:09:34.207 INFO:teuthology.orchestra.run.smithi039.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-21T03:09:34.207 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs-devel x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 32 k 2024-05-21T03:09:34.207 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs2 x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 746 k 2024-05-21T03:09:34.207 INFO:teuthology.orchestra.run.smithi039.stdout: librados-devel x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 127 k 2024-05-21T03:09:34.207 INFO:teuthology.orchestra.run.smithi039.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-21T03:09:34.207 INFO:teuthology.orchestra.run.smithi039.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-21T03:09:34.208 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-21T03:09:34.208 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-21T03:09:34.208 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-21T03:09:34.208 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-21T03:09:34.208 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cephfs x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 161 k 2024-05-21T03:09:34.208 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rados x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 320 k 2024-05-21T03:09:34.208 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rbd x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 300 k 2024-05-21T03:09:34.208 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rgw x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 99 k 2024-05-21T03:09:34.208 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-fuse x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 85 k 2024-05-21T03:09:34.208 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-mirror x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 3.1 M 2024-05-21T03:09:34.208 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-nbd x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 171 k 2024-05-21T03:09:34.208 INFO:teuthology.orchestra.run.smithi039.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-21T03:09:34.208 INFO:teuthology.orchestra.run.smithi039.stdout:Upgrading: 2024-05-21T03:09:34.209 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-21T03:09:34.209 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-21T03:09:34.209 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-21T03:09:34.209 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-21T03:09:34.209 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-21T03:09:34.209 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-21T03:09:34.209 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-21T03:09:34.209 INFO:teuthology.orchestra.run.smithi039.stdout: librados2 x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 3.4 M 2024-05-21T03:09:34.209 INFO:teuthology.orchestra.run.smithi039.stdout: librbd1 x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 3.1 M 2024-05-21T03:09:34.209 INFO:teuthology.orchestra.run.smithi039.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-21T03:09:34.209 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-21T03:09:34.209 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2024-05-21T03:09:34.209 INFO:teuthology.orchestra.run.smithi039.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-21T03:09:34.209 INFO:teuthology.orchestra.run.smithi039.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-21T03:09:34.210 INFO:teuthology.orchestra.run.smithi039.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-21T03:09:34.210 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-common x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 21 M 2024-05-21T03:09:34.210 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 34 k 2024-05-21T03:09:34.210 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mds x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 2.4 M 2024-05-21T03:09:34.210 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 274 k 2024-05-21T03:09:34.210 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mon x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 4.7 M 2024-05-21T03:09:34.210 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-osd x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 17 M 2024-05-21T03:09:34.210 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 16 k 2024-05-21T03:09:34.210 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-selinux x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 25 k 2024-05-21T03:09:34.210 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-21T03:09:34.210 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-21T03:09:34.210 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-21T03:09:34.210 INFO:teuthology.orchestra.run.smithi039.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-21T03:09:34.211 INFO:teuthology.orchestra.run.smithi039.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-21T03:09:34.211 INFO:teuthology.orchestra.run.smithi039.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-21T03:09:34.211 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-21T03:09:34.211 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-21T03:09:34.211 INFO:teuthology.orchestra.run.smithi039.stdout: libcephsqlite x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 163 k 2024-05-21T03:09:34.211 INFO:teuthology.orchestra.run.smithi039.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-21T03:09:34.211 INFO:teuthology.orchestra.run.smithi039.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-21T03:09:34.211 INFO:teuthology.orchestra.run.smithi039.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-21T03:09:34.211 INFO:teuthology.orchestra.run.smithi039.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-21T03:09:34.211 INFO:teuthology.orchestra.run.smithi039.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-21T03:09:34.211 INFO:teuthology.orchestra.run.smithi039.stdout: libradosstriper1 x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 501 k 2024-05-21T03:09:34.211 INFO:teuthology.orchestra.run.smithi039.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-21T03:09:34.211 INFO:teuthology.orchestra.run.smithi039.stdout: librgw2 x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 5.4 M 2024-05-21T03:09:34.212 INFO:teuthology.orchestra.run.smithi039.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-21T03:09:34.212 INFO:teuthology.orchestra.run.smithi039.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-21T03:09:34.212 INFO:teuthology.orchestra.run.smithi039.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-21T03:09:34.212 INFO:teuthology.orchestra.run.smithi039.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-21T03:09:34.212 INFO:teuthology.orchestra.run.smithi039.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-21T03:09:34.212 INFO:teuthology.orchestra.run.smithi039.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-21T03:09:34.212 INFO:teuthology.orchestra.run.smithi039.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-21T03:09:34.212 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-21T03:09:34.212 INFO:teuthology.orchestra.run.smithi039.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-21T03:09:34.212 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-21T03:09:34.212 INFO:teuthology.orchestra.run.smithi039.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-21T03:09:34.212 INFO:teuthology.orchestra.run.smithi039.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-21T03:09:34.212 INFO:teuthology.orchestra.run.smithi039.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-21T03:09:34.213 INFO:teuthology.orchestra.run.smithi039.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-21T03:09:34.213 INFO:teuthology.orchestra.run.smithi039.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-21T03:09:34.213 INFO:teuthology.orchestra.run.smithi039.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-21T03:09:34.213 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-21T03:09:34.213 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-argparse x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 45 k 2024-05-21T03:09:34.213 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-common x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 136 k 2024-05-21T03:09:34.213 INFO:teuthology.orchestra.run.smithi039.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-21T03:09:34.213 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-21T03:09:34.213 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-21T03:09:34.213 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-21T03:09:34.213 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-21T03:09:34.213 INFO:teuthology.orchestra.run.smithi039.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-21T03:09:34.213 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-21T03:09:34.213 INFO:teuthology.orchestra.run.smithi039.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-21T03:09:34.214 INFO:teuthology.orchestra.run.smithi039.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-21T03:09:34.214 INFO:teuthology.orchestra.run.smithi039.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-21T03:09:34.214 INFO:teuthology.orchestra.run.smithi039.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-21T03:09:34.214 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-21T03:09:34.214 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-21T03:09:34.214 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-21T03:09:34.214 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-21T03:09:34.214 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-21T03:09:34.214 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-21T03:09:34.214 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-21T03:09:34.214 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-21T03:09:34.214 INFO:teuthology.orchestra.run.smithi039.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-21T03:09:34.214 INFO:teuthology.orchestra.run.smithi039.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-21T03:09:34.215 INFO:teuthology.orchestra.run.smithi039.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-21T03:09:34.215 INFO:teuthology.orchestra.run.smithi039.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-21T03:09:34.215 INFO:teuthology.orchestra.run.smithi039.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-21T03:09:34.215 INFO:teuthology.orchestra.run.smithi039.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-21T03:09:34.215 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-21T03:09:34.215 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-21T03:09:34.215 INFO:teuthology.orchestra.run.smithi039.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-21T03:09:34.215 INFO:teuthology.orchestra.run.smithi039.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-21T03:09:34.215 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-21T03:09:34.215 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-21T03:09:34.215 INFO:teuthology.orchestra.run.smithi039.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-21T03:09:34.215 INFO:teuthology.orchestra.run.smithi039.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-21T03:09:34.215 INFO:teuthology.orchestra.run.smithi039.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-21T03:09:34.215 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-21T03:09:34.216 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-21T03:09:34.216 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-21T03:09:34.216 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-21T03:09:34.216 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-21T03:09:34.216 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-21T03:09:34.216 INFO:teuthology.orchestra.run.smithi039.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-21T03:09:34.216 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-21T03:09:34.216 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-21T03:09:34.216 INFO:teuthology.orchestra.run.smithi039.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-21T03:09:34.216 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-21T03:09:34.216 INFO:teuthology.orchestra.run.smithi039.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-21T03:09:34.216 INFO:teuthology.orchestra.run.smithi039.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-21T03:09:34.216 INFO:teuthology.orchestra.run.smithi039.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-21T03:09:34.217 INFO:teuthology.orchestra.run.smithi039.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-21T03:09:34.217 INFO:teuthology.orchestra.run.smithi039.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-21T03:09:34.217 INFO:teuthology.orchestra.run.smithi039.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-21T03:09:34.217 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-21T03:09:34.217 INFO:teuthology.orchestra.run.smithi039.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-21T03:09:34.217 INFO:teuthology.orchestra.run.smithi039.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-21T03:09:34.217 INFO:teuthology.orchestra.run.smithi039.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-21T03:09:34.217 INFO:teuthology.orchestra.run.smithi039.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-21T03:09:34.217 INFO:teuthology.orchestra.run.smithi039.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-21T03:09:34.217 INFO:teuthology.orchestra.run.smithi039.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-21T03:09:34.217 INFO:teuthology.orchestra.run.smithi039.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-21T03:09:34.217 INFO:teuthology.orchestra.run.smithi039.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-21T03:09:34.217 INFO:teuthology.orchestra.run.smithi039.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-21T03:09:34.217 INFO:teuthology.orchestra.run.smithi039.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-21T03:09:34.218 INFO:teuthology.orchestra.run.smithi039.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-21T03:09:34.218 INFO:teuthology.orchestra.run.smithi039.stdout:Installing weak dependencies: 2024-05-21T03:09:34.218 INFO:teuthology.orchestra.run.smithi039.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-21T03:09:34.218 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-21T03:09:34.218 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-05-21T03:09:34.218 INFO:teuthology.orchestra.run.smithi039.stdout:======================================================================================= 2024-05-21T03:09:34.218 INFO:teuthology.orchestra.run.smithi039.stdout:Install 148 Packages 2024-05-21T03:09:34.218 INFO:teuthology.orchestra.run.smithi039.stdout:Upgrade 11 Packages 2024-05-21T03:09:34.218 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-21T03:09:34.219 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 227 M 2024-05-21T03:09:34.219 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-05-21T03:09:34.286 INFO:teuthology.orchestra.run.smithi027.stdout:(5/159): ceph-mgr-19.0.0-3830.g3b748ea0.el9.x86 4.7 MB/s | 1.5 MB 00:00 2024-05-21T03:09:34.553 INFO:teuthology.orchestra.run.smithi027.stdout:(6/159): ceph-common-19.0.0-3830.g3b748ea0.el9. 14 MB/s | 21 MB 00:01 2024-05-21T03:09:34.812 INFO:teuthology.orchestra.run.smithi027.stdout:(7/159): ceph-base-19.0.0-3830.g3b748ea0.el9.x8 3.1 MB/s | 5.4 MB 00:01 2024-05-21T03:09:34.953 INFO:teuthology.orchestra.run.smithi027.stdout:(8/159): ceph-mon-19.0.0-3830.g3b748ea0.el9.x86 7.0 MB/s | 4.7 MB 00:00 2024-05-21T03:09:35.029 INFO:teuthology.orchestra.run.smithi027.stdout:(9/159): ceph-selinux-19.0.0-3830.g3b748ea0.el9 335 kB/s | 25 kB 00:00 2024-05-21T03:09:35.391 INFO:teuthology.orchestra.run.smithi039.stdout:(1/159): ceph-19.0.0-3830.g3b748ea0.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-05-21T03:09:35.729 INFO:teuthology.orchestra.run.smithi027.stdout:(10/159): ceph-osd-19.0.0-3830.g3b748ea0.el9.x8 14 MB/s | 17 MB 00:01 2024-05-21T03:09:35.792 INFO:teuthology.orchestra.run.smithi039.stdout:(2/159): ceph-fuse-19.0.0-3830.g3b748ea0.el9.x8 2.2 MB/s | 891 kB 00:00 2024-05-21T03:09:35.846 INFO:teuthology.orchestra.run.smithi027.stdout:(11/159): cephfs-mirror-19.0.0-3830.g3b748ea0.e 1.9 MB/s | 225 kB 00:00 2024-05-21T03:09:35.929 INFO:teuthology.orchestra.run.smithi027.stdout:(12/159): cephfs-shell-19.0.0-3830.g3b748ea0.el 324 kB/s | 27 kB 00:00 2024-05-21T03:09:36.013 INFO:teuthology.orchestra.run.smithi027.stdout:(13/159): libcephfs-devel-19.0.0-3830.g3b748ea0 381 kB/s | 32 kB 00:00 2024-05-21T03:09:36.263 INFO:teuthology.orchestra.run.smithi027.stdout:(14/159): ceph-radosgw-19.0.0-3830.g3b748ea0.el 7.3 MB/s | 11 MB 00:01 2024-05-21T03:09:36.325 INFO:teuthology.orchestra.run.smithi039.stdout:(3/159): ceph-base-19.0.0-3830.g3b748ea0.el9.x8 4.1 MB/s | 5.4 MB 00:01 2024-05-21T03:09:36.330 INFO:teuthology.orchestra.run.smithi027.stdout:(15/159): libcephsqlite-19.0.0-3830.g3b748ea0.e 2.4 MB/s | 163 kB 00:00 2024-05-21T03:09:36.355 INFO:teuthology.orchestra.run.smithi027.stdout:(16/159): libcephfs2-19.0.0-3830.g3b748ea0.el9. 2.1 MB/s | 746 kB 00:00 2024-05-21T03:09:36.400 INFO:teuthology.orchestra.run.smithi039.stdout:(4/159): ceph-immutable-object-cache-19.0.0-383 238 kB/s | 145 kB 00:00 2024-05-21T03:09:36.414 INFO:teuthology.orchestra.run.smithi027.stdout:(17/159): librados-devel-19.0.0-3830.g3b748ea0. 1.5 MB/s | 127 kB 00:00 2024-05-21T03:09:36.439 INFO:teuthology.orchestra.run.smithi027.stdout:(18/159): libradosstriper1-19.0.0-3830.g3b748ea 5.9 MB/s | 501 kB 00:00 2024-05-21T03:09:36.523 INFO:teuthology.orchestra.run.smithi027.stdout:(19/159): python3-ceph-argparse-19.0.0-3830.g3b 541 kB/s | 45 kB 00:00 2024-05-21T03:09:36.615 INFO:teuthology.orchestra.run.smithi027.stdout:(20/159): python3-ceph-common-19.0.0-3830.g3b74 1.4 MB/s | 136 kB 00:00 2024-05-21T03:09:36.690 INFO:teuthology.orchestra.run.smithi027.stdout:(21/159): python3-cephfs-19.0.0-3830.g3b748ea0. 2.1 MB/s | 161 kB 00:00 2024-05-21T03:09:36.710 INFO:teuthology.orchestra.run.smithi039.stdout:(5/159): ceph-mds-19.0.0-3830.g3b748ea0.el9.x86 6.2 MB/s | 2.4 MB 00:00 2024-05-21T03:09:36.743 INFO:teuthology.orchestra.run.smithi039.stdout:(6/159): ceph-mgr-19.0.0-3830.g3b748ea0.el9.x86 4.4 MB/s | 1.5 MB 00:00 2024-05-21T03:09:36.757 INFO:teuthology.orchestra.run.smithi027.stdout:(22/159): python3-rados-19.0.0-3830.g3b748ea0.e 4.7 MB/s | 320 kB 00:00 2024-05-21T03:09:36.858 INFO:teuthology.orchestra.run.smithi027.stdout:(23/159): python3-rbd-19.0.0-3830.g3b748ea0.el9 2.9 MB/s | 300 kB 00:00 2024-05-21T03:09:36.966 INFO:teuthology.orchestra.run.smithi027.stdout:(24/159): python3-rgw-19.0.0-3830.g3b748ea0.el9 918 kB/s | 99 kB 00:00 2024-05-21T03:09:37.058 INFO:teuthology.orchestra.run.smithi027.stdout:(25/159): rbd-fuse-19.0.0-3830.g3b748ea0.el9.x8 929 kB/s | 85 kB 00:00 2024-05-21T03:09:37.159 INFO:teuthology.orchestra.run.smithi027.stdout:(26/159): librgw2-19.0.0-3830.g3b748ea0.el9.x86 7.2 MB/s | 5.4 MB 00:00 2024-05-21T03:09:37.267 INFO:teuthology.orchestra.run.smithi027.stdout:(27/159): rbd-nbd-19.0.0-3830.g3b748ea0.el9.x86 1.5 MB/s | 171 kB 00:00 2024-05-21T03:09:37.368 INFO:teuthology.orchestra.run.smithi027.stdout:(28/159): ceph-grafana-dashboards-19.0.0-3830.g 343 kB/s | 34 kB 00:00 2024-05-21T03:09:37.401 INFO:teuthology.orchestra.run.smithi039.stdout:(7/159): ceph-mon-19.0.0-3830.g3b748ea0.el9.x86 6.8 MB/s | 4.7 MB 00:00 2024-05-21T03:09:37.434 INFO:teuthology.orchestra.run.smithi027.stdout:(29/159): ceph-mgr-cephadm-19.0.0-3830.g3b748ea 2.1 MB/s | 143 kB 00:00 2024-05-21T03:09:37.543 INFO:teuthology.orchestra.run.smithi027.stdout:(30/159): rbd-mirror-19.0.0-3830.g3b748ea0.el9. 6.4 MB/s | 3.1 MB 00:00 2024-05-21T03:09:37.985 INFO:teuthology.orchestra.run.smithi027.stdout:(31/159): ceph-mgr-dashboard-19.0.0-3830.g3b748 6.4 MB/s | 3.5 MB 00:00 2024-05-21T03:09:38.077 INFO:teuthology.orchestra.run.smithi027.stdout:(32/159): ceph-mgr-modules-core-19.0.0-3830.g3b 2.9 MB/s | 274 kB 00:00 2024-05-21T03:09:38.169 INFO:teuthology.orchestra.run.smithi027.stdout:(33/159): ceph-mgr-rook-19.0.0-3830.g3b748ea0.e 537 kB/s | 49 kB 00:00 2024-05-21T03:09:38.253 INFO:teuthology.orchestra.run.smithi027.stdout:(34/159): ceph-prometheus-alerts-19.0.0-3830.g3 197 kB/s | 16 kB 00:00 2024-05-21T03:09:38.345 INFO:teuthology.orchestra.run.smithi027.stdout:(35/159): ceph-volume-19.0.0-3830.g3b748ea0.el9 2.9 MB/s | 273 kB 00:00 2024-05-21T03:09:38.520 INFO:teuthology.orchestra.run.smithi027.stdout:(36/159): cephadm-19.0.0-3830.g3b748ea0.el9.noa 4.3 MB/s | 764 kB 00:00 2024-05-21T03:09:38.612 INFO:teuthology.orchestra.run.smithi027.stdout:(37/159): cephfs-top-19.0.0-3830.g3b748ea0.el9. 199 kB/s | 18 kB 00:00 2024-05-21T03:09:38.871 INFO:teuthology.orchestra.run.smithi027.stdout:(38/159): ceph-mgr-diskprediction-local-19.0.0- 5.6 MB/s | 7.4 MB 00:01 2024-05-21T03:09:38.935 INFO:teuthology.orchestra.run.smithi039.stdout:(8/159): ceph-radosgw-19.0.0-3830.g3b748ea0.el9 6.9 MB/s | 11 MB 00:01 2024-05-21T03:09:39.010 INFO:teuthology.orchestra.run.smithi039.stdout:(9/159): ceph-selinux-19.0.0-3830.g3b748ea0.el9 335 kB/s | 25 kB 00:00 2024-05-21T03:09:39.063 INFO:teuthology.orchestra.run.smithi027.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 178 kB/s | 80 kB 00:00 2024-05-21T03:09:39.438 INFO:teuthology.orchestra.run.smithi027.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 521 kB/s | 195 kB 00:00 2024-05-21T03:09:39.621 INFO:teuthology.orchestra.run.smithi027.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.4 MB/s | 253 kB 00:00 2024-05-21T03:09:39.755 INFO:teuthology.orchestra.run.smithi027.stdout:(42/159): libgfortran-11.4.1-3.el9.x86_64.rpm 908 kB/s | 803 kB 00:00 2024-05-21T03:09:39.864 INFO:teuthology.orchestra.run.smithi027.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 984 kB/s | 106 kB 00:00 2024-05-21T03:09:39.906 INFO:teuthology.orchestra.run.smithi027.stdout:(44/159): python3-cryptography-36.0.1-4.el9.x86 4.3 MB/s | 1.2 MB 00:00 2024-05-21T03:09:39.965 INFO:teuthology.orchestra.run.smithi027.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-05-21T03:09:40.023 INFO:teuthology.orchestra.run.smithi027.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-05-21T03:09:40.073 INFO:teuthology.orchestra.run.smithi027.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-05-21T03:09:40.140 INFO:teuthology.orchestra.run.smithi027.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 5.7 MB/s | 689 kB 00:00 2024-05-21T03:09:40.160 INFO:teuthology.orchestra.run.smithi039.stdout:(10/159): ceph-osd-19.0.0-3830.g3b748ea0.el9.x8 4.9 MB/s | 17 MB 00:03 2024-05-21T03:09:40.244 INFO:teuthology.orchestra.run.smithi039.stdout:(11/159): cephfs-mirror-19.0.0-3830.g3b748ea0.e 2.6 MB/s | 225 kB 00:00 2024-05-21T03:09:40.249 INFO:teuthology.orchestra.run.smithi027.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 987 kB/s | 107 kB 00:00 2024-05-21T03:09:40.319 INFO:teuthology.orchestra.run.smithi039.stdout:(12/159): cephfs-shell-19.0.0-3830.g3b748ea0.el 361 kB/s | 27 kB 00:00 2024-05-21T03:09:40.358 INFO:teuthology.orchestra.run.smithi027.stdout:(50/159): elfutils-libelf-devel-0.191-4.el9.x86 200 kB/s | 22 kB 00:00 2024-05-21T03:09:40.394 INFO:teuthology.orchestra.run.smithi039.stdout:(13/159): libcephfs-devel-19.0.0-3830.g3b748ea0 424 kB/s | 32 kB 00:00 2024-05-21T03:09:40.399 INFO:teuthology.orchestra.run.smithi027.stdout:(51/159): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-05-21T03:09:40.492 INFO:teuthology.orchestra.run.smithi027.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-05-21T03:09:40.517 INFO:teuthology.orchestra.run.smithi027.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 279 kB/s | 33 kB 00:00 2024-05-21T03:09:40.570 INFO:teuthology.orchestra.run.smithi039.stdout:(14/159): libcephfs2-19.0.0-3830.g3b748ea0.el9. 4.2 MB/s | 746 kB 00:00 2024-05-21T03:09:40.626 INFO:teuthology.orchestra.run.smithi027.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-05-21T03:09:40.653 INFO:teuthology.orchestra.run.smithi039.stdout:(15/159): libcephsqlite-19.0.0-3830.g3b748ea0.e 1.9 MB/s | 163 kB 00:00 2024-05-21T03:09:40.734 INFO:teuthology.orchestra.run.smithi027.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 11 kB 00:00 2024-05-21T03:09:40.843 INFO:teuthology.orchestra.run.smithi027.stdout:(56/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-05-21T03:09:40.870 INFO:teuthology.orchestra.run.smithi039.stdout:(16/159): ceph-common-19.0.0-3830.g3b748ea0.el9 3.6 MB/s | 21 MB 00:05 2024-05-21T03:09:40.877 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - CRB 1.7 MB/s | 6.2 MB 00:03 2024-05-21T03:09:40.896 INFO:teuthology.orchestra.run.smithi039.stdout:(17/159): librados-devel-19.0.0-3830.g3b748ea0. 523 kB/s | 127 kB 00:00 2024-05-21T03:09:40.951 INFO:teuthology.orchestra.run.smithi027.stdout:(57/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 418 kB/s | 45 kB 00:00 2024-05-21T03:09:41.046 INFO:teuthology.orchestra.run.smithi039.stdout:(18/159): libradosstriper1-19.0.0-3830.g3b748ea 2.8 MB/s | 501 kB 00:00 2024-05-21T03:09:41.068 INFO:teuthology.orchestra.run.smithi027.stdout:(58/159): librdkafka-1.6.1-102.el9.x86_64.rpm 5.6 MB/s | 662 kB 00:00 2024-05-21T03:09:41.121 INFO:teuthology.orchestra.run.smithi039.stdout:(19/159): python3-ceph-argparse-19.0.0-3830.g3b 602 kB/s | 45 kB 00:00 2024-05-21T03:09:41.194 INFO:teuthology.orchestra.run.smithi027.stdout:(59/159): libtool-2.4.6-45.el9.x86_64.rpm 4.5 MB/s | 582 kB 00:00 2024-05-21T03:09:41.196 INFO:teuthology.orchestra.run.smithi039.stdout:(20/159): python3-ceph-common-19.0.0-3830.g3b74 1.8 MB/s | 136 kB 00:00 2024-05-21T03:09:41.280 INFO:teuthology.orchestra.run.smithi039.stdout:(21/159): python3-cephfs-19.0.0-3830.g3b748ea0. 1.9 MB/s | 161 kB 00:00 2024-05-21T03:09:41.302 INFO:teuthology.orchestra.run.smithi027.stdout:(60/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 235 kB/s | 25 kB 00:00 2024-05-21T03:09:41.406 INFO:teuthology.orchestra.run.smithi039.stdout:(22/159): python3-rados-19.0.0-3830.g3b748ea0.e 2.5 MB/s | 320 kB 00:00 2024-05-21T03:09:41.411 INFO:teuthology.orchestra.run.smithi027.stdout:(61/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 437 kB/s | 47 kB 00:00 2024-05-21T03:09:41.564 INFO:teuthology.orchestra.run.smithi039.stdout:(23/159): python3-rbd-19.0.0-3830.g3b748ea0.el9 1.9 MB/s | 300 kB 00:00 2024-05-21T03:09:41.570 INFO:teuthology.orchestra.run.smithi027.stdout:(62/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.8 MB/s | 292 kB 00:00 2024-05-21T03:09:41.639 INFO:teuthology.orchestra.run.smithi039.stdout:(24/159): python3-rgw-19.0.0-3830.g3b748ea0.el9 1.3 MB/s | 99 kB 00:00 2024-05-21T03:09:41.678 INFO:teuthology.orchestra.run.smithi027.stdout:(63/159): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-05-21T03:09:41.715 INFO:teuthology.orchestra.run.smithi039.stdout:(25/159): rbd-fuse-19.0.0-3830.g3b748ea0.el9.x8 1.1 MB/s | 85 kB 00:00 2024-05-21T03:09:41.787 INFO:teuthology.orchestra.run.smithi027.stdout:(64/159): openblas-0.3.21-2.el9.x86_64.rpm 325 kB/s | 35 kB 00:00 2024-05-21T03:09:41.890 INFO:teuthology.orchestra.run.smithi039.stdout:(26/159): librgw2-19.0.0-3830.g3b748ea0.el9.x86 5.4 MB/s | 5.4 MB 00:00 2024-05-21T03:09:41.957 INFO:teuthology.orchestra.run.smithi039.stdout:(27/159): rbd-nbd-19.0.0-3830.g3b748ea0.el9.x86 2.5 MB/s | 171 kB 00:00 2024-05-21T03:09:42.024 INFO:teuthology.orchestra.run.smithi039.stdout:(28/159): ceph-grafana-dashboards-19.0.0-3830.g 514 kB/s | 34 kB 00:00 2024-05-21T03:09:42.099 INFO:teuthology.orchestra.run.smithi039.stdout:(29/159): ceph-mgr-cephadm-19.0.0-3830.g3b748ea 1.9 MB/s | 143 kB 00:00 2024-05-21T03:09:42.425 INFO:teuthology.orchestra.run.smithi039.stdout:(30/159): rbd-mirror-19.0.0-3830.g3b748ea0.el9. 4.4 MB/s | 3.1 MB 00:00 2024-05-21T03:09:42.716 INFO:teuthology.orchestra.run.smithi039.stdout:(31/159): ceph-mgr-dashboard-19.0.0-3830.g3b748 5.7 MB/s | 3.5 MB 00:00 2024-05-21T03:09:42.735 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - Extras packages 23 kB/s | 16 kB 00:00 2024-05-21T03:09:42.800 INFO:teuthology.orchestra.run.smithi039.stdout:(32/159): ceph-mgr-modules-core-19.0.0-3830.g3b 3.2 MB/s | 274 kB 00:00 2024-05-21T03:09:42.859 INFO:teuthology.orchestra.run.smithi039.stdout:(33/159): ceph-mgr-rook-19.0.0-3830.g3b748ea0.e 848 kB/s | 49 kB 00:00 2024-05-21T03:09:42.871 INFO:teuthology.orchestra.run.smithi027.stdout:(65/159): ceph-test-19.0.0-3830.g3b748ea0.el9.x 6.3 MB/s | 49 MB 00:07 2024-05-21T03:09:42.909 INFO:teuthology.orchestra.run.smithi039.stdout:(34/159): ceph-prometheus-alerts-19.0.0-3830.g3 328 kB/s | 16 kB 00:00 2024-05-21T03:09:42.929 INFO:teuthology.orchestra.run.smithi027.stdout:(66/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 1.2 MB/s | 3.0 MB 00:02 2024-05-21T03:09:42.984 INFO:teuthology.orchestra.run.smithi039.stdout:(35/159): ceph-volume-19.0.0-3830.g3b748ea0.el9 3.6 MB/s | 273 kB 00:00 2024-05-21T03:09:43.063 INFO:teuthology.orchestra.run.smithi027.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-05-21T03:09:43.159 INFO:teuthology.orchestra.run.smithi039.stdout:(36/159): cephadm-19.0.0-3830.g3b748ea0.el9.noa 4.3 MB/s | 764 kB 00:00 2024-05-21T03:09:43.235 INFO:teuthology.orchestra.run.smithi039.stdout:(37/159): cephfs-top-19.0.0-3830.g3b748ea0.el9. 243 kB/s | 18 kB 00:00 2024-05-21T03:09:43.852 INFO:teuthology.orchestra.run.smithi039.stdout:(38/159): ceph-test-19.0.0-3830.g3b748ea0.el9.x 10 MB/s | 49 MB 00:04 2024-05-21T03:09:43.869 INFO:teuthology.orchestra.run.smithi039.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 127 kB/s | 80 kB 00:00 2024-05-21T03:09:43.961 INFO:teuthology.orchestra.run.smithi039.stdout:(40/159): ceph-mgr-diskprediction-local-19.0.0- 4.8 MB/s | 7.4 MB 00:01 2024-05-21T03:09:44.038 INFO:teuthology.orchestra.run.smithi027.stdout:(68/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.8 MB/s | 4.4 MB 00:01 2024-05-21T03:09:44.147 INFO:teuthology.orchestra.run.smithi027.stdout:(69/159): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-05-21T03:09:44.219 INFO:teuthology.orchestra.run.smithi039.stdout:(41/159): libquadmath-11.4.1-3.el9.x86_64.rpm 558 kB/s | 195 kB 00:00 2024-05-21T03:09:44.380 INFO:teuthology.orchestra.run.smithi027.stdout:(70/159): postgresql-contrib-13.14-1.el9.x86_64 692 kB/s | 911 kB 00:01 2024-05-21T03:09:44.561 INFO:teuthology.orchestra.run.smithi039.stdout:(42/159): libgfortran-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 803 kB 00:00 2024-05-21T03:09:44.586 INFO:teuthology.orchestra.run.smithi039.stdout:(43/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 405 kB/s | 253 kB 00:00 2024-05-21T03:09:44.653 INFO:teuthology.orchestra.run.smithi039.stdout:(44/159): python3-cryptography-36.0.1-4.el9.x86 2.8 MB/s | 1.2 MB 00:00 2024-05-21T03:09:44.679 INFO:teuthology.orchestra.run.smithi039.stdout:(45/159): python3-ply-3.11-14.el9.noarch.rpm 908 kB/s | 106 kB 00:00 2024-05-21T03:09:44.704 INFO:teuthology.orchestra.run.smithi039.stdout:(46/159): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-05-21T03:09:44.705 INFO:teuthology.orchestra.run.smithi027.stdout:(71/159): openblas-openmp-0.3.21-2.el9.x86_64.r 1.6 MB/s | 4.7 MB 00:02 2024-05-21T03:09:44.762 INFO:teuthology.orchestra.run.smithi039.stdout:(47/159): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-05-21T03:09:44.788 INFO:teuthology.orchestra.run.smithi039.stdout:(48/159): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-05-21T03:09:44.880 INFO:teuthology.orchestra.run.smithi039.stdout:(49/159): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-05-21T03:09:44.896 INFO:teuthology.orchestra.run.smithi039.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-21T03:09:44.922 INFO:teuthology.orchestra.run.smithi027.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 404 kB/s | 87 kB 00:00 2024-05-21T03:09:44.930 INFO:teuthology.orchestra.run.smithi039.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 753 kB/s | 107 kB 00:00 2024-05-21T03:09:44.972 INFO:teuthology.orchestra.run.smithi039.stdout:(51/159): flex-2.6.4-9.el9.x86_64.rpm 7.4 MB/s | 313 kB 00:00 2024-05-21T03:09:45.006 INFO:teuthology.orchestra.run.smithi039.stdout:(52/159): bison-3.7.4-5.el9.x86_64.rpm 3.8 MB/s | 943 kB 00:00 2024-05-21T03:09:45.031 INFO:teuthology.orchestra.run.smithi039.stdout:(53/159): elfutils-libelf-devel-0.191-4.el9.x86 144 kB/s | 22 kB 00:00 2024-05-21T03:09:45.056 INFO:teuthology.orchestra.run.smithi039.stdout:(54/159): flexiblas-3.0.4-8.el9.x86_64.rpm 389 kB/s | 33 kB 00:00 2024-05-21T03:09:45.082 INFO:teuthology.orchestra.run.smithi039.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-05-21T03:09:45.107 INFO:teuthology.orchestra.run.smithi039.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-05-21T03:09:45.132 INFO:teuthology.orchestra.run.smithi039.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-21T03:09:45.158 INFO:teuthology.orchestra.run.smithi039.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-05-21T03:09:45.200 INFO:teuthology.orchestra.run.smithi039.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-05-21T03:09:45.258 INFO:teuthology.orchestra.run.smithi039.stdout:(60/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-05-21T03:09:45.283 INFO:teuthology.orchestra.run.smithi039.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 304 kB/s | 25 kB 00:00 2024-05-21T03:09:45.309 INFO:teuthology.orchestra.run.smithi039.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-05-21T03:09:45.334 INFO:teuthology.orchestra.run.smithi039.stdout:(63/159): libtool-2.4.6-45.el9.x86_64.rpm 3.2 MB/s | 582 kB 00:00 2024-05-21T03:09:45.359 INFO:teuthology.orchestra.run.smithi039.stdout:(64/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-05-21T03:09:45.385 INFO:teuthology.orchestra.run.smithi039.stdout:(65/159): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-05-21T03:09:45.403 INFO:teuthology.orchestra.run.smithi165.stdout:Extra Packages for Enterprise Linux 8.3 MB/s | 21 MB 00:02 2024-05-21T03:09:45.410 INFO:teuthology.orchestra.run.smithi039.stdout:(66/159): openblas-0.3.21-2.el9.x86_64.rpm 467 kB/s | 35 kB 00:00 2024-05-21T03:09:45.527 INFO:teuthology.orchestra.run.smithi039.stdout:(67/159): openblas-openmp-0.3.21-2.el9.x86_64.r 28 MB/s | 4.7 MB 00:00 2024-05-21T03:09:45.560 INFO:teuthology.orchestra.run.smithi039.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-21T03:09:45.627 INFO:teuthology.orchestra.run.smithi039.stdout:(69/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-05-21T03:09:45.652 INFO:teuthology.orchestra.run.smithi039.stdout:(70/159): postgresql-contrib-13.14-1.el9.x86_64 7.1 MB/s | 911 kB 00:00 2024-05-21T03:09:45.677 INFO:teuthology.orchestra.run.smithi039.stdout:(71/159): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-05-21T03:09:45.719 INFO:teuthology.orchestra.run.smithi039.stdout:(72/159): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-05-21T03:09:45.745 INFO:teuthology.orchestra.run.smithi039.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-05-21T03:09:45.763 INFO:teuthology.orchestra.run.smithi039.stdout:[MIRROR] python3-colorama-0.4.6-3.el9.noarch.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-colorama-0.4.6-3.el9.noarch.rpm (IP: 199.223.246.113) 2024-05-21T03:09:45.845 INFO:teuthology.orchestra.run.smithi039.stdout:(74/159): postgresql-server-13.14-1.el9.x86_64. 30 MB/s | 5.8 MB 00:00 2024-05-21T03:09:45.870 INFO:teuthology.orchestra.run.smithi039.stdout:(75/159): python3-colorama-0.4.6-3.el9.noarch.r 489 kB/s | 61 kB 00:00 2024-05-21T03:09:45.896 INFO:teuthology.orchestra.run.smithi039.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-05-21T03:09:45.896 INFO:teuthology.orchestra.run.smithi039.stdout:[MIRROR] python3-jinja2-2.11.3-6.el9.noarch.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-jinja2-2.11.3-6.el9.noarch.rpm (IP: 199.223.246.113) 2024-05-21T03:09:45.965 INFO:teuthology.orchestra.run.smithi027.stdout:(73/159): protobuf-3.14.0-13.el9.x86_64.rpm 652 kB/s | 1.0 MB 00:01 2024-05-21T03:09:45.971 INFO:teuthology.orchestra.run.smithi039.stdout:(77/159): python3-babel-2.9.1-2.el9.noarch.rpm 24 MB/s | 6.0 MB 00:00 2024-05-21T03:09:45.996 INFO:teuthology.orchestra.run.smithi039.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-05-21T03:09:46.022 INFO:teuthology.orchestra.run.smithi039.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 379 kB/s | 19 kB 00:00 2024-05-21T03:09:46.047 INFO:teuthology.orchestra.run.smithi039.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-05-21T03:09:46.072 INFO:teuthology.orchestra.run.smithi039.stdout:(81/159): python3-jinja2-2.11.3-6.el9.noarch.rp 1.2 MB/s | 248 kB 00:00 2024-05-21T03:09:46.097 INFO:teuthology.orchestra.run.smithi039.stdout:(82/159): python3-markupsafe-1.1.1-12.el9.x86_6 462 kB/s | 35 kB 00:00 2024-05-21T03:09:46.115 INFO:teuthology.orchestra.run.smithi027.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 409 kB/s | 61 kB 00:00 2024-05-21T03:09:46.190 INFO:teuthology.orchestra.run.smithi039.stdout:(83/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 5.1 MB 00:00 2024-05-21T03:09:46.215 INFO:teuthology.orchestra.run.smithi039.stdout:(84/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-05-21T03:09:46.240 INFO:teuthology.orchestra.run.smithi027.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 1.9 MB/s | 244 kB 00:00 2024-05-21T03:09:46.323 INFO:teuthology.orchestra.run.smithi039.stdout:(85/159): python3-prettytable-0.7.2-27.el9.noar 390 kB/s | 42 kB 00:00 2024-05-21T03:09:46.373 INFO:teuthology.orchestra.run.smithi039.stdout:(86/159): python3-oauthlib-3.1.1-5.el9.noarch.r 806 kB/s | 222 kB 00:00 2024-05-21T03:09:46.432 INFO:teuthology.orchestra.run.smithi039.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-05-21T03:09:46.482 INFO:teuthology.orchestra.run.smithi039.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-05-21T03:09:46.549 INFO:teuthology.orchestra.run.smithi039.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-05-21T03:09:46.574 INFO:teuthology.orchestra.run.smithi039.stdout:(90/159): python3-packaging-20.9-5.el9.noarch.r 200 kB/s | 77 kB 00:00 2024-05-21T03:09:46.590 INFO:teuthology.orchestra.run.smithi027.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 709 kB/s | 248 kB 00:00 2024-05-21T03:09:46.600 INFO:teuthology.orchestra.run.smithi039.stdout:(91/159): python3-pytz-2021.1-5.el9.noarch.rpm 435 kB/s | 51 kB 00:00 2024-05-21T03:09:46.658 INFO:teuthology.orchestra.run.smithi039.stdout:(92/159): python3-requests-oauthlib-1.3.0-12.el 493 kB/s | 54 kB 00:00 2024-05-21T03:09:46.707 INFO:teuthology.orchestra.run.smithi027.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 220 kB/s | 26 kB 00:00 2024-05-21T03:09:46.717 INFO:teuthology.orchestra.run.smithi039.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 356 kB/s | 42 kB 00:00 2024-05-21T03:09:46.775 INFO:teuthology.orchestra.run.smithi039.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 378 kB/s | 44 kB 00:00 2024-05-21T03:09:46.816 INFO:teuthology.orchestra.run.smithi027.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 177 kB/s | 19 kB 00:00 2024-05-21T03:09:46.826 INFO:teuthology.orchestra.run.smithi039.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-05-21T03:09:46.901 INFO:teuthology.orchestra.run.smithi039.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 286 kB/s | 36 kB 00:00 2024-05-21T03:09:46.943 INFO:teuthology.orchestra.run.smithi039.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 405 kB/s | 47 kB 00:00 2024-05-21T03:09:47.035 INFO:teuthology.orchestra.run.smithi039.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 2.2 MB/s | 305 kB 00:00 2024-05-21T03:09:47.060 INFO:teuthology.orchestra.run.smithi039.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 494 kB/s | 58 kB 00:00 2024-05-21T03:09:47.117 INFO:teuthology.orchestra.run.smithi027.stdout:(79/159): python3-mako-1.1.4-6.el9.noarch.rpm 574 kB/s | 172 kB 00:00 2024-05-21T03:09:47.145 INFO:teuthology.orchestra.run.smithi039.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 863 kB/s | 94 kB 00:00 2024-05-21T03:09:47.169 INFO:teuthology.orchestra.run.smithi039.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 586 kB/s | 64 kB 00:00 2024-05-21T03:09:47.225 INFO:teuthology.orchestra.run.smithi027.stdout:(80/159): python3-markupsafe-1.1.1-12.el9.x86_6 321 kB/s | 35 kB 00:00 2024-05-21T03:09:47.253 INFO:teuthology.orchestra.run.smithi039.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 413 kB/s | 45 kB 00:00 2024-05-21T03:09:47.295 INFO:teuthology.orchestra.run.smithi039.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 178 kB/s | 22 kB 00:00 2024-05-21T03:09:47.478 INFO:teuthology.orchestra.run.smithi039.stdout:(104/159): abseil-cpp-20211102.0-3.el9.x86_64.r 2.9 MB/s | 548 kB 00:00 2024-05-21T03:09:47.529 INFO:teuthology.orchestra.run.smithi039.stdout:(105/159): protobuf-compiler-3.14.0-13.el9.x86_ 3.1 MB/s | 863 kB 00:00 2024-05-21T03:09:47.554 INFO:teuthology.orchestra.run.smithi039.stdout:(106/159): gperftools-libs-2.9.1-2.el9.x86_64.r 4.0 MB/s | 309 kB 00:00 2024-05-21T03:09:47.612 INFO:teuthology.orchestra.run.smithi039.stdout:(107/159): grpc-data-1.46.7-10.el9.noarch.rpm 233 kB/s | 19 kB 00:00 2024-05-21T03:09:47.705 INFO:teuthology.orchestra.run.smithi039.stdout:(108/159): libarrow-9.0.0-11.el9.x86_64.rpm 29 MB/s | 4.4 MB 00:00 2024-05-21T03:09:47.738 INFO:teuthology.orchestra.run.smithi039.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 207 kB/s | 26 kB 00:00 2024-05-21T03:09:47.764 INFO:teuthology.orchestra.run.smithi039.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 829 kB/s | 49 kB 00:00 2024-05-21T03:09:47.806 INFO:teuthology.orchestra.run.smithi039.stdout:(111/159): luarocks-3.9.2-5.el9.noarch.rpm 3.6 MB/s | 151 kB 00:00 2024-05-21T03:09:47.831 INFO:teuthology.orchestra.run.smithi039.stdout:(112/159): libunwind-1.6.2-1.el9.x86_64.rpm 731 kB/s | 67 kB 00:00 2024-05-21T03:09:47.864 INFO:teuthology.orchestra.run.smithi039.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-05-21T03:09:47.898 INFO:teuthology.orchestra.run.smithi039.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-05-21T03:09:47.932 INFO:teuthology.orchestra.run.smithi039.stdout:(115/159): python3-cachetools-4.2.4-1.el9.noarc 964 kB/s | 32 kB 00:00 2024-05-21T03:09:47.957 INFO:teuthology.orchestra.run.smithi039.stdout:(116/159): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-05-21T03:09:47.982 INFO:teuthology.orchestra.run.smithi039.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-05-21T03:09:48.007 INFO:teuthology.orchestra.run.smithi039.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-05-21T03:09:48.033 INFO:teuthology.orchestra.run.smithi039.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-05-21T03:09:48.058 INFO:teuthology.orchestra.run.smithi039.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-05-21T03:09:48.083 INFO:teuthology.orchestra.run.smithi039.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-05-21T03:09:48.134 INFO:teuthology.orchestra.run.smithi039.stdout:(122/159): python3-grpcio-1.46.7-10.el9.x86_64. 27 MB/s | 2.0 MB 00:00 2024-05-21T03:09:48.159 INFO:teuthology.orchestra.run.smithi039.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-05-21T03:09:48.184 INFO:teuthology.orchestra.run.smithi039.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-05-21T03:09:48.210 INFO:teuthology.orchestra.run.smithi039.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-05-21T03:09:48.234 INFO:teuthology.orchestra.run.smithi039.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-05-21T03:09:48.260 INFO:teuthology.orchestra.run.smithi039.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-05-21T03:09:48.285 INFO:teuthology.orchestra.run.smithi039.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-05-21T03:09:48.319 INFO:teuthology.orchestra.run.smithi039.stdout:(129/159): python3-kubernetes-26.1.0-2.el9.noar 17 MB/s | 1.0 MB 00:00 2024-05-21T03:09:48.344 INFO:teuthology.orchestra.run.smithi039.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 793 kB/s | 46 kB 00:00 2024-05-21T03:09:48.369 INFO:teuthology.orchestra.run.smithi039.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-05-21T03:09:48.395 INFO:teuthology.orchestra.run.smithi039.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-05-21T03:09:48.420 INFO:teuthology.orchestra.run.smithi039.stdout:(133/159): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-05-21T03:09:48.445 INFO:teuthology.orchestra.run.smithi039.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-05-21T03:09:48.462 INFO:teuthology.orchestra.run.smithi039.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.1 MB/s | 90 kB 00:00 2024-05-21T03:09:48.487 INFO:teuthology.orchestra.run.smithi039.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 678 kB/s | 28 kB 00:00 2024-05-21T03:09:48.513 INFO:teuthology.orchestra.run.smithi039.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-05-21T03:09:48.538 INFO:teuthology.orchestra.run.smithi039.stdout:(138/159): python3-routes-2.5.1-5.el9.noarch.rp 3.6 MB/s | 188 kB 00:00 2024-05-21T03:09:48.563 INFO:teuthology.orchestra.run.smithi039.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-05-21T03:09:48.589 INFO:teuthology.orchestra.run.smithi039.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-05-21T03:09:48.614 INFO:teuthology.orchestra.run.smithi039.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-05-21T03:09:48.642 INFO:teuthology.orchestra.run.smithi039.stdout:(142/159): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-05-21T03:09:48.665 INFO:teuthology.orchestra.run.smithi039.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-05-21T03:09:48.698 INFO:teuthology.orchestra.run.smithi039.stdout:(144/159): python3-werkzeug-2.0.3-3.el9.1.noarc 7.1 MB/s | 427 kB 00:00 2024-05-21T03:09:48.724 INFO:teuthology.orchestra.run.smithi039.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-05-21T03:09:48.749 INFO:teuthology.orchestra.run.smithi039.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-05-21T03:09:48.791 INFO:teuthology.orchestra.run.smithi039.stdout:(147/159): thrift-0.15.0-2.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-05-21T03:09:49.241 INFO:teuthology.orchestra.run.smithi039.stdout:(148/159): librbd1-19.0.0-3830.g3b748ea0.el9.x8 7.0 MB/s | 3.1 MB 00:00 2024-05-21T03:09:49.366 INFO:teuthology.orchestra.run.smithi039.stdout:(149/159): librados2-19.0.0-3830.g3b748ea0.el9. 5.5 MB/s | 3.4 MB 00:00 2024-05-21T03:09:50.109 INFO:teuthology.orchestra.run.smithi027.stdout:(81/159): postgresql-server-13.14-1.el9.x86_64. 993 kB/s | 5.8 MB 00:05 2024-05-21T03:09:50.133 INFO:teuthology.orchestra.run.smithi039.stdout:(150/159): NetworkManager-1.47.90-1.el9.x86_64. 2.6 MB/s | 2.3 MB 00:00 2024-05-21T03:09:50.242 INFO:teuthology.orchestra.run.smithi039.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 383 kB/s | 41 kB 00:00 2024-05-21T03:09:50.284 INFO:teuthology.orchestra.run.smithi039.stdout:(152/159): NetworkManager-libnm-1.47.90-1.el9.x 2.0 MB/s | 1.8 MB 00:00 2024-05-21T03:09:50.351 INFO:teuthology.orchestra.run.smithi039.stdout:(153/159): NetworkManager-tui-1.47.90-1.el9.x86 2.2 MB/s | 248 kB 00:00 2024-05-21T03:09:50.392 INFO:teuthology.orchestra.run.smithi039.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 338 kB/s | 37 kB 00:00 2024-05-21T03:09:50.459 INFO:teuthology.orchestra.run.smithi039.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 1.9 MB/s | 207 kB 00:00 2024-05-21T03:09:50.467 INFO:teuthology.orchestra.run.smithi027.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 700 kB/s | 250 kB 00:00 2024-05-21T03:09:50.501 INFO:teuthology.orchestra.run.smithi039.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 2.3 MB/s | 260 kB 00:00 2024-05-21T03:09:50.536 INFO:teuthology.orchestra.run.smithi165.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-05-21T03:09:50.593 INFO:teuthology.orchestra.run.smithi039.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 10 MB/s | 1.3 MB 00:00 2024-05-21T03:09:50.744 INFO:teuthology.orchestra.run.smithi039.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 9.9 MB/s | 2.4 MB 00:00 2024-05-21T03:09:50.835 INFO:teuthology.orchestra.run.smithi027.stdout:(83/159): python3-oauthlib-3.1.1-5.el9.noarch.r 605 kB/s | 222 kB 00:00 2024-05-21T03:09:51.018 INFO:teuthology.orchestra.run.smithi027.stdout:(84/159): python3-packaging-20.9-5.el9.noarch.r 421 kB/s | 77 kB 00:00 2024-05-21T03:09:51.135 INFO:teuthology.orchestra.run.smithi027.stdout:(85/159): python3-prettytable-0.7.2-27.el9.noar 362 kB/s | 42 kB 00:00 2024-05-21T03:09:51.577 INFO:teuthology.orchestra.run.smithi027.stdout:(86/159): python3-protobuf-3.14.0-13.el9.noarch 610 kB/s | 269 kB 00:00 2024-05-21T03:09:51.844 INFO:teuthology.orchestra.run.smithi027.stdout:(87/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 595 kB/s | 159 kB 00:00 2024-05-21T03:09:52.296 INFO:teuthology.orchestra.run.smithi027.stdout:(88/159): python3-pyasn1-modules-0.4.8-6.el9.no 618 kB/s | 279 kB 00:00 2024-05-21T03:09:52.403 INFO:teuthology.orchestra.run.smithi027.stdout:(89/159): python3-pytz-2021.1-5.el9.noarch.rpm 476 kB/s | 51 kB 00:00 2024-05-21T03:09:52.512 INFO:teuthology.orchestra.run.smithi027.stdout:(90/159): python3-requests-oauthlib-1.3.0-12.el 495 kB/s | 54 kB 00:00 2024-05-21T03:09:52.654 INFO:teuthology.orchestra.run.smithi165.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-21T03:09:52.655 INFO:teuthology.orchestra.run.smithi165.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-21T03:09:52.665 INFO:teuthology.orchestra.run.smithi165.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-21T03:09:52.666 INFO:teuthology.orchestra.run.smithi165.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-21T03:09:52.667 INFO:teuthology.orchestra.run.smithi165.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-21T03:09:52.668 INFO:teuthology.orchestra.run.smithi165.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-21T03:09:52.669 INFO:teuthology.orchestra.run.smithi165.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-21T03:09:52.671 INFO:teuthology.orchestra.run.smithi165.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-21T03:09:52.672 INFO:teuthology.orchestra.run.smithi165.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-21T03:09:52.733 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-05-21T03:09:52.758 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================= 2024-05-21T03:09:52.758 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-05-21T03:09:52.758 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================= 2024-05-21T03:09:52.758 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-05-21T03:09:52.759 INFO:teuthology.orchestra.run.smithi165.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-21T03:09:52.759 INFO:teuthology.orchestra.run.smithi165.stdout: ceph x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 6.6 k 2024-05-21T03:09:52.759 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-base x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 5.4 M 2024-05-21T03:09:52.759 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-fuse x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 891 k 2024-05-21T03:09:52.759 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 145 k 2024-05-21T03:09:52.759 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 1.5 M 2024-05-21T03:09:52.759 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 143 k 2024-05-21T03:09:52.759 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 3.5 M 2024-05-21T03:09:52.759 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 7.4 M 2024-05-21T03:09:52.759 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-rook noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 49 k 2024-05-21T03:09:52.759 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-radosgw x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 11 M 2024-05-21T03:09:52.759 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-test x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 49 M 2024-05-21T03:09:52.759 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-volume noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 273 k 2024-05-21T03:09:52.760 INFO:teuthology.orchestra.run.smithi165.stdout: cephadm noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 764 k 2024-05-21T03:09:52.760 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-mirror x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 225 k 2024-05-21T03:09:52.760 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-shell x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 27 k 2024-05-21T03:09:52.760 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-top noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 18 k 2024-05-21T03:09:52.760 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-21T03:09:52.760 INFO:teuthology.orchestra.run.smithi165.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-21T03:09:52.760 INFO:teuthology.orchestra.run.smithi165.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-21T03:09:52.760 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs-devel x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 32 k 2024-05-21T03:09:52.760 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs2 x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 746 k 2024-05-21T03:09:52.760 INFO:teuthology.orchestra.run.smithi165.stdout: librados-devel x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 127 k 2024-05-21T03:09:52.760 INFO:teuthology.orchestra.run.smithi165.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-21T03:09:52.760 INFO:teuthology.orchestra.run.smithi165.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-21T03:09:52.760 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-21T03:09:52.761 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-21T03:09:52.761 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-21T03:09:52.761 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-21T03:09:52.761 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cephfs x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 161 k 2024-05-21T03:09:52.761 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rados x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 320 k 2024-05-21T03:09:52.761 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rbd x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 300 k 2024-05-21T03:09:52.761 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rgw x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 99 k 2024-05-21T03:09:52.761 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-fuse x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 85 k 2024-05-21T03:09:52.761 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-mirror x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 3.1 M 2024-05-21T03:09:52.761 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-nbd x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 171 k 2024-05-21T03:09:52.761 INFO:teuthology.orchestra.run.smithi165.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-21T03:09:52.761 INFO:teuthology.orchestra.run.smithi165.stdout:Upgrading: 2024-05-21T03:09:52.762 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-21T03:09:52.762 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-21T03:09:52.762 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-21T03:09:52.762 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-21T03:09:52.762 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-21T03:09:52.762 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-21T03:09:52.762 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-21T03:09:52.762 INFO:teuthology.orchestra.run.smithi165.stdout: librados2 x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 3.4 M 2024-05-21T03:09:52.762 INFO:teuthology.orchestra.run.smithi165.stdout: librbd1 x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 3.1 M 2024-05-21T03:09:52.762 INFO:teuthology.orchestra.run.smithi165.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-21T03:09:52.762 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-21T03:09:52.762 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-05-21T03:09:52.762 INFO:teuthology.orchestra.run.smithi165.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-21T03:09:52.763 INFO:teuthology.orchestra.run.smithi165.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-21T03:09:52.763 INFO:teuthology.orchestra.run.smithi165.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-21T03:09:52.763 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-common x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 21 M 2024-05-21T03:09:52.763 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 34 k 2024-05-21T03:09:52.763 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mds x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 2.4 M 2024-05-21T03:09:52.763 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 274 k 2024-05-21T03:09:52.763 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mon x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 4.7 M 2024-05-21T03:09:52.763 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-osd x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 17 M 2024-05-21T03:09:52.763 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 16 k 2024-05-21T03:09:52.763 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-selinux x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 25 k 2024-05-21T03:09:52.763 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-21T03:09:52.763 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-21T03:09:52.763 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-21T03:09:52.764 INFO:teuthology.orchestra.run.smithi165.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-21T03:09:52.764 INFO:teuthology.orchestra.run.smithi165.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-21T03:09:52.764 INFO:teuthology.orchestra.run.smithi165.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-21T03:09:52.764 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-21T03:09:52.764 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-21T03:09:52.764 INFO:teuthology.orchestra.run.smithi165.stdout: libcephsqlite x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 163 k 2024-05-21T03:09:52.764 INFO:teuthology.orchestra.run.smithi165.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-21T03:09:52.764 INFO:teuthology.orchestra.run.smithi165.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-21T03:09:52.764 INFO:teuthology.orchestra.run.smithi165.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-21T03:09:52.764 INFO:teuthology.orchestra.run.smithi165.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-21T03:09:52.764 INFO:teuthology.orchestra.run.smithi165.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-21T03:09:52.764 INFO:teuthology.orchestra.run.smithi165.stdout: libradosstriper1 x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 501 k 2024-05-21T03:09:52.764 INFO:teuthology.orchestra.run.smithi165.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-21T03:09:52.765 INFO:teuthology.orchestra.run.smithi165.stdout: librgw2 x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 5.4 M 2024-05-21T03:09:52.765 INFO:teuthology.orchestra.run.smithi165.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-21T03:09:52.765 INFO:teuthology.orchestra.run.smithi165.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-21T03:09:52.765 INFO:teuthology.orchestra.run.smithi165.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-21T03:09:52.765 INFO:teuthology.orchestra.run.smithi165.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-21T03:09:52.765 INFO:teuthology.orchestra.run.smithi165.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-21T03:09:52.765 INFO:teuthology.orchestra.run.smithi165.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-21T03:09:52.765 INFO:teuthology.orchestra.run.smithi165.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-21T03:09:52.765 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-21T03:09:52.765 INFO:teuthology.orchestra.run.smithi165.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-21T03:09:52.765 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-21T03:09:52.765 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-21T03:09:52.765 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-21T03:09:52.765 INFO:teuthology.orchestra.run.smithi165.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-21T03:09:52.766 INFO:teuthology.orchestra.run.smithi165.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-21T03:09:52.766 INFO:teuthology.orchestra.run.smithi165.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-21T03:09:52.766 INFO:teuthology.orchestra.run.smithi165.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-21T03:09:52.766 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-21T03:09:52.766 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-argparse x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 45 k 2024-05-21T03:09:52.766 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-common x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 136 k 2024-05-21T03:09:52.766 INFO:teuthology.orchestra.run.smithi165.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-21T03:09:52.766 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-21T03:09:52.766 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-21T03:09:52.766 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-21T03:09:52.766 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-21T03:09:52.766 INFO:teuthology.orchestra.run.smithi165.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-21T03:09:52.767 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-21T03:09:52.767 INFO:teuthology.orchestra.run.smithi165.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-21T03:09:52.767 INFO:teuthology.orchestra.run.smithi165.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-21T03:09:52.767 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-21T03:09:52.767 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-21T03:09:52.767 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-21T03:09:52.767 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-21T03:09:52.767 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-21T03:09:52.767 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-21T03:09:52.767 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-21T03:09:52.767 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-21T03:09:52.767 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-21T03:09:52.767 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-21T03:09:52.767 INFO:teuthology.orchestra.run.smithi165.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-21T03:09:52.768 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-21T03:09:52.768 INFO:teuthology.orchestra.run.smithi165.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-21T03:09:52.768 INFO:teuthology.orchestra.run.smithi165.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-21T03:09:52.768 INFO:teuthology.orchestra.run.smithi165.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-21T03:09:52.768 INFO:teuthology.orchestra.run.smithi165.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-21T03:09:52.768 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-21T03:09:52.768 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-21T03:09:52.768 INFO:teuthology.orchestra.run.smithi165.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-21T03:09:52.768 INFO:teuthology.orchestra.run.smithi165.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-21T03:09:52.768 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-21T03:09:52.768 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-21T03:09:52.768 INFO:teuthology.orchestra.run.smithi165.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-21T03:09:52.768 INFO:teuthology.orchestra.run.smithi165.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-21T03:09:52.768 INFO:teuthology.orchestra.run.smithi165.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-21T03:09:52.769 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-21T03:09:52.769 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-21T03:09:52.769 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-21T03:09:52.769 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-21T03:09:52.769 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-21T03:09:52.769 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-21T03:09:52.769 INFO:teuthology.orchestra.run.smithi165.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-21T03:09:52.769 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-21T03:09:52.769 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-21T03:09:52.769 INFO:teuthology.orchestra.run.smithi165.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-21T03:09:52.769 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-21T03:09:52.769 INFO:teuthology.orchestra.run.smithi165.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-21T03:09:52.769 INFO:teuthology.orchestra.run.smithi165.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-21T03:09:52.769 INFO:teuthology.orchestra.run.smithi165.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-21T03:09:52.770 INFO:teuthology.orchestra.run.smithi165.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-21T03:09:52.770 INFO:teuthology.orchestra.run.smithi165.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-21T03:09:52.770 INFO:teuthology.orchestra.run.smithi165.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-21T03:09:52.770 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-21T03:09:52.770 INFO:teuthology.orchestra.run.smithi165.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-21T03:09:52.770 INFO:teuthology.orchestra.run.smithi165.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-21T03:09:52.770 INFO:teuthology.orchestra.run.smithi165.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-21T03:09:52.770 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-21T03:09:52.770 INFO:teuthology.orchestra.run.smithi165.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-21T03:09:52.770 INFO:teuthology.orchestra.run.smithi165.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-21T03:09:52.770 INFO:teuthology.orchestra.run.smithi165.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-21T03:09:52.770 INFO:teuthology.orchestra.run.smithi165.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-21T03:09:52.770 INFO:teuthology.orchestra.run.smithi165.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-21T03:09:52.771 INFO:teuthology.orchestra.run.smithi165.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-21T03:09:52.771 INFO:teuthology.orchestra.run.smithi165.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-21T03:09:52.771 INFO:teuthology.orchestra.run.smithi165.stdout:Installing weak dependencies: 2024-05-21T03:09:52.771 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-21T03:09:52.771 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-21T03:09:52.771 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-05-21T03:09:52.771 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================= 2024-05-21T03:09:52.771 INFO:teuthology.orchestra.run.smithi165.stdout:Install 148 Packages 2024-05-21T03:09:52.771 INFO:teuthology.orchestra.run.smithi165.stdout:Upgrade 11 Packages 2024-05-21T03:09:52.771 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-21T03:09:52.772 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 227 M 2024-05-21T03:09:52.772 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-05-21T03:09:53.138 INFO:teuthology.orchestra.run.smithi027.stdout:(91/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 883 kB/s | 5.1 MB 00:05 2024-05-21T03:09:53.246 INFO:teuthology.orchestra.run.smithi027.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 384 kB/s | 42 kB 00:00 2024-05-21T03:09:53.354 INFO:teuthology.orchestra.run.smithi027.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 408 kB/s | 44 kB 00:00 2024-05-21T03:09:53.638 INFO:teuthology.orchestra.run.smithi165.stdout:(1/159): ceph-19.0.0-3830.g3b748ea0.el9.x86_64. 38 kB/s | 6.6 kB 00:00 2024-05-21T03:09:53.671 INFO:teuthology.orchestra.run.smithi027.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 697 kB/s | 221 kB 00:00 2024-05-21T03:09:53.750 INFO:teuthology.orchestra.run.smithi027.stdout:(95/159): python3-babel-2.9.1-2.el9.noarch.rpm 692 kB/s | 6.0 MB 00:08 2024-05-21T03:09:53.772 INFO:teuthology.orchestra.run.smithi027.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 357 kB/s | 36 kB 00:00 2024-05-21T03:09:53.847 INFO:teuthology.orchestra.run.smithi027.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-05-21T03:09:53.888 INFO:teuthology.orchestra.run.smithi165.stdout:(2/159): ceph-fuse-19.0.0-3830.g3b748ea0.el9.x8 3.5 MB/s | 891 kB 00:00 2024-05-21T03:09:53.947 INFO:teuthology.orchestra.run.smithi027.stdout:(98/159): uuid-1.6.2-55.el9.x86_64.rpm 579 kB/s | 58 kB 00:00 2024-05-21T03:09:53.963 INFO:teuthology.orchestra.run.smithi165.stdout:(3/159): ceph-immutable-object-cache-19.0.0-383 1.9 MB/s | 145 kB 00:00 2024-05-21T03:09:54.113 INFO:teuthology.orchestra.run.smithi165.stdout:(4/159): ceph-base-19.0.0-3830.g3b748ea0.el9.x8 8.3 MB/s | 5.4 MB 00:00 2024-05-21T03:09:54.148 INFO:teuthology.orchestra.run.smithi027.stdout:(99/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 469 kB/s | 94 kB 00:00 2024-05-21T03:09:54.215 INFO:teuthology.orchestra.run.smithi027.stdout:(100/159): socat-1.7.4.1-5.el9.x86_64.rpm 689 kB/s | 305 kB 00:00 2024-05-21T03:09:54.288 INFO:teuthology.orchestra.run.smithi165.stdout:(5/159): ceph-mgr-19.0.0-3830.g3b748ea0.el9.x86 8.7 MB/s | 1.5 MB 00:00 2024-05-21T03:09:54.315 INFO:teuthology.orchestra.run.smithi027.stdout:(101/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 448 kB/s | 45 kB 00:00 2024-05-21T03:09:54.332 INFO:teuthology.orchestra.run.smithi027.stdout:(102/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 346 kB/s | 64 kB 00:00 2024-05-21T03:09:54.347 INFO:teuthology.orchestra.run.smithi165.stdout:(6/159): ceph-mds-19.0.0-3830.g3b748ea0.el9.x86 6.2 MB/s | 2.4 MB 00:00 2024-05-21T03:09:54.416 INFO:teuthology.orchestra.run.smithi027.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-05-21T03:09:54.666 INFO:teuthology.orchestra.run.smithi027.stdout:(104/159): abseil-cpp-20211102.0-3.el9.x86_64.r 2.1 MB/s | 548 kB 00:00 2024-05-21T03:09:54.716 INFO:teuthology.orchestra.run.smithi027.stdout:(105/159): gperftools-libs-2.9.1-2.el9.x86_64.r 6.0 MB/s | 309 kB 00:00 2024-05-21T03:09:54.747 INFO:teuthology.orchestra.run.smithi165.stdout:(7/159): ceph-mon-19.0.0-3830.g3b748ea0.el9.x86 10 MB/s | 4.7 MB 00:00 2024-05-21T03:09:54.766 INFO:teuthology.orchestra.run.smithi027.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 390 kB/s | 19 kB 00:00 2024-05-21T03:09:54.925 INFO:teuthology.orchestra.run.smithi027.stdout:(107/159): libarrow-9.0.0-11.el9.x86_64.rpm 28 MB/s | 4.4 MB 00:00 2024-05-21T03:09:54.984 INFO:teuthology.orchestra.run.smithi027.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 446 kB/s | 26 kB 00:00 2024-05-21T03:09:55.034 INFO:teuthology.orchestra.run.smithi027.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 969 kB/s | 49 kB 00:00 2024-05-21T03:09:55.076 INFO:teuthology.orchestra.run.smithi027.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-05-21T03:09:55.126 INFO:teuthology.orchestra.run.smithi027.stdout:(111/159): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-05-21T03:09:55.186 INFO:teuthology.orchestra.run.smithi027.stdout:(112/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-05-21T03:09:55.252 INFO:teuthology.orchestra.run.smithi027.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-05-21T03:09:55.302 INFO:teuthology.orchestra.run.smithi027.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 869 kB/s | 43 kB 00:00 2024-05-21T03:09:55.381 INFO:teuthology.orchestra.run.smithi027.stdout:(115/159): python3-cachetools-4.2.4-1.el9.noarc 413 kB/s | 32 kB 00:00 2024-05-21T03:09:55.436 INFO:teuthology.orchestra.run.smithi027.stdout:(116/159): python3-certifi-2021.10.8-2.el9.noar 270 kB/s | 15 kB 00:00 2024-05-21T03:09:55.487 INFO:teuthology.orchestra.run.smithi027.stdout:(117/159): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-05-21T03:09:55.545 INFO:teuthology.orchestra.run.smithi027.stdout:(118/159): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-05-21T03:09:55.596 INFO:teuthology.orchestra.run.smithi027.stdout:(119/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-05-21T03:09:55.629 INFO:teuthology.orchestra.run.smithi027.stdout:(120/159): protobuf-compiler-3.14.0-13.el9.x86_ 666 kB/s | 863 kB 00:01 2024-05-21T03:09:55.654 INFO:teuthology.orchestra.run.smithi027.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 3.4 MB/s | 206 kB 00:00 2024-05-21T03:09:55.730 INFO:teuthology.orchestra.run.smithi027.stdout:(122/159): python3-grpcio-1.46.7-10.el9.x86_64. 20 MB/s | 2.0 MB 00:00 2024-05-21T03:09:55.755 INFO:teuthology.orchestra.run.smithi027.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-05-21T03:09:55.764 INFO:teuthology.orchestra.run.smithi165.stdout:(8/159): ceph-radosgw-19.0.0-3830.g3b748ea0.el9 10 MB/s | 11 MB 00:01 2024-05-21T03:09:55.789 INFO:teuthology.orchestra.run.smithi027.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 184 kB/s | 11 kB 00:00 2024-05-21T03:09:55.814 INFO:teuthology.orchestra.run.smithi027.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 302 kB/s | 18 kB 00:00 2024-05-21T03:09:55.839 INFO:teuthology.orchestra.run.smithi027.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-05-21T03:09:55.865 INFO:teuthology.orchestra.run.smithi027.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-05-21T03:09:55.965 INFO:teuthology.orchestra.run.smithi165.stdout:(9/159): ceph-common-19.0.0-3830.g3b748ea0.el9. 8.5 MB/s | 21 MB 00:02 2024-05-21T03:09:55.990 INFO:teuthology.orchestra.run.smithi165.stdout:(10/159): ceph-selinux-19.0.0-3830.g3b748ea0.el 112 kB/s | 25 kB 00:00 2024-05-21T03:09:55.990 INFO:teuthology.orchestra.run.smithi027.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 131 kB/s | 20 kB 00:00 2024-05-21T03:09:56.032 INFO:teuthology.orchestra.run.smithi027.stdout:(129/159): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-05-21T03:09:56.057 INFO:teuthology.orchestra.run.smithi027.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 714 kB/s | 46 kB 00:00 2024-05-21T03:09:56.065 INFO:teuthology.orchestra.run.smithi165.stdout:(11/159): cephfs-mirror-19.0.0-3830.g3b748ea0.e 2.9 MB/s | 225 kB 00:00 2024-05-21T03:09:56.083 INFO:teuthology.orchestra.run.smithi027.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-05-21T03:09:56.108 INFO:teuthology.orchestra.run.smithi027.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-05-21T03:09:56.132 INFO:teuthology.orchestra.run.smithi165.stdout:(12/159): cephfs-shell-19.0.0-3830.g3b748ea0.el 406 kB/s | 27 kB 00:00 2024-05-21T03:09:56.141 INFO:teuthology.orchestra.run.smithi027.stdout:(133/159): python3-pecan-1.4.2-3.el9.noarch.rpm 4.6 MB/s | 272 kB 00:00 2024-05-21T03:09:56.167 INFO:teuthology.orchestra.run.smithi027.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-05-21T03:09:56.199 INFO:teuthology.orchestra.run.smithi165.stdout:(13/159): libcephfs-devel-19.0.0-3830.g3b748ea0 477 kB/s | 32 kB 00:00 2024-05-21T03:09:56.200 INFO:teuthology.orchestra.run.smithi027.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-05-21T03:09:56.217 INFO:teuthology.orchestra.run.smithi027.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-05-21T03:09:56.243 INFO:teuthology.orchestra.run.smithi027.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 733 kB/s | 31 kB 00:00 2024-05-21T03:09:56.260 INFO:teuthology.orchestra.run.smithi027.stdout:(138/159): python3-routes-2.5.1-5.el9.noarch.rp 4.4 MB/s | 188 kB 00:00 2024-05-21T03:09:56.285 INFO:teuthology.orchestra.run.smithi027.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-05-21T03:09:56.299 INFO:teuthology.orchestra.run.smithi165.stdout:(14/159): libcephfs2-19.0.0-3830.g3b748ea0.el9. 7.3 MB/s | 746 kB 00:00 2024-05-21T03:09:56.302 INFO:teuthology.orchestra.run.smithi027.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 853 kB/s | 36 kB 00:00 2024-05-21T03:09:56.327 INFO:teuthology.orchestra.run.smithi027.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 1.2 MB/s | 51 kB 00:00 2024-05-21T03:09:56.361 INFO:teuthology.orchestra.run.smithi027.stdout:(142/159): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-05-21T03:09:56.374 INFO:teuthology.orchestra.run.smithi165.stdout:(15/159): libcephsqlite-19.0.0-3830.g3b748ea0.e 2.1 MB/s | 163 kB 00:00 2024-05-21T03:09:56.378 INFO:teuthology.orchestra.run.smithi027.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-05-21T03:09:56.420 INFO:teuthology.orchestra.run.smithi027.stdout:(144/159): python3-werkzeug-2.0.3-3.el9.1.noarc 7.2 MB/s | 427 kB 00:00 2024-05-21T03:09:56.445 INFO:teuthology.orchestra.run.smithi027.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 299 kB/s | 20 kB 00:00 2024-05-21T03:09:56.449 INFO:teuthology.orchestra.run.smithi165.stdout:(16/159): librados-devel-19.0.0-3830.g3b748ea0. 1.7 MB/s | 127 kB 00:00 2024-05-21T03:09:56.479 INFO:teuthology.orchestra.run.smithi027.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-05-21T03:09:56.537 INFO:teuthology.orchestra.run.smithi027.stdout:(147/159): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-05-21T03:09:56.618 INFO:teuthology.orchestra.run.smithi165.stdout:(17/159): ceph-osd-19.0.0-3830.g3b748ea0.el9.x8 7.4 MB/s | 17 MB 00:02 2024-05-21T03:09:56.650 INFO:teuthology.orchestra.run.smithi165.stdout:(18/159): libradosstriper1-19.0.0-3830.g3b748ea 2.4 MB/s | 501 kB 00:00 2024-05-21T03:09:56.734 INFO:teuthology.orchestra.run.smithi165.stdout:(19/159): python3-ceph-argparse-19.0.0-3830.g3b 541 kB/s | 45 kB 00:00 2024-05-21T03:09:56.817 INFO:teuthology.orchestra.run.smithi165.stdout:(20/159): python3-ceph-common-19.0.0-3830.g3b74 1.6 MB/s | 136 kB 00:00 2024-05-21T03:09:56.901 INFO:teuthology.orchestra.run.smithi165.stdout:(21/159): python3-cephfs-19.0.0-3830.g3b748ea0. 1.9 MB/s | 161 kB 00:00 2024-05-21T03:09:56.993 INFO:teuthology.orchestra.run.smithi165.stdout:(22/159): python3-rados-19.0.0-3830.g3b748ea0.e 3.4 MB/s | 320 kB 00:00 2024-05-21T03:09:57.085 INFO:teuthology.orchestra.run.smithi165.stdout:(23/159): python3-rbd-19.0.0-3830.g3b748ea0.el9 3.2 MB/s | 300 kB 00:00 2024-05-21T03:09:57.170 INFO:teuthology.orchestra.run.smithi165.stdout:(24/159): python3-rgw-19.0.0-3830.g3b748ea0.el9 1.2 MB/s | 99 kB 00:00 2024-05-21T03:09:57.196 INFO:teuthology.orchestra.run.smithi027.stdout:(148/159): librados2-19.0.0-3830.g3b748ea0.el9. 4.7 MB/s | 3.4 MB 00:00 2024-05-21T03:09:57.244 INFO:teuthology.orchestra.run.smithi165.stdout:(25/159): rbd-fuse-19.0.0-3830.g3b748ea0.el9.x8 1.1 MB/s | 85 kB 00:00 2024-05-21T03:09:57.319 INFO:teuthology.orchestra.run.smithi165.stdout:(26/159): librgw2-19.0.0-3830.g3b748ea0.el9.x86 7.7 MB/s | 5.4 MB 00:00 2024-05-21T03:09:57.411 INFO:teuthology.orchestra.run.smithi165.stdout:(27/159): rbd-nbd-19.0.0-3830.g3b748ea0.el9.x86 1.8 MB/s | 171 kB 00:00 2024-05-21T03:09:57.528 INFO:teuthology.orchestra.run.smithi165.stdout:(28/159): rbd-mirror-19.0.0-3830.g3b748ea0.el9. 11 MB/s | 3.1 MB 00:00 2024-05-21T03:09:57.545 INFO:teuthology.orchestra.run.smithi165.stdout:(29/159): ceph-grafana-dashboards-19.0.0-3830.g 256 kB/s | 34 kB 00:00 2024-05-21T03:09:57.579 INFO:teuthology.orchestra.run.smithi165.stdout:(30/159): ceph-mgr-cephadm-19.0.0-3830.g3b748ea 2.8 MB/s | 143 kB 00:00 2024-05-21T03:09:58.096 INFO:teuthology.orchestra.run.smithi165.stdout:(31/159): ceph-mgr-dashboard-19.0.0-3830.g3b748 6.4 MB/s | 3.5 MB 00:00 2024-05-21T03:09:58.104 INFO:teuthology.orchestra.run.smithi027.stdout:(149/159): NetworkManager-1.47.90-1.el9.x86_64. 2.5 MB/s | 2.3 MB 00:00 2024-05-21T03:09:58.196 INFO:teuthology.orchestra.run.smithi165.stdout:(32/159): ceph-mgr-modules-core-19.0.0-3830.g3b 2.7 MB/s | 274 kB 00:00 2024-05-21T03:09:58.288 INFO:teuthology.orchestra.run.smithi165.stdout:(33/159): ceph-mgr-diskprediction-local-19.0.0- 10 MB/s | 7.4 MB 00:00 2024-05-21T03:09:58.313 INFO:teuthology.orchestra.run.smithi165.stdout:(34/159): ceph-mgr-rook-19.0.0-3830.g3b748ea0.e 422 kB/s | 49 kB 00:00 2024-05-21T03:09:58.338 INFO:teuthology.orchestra.run.smithi165.stdout:(35/159): ceph-prometheus-alerts-19.0.0-3830.g3 329 kB/s | 16 kB 00:00 2024-05-21T03:09:58.372 INFO:teuthology.orchestra.run.smithi165.stdout:(36/159): ceph-volume-19.0.0-3830.g3b748ea0.el9 4.6 MB/s | 273 kB 00:00 2024-05-21T03:09:58.458 INFO:teuthology.orchestra.run.smithi027.stdout:(150/159): NetworkManager-libnm-1.47.90-1.el9.x 5.2 MB/s | 1.8 MB 00:00 2024-05-21T03:09:58.505 INFO:teuthology.orchestra.run.smithi165.stdout:(37/159): cephadm-19.0.0-3830.g3b748ea0.el9.noa 4.5 MB/s | 764 kB 00:00 2024-05-21T03:09:58.531 INFO:teuthology.orchestra.run.smithi165.stdout:(38/159): cephfs-top-19.0.0-3830.g3b748ea0.el9. 115 kB/s | 18 kB 00:00 2024-05-21T03:09:58.597 INFO:teuthology.orchestra.run.smithi027.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 293 kB/s | 41 kB 00:00 2024-05-21T03:09:58.631 INFO:teuthology.orchestra.run.smithi165.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 642 kB/s | 80 kB 00:00 2024-05-21T03:09:58.698 INFO:teuthology.orchestra.run.smithi165.stdout:(40/159): libgfortran-11.4.1-3.el9.x86_64.rpm 4.7 MB/s | 803 kB 00:00 2024-05-21T03:09:58.705 INFO:teuthology.orchestra.run.smithi027.stdout:(152/159): NetworkManager-tui-1.47.90-1.el9.x86 2.2 MB/s | 248 kB 00:00 2024-05-21T03:09:58.723 INFO:teuthology.orchestra.run.smithi165.stdout:(41/159): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-05-21T03:09:58.749 INFO:teuthology.orchestra.run.smithi165.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-05-21T03:09:58.790 INFO:teuthology.orchestra.run.smithi165.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-05-21T03:09:58.814 INFO:teuthology.orchestra.run.smithi027.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 339 kB/s | 37 kB 00:00 2024-05-21T03:09:58.832 INFO:teuthology.orchestra.run.smithi165.stdout:(44/159): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-05-21T03:09:58.858 INFO:teuthology.orchestra.run.smithi165.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-05-21T03:09:58.883 INFO:teuthology.orchestra.run.smithi165.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-05-21T03:09:58.935 INFO:teuthology.orchestra.run.smithi165.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 2.7 MB/s | 215 kB 00:00 2024-05-21T03:09:58.967 INFO:teuthology.orchestra.run.smithi165.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 8.1 MB/s | 689 kB 00:00 2024-05-21T03:09:59.000 INFO:teuthology.orchestra.run.smithi165.stdout:(49/159): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-05-21T03:09:59.026 INFO:teuthology.orchestra.run.smithi165.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-05-21T03:09:59.026 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-21T03:09:59.071 INFO:teuthology.orchestra.run.smithi165.stdout:(51/159): flex-2.6.4-9.el9.x86_64.rpm 7.4 MB/s | 313 kB 00:00 2024-05-21T03:09:59.110 INFO:teuthology.orchestra.run.smithi165.stdout:(52/159): flexiblas-3.0.4-8.el9.x86_64.rpm 790 kB/s | 33 kB 00:00 2024-05-21T03:09:59.173 INFO:teuthology.orchestra.run.smithi027.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 577 kB/s | 207 kB 00:00 2024-05-21T03:09:59.552 INFO:teuthology.orchestra.run.smithi165.stdout:(53/159): ceph-test-19.0.0-3830.g3b748ea0.el9.x 14 MB/s | 49 MB 00:03 2024-05-21T03:09:59.577 INFO:teuthology.orchestra.run.smithi165.stdout:(54/159): elfutils-libelf-devel-0.191-4.el9.x86 38 kB/s | 22 kB 00:00 2024-05-21T03:09:59.602 INFO:teuthology.orchestra.run.smithi165.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-05-21T03:09:59.614 INFO:teuthology.orchestra.run.smithi027.stdout:(155/159): elfutils-libs-0.191-4.el9.x86_64.rpm 588 kB/s | 260 kB 00:00 2024-05-21T03:09:59.661 INFO:teuthology.orchestra.run.smithi165.stdout:(56/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.4 MB/s | 3.0 MB 00:00 2024-05-21T03:09:59.687 INFO:teuthology.orchestra.run.smithi165.stdout:(57/159): libaio-devel-0.3.111-13.el9.x86_64.rp 102 kB/s | 11 kB 00:00 2024-05-21T03:09:59.711 INFO:teuthology.orchestra.run.smithi165.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 899 kB/s | 45 kB 00:00 2024-05-21T03:09:59.737 INFO:teuthology.orchestra.run.smithi165.stdout:(59/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-05-21T03:09:59.762 INFO:teuthology.orchestra.run.smithi165.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 582 kB 00:00 2024-05-21T03:09:59.787 INFO:teuthology.orchestra.run.smithi165.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 507 kB/s | 25 kB 00:00 2024-05-21T03:09:59.812 INFO:teuthology.orchestra.run.smithi165.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-05-21T03:09:59.838 INFO:teuthology.orchestra.run.smithi165.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-05-21T03:09:59.863 INFO:teuthology.orchestra.run.smithi165.stdout:(64/159): librdkafka-1.6.1-102.el9.x86_64.rpm 3.7 MB/s | 662 kB 00:00 2024-05-21T03:09:59.890 INFO:teuthology.orchestra.run.smithi165.stdout:(65/159): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-05-21T03:09:59.905 INFO:teuthology.orchestra.run.smithi165.stdout:(66/159): openblas-0.3.21-2.el9.x86_64.rpm 526 kB/s | 35 kB 00:00 2024-05-21T03:09:59.989 INFO:teuthology.orchestra.run.smithi165.stdout:(67/159): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-05-21T03:10:00.039 INFO:teuthology.orchestra.run.smithi165.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-05-21T03:10:00.081 INFO:teuthology.orchestra.run.smithi165.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 9.7 MB/s | 911 kB 00:00 2024-05-21T03:10:00.106 INFO:teuthology.orchestra.run.smithi165.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-05-21T03:10:00.173 INFO:teuthology.orchestra.run.smithi165.stdout:(71/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-05-21T03:10:00.215 INFO:teuthology.orchestra.run.smithi165.stdout:(72/159): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-05-21T03:10:00.240 INFO:teuthology.orchestra.run.smithi165.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-05-21T03:10:00.257 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] python3-colorama-0.4.6-3.el9.noarch.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-colorama-0.4.6-3.el9.noarch.rpm (IP: 199.223.246.113) 2024-05-21T03:10:00.332 INFO:teuthology.orchestra.run.smithi165.stdout:(74/159): postgresql-server-13.14-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-05-21T03:10:00.416 INFO:teuthology.orchestra.run.smithi165.stdout:(75/159): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 6.0 MB 00:00 2024-05-21T03:10:00.441 INFO:teuthology.orchestra.run.smithi165.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-05-21T03:10:00.441 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] python3-jinja2-2.11.3-6.el9.noarch.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-jinja2-2.11.3-6.el9.noarch.rpm (IP: 199.223.246.113) 2024-05-21T03:10:00.483 INFO:teuthology.orchestra.run.smithi165.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 615 kB/s | 26 kB 00:00 2024-05-21T03:10:00.525 INFO:teuthology.orchestra.run.smithi165.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 460 kB/s | 19 kB 00:00 2024-05-21T03:10:00.550 INFO:teuthology.orchestra.run.smithi165.stdout:(79/159): python3-colorama-0.4.6-3.el9.noarch.r 198 kB/s | 61 kB 00:00 2024-05-21T03:10:00.567 INFO:teuthology.orchestra.run.smithi165.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-05-21T03:10:00.593 INFO:teuthology.orchestra.run.smithi165.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 829 kB/s | 35 kB 00:00 2024-05-21T03:10:00.634 INFO:teuthology.orchestra.run.smithi165.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-05-21T03:10:00.693 INFO:teuthology.orchestra.run.smithi165.stdout:(83/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 41 MB/s | 5.1 MB 00:00 2024-05-21T03:10:00.718 INFO:teuthology.orchestra.run.smithi165.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-05-21T03:10:00.743 INFO:teuthology.orchestra.run.smithi165.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-21T03:10:00.769 INFO:teuthology.orchestra.run.smithi165.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 845 kB/s | 42 kB 00:00 2024-05-21T03:10:00.794 INFO:teuthology.orchestra.run.smithi165.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-05-21T03:10:00.819 INFO:teuthology.orchestra.run.smithi165.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-21T03:10:00.853 INFO:teuthology.orchestra.run.smithi165.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-05-21T03:10:00.865 INFO:teuthology.orchestra.run.smithi027.stdout:(156/159): librbd1-19.0.0-3830.g3b748ea0.el9.x8 744 kB/s | 3.1 MB 00:04 2024-05-21T03:10:00.878 INFO:teuthology.orchestra.run.smithi165.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 867 kB/s | 51 kB 00:00 2024-05-21T03:10:00.904 INFO:teuthology.orchestra.run.smithi165.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-21T03:10:00.946 INFO:teuthology.orchestra.run.smithi165.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-05-21T03:10:00.987 INFO:teuthology.orchestra.run.smithi165.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-05-21T03:10:01.030 INFO:teuthology.orchestra.run.smithi165.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-05-21T03:10:01.055 INFO:teuthology.orchestra.run.smithi165.stdout:(95/159): python3-jinja2-2.11.3-6.el9.noarch.rp 389 kB/s | 248 kB 00:00 2024-05-21T03:10:01.080 INFO:teuthology.orchestra.run.smithi165.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 718 kB/s | 36 kB 00:00 2024-05-21T03:10:01.255 INFO:teuthology.orchestra.run.smithi165.stdout:(97/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 43 MB/s | 16 MB 00:00 2024-05-21T03:10:01.281 INFO:teuthology.orchestra.run.smithi165.stdout:(98/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 210 kB/s | 47 kB 00:00 2024-05-21T03:10:01.314 INFO:teuthology.orchestra.run.smithi165.stdout:(99/159): socat-1.7.4.1-5.el9.x86_64.rpm 1.3 MB/s | 305 kB 00:00 2024-05-21T03:10:01.414 INFO:teuthology.orchestra.run.smithi165.stdout:(100/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 638 kB/s | 64 kB 00:00 2024-05-21T03:10:01.473 INFO:teuthology.orchestra.run.smithi165.stdout:(101/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 488 kB/s | 94 kB 00:00 2024-05-21T03:10:01.482 INFO:teuthology.orchestra.run.smithi027.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 740 kB/s | 1.3 MB 00:01 2024-05-21T03:10:01.506 INFO:teuthology.orchestra.run.smithi165.stdout:(102/159): lua-devel-5.4.4-4.el9.x86_64.rpm 672 kB/s | 22 kB 00:00 2024-05-21T03:10:01.523 INFO:teuthology.orchestra.run.smithi165.stdout:(103/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 412 kB/s | 45 kB 00:00 2024-05-21T03:10:01.557 INFO:teuthology.orchestra.run.smithi165.stdout:(104/159): protobuf-compiler-3.14.0-13.el9.x86_ 17 MB/s | 863 kB 00:00 2024-05-21T03:10:01.599 INFO:teuthology.orchestra.run.smithi165.stdout:(105/159): uuid-1.6.2-55.el9.x86_64.rpm 168 kB/s | 58 kB 00:00 2024-05-21T03:10:01.883 INFO:teuthology.orchestra.run.smithi165.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 69 kB/s | 19 kB 00:00 2024-05-21T03:10:02.074 INFO:teuthology.orchestra.run.smithi027.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.0 MB/s | 2.4 MB 00:01 2024-05-21T03:10:02.183 INFO:teuthology.orchestra.run.smithi165.stdout:(107/159): gperftools-libs-2.9.1-2.el9.x86_64.r 494 kB/s | 309 kB 00:00 2024-05-21T03:10:02.208 INFO:teuthology.orchestra.run.smithi165.stdout:(108/159): abseil-cpp-20211102.0-3.el9.x86_64.r 801 kB/s | 548 kB 00:00 2024-05-21T03:10:02.325 INFO:teuthology.orchestra.run.smithi165.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 183 kB/s | 26 kB 00:00 2024-05-21T03:10:02.350 INFO:teuthology.orchestra.run.smithi165.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 343 kB/s | 49 kB 00:00 2024-05-21T03:10:02.467 INFO:teuthology.orchestra.run.smithi165.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 475 kB/s | 67 kB 00:00 2024-05-21T03:10:02.509 INFO:teuthology.orchestra.run.smithi165.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 954 kB/s | 151 kB 00:00 2024-05-21T03:10:02.709 INFO:teuthology.orchestra.run.smithi165.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 2.7 MB/s | 548 kB 00:00 2024-05-21T03:10:02.768 INFO:teuthology.orchestra.run.smithi165.stdout:(114/159): libarrow-9.0.0-11.el9.x86_64.rpm 5.0 MB/s | 4.4 MB 00:00 2024-05-21T03:10:02.793 INFO:teuthology.orchestra.run.smithi165.stdout:(115/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.5 MB/s | 839 kB 00:00 2024-05-21T03:10:02.818 INFO:teuthology.orchestra.run.smithi165.stdout:(116/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 400 kB/s | 43 kB 00:00 2024-05-21T03:10:02.868 INFO:teuthology.orchestra.run.smithi165.stdout:(117/159): python3-cachetools-4.2.4-1.el9.noarc 320 kB/s | 32 kB 00:00 2024-05-21T03:10:02.894 INFO:teuthology.orchestra.run.smithi165.stdout:(118/159): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-05-21T03:10:02.927 INFO:teuthology.orchestra.run.smithi165.stdout:(119/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-05-21T03:10:02.994 INFO:teuthology.orchestra.run.smithi165.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-05-21T03:10:03.020 INFO:teuthology.orchestra.run.smithi165.stdout:(121/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-05-21T03:10:03.062 INFO:teuthology.orchestra.run.smithi165.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 1.5 MB/s | 206 kB 00:00 2024-05-21T03:10:03.129 INFO:teuthology.orchestra.run.smithi165.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-05-21T03:10:03.171 INFO:teuthology.orchestra.run.smithi165.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 98 kB/s | 11 kB 00:00 2024-05-21T03:10:03.221 INFO:teuthology.orchestra.run.smithi165.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-05-21T03:10:03.279 INFO:teuthology.orchestra.run.smithi165.stdout:(126/159): python3-grpcio-1.46.7-10.el9.x86_64. 7.2 MB/s | 2.0 MB 00:00 2024-05-21T03:10:03.305 INFO:teuthology.orchestra.run.smithi165.stdout:(127/159): python3-jaraco-collections-3.0.0-8.e 173 kB/s | 23 kB 00:00 2024-05-21T03:10:03.330 INFO:teuthology.orchestra.run.smithi165.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 179 kB/s | 19 kB 00:00 2024-05-21T03:10:03.405 INFO:teuthology.orchestra.run.smithi165.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 157 kB/s | 20 kB 00:00 2024-05-21T03:10:03.481 INFO:teuthology.orchestra.run.smithi165.stdout:(130/159): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-05-21T03:10:03.506 INFO:teuthology.orchestra.run.smithi165.stdout:(131/159): python3-logutils-0.3.5-21.el9.noarch 264 kB/s | 46 kB 00:00 2024-05-21T03:10:03.556 INFO:teuthology.orchestra.run.smithi165.stdout:(132/159): python3-more-itertools-8.12.0-2.el9. 524 kB/s | 79 kB 00:00 2024-05-21T03:10:03.590 INFO:teuthology.orchestra.run.smithi165.stdout:(133/159): python3-natsort-7.1.1-5.el9.noarch.r 531 kB/s | 58 kB 00:00 2024-05-21T03:10:03.615 INFO:teuthology.orchestra.run.smithi165.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-05-21T03:10:03.690 INFO:teuthology.orchestra.run.smithi165.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 122 kB/s | 16 kB 00:00 2024-05-21T03:10:03.715 INFO:teuthology.orchestra.run.smithi165.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 718 kB/s | 90 kB 00:00 2024-05-21T03:10:03.741 INFO:teuthology.orchestra.run.smithi165.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 225 kB/s | 28 kB 00:00 2024-05-21T03:10:03.816 INFO:teuthology.orchestra.run.smithi165.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 245 kB/s | 31 kB 00:00 2024-05-21T03:10:03.841 INFO:teuthology.orchestra.run.smithi165.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-05-21T03:10:03.875 INFO:teuthology.orchestra.run.smithi165.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 442 kB/s | 59 kB 00:00 2024-05-21T03:10:03.925 INFO:teuthology.orchestra.run.smithi165.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 328 kB/s | 36 kB 00:00 2024-05-21T03:10:03.950 INFO:teuthology.orchestra.run.smithi165.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 469 kB/s | 51 kB 00:00 2024-05-21T03:10:03.984 INFO:teuthology.orchestra.run.smithi165.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-05-21T03:10:03.984 INFO:teuthology.orchestra.run.smithi039.stdout:(159/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 956 kB/s | 16 MB 00:17 2024-05-21T03:10:03.987 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-05-21T03:10:03.987 INFO:teuthology.orchestra.run.smithi039.stdout:Total 7.6 MB/s | 227 MB 00:29 2024-05-21T03:10:04.059 INFO:teuthology.orchestra.run.smithi165.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 671 kB/s | 90 kB 00:00 2024-05-21T03:10:04.092 INFO:teuthology.orchestra.run.smithi165.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-05-21T03:10:04.160 INFO:teuthology.orchestra.run.smithi165.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 114 kB/s | 20 kB 00:00 2024-05-21T03:10:04.185 INFO:teuthology.orchestra.run.smithi165.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-05-21T03:10:04.285 INFO:teuthology.orchestra.run.smithi165.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 8.2 MB/s | 1.6 MB 00:00 2024-05-21T03:10:04.468 INFO:teuthology.orchestra.run.smithi165.stdout:(149/159): NetworkManager-1.47.90-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-05-21T03:10:04.544 INFO:teuthology.orchestra.run.smithi165.stdout:(150/159): librados2-19.0.0-3830.g3b748ea0.el9. 8.9 MB/s | 3.4 MB 00:00 2024-05-21T03:10:04.594 INFO:teuthology.orchestra.run.smithi165.stdout:(151/159): librbd1-19.0.0-3830.g3b748ea0.el9.x8 7.7 MB/s | 3.1 MB 00:00 2024-05-21T03:10:04.636 INFO:teuthology.orchestra.run.smithi165.stdout:(152/159): NetworkManager-libnm-1.47.90-1.el9.x 11 MB/s | 1.8 MB 00:00 2024-05-21T03:10:04.678 INFO:teuthology.orchestra.run.smithi165.stdout:(153/159): NetworkManager-team-1.47.90-1.el9.x8 309 kB/s | 41 kB 00:00 2024-05-21T03:10:04.679 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-21T03:10:04.695 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] elfutils-libelf-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libelf-0.191-4.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-21T03:10:04.720 INFO:teuthology.orchestra.run.smithi165.stdout:(154/159): NetworkManager-tui-1.47.90-1.el9.x86 1.9 MB/s | 248 kB 00:00 2024-05-21T03:10:04.871 INFO:teuthology.orchestra.run.smithi165.stdout:(155/159): elfutils-debuginfod-client-0.191-4.e 157 kB/s | 37 kB 00:00 2024-05-21T03:10:05.137 INFO:teuthology.orchestra.run.smithi165.stdout:(156/159): elfutils-libelf-0.191-4.el9.x86_64.r 451 kB/s | 207 kB 00:00 2024-05-21T03:10:05.245 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-05-21T03:10:05.254 INFO:teuthology.orchestra.run.smithi165.stdout:(157/159): elfutils-libs-0.191-4.el9.x86_64.rpm 486 kB/s | 260 kB 00:00 2024-05-21T03:10:05.421 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-05-21T03:10:05.421 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-05-21T03:10:05.671 INFO:teuthology.orchestra.run.smithi165.stdout:(158/159): openssl-3.2.1-1.el9.x86_64.rpm 1.7 MB/s | 1.3 MB 00:00 2024-05-21T03:10:05.946 INFO:teuthology.orchestra.run.smithi165.stdout:(159/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 3.0 MB/s | 2.4 MB 00:00 2024-05-21T03:10:05.949 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-05-21T03:10:05.949 INFO:teuthology.orchestra.run.smithi165.stdout:Total 17 MB/s | 227 MB 00:13 2024-05-21T03:10:07.131 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-05-21T03:10:07.288 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-05-21T03:10:07.288 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-05-21T03:10:07.341 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-05-21T03:10:07.341 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-05-21T03:10:09.161 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-05-21T03:10:09.162 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-05-21T03:10:12.807 INFO:teuthology.orchestra.run.smithi027.stdout:(159/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 820 kB/s | 16 MB 00:20 2024-05-21T03:10:12.811 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-05-21T03:10:12.812 INFO:teuthology.orchestra.run.smithi027.stdout:Total 5.6 MB/s | 227 MB 00:40 2024-05-21T03:10:13.830 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-05-21T03:10:14.041 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-21T03:10:14.045 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-05-21T03:10:14.080 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-21T03:10:14.134 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-21T03:10:14.184 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-21T03:10:14.208 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-21T03:10:14.224 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-05-21T03:10:14.225 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-05-21T03:10:14.253 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-21T03:10:14.600 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-21T03:10:14.718 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : librados2-2:19.0.0-3830.g3b748ea0.el9.x86_64 8/170 2024-05-21T03:10:14.843 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librados2-2:19.0.0-3830.g3b748ea0.el9.x86_64 8/170 2024-05-21T03:10:14.985 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephfs2-2:19.0.0-3830.g3b748ea0.el9.x86_64 9/170 2024-05-21T03:10:15.070 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libcephfs2-2:19.0.0-3830.g3b748ea0.el9.x86_64 9/170 2024-05-21T03:10:15.098 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rados-2:19.0.0-3830.g3b748ea0.el9.x86_64 10/170 2024-05-21T03:10:15.131 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-21T03:10:15.336 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-21T03:10:15.388 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-21T03:10:15.415 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephsqlite-2:19.0.0-3830.g3b748ea0.el9.x86_64 14/170 2024-05-21T03:10:15.510 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-05-21T03:10:15.521 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3830.g3b748ea0.el9.x86_64 14/170 2024-05-21T03:10:15.550 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libradosstriper1-2:19.0.0-3830.g3b748ea0.el9.x86 15/170 2024-05-21T03:10:15.592 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-21T03:10:15.596 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3830.g3b748ea0.el9.x86 15/170 2024-05-21T03:10:15.609 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-21T03:10:15.630 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-21T03:10:15.635 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-21T03:10:15.683 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-21T03:10:15.717 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-21T03:10:15.743 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-21T03:10:15.785 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-21T03:10:15.785 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-21T03:10:15.921 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-21T03:10:16.073 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-21T03:10:16.114 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-21T03:10:16.133 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-21T03:10:16.210 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : librados2-2:19.0.0-3830.g3b748ea0.el9.x86_64 8/170 2024-05-21T03:10:16.227 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T03:10:16.309 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T03:10:16.332 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados2-2:19.0.0-3830.g3b748ea0.el9.x86_64 8/170 2024-05-21T03:10:16.353 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephfs2-2:19.0.0-3830.g3b748ea0.el9.x86_64 9/170 2024-05-21T03:10:16.444 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephfs2-2:19.0.0-3830.g3b748ea0.el9.x86_64 9/170 2024-05-21T03:10:16.474 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rados-2:19.0.0-3830.g3b748ea0.el9.x86_64 10/170 2024-05-21T03:10:16.507 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-21T03:10:16.508 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-05-21T03:10:16.509 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-05-21T03:10:16.518 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T03:10:16.629 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-21T03:10:16.658 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-21T03:10:16.707 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-21T03:10:16.710 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/170 2024-05-21T03:10:16.772 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-21T03:10:16.791 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephsqlite-2:19.0.0-3830.g3b748ea0.el9.x86_64 14/170 2024-05-21T03:10:16.794 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/170 2024-05-21T03:10:16.835 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/170 2024-05-21T03:10:16.878 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/170 2024-05-21T03:10:16.895 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3830.g3b748ea0.el9.x86_64 14/170 2024-05-21T03:10:16.911 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/170 2024-05-21T03:10:16.917 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libradosstriper1-2:19.0.0-3830.g3b748ea0.el9.x86 15/170 2024-05-21T03:10:16.939 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/170 2024-05-21T03:10:16.982 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3830.g3b748ea0.el9.x86 15/170 2024-05-21T03:10:17.004 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-21T03:10:17.010 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/170 2024-05-21T03:10:17.028 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-21T03:10:17.070 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/170 2024-05-21T03:10:17.099 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/170 2024-05-21T03:10:17.152 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/170 2024-05-21T03:10:17.220 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-21T03:10:17.231 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/170 2024-05-21T03:10:17.263 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ceph-common-2:19.0.0-3830.g3b748ea0.el9. 33/170 2024-05-21T03:10:17.307 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ceph-argparse-2:19.0.0-3830.g3b748ea0.el 34/170 2024-05-21T03:10:17.335 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cephfs-2:19.0.0-3830.g3b748ea0.el9.x86_6 35/170 2024-05-21T03:10:17.410 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/170 2024-05-21T03:10:17.413 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-21T03:10:17.459 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/170 2024-05-21T03:10:17.537 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/170 2024-05-21T03:10:17.575 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-21T03:10:17.635 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-21T03:10:17.683 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/170 2024-05-21T03:10:17.736 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T03:10:17.737 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/170 2024-05-21T03:10:17.828 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T03:10:18.026 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T03:10:18.135 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-21T03:10:18.157 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-21T03:10:18.193 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/170 2024-05-21T03:10:18.265 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/170 2024-05-21T03:10:18.300 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/170 2024-05-21T03:10:18.303 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/170 2024-05-21T03:10:18.343 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/170 2024-05-21T03:10:18.376 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/170 2024-05-21T03:10:18.394 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/170 2024-05-21T03:10:18.404 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/170 2024-05-21T03:10:18.429 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/170 2024-05-21T03:10:18.464 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/170 2024-05-21T03:10:18.464 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/170 2024-05-21T03:10:18.513 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 45/170 2024-05-21T03:10:18.526 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/170 2024-05-21T03:10:18.552 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librados-devel-2:19.0.0-3830.g3b748ea0.el9.x86_6 46/170 2024-05-21T03:10:18.556 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/170 2024-05-21T03:10:18.592 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/170 2024-05-21T03:10:18.594 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/170 2024-05-21T03:10:18.668 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/170 2024-05-21T03:10:18.704 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ceph-common-2:19.0.0-3830.g3b748ea0.el9. 33/170 2024-05-21T03:10:18.746 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ceph-argparse-2:19.0.0-3830.g3b748ea0.el 34/170 2024-05-21T03:10:18.770 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 48/170 2024-05-21T03:10:18.775 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cephfs-2:19.0.0-3830.g3b748ea0.el9.x86_6 35/170 2024-05-21T03:10:18.845 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/170 2024-05-21T03:10:18.856 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 49/170 2024-05-21T03:10:18.890 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/170 2024-05-21T03:10:18.953 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/170 2024-05-21T03:10:18.965 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/170 2024-05-21T03:10:19.022 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/170 2024-05-21T03:10:19.101 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/170 2024-05-21T03:10:19.105 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-21T03:10:19.140 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-21T03:10:19.158 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/170 2024-05-21T03:10:19.177 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-21T03:10:19.225 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-21T03:10:19.259 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-21T03:10:19.304 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-21T03:10:19.346 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-21T03:10:19.491 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-21T03:10:19.539 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-21T03:10:19.581 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-21T03:10:19.681 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-21T03:10:19.681 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/170 2024-05-21T03:10:19.714 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-21T03:10:19.754 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-21T03:10:19.771 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/170 2024-05-21T03:10:19.779 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-21T03:10:19.804 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-21T03:10:19.808 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/170 2024-05-21T03:10:19.844 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/170 2024-05-21T03:10:19.884 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-21T03:10:19.893 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 45/170 2024-05-21T03:10:19.931 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librados-devel-2:19.0.0-3830.g3b748ea0.el9.x86_6 46/170 2024-05-21T03:10:19.973 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/170 2024-05-21T03:10:20.014 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-21T03:10:20.045 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-21T03:10:20.065 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-21T03:10:20.099 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-21T03:10:20.129 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 48/170 2024-05-21T03:10:20.138 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-21T03:10:20.179 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-21T03:10:20.183 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 49/170 2024-05-21T03:10:20.208 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-21T03:10:20.243 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-21T03:10:20.275 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/170 2024-05-21T03:10:20.276 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-21T03:10:20.319 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-21T03:10:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/170 2024-05-21T03:10:20.361 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-21T03:10:20.412 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-21T03:10:20.443 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-21T03:10:20.453 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-21T03:10:20.480 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-21T03:10:20.501 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-21T03:10:20.527 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-21T03:10:20.539 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-21T03:10:20.563 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-21T03:10:20.588 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-21T03:10:20.605 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-21T03:10:20.647 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-21T03:10:20.699 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-21T03:10:20.751 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-21T03:10:20.784 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-21T03:10:20.824 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-21T03:10:20.866 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-21T03:10:20.959 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-21T03:10:21.244 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-21T03:10:21.300 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-21T03:10:21.340 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-21T03:10:21.365 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-21T03:10:21.448 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-21T03:10:21.541 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-21T03:10:21.577 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-21T03:10:21.589 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-21T03:10:21.613 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-21T03:10:21.622 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-21T03:10:21.641 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-21T03:10:21.675 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-21T03:10:21.714 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-21T03:10:21.729 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-21T03:10:21.746 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-21T03:10:21.776 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-21T03:10:21.796 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-21T03:10:21.810 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-21T03:10:21.841 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-21T03:10:21.843 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-21T03:10:21.884 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-21T03:10:21.919 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-21T03:10:21.933 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-21T03:10:21.975 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-21T03:10:22.031 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-21T03:10:22.075 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-21T03:10:22.106 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-21T03:10:22.154 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-21T03:10:22.161 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-21T03:10:22.198 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-21T03:10:22.252 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-21T03:10:22.278 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-21T03:10:22.307 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-21T03:10:22.948 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-05-21T03:10:22.981 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-21T03:10:23.030 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-21T03:10:23.045 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-21T03:10:23.079 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-21T03:10:23.088 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-21T03:10:23.111 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-21T03:10:23.113 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-21T03:10:23.155 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-21T03:10:23.195 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-21T03:10:23.202 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-21T03:10:23.220 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-21T03:10:23.251 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-21T03:10:23.273 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-21T03:10:23.296 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-21T03:10:23.384 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-21T03:10:23.432 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-21T03:10:23.612 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-21T03:10:23.620 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-21T03:10:23.637 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-21T03:10:23.716 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-21T03:10:23.730 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librados2-2:19.0.0-3830.g3b748ea0.el9.x86_64 8/170 2024-05-21T03:10:23.855 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:19.0.0-3830.g3b748ea0.el9.x86_64 8/170 2024-05-21T03:10:23.880 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs2-2:19.0.0-3830.g3b748ea0.el9.x86_64 9/170 2024-05-21T03:10:23.974 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephfs2-2:19.0.0-3830.g3b748ea0.el9.x86_64 9/170 2024-05-21T03:10:24.002 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rados-2:19.0.0-3830.g3b748ea0.el9.x86_64 10/170 2024-05-21T03:10:24.051 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-21T03:10:24.266 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-21T03:10:24.310 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-21T03:10:24.336 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephsqlite-2:19.0.0-3830.g3b748ea0.el9.x86_64 14/170 2024-05-21T03:10:24.353 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-21T03:10:24.443 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3830.g3b748ea0.el9.x86_64 14/170 2024-05-21T03:10:24.451 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-21T03:10:24.469 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-21T03:10:24.471 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libradosstriper1-2:19.0.0-3830.g3b748ea0.el9.x86 15/170 2024-05-21T03:10:24.526 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3830.g3b748ea0.el9.x86 15/170 2024-05-21T03:10:24.528 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-21T03:10:24.547 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-21T03:10:24.565 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-21T03:10:24.571 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-21T03:10:24.589 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-21T03:10:24.767 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-21T03:10:24.920 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-21T03:10:24.933 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-21T03:10:24.961 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : librbd1-2:19.0.0-3830.g3b748ea0.el9.x86_64 103/170 2024-05-21T03:10:25.036 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librbd1-2:19.0.0-3830.g3b748ea0.el9.x86_64 103/170 2024-05-21T03:10:25.066 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 104/170 2024-05-21T03:10:25.094 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-21T03:10:25.102 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-21T03:10:25.155 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-21T03:10:25.258 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T03:10:25.323 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T03:10:25.541 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T03:10:25.611 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-21T03:10:25.639 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librgw2-2:19.0.0-3830.g3b748ea0.el9.x86_64 107/170 2024-05-21T03:10:25.696 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-21T03:10:25.697 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-21T03:10:25.702 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librgw2-2:19.0.0-3830.g3b748ea0.el9.x86_64 107/170 2024-05-21T03:10:25.727 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-21T03:10:25.733 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 108/170 2024-05-21T03:10:25.770 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/170 2024-05-21T03:10:25.782 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-21T03:10:25.818 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-21T03:10:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/170 2024-05-21T03:10:25.851 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-21T03:10:25.886 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/170 2024-05-21T03:10:25.930 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/170 2024-05-21T03:10:25.963 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/170 2024-05-21T03:10:25.990 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/170 2024-05-21T03:10:26.062 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/170 2024-05-21T03:10:26.122 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/170 2024-05-21T03:10:26.151 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/170 2024-05-21T03:10:26.164 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-21T03:10:26.187 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/170 2024-05-21T03:10:26.206 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : librbd1-2:19.0.0-3830.g3b748ea0.el9.x86_64 103/170 2024-05-21T03:10:26.266 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/170 2024-05-21T03:10:26.280 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librbd1-2:19.0.0-3830.g3b748ea0.el9.x86_64 103/170 2024-05-21T03:10:26.306 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-common-2:19.0.0-3830.g3b748ea0.el9. 33/170 2024-05-21T03:10:26.310 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 104/170 2024-05-21T03:10:26.347 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-21T03:10:26.350 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-argparse-2:19.0.0-3830.g3b748ea0.el 34/170 2024-05-21T03:10:26.378 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cephfs-2:19.0.0-3830.g3b748ea0.el9.x86_6 35/170 2024-05-21T03:10:26.451 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/170 2024-05-21T03:10:26.502 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/170 2024-05-21T03:10:26.580 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/170 2024-05-21T03:10:26.724 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/170 2024-05-21T03:10:26.780 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/170 2024-05-21T03:10:26.829 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-21T03:10:26.859 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librgw2-2:19.0.0-3830.g3b748ea0.el9.x86_64 107/170 2024-05-21T03:10:26.913 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librgw2-2:19.0.0-3830.g3b748ea0.el9.x86_64 107/170 2024-05-21T03:10:26.935 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 108/170 2024-05-21T03:10:27.333 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/170 2024-05-21T03:10:27.419 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/170 2024-05-21T03:10:27.462 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/170 2024-05-21T03:10:27.500 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/170 2024-05-21T03:10:27.548 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 45/170 2024-05-21T03:10:27.586 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librados-devel-2:19.0.0-3830.g3b748ea0.el9.x86_6 46/170 2024-05-21T03:10:27.657 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/170 2024-05-21T03:10:27.820 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 48/170 2024-05-21T03:10:27.872 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 49/170 2024-05-21T03:10:27.967 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/170 2024-05-21T03:10:28.030 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/170 2024-05-21T03:10:28.080 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 109/170 2024-05-21T03:10:28.103 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-21T03:10:28.113 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 109/170 2024-05-21T03:10:28.140 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-21T03:10:28.177 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-21T03:10:28.224 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-21T03:10:28.260 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-21T03:10:28.312 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-21T03:10:28.353 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-21T03:10:28.496 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-21T03:10:28.538 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-21T03:10:28.581 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-21T03:10:28.678 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-21T03:10:28.684 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 109/170 2024-05-21T03:10:28.716 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-21T03:10:28.725 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-base-2:19.0.0-3830.g3b748ea0.el9.x86_64 110/170 2024-05-21T03:10:28.757 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-21T03:10:28.789 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-21T03:10:28.801 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-base-2:19.0.0-3830.g3b748ea0.el9.x86_64 110/170 2024-05-21T03:10:28.801 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-21T03:10:28.801 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-21T03:10:28.802 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-21T03:10:28.813 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-21T03:10:28.834 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-selinux-2:19.0.0-3830.g3b748ea0.el9.x86_64 111/170 2024-05-21T03:10:28.913 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-21T03:10:29.045 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-21T03:10:29.088 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-21T03:10:29.117 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-21T03:10:29.151 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-21T03:10:29.190 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-21T03:10:29.221 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-21T03:10:29.250 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-21T03:10:29.252 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 109/170 2024-05-21T03:10:29.285 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-21T03:10:29.318 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-21T03:10:29.361 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-21T03:10:29.394 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-21T03:10:29.485 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-21T03:10:29.540 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 109/170 2024-05-21T03:10:29.542 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-21T03:10:29.573 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-21T03:10:29.620 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-21T03:10:29.731 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-21T03:10:29.784 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-21T03:10:30.084 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 109/170 2024-05-21T03:10:30.118 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-base-2:19.0.0-3830.g3b748ea0.el9.x86_64 110/170 2024-05-21T03:10:30.209 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-base-2:19.0.0-3830.g3b748ea0.el9.x86_64 110/170 2024-05-21T03:10:30.209 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-21T03:10:30.209 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-21T03:10:30.209 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-21T03:10:30.235 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-selinux-2:19.0.0-3830.g3b748ea0.el9.x86_64 111/170 2024-05-21T03:10:30.546 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-21T03:10:30.597 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-21T03:10:30.630 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-21T03:10:30.733 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-21T03:10:30.786 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-21T03:10:30.831 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-21T03:10:30.921 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-21T03:10:30.958 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-21T03:10:31.143 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-21T03:10:31.172 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-21T03:10:31.256 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-21T03:10:31.947 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-21T03:10:32.066 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-21T03:10:33.384 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-21T03:10:33.467 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-21T03:10:33.495 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-21T03:10:33.528 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-21T03:10:33.864 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-21T03:10:33.908 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librbd1-2:19.0.0-3830.g3b748ea0.el9.x86_64 103/170 2024-05-21T03:10:34.000 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:19.0.0-3830.g3b748ea0.el9.x86_64 103/170 2024-05-21T03:10:34.029 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 104/170 2024-05-21T03:10:34.066 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-21T03:10:34.572 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-21T03:10:34.611 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librgw2-2:19.0.0-3830.g3b748ea0.el9.x86_64 107/170 2024-05-21T03:10:34.683 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librgw2-2:19.0.0-3830.g3b748ea0.el9.x86_64 107/170 2024-05-21T03:10:34.712 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 108/170 2024-05-21T03:10:37.070 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 109/170 2024-05-21T03:10:37.100 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 109/170 2024-05-21T03:10:37.674 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 109/170 2024-05-21T03:10:37.712 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-base-2:19.0.0-3830.g3b748ea0.el9.x86_64 110/170 2024-05-21T03:10:37.802 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-base-2:19.0.0-3830.g3b748ea0.el9.x86_64 110/170 2024-05-21T03:10:37.803 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-21T03:10:37.803 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-21T03:10:37.803 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:10:37.829 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-selinux-2:19.0.0-3830.g3b748ea0.el9.x86_64 111/170 2024-05-21T03:10:41.058 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3830.g3b748ea0.el9.x86_64 111/170 2024-05-21T03:10:41.058 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /sys 2024-05-21T03:10:41.058 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /proc 2024-05-21T03:10:41.058 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /mnt 2024-05-21T03:10:41.058 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /var/tmp 2024-05-21T03:10:41.058 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /home 2024-05-21T03:10:41.058 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /root 2024-05-21T03:10:41.058 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /tmp 2024-05-21T03:10:41.059 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-21T03:10:41.230 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-volume-2:19.0.0-3830.g3b748ea0.el9.noarch 112/170 2024-05-21T03:10:41.247 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-volume-2:19.0.0-3830.g3b748ea0.el9.noarch 112/170 2024-05-21T03:10:41.247 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T03:10:41.247 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-21T03:10:41.247 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-21T03:10:41.687 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3830.g3b748ea0.el9.x86_64 111/170 2024-05-21T03:10:41.687 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /sys 2024-05-21T03:10:41.687 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /proc 2024-05-21T03:10:41.687 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /mnt 2024-05-21T03:10:41.687 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /var/tmp 2024-05-21T03:10:41.687 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /home 2024-05-21T03:10:41.687 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /root 2024-05-21T03:10:41.687 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /tmp 2024-05-21T03:10:41.687 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-21T03:10:41.874 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-volume-2:19.0.0-3830.g3b748ea0.el9.noarch 112/170 2024-05-21T03:10:41.890 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-volume-2:19.0.0-3830.g3b748ea0.el9.noarch 112/170 2024-05-21T03:10:41.890 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T03:10:41.890 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-21T03:10:41.890 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-21T03:10:42.800 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-osd-2:19.0.0-3830.g3b748ea0.el9.x86_64 113/170 2024-05-21T03:10:42.828 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-osd-2:19.0.0-3830.g3b748ea0.el9.x86_64 113/170 2024-05-21T03:10:42.828 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T03:10:42.828 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-21T03:10:42.828 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T03:10:42.828 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T03:10:42.828 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-21T03:10:43.067 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mds-2:19.0.0-3830.g3b748ea0.el9.x86_64 114/170 2024-05-21T03:10:43.091 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mds-2:19.0.0-3830.g3b748ea0.el9.x86_64 114/170 2024-05-21T03:10:43.091 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T03:10:43.091 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-21T03:10:43.091 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T03:10:43.091 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T03:10:43.091 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-21T03:10:43.377 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-osd-2:19.0.0-3830.g3b748ea0.el9.x86_64 113/170 2024-05-21T03:10:43.402 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-osd-2:19.0.0-3830.g3b748ea0.el9.x86_64 113/170 2024-05-21T03:10:43.403 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T03:10:43.403 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-21T03:10:43.403 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T03:10:43.403 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T03:10:43.403 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-21T03:10:43.559 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mon-2:19.0.0-3830.g3b748ea0.el9.x86_64 115/170 2024-05-21T03:10:43.583 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mon-2:19.0.0-3830.g3b748ea0.el9.x86_64 115/170 2024-05-21T03:10:43.584 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T03:10:43.584 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-21T03:10:43.584 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T03:10:43.584 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T03:10:43.584 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-21T03:10:43.628 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mds-2:19.0.0-3830.g3b748ea0.el9.x86_64 114/170 2024-05-21T03:10:43.650 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mds-2:19.0.0-3830.g3b748ea0.el9.x86_64 114/170 2024-05-21T03:10:43.650 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T03:10:43.651 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-21T03:10:43.651 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T03:10:43.651 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T03:10:43.651 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-21T03:10:43.707 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-21T03:10:43.765 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-21T03:10:43.830 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-21T03:10:44.027 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-21T03:10:44.077 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-21T03:10:44.095 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mon-2:19.0.0-3830.g3b748ea0.el9.x86_64 115/170 2024-05-21T03:10:44.118 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mon-2:19.0.0-3830.g3b748ea0.el9.x86_64 115/170 2024-05-21T03:10:44.118 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T03:10:44.118 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-21T03:10:44.118 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T03:10:44.118 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T03:10:44.118 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-21T03:10:44.159 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-21T03:10:44.242 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-21T03:10:44.249 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-21T03:10:44.290 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-21T03:10:44.345 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-21T03:10:44.410 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-21T03:10:44.453 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-21T03:10:44.484 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-21T03:10:44.511 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-21T03:10:44.528 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-21T03:10:44.579 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-21T03:10:44.656 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-21T03:10:44.744 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-21T03:10:44.886 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-21T03:10:44.922 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-21T03:10:44.949 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: cephadm-2:19.0.0-3830.g3b748ea0.el9.noarch 127/170 2024-05-21T03:10:44.962 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-21T03:10:44.985 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephadm-2:19.0.0-3830.g3b748ea0.el9.noarch 127/170 2024-05-21T03:10:44.998 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-21T03:10:45.018 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3830.g3b748ea0.e 128/170 2024-05-21T03:10:45.091 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3830.g3b748ea0. 129/170 2024-05-21T03:10:45.116 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3830.g3b748ea0.el9.noa 130/170 2024-05-21T03:10:45.418 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: cephadm-2:19.0.0-3830.g3b748ea0.el9.noarch 127/170 2024-05-21T03:10:45.457 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephadm-2:19.0.0-3830.g3b748ea0.el9.noarch 127/170 2024-05-21T03:10:45.496 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3830.g3b748ea0.e 128/170 2024-05-21T03:10:45.574 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3830.g3b748ea0. 129/170 2024-05-21T03:10:45.604 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3830.g3b748ea0.el9.noa 130/170 2024-05-21T03:10:46.226 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3830.g3b748ea0.el9.noa 130/170 2024-05-21T03:10:46.268 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3830.g3b748ea0.el9.n 131/170 2024-05-21T03:10:46.643 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3830.g3b748ea0.el9.noa 130/170 2024-05-21T03:10:46.690 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3830.g3b748ea0.el9.n 131/170 2024-05-21T03:10:47.286 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3830.g3b748ea0.el9.n 131/170 2024-05-21T03:10:47.318 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3830.g3b7 132/170 2024-05-21T03:10:47.438 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3830.g3b7 132/170 2024-05-21T03:10:47.621 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3830.g3b748ea0.el 133/170 2024-05-21T03:10:47.645 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-2:19.0.0-3830.g3b748ea0.el9.x86_64 134/170 2024-05-21T03:10:47.656 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3830.g3b748ea0.el9.n 131/170 2024-05-21T03:10:47.670 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3830.g3b748ea0.el9.x86_64 134/170 2024-05-21T03:10:47.670 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T03:10:47.671 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-21T03:10:47.671 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T03:10:47.671 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T03:10:47.671 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-21T03:10:47.689 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3830.g3b7 132/170 2024-05-21T03:10:47.721 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-rook-2:19.0.0-3830.g3b748ea0.el9.noarch 135/170 2024-05-21T03:10:47.732 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3830.g3b748ea0.el9.noarch 135/170 2024-05-21T03:10:47.800 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3830.g3b7 132/170 2024-05-21T03:10:47.834 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-2:19.0.0-3830.g3b748ea0.el9.x86_64 136/170 2024-05-21T03:10:47.887 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 137/170 2024-05-21T03:10:47.967 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3830.g3b748ea0.el 133/170 2024-05-21T03:10:48.000 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-2:19.0.0-3830.g3b748ea0.el9.x86_64 134/170 2024-05-21T03:10:48.023 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3830.g3b748ea0.el9.x86_64 134/170 2024-05-21T03:10:48.024 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T03:10:48.024 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-21T03:10:48.024 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T03:10:48.024 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T03:10:48.024 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-21T03:10:48.068 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-rook-2:19.0.0-3830.g3b748ea0.el9.noarch 135/170 2024-05-21T03:10:48.079 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3830.g3b748ea0.el9.noarch 135/170 2024-05-21T03:10:48.176 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-2:19.0.0-3830.g3b748ea0.el9.x86_64 136/170 2024-05-21T03:10:48.224 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 137/170 2024-05-21T03:10:48.870 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-21T03:10:48.899 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-radosgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 139/170 2024-05-21T03:10:48.923 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 139/170 2024-05-21T03:10:48.923 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T03:10:48.923 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-21T03:10:48.923 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T03:10:48.923 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T03:10:48.924 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-21T03:10:48.966 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3830.g3b748 140/170 2024-05-21T03:10:48.990 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3830.g3b748 140/170 2024-05-21T03:10:48.990 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T03:10:48.990 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-21T03:10:48.990 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-21T03:10:49.050 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephfs-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 141/170 2024-05-21T03:10:49.074 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 141/170 2024-05-21T03:10:49.074 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T03:10:49.075 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-21T03:10:49.075 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-21T03:10:49.075 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-21T03:10:49.075 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-21T03:10:49.156 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-21T03:10:49.187 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-radosgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 139/170 2024-05-21T03:10:49.209 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 139/170 2024-05-21T03:10:49.210 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T03:10:49.210 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-21T03:10:49.210 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T03:10:49.210 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T03:10:49.210 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-21T03:10:49.254 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3830.g3b748 140/170 2024-05-21T03:10:49.276 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3830.g3b748 140/170 2024-05-21T03:10:49.277 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T03:10:49.277 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-21T03:10:49.277 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-21T03:10:49.330 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephfs-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 141/170 2024-05-21T03:10:49.354 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 141/170 2024-05-21T03:10:49.354 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T03:10:49.354 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-21T03:10:49.354 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-21T03:10:49.354 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-21T03:10:49.354 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-21T03:10:49.392 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 142/170 2024-05-21T03:10:49.416 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 142/170 2024-05-21T03:10:49.416 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T03:10:49.417 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-21T03:10:49.417 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T03:10:49.417 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T03:10:49.417 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-21T03:10:49.664 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 142/170 2024-05-21T03:10:49.686 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 142/170 2024-05-21T03:10:49.686 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T03:10:49.687 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-21T03:10:49.687 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T03:10:49.687 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T03:10:49.687 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-21T03:10:49.958 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3830.g3b748ea0.el9.x86_64 111/170 2024-05-21T03:10:49.958 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /sys 2024-05-21T03:10:49.958 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /proc 2024-05-21T03:10:49.958 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /mnt 2024-05-21T03:10:49.958 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /var/tmp 2024-05-21T03:10:49.958 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /home 2024-05-21T03:10:49.958 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /root 2024-05-21T03:10:49.958 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /tmp 2024-05-21T03:10:49.958 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:10:50.135 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-volume-2:19.0.0-3830.g3b748ea0.el9.noarch 112/170 2024-05-21T03:10:50.151 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-volume-2:19.0.0-3830.g3b748ea0.el9.noarch 112/170 2024-05-21T03:10:50.152 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T03:10:50.152 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-21T03:10:50.152 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:10:51.696 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-osd-2:19.0.0-3830.g3b748ea0.el9.x86_64 113/170 2024-05-21T03:10:51.725 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-osd-2:19.0.0-3830.g3b748ea0.el9.x86_64 113/170 2024-05-21T03:10:51.725 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T03:10:51.725 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-21T03:10:51.725 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T03:10:51.725 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T03:10:51.725 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:10:51.963 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mds-2:19.0.0-3830.g3b748ea0.el9.x86_64 114/170 2024-05-21T03:10:51.988 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mds-2:19.0.0-3830.g3b748ea0.el9.x86_64 114/170 2024-05-21T03:10:51.988 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T03:10:51.988 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-21T03:10:51.988 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T03:10:51.988 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T03:10:51.989 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:10:52.456 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mon-2:19.0.0-3830.g3b748ea0.el9.x86_64 115/170 2024-05-21T03:10:52.480 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mon-2:19.0.0-3830.g3b748ea0.el9.x86_64 115/170 2024-05-21T03:10:52.481 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T03:10:52.481 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-21T03:10:52.481 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T03:10:52.481 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T03:10:52.481 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:10:52.586 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-21T03:10:52.644 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-21T03:10:52.708 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-21T03:10:52.910 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-21T03:10:52.957 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-21T03:10:53.038 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-21T03:10:53.127 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-21T03:10:53.285 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-21T03:10:53.323 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-21T03:10:53.356 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-21T03:10:53.383 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-21T03:10:53.838 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephadm-2:19.0.0-3830.g3b748ea0.el9.noarch 127/170 2024-05-21T03:10:53.865 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephadm-2:19.0.0-3830.g3b748ea0.el9.noarch 127/170 2024-05-21T03:10:53.897 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3830.g3b748ea0.e 128/170 2024-05-21T03:10:53.960 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3830.g3b748ea0. 129/170 2024-05-21T03:10:53.988 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3830.g3b748ea0.el9.noa 130/170 2024-05-21T03:10:54.272 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-test-2:19.0.0-3830.g3b748ea0.el9.x86_64 143/170 2024-05-21T03:10:54.301 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-test-2:19.0.0-3830.g3b748ea0.el9.x86_64 143/170 2024-05-21T03:10:54.314 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 144/170 2024-05-21T03:10:54.343 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 144/170 2024-05-21T03:10:54.354 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-nbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 145/170 2024-05-21T03:10:54.383 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-nbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 145/170 2024-05-21T03:10:54.386 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-21T03:10:54.412 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-21T03:10:54.416 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-21T03:10:54.443 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-21T03:10:54.617 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephfs-shell-2:19.0.0-3830.g3b748ea0.el9.x86_64 148/170 2024-05-21T03:10:54.625 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephfs-shell-2:19.0.0-3830.g3b748ea0.el9.x86_64 148/170 2024-05-21T03:10:54.667 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-21T03:10:54.672 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-21T03:10:54.705 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephfs-devel-2:19.0.0-3830.g3b748ea0.el9.x86_ 150/170 2024-05-21T03:10:54.708 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephfs-devel-2:19.0.0-3830.g3b748ea0.el9.x86_ 150/170 2024-05-21T03:10:54.748 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-21T03:10:54.771 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-21T03:10:54.778 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-21T03:10:54.801 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-21T03:10:54.816 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-21T03:10:54.847 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-21T03:10:55.088 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3830.g3b748ea0.el9.noa 130/170 2024-05-21T03:10:55.122 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3830.g3b748ea0.el9.n 131/170 2024-05-21T03:10:55.387 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T03:10:55.406 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T03:10:55.430 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T03:10:55.451 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T03:10:55.454 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T03:10:55.473 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T03:10:56.139 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3830.g3b748ea0.el9.n 131/170 2024-05-21T03:10:56.165 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3830.g3b7 132/170 2024-05-21T03:10:56.284 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3830.g3b7 132/170 2024-05-21T03:10:56.310 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephfs-top-2:19.0.0-3830.g3b748ea0.el9.noarch 155/170 2024-05-21T03:10:56.366 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephfs-top-2:19.0.0-3830.g3b748ea0.el9.noarch 155/170 2024-05-21T03:10:56.401 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-21T03:10:56.442 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-21T03:10:56.458 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3830.g3b748ea0.el 133/170 2024-05-21T03:10:56.460 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-21T03:10:56.492 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-2:19.0.0-3830.g3b748ea0.el9.x86_64 134/170 2024-05-21T03:10:56.502 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-21T03:10:56.517 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3830.g3b748ea0.el9.x86_64 134/170 2024-05-21T03:10:56.517 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T03:10:56.517 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-21T03:10:56.517 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T03:10:56.517 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T03:10:56.518 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:10:56.523 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-21T03:10:56.567 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-rook-2:19.0.0-3830.g3b748ea0.el9.noarch 135/170 2024-05-21T03:10:56.571 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-21T03:10:56.579 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3830.g3b748ea0.el9.noarch 135/170 2024-05-21T03:10:56.595 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-21T03:10:56.596 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-21T03:10:56.621 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-21T03:10:56.629 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-21T03:10:56.648 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-21T03:10:56.648 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-21T03:10:56.654 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-21T03:10:56.680 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-21T03:10:56.687 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-2:19.0.0-3830.g3b748ea0.el9.x86_64 136/170 2024-05-21T03:10:56.706 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-21T03:10:56.706 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-21T03:10:56.732 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-21T03:10:56.749 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 137/170 2024-05-21T03:10:56.766 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-21T03:10:56.766 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-21T03:10:56.783 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-21T03:10:56.816 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-21T03:10:56.816 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-21T03:10:56.850 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-21T03:10:56.875 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-21T03:10:56.880 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T03:10:56.880 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T03:10:56.883 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-21T03:10:56.908 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-21T03:10:56.915 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T03:10:56.915 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T03:10:57.060 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T03:10:57.061 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-21T03:10:57.102 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T03:10:57.102 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-21T03:10:57.136 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-21T03:10:57.161 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-21T03:10:57.186 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-21T03:10:57.211 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-21T03:10:57.225 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-21T03:10:57.277 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-21T03:10:57.737 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-21T03:10:57.770 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-radosgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 139/170 2024-05-21T03:10:57.795 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 139/170 2024-05-21T03:10:57.795 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T03:10:57.795 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-21T03:10:57.795 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T03:10:57.795 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T03:10:57.795 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:10:57.837 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3830.g3b748 140/170 2024-05-21T03:10:57.861 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3830.g3b748 140/170 2024-05-21T03:10:57.862 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T03:10:57.862 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-21T03:10:57.862 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:10:57.913 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 141/170 2024-05-21T03:10:57.937 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 141/170 2024-05-21T03:10:57.937 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T03:10:57.937 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-21T03:10:57.937 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-21T03:10:57.937 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-21T03:10:57.937 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:10:58.255 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 142/170 2024-05-21T03:10:58.279 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 142/170 2024-05-21T03:10:58.280 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T03:10:58.280 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-21T03:10:58.280 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T03:10:58.280 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T03:10:58.280 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:11:02.784 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-21T03:11:02.785 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-2:19.0.0-3830.g3b748ea0.el9.x86_64 1/170 2024-05-21T03:11:02.785 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-base-2:19.0.0-3830.g3b748ea0.el9.x86_64 2/170 2024-05-21T03:11:02.785 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 3/170 2024-05-21T03:11:02.785 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 4/170 2024-05-21T03:11:02.785 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3830.g3b748 5/170 2024-05-21T03:11:02.785 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mds-2:19.0.0-3830.g3b748ea0.el9.x86_64 6/170 2024-05-21T03:11:02.785 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-2:19.0.0-3830.g3b748ea0.el9.x86_64 7/170 2024-05-21T03:11:02.785 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mon-2:19.0.0-3830.g3b748ea0.el9.x86_64 8/170 2024-05-21T03:11:02.785 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-osd-2:19.0.0-3830.g3b748ea0.el9.x86_64 9/170 2024-05-21T03:11:02.785 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-radosgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 10/170 2024-05-21T03:11:02.786 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-selinux-2:19.0.0-3830.g3b748ea0.el9.x86_64 11/170 2024-05-21T03:11:02.786 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-test-2:19.0.0-3830.g3b748ea0.el9.x86_64 12/170 2024-05-21T03:11:02.786 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephfs-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 13/170 2024-05-21T03:11:02.786 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephfs-shell-2:19.0.0-3830.g3b748ea0.el9.x86_64 14/170 2024-05-21T03:11:02.786 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephfs-devel-2:19.0.0-3830.g3b748ea0.el9.x86_ 15/170 2024-05-21T03:11:02.786 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephfs2-2:19.0.0-3830.g3b748ea0.el9.x86_64 16/170 2024-05-21T03:11:02.786 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephsqlite-2:19.0.0-3830.g3b748ea0.el9.x86_64 17/170 2024-05-21T03:11:02.786 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados-devel-2:19.0.0-3830.g3b748ea0.el9.x86_6 18/170 2024-05-21T03:11:02.786 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libradosstriper1-2:19.0.0-3830.g3b748ea0.el9.x86 19/170 2024-05-21T03:11:02.786 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librgw2-2:19.0.0-3830.g3b748ea0.el9.x86_64 20/170 2024-05-21T03:11:02.787 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3830.g3b748ea0.el 21/170 2024-05-21T03:11:02.787 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ceph-common-2:19.0.0-3830.g3b748ea0.el9. 22/170 2024-05-21T03:11:02.787 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cephfs-2:19.0.0-3830.g3b748ea0.el9.x86_6 23/170 2024-05-21T03:11:02.787 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rados-2:19.0.0-3830.g3b748ea0.el9.x86_64 24/170 2024-05-21T03:11:02.787 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 25/170 2024-05-21T03:11:02.787 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 26/170 2024-05-21T03:11:02.787 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 27/170 2024-05-21T03:11:02.787 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 28/170 2024-05-21T03:11:02.787 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-nbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 29/170 2024-05-21T03:11:02.787 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3830.g3b748ea0. 30/170 2024-05-21T03:11:02.787 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3830.g3b748ea0.el9.noa 31/170 2024-05-21T03:11:02.787 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3830.g3b748ea0.el9.n 32/170 2024-05-21T03:11:02.787 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3830.g3b7 33/170 2024-05-21T03:11:02.788 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3830.g3b748ea0.el 34/170 2024-05-21T03:11:02.788 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3830.g3b748ea0.el9.noarch 35/170 2024-05-21T03:11:02.788 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3830.g3b748ea0.e 36/170 2024-05-21T03:11:02.788 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-volume-2:19.0.0-3830.g3b748ea0.el9.noarch 37/170 2024-05-21T03:11:02.788 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephadm-2:19.0.0-3830.g3b748ea0.el9.noarch 38/170 2024-05-21T03:11:02.788 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephfs-top-2:19.0.0-3830.g3b748ea0.el9.noarch 39/170 2024-05-21T03:11:02.788 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-21T03:11:02.788 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-21T03:11:02.788 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-21T03:11:02.788 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-21T03:11:02.788 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-21T03:11:02.788 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-21T03:11:02.788 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-21T03:11:02.789 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-21T03:11:02.789 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-21T03:11:02.789 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-21T03:11:02.789 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-21T03:11:02.789 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-21T03:11:02.789 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-21T03:11:02.789 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-21T03:11:02.789 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-21T03:11:02.789 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-21T03:11:02.789 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-21T03:11:02.789 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-21T03:11:02.790 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-21T03:11:02.790 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-21T03:11:02.790 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-21T03:11:02.790 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-21T03:11:02.790 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-21T03:11:02.790 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-21T03:11:02.790 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-21T03:11:02.790 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-21T03:11:02.790 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-21T03:11:02.791 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-21T03:11:02.791 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-21T03:11:02.791 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-21T03:11:02.791 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-21T03:11:02.791 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-21T03:11:02.791 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-21T03:11:02.791 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-21T03:11:02.791 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-21T03:11:02.791 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-21T03:11:02.791 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-21T03:11:02.791 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-21T03:11:02.791 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-21T03:11:02.791 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-21T03:11:02.791 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-21T03:11:02.792 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-21T03:11:02.792 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-21T03:11:02.792 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-21T03:11:02.792 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-21T03:11:02.792 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-21T03:11:02.792 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-21T03:11:02.792 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-21T03:11:02.792 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-21T03:11:02.792 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-21T03:11:02.792 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-21T03:11:02.792 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-21T03:11:02.792 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-21T03:11:02.792 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-21T03:11:02.793 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-21T03:11:02.793 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-21T03:11:02.793 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-21T03:11:02.793 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-21T03:11:02.793 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-21T03:11:02.793 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-21T03:11:02.793 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-21T03:11:02.793 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-21T03:11:02.793 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-21T03:11:02.793 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-21T03:11:02.793 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-21T03:11:02.793 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-21T03:11:02.793 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-21T03:11:02.794 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-21T03:11:02.794 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-21T03:11:02.794 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-21T03:11:02.794 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-21T03:11:02.794 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-21T03:11:02.794 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-21T03:11:02.794 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-21T03:11:02.794 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-21T03:11:02.794 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-21T03:11:02.794 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-21T03:11:02.794 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-21T03:11:02.794 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-21T03:11:02.794 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-21T03:11:02.795 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-21T03:11:02.795 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-21T03:11:02.795 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-21T03:11:02.795 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-21T03:11:02.795 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-21T03:11:02.795 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-21T03:11:02.795 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-21T03:11:02.795 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-21T03:11:02.795 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-21T03:11:02.795 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-21T03:11:02.795 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-21T03:11:02.796 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-21T03:11:02.796 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-21T03:11:02.796 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-21T03:11:02.796 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-21T03:11:02.796 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-21T03:11:02.796 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-21T03:11:02.796 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-21T03:11:02.796 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-21T03:11:02.796 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-21T03:11:02.796 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-21T03:11:02.796 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-21T03:11:02.796 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-21T03:11:02.796 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-21T03:11:02.797 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-21T03:11:02.797 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-21T03:11:02.797 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-21T03:11:02.797 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-21T03:11:02.797 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-21T03:11:02.797 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados2-2:19.0.0-3830.g3b748ea0.el9.x86_64 149/170 2024-05-21T03:11:02.797 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-21T03:11:02.797 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librbd1-2:19.0.0-3830.g3b748ea0.el9.x86_64 151/170 2024-05-21T03:11:02.797 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-21T03:11:02.797 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-21T03:11:02.797 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-21T03:11:02.797 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-21T03:11:02.797 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-21T03:11:02.798 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-21T03:11:02.798 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-21T03:11:02.798 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-21T03:11:02.798 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-21T03:11:02.798 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-21T03:11:02.798 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-21T03:11:02.798 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-21T03:11:02.798 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-21T03:11:02.798 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-21T03:11:02.798 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-21T03:11:02.798 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-21T03:11:02.798 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-21T03:11:02.798 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-21T03:11:02.966 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-21T03:11:02.967 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-2:19.0.0-3830.g3b748ea0.el9.x86_64 1/170 2024-05-21T03:11:02.967 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-base-2:19.0.0-3830.g3b748ea0.el9.x86_64 2/170 2024-05-21T03:11:02.967 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 3/170 2024-05-21T03:11:02.967 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 4/170 2024-05-21T03:11:02.967 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3830.g3b748 5/170 2024-05-21T03:11:02.967 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mds-2:19.0.0-3830.g3b748ea0.el9.x86_64 6/170 2024-05-21T03:11:02.967 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-2:19.0.0-3830.g3b748ea0.el9.x86_64 7/170 2024-05-21T03:11:02.967 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mon-2:19.0.0-3830.g3b748ea0.el9.x86_64 8/170 2024-05-21T03:11:02.967 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-osd-2:19.0.0-3830.g3b748ea0.el9.x86_64 9/170 2024-05-21T03:11:02.968 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-radosgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 10/170 2024-05-21T03:11:02.968 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-selinux-2:19.0.0-3830.g3b748ea0.el9.x86_64 11/170 2024-05-21T03:11:02.968 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-test-2:19.0.0-3830.g3b748ea0.el9.x86_64 12/170 2024-05-21T03:11:02.968 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephfs-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 13/170 2024-05-21T03:11:02.968 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephfs-shell-2:19.0.0-3830.g3b748ea0.el9.x86_64 14/170 2024-05-21T03:11:02.968 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephfs-devel-2:19.0.0-3830.g3b748ea0.el9.x86_ 15/170 2024-05-21T03:11:02.968 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephfs2-2:19.0.0-3830.g3b748ea0.el9.x86_64 16/170 2024-05-21T03:11:02.968 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephsqlite-2:19.0.0-3830.g3b748ea0.el9.x86_64 17/170 2024-05-21T03:11:02.968 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados-devel-2:19.0.0-3830.g3b748ea0.el9.x86_6 18/170 2024-05-21T03:11:02.968 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libradosstriper1-2:19.0.0-3830.g3b748ea0.el9.x86 19/170 2024-05-21T03:11:02.968 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librgw2-2:19.0.0-3830.g3b748ea0.el9.x86_64 20/170 2024-05-21T03:11:02.968 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3830.g3b748ea0.el 21/170 2024-05-21T03:11:02.969 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ceph-common-2:19.0.0-3830.g3b748ea0.el9. 22/170 2024-05-21T03:11:02.969 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cephfs-2:19.0.0-3830.g3b748ea0.el9.x86_6 23/170 2024-05-21T03:11:02.969 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rados-2:19.0.0-3830.g3b748ea0.el9.x86_64 24/170 2024-05-21T03:11:02.969 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 25/170 2024-05-21T03:11:02.969 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 26/170 2024-05-21T03:11:02.969 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 27/170 2024-05-21T03:11:02.969 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 28/170 2024-05-21T03:11:02.969 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-nbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 29/170 2024-05-21T03:11:02.969 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3830.g3b748ea0. 30/170 2024-05-21T03:11:02.969 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3830.g3b748ea0.el9.noa 31/170 2024-05-21T03:11:02.969 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3830.g3b748ea0.el9.n 32/170 2024-05-21T03:11:02.969 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3830.g3b7 33/170 2024-05-21T03:11:02.969 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3830.g3b748ea0.el 34/170 2024-05-21T03:11:02.970 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3830.g3b748ea0.el9.noarch 35/170 2024-05-21T03:11:02.970 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3830.g3b748ea0.e 36/170 2024-05-21T03:11:02.970 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-volume-2:19.0.0-3830.g3b748ea0.el9.noarch 37/170 2024-05-21T03:11:02.970 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephadm-2:19.0.0-3830.g3b748ea0.el9.noarch 38/170 2024-05-21T03:11:02.970 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephfs-top-2:19.0.0-3830.g3b748ea0.el9.noarch 39/170 2024-05-21T03:11:02.970 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-21T03:11:02.970 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-21T03:11:02.970 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-21T03:11:02.970 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-21T03:11:02.970 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-21T03:11:02.971 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-21T03:11:02.971 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-21T03:11:02.971 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-21T03:11:02.971 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-21T03:11:02.971 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-21T03:11:02.971 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-21T03:11:02.971 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-21T03:11:02.971 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-21T03:11:02.971 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-21T03:11:02.971 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-21T03:11:02.971 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-21T03:11:02.971 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-21T03:11:02.971 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-21T03:11:02.972 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-21T03:11:02.972 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-21T03:11:02.972 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-21T03:11:02.972 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-21T03:11:02.972 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-21T03:11:02.972 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-21T03:11:02.972 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-21T03:11:02.972 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-21T03:11:02.972 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-21T03:11:02.972 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-21T03:11:02.972 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-21T03:11:02.972 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-21T03:11:02.972 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-21T03:11:02.972 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-21T03:11:02.972 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-21T03:11:02.973 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-21T03:11:02.973 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-21T03:11:02.973 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-21T03:11:02.973 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-21T03:11:02.973 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-21T03:11:02.973 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-21T03:11:02.973 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-21T03:11:02.973 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-21T03:11:02.973 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-21T03:11:02.973 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-21T03:11:02.973 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-21T03:11:02.973 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-21T03:11:02.973 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-21T03:11:02.974 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-21T03:11:02.974 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-21T03:11:02.974 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-21T03:11:02.974 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-21T03:11:02.974 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-21T03:11:02.974 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-21T03:11:02.974 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-21T03:11:02.974 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-21T03:11:02.974 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-21T03:11:02.974 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-21T03:11:02.974 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-21T03:11:02.974 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-21T03:11:02.975 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-21T03:11:02.975 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-21T03:11:02.975 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-21T03:11:02.975 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-21T03:11:02.975 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-21T03:11:02.975 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-21T03:11:02.975 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-21T03:11:02.975 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-21T03:11:02.975 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-21T03:11:02.975 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-21T03:11:02.975 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-21T03:11:02.975 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-21T03:11:02.975 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-21T03:11:02.975 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-21T03:11:02.975 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-21T03:11:02.976 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-21T03:11:02.976 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-21T03:11:02.976 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-21T03:11:02.976 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-21T03:11:02.976 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-21T03:11:02.976 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-21T03:11:02.976 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-21T03:11:02.976 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-21T03:11:02.976 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-21T03:11:02.976 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-21T03:11:02.976 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-21T03:11:02.976 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-21T03:11:02.976 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-21T03:11:02.977 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-21T03:11:02.977 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-21T03:11:02.977 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-21T03:11:02.977 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-21T03:11:02.977 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-21T03:11:02.977 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-21T03:11:02.977 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-21T03:11:02.977 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-21T03:11:02.977 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-21T03:11:02.977 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-21T03:11:02.977 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-21T03:11:02.977 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-21T03:11:02.978 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-21T03:11:02.978 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-21T03:11:02.978 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-21T03:11:02.978 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-21T03:11:02.978 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-21T03:11:02.978 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-21T03:11:02.978 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-21T03:11:02.978 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-21T03:11:02.978 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-21T03:11:02.978 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-21T03:11:02.978 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-21T03:11:02.978 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados2-2:19.0.0-3830.g3b748ea0.el9.x86_64 149/170 2024-05-21T03:11:02.979 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-21T03:11:02.979 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librbd1-2:19.0.0-3830.g3b748ea0.el9.x86_64 151/170 2024-05-21T03:11:02.979 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-21T03:11:02.979 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-21T03:11:02.979 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-21T03:11:02.979 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-21T03:11:02.979 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-21T03:11:02.979 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-21T03:11:02.979 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-21T03:11:02.979 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-21T03:11:02.979 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-21T03:11:02.980 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-21T03:11:02.980 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-21T03:11:02.980 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-21T03:11:02.980 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-21T03:11:02.980 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-21T03:11:02.980 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-21T03:11:02.980 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-21T03:11:02.980 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-21T03:11:02.980 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-21T03:11:03.133 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-test-2:19.0.0-3830.g3b748ea0.el9.x86_64 143/170 2024-05-21T03:11:03.176 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 144/170 2024-05-21T03:11:03.217 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-nbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 145/170 2024-05-21T03:11:03.248 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-21T03:11:03.275 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-21T03:11:03.482 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-shell-2:19.0.0-3830.g3b748ea0.el9.x86_64 148/170 2024-05-21T03:11:03.529 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-21T03:11:03.554 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs-devel-2:19.0.0-3830.g3b748ea0.el9.x86_ 150/170 2024-05-21T03:11:03.585 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-21T03:11:03.615 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-21T03:11:03.662 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-21T03:11:04.290 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T03:11:04.326 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T03:11:04.347 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T03:11:05.259 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-top-2:19.0.0-3830.g3b748ea0.el9.noarch 155/170 2024-05-21T03:11:05.364 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-21T03:11:05.407 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-21T03:11:05.476 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-21T03:11:05.477 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-21T03:11:05.477 INFO:teuthology.orchestra.run.smithi165.stdout:Upgraded: 2024-05-21T03:11:05.477 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-21T03:11:05.477 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-21T03:11:05.477 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-21T03:11:05.477 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-21T03:11:05.477 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-21T03:11:05.477 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-21T03:11:05.477 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-21T03:11:05.477 INFO:teuthology.orchestra.run.smithi165.stdout: librados2-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.477 INFO:teuthology.orchestra.run.smithi165.stdout: librbd1-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.477 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-21T03:11:05.478 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-21T03:11:05.478 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-05-21T03:11:05.478 INFO:teuthology.orchestra.run.smithi165.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-21T03:11:05.478 INFO:teuthology.orchestra.run.smithi165.stdout: automake-1.16.2-8.el9.noarch 2024-05-21T03:11:05.478 INFO:teuthology.orchestra.run.smithi165.stdout: bison-3.7.4-5.el9.x86_64 2024-05-21T03:11:05.478 INFO:teuthology.orchestra.run.smithi165.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-21T03:11:05.478 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.478 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-base-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.478 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.478 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.478 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-grafana-dashboards-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T03:11:05.478 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-immutable-object-cache-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.479 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mds-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.479 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.479 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-cephadm-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T03:11:05.479 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-dashboard-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T03:11:05.479 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T03:11:05.479 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-modules-core-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T03:11:05.479 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-rook-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T03:11:05.479 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mon-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.479 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-osd-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.479 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-prometheus-alerts-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T03:11:05.479 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-radosgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.479 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-selinux-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.479 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-test-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.480 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-volume-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T03:11:05.480 INFO:teuthology.orchestra.run.smithi165.stdout: cephadm-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T03:11:05.480 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.480 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-shell-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.480 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-top-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T03:11:05.480 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-21T03:11:05.480 INFO:teuthology.orchestra.run.smithi165.stdout: flex-2.6.4-9.el9.x86_64 2024-05-21T03:11:05.480 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-21T03:11:05.480 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-21T03:11:05.480 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-21T03:11:05.480 INFO:teuthology.orchestra.run.smithi165.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-21T03:11:05.480 INFO:teuthology.orchestra.run.smithi165.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-21T03:11:05.480 INFO:teuthology.orchestra.run.smithi165.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-21T03:11:05.481 INFO:teuthology.orchestra.run.smithi165.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-21T03:11:05.481 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-21T03:11:05.481 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-21T03:11:05.481 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs-devel-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.481 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs2-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.481 INFO:teuthology.orchestra.run.smithi165.stdout: libcephsqlite-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.481 INFO:teuthology.orchestra.run.smithi165.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-21T03:11:05.481 INFO:teuthology.orchestra.run.smithi165.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-21T03:11:05.481 INFO:teuthology.orchestra.run.smithi165.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-21T03:11:05.481 INFO:teuthology.orchestra.run.smithi165.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-21T03:11:05.481 INFO:teuthology.orchestra.run.smithi165.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-21T03:11:05.481 INFO:teuthology.orchestra.run.smithi165.stdout: librados-devel-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.482 INFO:teuthology.orchestra.run.smithi165.stdout: libradosstriper1-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.482 INFO:teuthology.orchestra.run.smithi165.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-21T03:11:05.482 INFO:teuthology.orchestra.run.smithi165.stdout: librgw2-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.482 INFO:teuthology.orchestra.run.smithi165.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-21T03:11:05.482 INFO:teuthology.orchestra.run.smithi165.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-21T03:11:05.482 INFO:teuthology.orchestra.run.smithi165.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-21T03:11:05.482 INFO:teuthology.orchestra.run.smithi165.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-21T03:11:05.482 INFO:teuthology.orchestra.run.smithi165.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-21T03:11:05.482 INFO:teuthology.orchestra.run.smithi165.stdout: lua-5.4.4-4.el9.x86_64 2024-05-21T03:11:05.482 INFO:teuthology.orchestra.run.smithi165.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-21T03:11:05.482 INFO:teuthology.orchestra.run.smithi165.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-21T03:11:05.482 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-21T03:11:05.482 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-21T03:11:05.483 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-21T03:11:05.483 INFO:teuthology.orchestra.run.smithi165.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-21T03:11:05.483 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-13.14-1.el9.x86_64 2024-05-21T03:11:05.483 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-21T03:11:05.483 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-21T03:11:05.483 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-21T03:11:05.483 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-21T03:11:05.483 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-21T03:11:05.483 INFO:teuthology.orchestra.run.smithi165.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-21T03:11:05.483 INFO:teuthology.orchestra.run.smithi165.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-21T03:11:05.483 INFO:teuthology.orchestra.run.smithi165.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-21T03:11:05.483 INFO:teuthology.orchestra.run.smithi165.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-21T03:11:05.483 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-21T03:11:05.484 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-argparse-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.484 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.484 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cephfs-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.484 INFO:teuthology.orchestra.run.smithi165.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-21T03:11:05.484 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-21T03:11:05.484 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-21T03:11:05.484 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-21T03:11:05.484 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-21T03:11:05.484 INFO:teuthology.orchestra.run.smithi165.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-21T03:11:05.484 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-21T03:11:05.484 INFO:teuthology.orchestra.run.smithi165.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-21T03:11:05.484 INFO:teuthology.orchestra.run.smithi165.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-21T03:11:05.484 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-21T03:11:05.484 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-21T03:11:05.485 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-21T03:11:05.485 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-21T03:11:05.485 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-21T03:11:05.485 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-21T03:11:05.485 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-21T03:11:05.485 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-21T03:11:05.485 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-21T03:11:05.485 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-21T03:11:05.485 INFO:teuthology.orchestra.run.smithi165.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-21T03:11:05.485 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-21T03:11:05.485 INFO:teuthology.orchestra.run.smithi165.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-21T03:11:05.485 INFO:teuthology.orchestra.run.smithi165.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-21T03:11:05.485 INFO:teuthology.orchestra.run.smithi165.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-21T03:11:05.485 INFO:teuthology.orchestra.run.smithi165.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-21T03:11:05.486 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-21T03:11:05.486 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-21T03:11:05.486 INFO:teuthology.orchestra.run.smithi165.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-21T03:11:05.486 INFO:teuthology.orchestra.run.smithi165.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-21T03:11:05.486 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-21T03:11:05.486 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ply-3.11-14.el9.noarch 2024-05-21T03:11:05.486 INFO:teuthology.orchestra.run.smithi165.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-21T03:11:05.486 INFO:teuthology.orchestra.run.smithi165.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-21T03:11:05.486 INFO:teuthology.orchestra.run.smithi165.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-21T03:11:05.486 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-21T03:11:05.486 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-21T03:11:05.486 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-21T03:11:05.486 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-21T03:11:05.487 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-21T03:11:05.487 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-21T03:11:05.487 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rados-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.487 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.487 INFO:teuthology.orchestra.run.smithi165.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-21T03:11:05.487 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-21T03:11:05.487 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-21T03:11:05.487 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.487 INFO:teuthology.orchestra.run.smithi165.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-21T03:11:05.487 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-21T03:11:05.487 INFO:teuthology.orchestra.run.smithi165.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-21T03:11:05.487 INFO:teuthology.orchestra.run.smithi165.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-21T03:11:05.487 INFO:teuthology.orchestra.run.smithi165.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-21T03:11:05.488 INFO:teuthology.orchestra.run.smithi165.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-21T03:11:05.488 INFO:teuthology.orchestra.run.smithi165.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-21T03:11:05.488 INFO:teuthology.orchestra.run.smithi165.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-21T03:11:05.488 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-21T03:11:05.488 INFO:teuthology.orchestra.run.smithi165.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-21T03:11:05.488 INFO:teuthology.orchestra.run.smithi165.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-21T03:11:05.488 INFO:teuthology.orchestra.run.smithi165.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-21T03:11:05.488 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-21T03:11:05.488 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-21T03:11:05.488 INFO:teuthology.orchestra.run.smithi165.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-21T03:11:05.488 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.488 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.488 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-nbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.488 INFO:teuthology.orchestra.run.smithi165.stdout: re2-1:20211101-3.el9.x86_64 2024-05-21T03:11:05.488 INFO:teuthology.orchestra.run.smithi165.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-21T03:11:05.489 INFO:teuthology.orchestra.run.smithi165.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-21T03:11:05.489 INFO:teuthology.orchestra.run.smithi165.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-21T03:11:05.489 INFO:teuthology.orchestra.run.smithi165.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-21T03:11:05.489 INFO:teuthology.orchestra.run.smithi165.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-21T03:11:05.489 INFO:teuthology.orchestra.run.smithi165.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-21T03:11:05.489 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-21T03:11:05.489 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-05-21T03:11:05.510 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-21T03:11:05.550 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-21T03:11:05.576 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-21T03:11:05.601 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-21T03:11:05.627 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-21T03:11:05.627 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-21T03:11:05.715 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-21T03:11:05.745 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-21T03:11:05.745 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-21T03:11:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-21T03:11:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-21T03:11:05.790 INFO:teuthology.orchestra.run.smithi039.stdout:Upgraded: 2024-05-21T03:11:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-21T03:11:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-21T03:11:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-21T03:11:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-21T03:11:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-21T03:11:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-21T03:11:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-21T03:11:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: librados2-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: librbd1-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-21T03:11:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-21T03:11:05.791 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-05-21T03:11:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-21T03:11:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: automake-1.16.2-8.el9.noarch 2024-05-21T03:11:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: bison-3.7.4-5.el9.x86_64 2024-05-21T03:11:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-21T03:11:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-base-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-grafana-dashboards-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T03:11:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-immutable-object-cache-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mds-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-cephadm-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T03:11:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-dashboard-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T03:11:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T03:11:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-modules-core-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T03:11:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-rook-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T03:11:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mon-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-osd-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-prometheus-alerts-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T03:11:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-radosgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-selinux-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-test-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-volume-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T03:11:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: cephadm-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T03:11:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-shell-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-top-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T03:11:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-21T03:11:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: flex-2.6.4-9.el9.x86_64 2024-05-21T03:11:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-21T03:11:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-21T03:11:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-21T03:11:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-21T03:11:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-21T03:11:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-21T03:11:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-21T03:11:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-21T03:11:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-21T03:11:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs-devel-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs2-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: libcephsqlite-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-21T03:11:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-21T03:11:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-21T03:11:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-21T03:11:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-21T03:11:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: librados-devel-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: libradosstriper1-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-21T03:11:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: librgw2-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-21T03:11:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-21T03:11:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-21T03:11:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-21T03:11:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-21T03:11:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: lua-5.4.4-4.el9.x86_64 2024-05-21T03:11:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-21T03:11:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-21T03:11:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-21T03:11:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-21T03:11:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-21T03:11:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-21T03:11:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-13.14-1.el9.x86_64 2024-05-21T03:11:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-21T03:11:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-21T03:11:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-21T03:11:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-21T03:11:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-21T03:11:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-21T03:11:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-21T03:11:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-21T03:11:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-21T03:11:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-21T03:11:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-argparse-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cephfs-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-21T03:11:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-21T03:11:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-21T03:11:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-21T03:11:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-21T03:11:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-21T03:11:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-21T03:11:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-21T03:11:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-21T03:11:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-21T03:11:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-21T03:11:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-21T03:11:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-21T03:11:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-21T03:11:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-21T03:11:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-21T03:11:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-21T03:11:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-21T03:11:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-21T03:11:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-21T03:11:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-21T03:11:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-21T03:11:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-21T03:11:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-21T03:11:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-21T03:11:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-21T03:11:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-21T03:11:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-21T03:11:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-21T03:11:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-21T03:11:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ply-3.11-14.el9.noarch 2024-05-21T03:11:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-21T03:11:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-21T03:11:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-21T03:11:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-21T03:11:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-21T03:11:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-21T03:11:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-21T03:11:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-21T03:11:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-21T03:11:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rados-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-21T03:11:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-21T03:11:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-21T03:11:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-21T03:11:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-21T03:11:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-21T03:11:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-21T03:11:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-21T03:11:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-21T03:11:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-21T03:11:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-21T03:11:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-21T03:11:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-21T03:11:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-21T03:11:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-21T03:11:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-21T03:11:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-21T03:11:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-21T03:11:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-nbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: re2-1:20211101-3.el9.x86_64 2024-05-21T03:11:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-21T03:11:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-21T03:11:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-21T03:11:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-21T03:11:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-21T03:11:05.804 INFO:teuthology.orchestra.run.smithi039.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-21T03:11:05.804 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-21T03:11:05.804 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-05-21T03:11:05.821 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-21T03:11:05.846 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-21T03:11:05.852 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T03:11:05.852 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T03:11:05.929 DEBUG:teuthology.parallel:result is None 2024-05-21T03:11:06.040 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T03:11:06.040 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-21T03:11:06.124 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-21T03:11:06.149 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-21T03:11:06.218 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-21T03:11:06.226 DEBUG:teuthology.parallel:result is None 2024-05-21T03:11:11.894 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-21T03:11:11.894 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-2:19.0.0-3830.g3b748ea0.el9.x86_64 1/170 2024-05-21T03:11:11.894 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-base-2:19.0.0-3830.g3b748ea0.el9.x86_64 2/170 2024-05-21T03:11:11.894 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 3/170 2024-05-21T03:11:11.894 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 4/170 2024-05-21T03:11:11.895 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3830.g3b748 5/170 2024-05-21T03:11:11.895 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mds-2:19.0.0-3830.g3b748ea0.el9.x86_64 6/170 2024-05-21T03:11:11.895 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-2:19.0.0-3830.g3b748ea0.el9.x86_64 7/170 2024-05-21T03:11:11.895 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mon-2:19.0.0-3830.g3b748ea0.el9.x86_64 8/170 2024-05-21T03:11:11.895 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-osd-2:19.0.0-3830.g3b748ea0.el9.x86_64 9/170 2024-05-21T03:11:11.895 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-radosgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 10/170 2024-05-21T03:11:11.895 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-selinux-2:19.0.0-3830.g3b748ea0.el9.x86_64 11/170 2024-05-21T03:11:11.895 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-test-2:19.0.0-3830.g3b748ea0.el9.x86_64 12/170 2024-05-21T03:11:11.895 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 13/170 2024-05-21T03:11:11.895 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-shell-2:19.0.0-3830.g3b748ea0.el9.x86_64 14/170 2024-05-21T03:11:11.896 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs-devel-2:19.0.0-3830.g3b748ea0.el9.x86_ 15/170 2024-05-21T03:11:11.896 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs2-2:19.0.0-3830.g3b748ea0.el9.x86_64 16/170 2024-05-21T03:11:11.896 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephsqlite-2:19.0.0-3830.g3b748ea0.el9.x86_64 17/170 2024-05-21T03:11:11.896 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados-devel-2:19.0.0-3830.g3b748ea0.el9.x86_6 18/170 2024-05-21T03:11:11.896 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libradosstriper1-2:19.0.0-3830.g3b748ea0.el9.x86 19/170 2024-05-21T03:11:11.896 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librgw2-2:19.0.0-3830.g3b748ea0.el9.x86_64 20/170 2024-05-21T03:11:11.896 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3830.g3b748ea0.el 21/170 2024-05-21T03:11:11.896 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-common-2:19.0.0-3830.g3b748ea0.el9. 22/170 2024-05-21T03:11:11.896 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cephfs-2:19.0.0-3830.g3b748ea0.el9.x86_6 23/170 2024-05-21T03:11:11.896 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rados-2:19.0.0-3830.g3b748ea0.el9.x86_64 24/170 2024-05-21T03:11:11.896 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 25/170 2024-05-21T03:11:11.896 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 26/170 2024-05-21T03:11:11.896 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 27/170 2024-05-21T03:11:11.897 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 28/170 2024-05-21T03:11:11.897 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-nbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 29/170 2024-05-21T03:11:11.897 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3830.g3b748ea0. 30/170 2024-05-21T03:11:11.897 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3830.g3b748ea0.el9.noa 31/170 2024-05-21T03:11:11.897 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3830.g3b748ea0.el9.n 32/170 2024-05-21T03:11:11.897 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3830.g3b7 33/170 2024-05-21T03:11:11.897 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3830.g3b748ea0.el 34/170 2024-05-21T03:11:11.897 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3830.g3b748ea0.el9.noarch 35/170 2024-05-21T03:11:11.897 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3830.g3b748ea0.e 36/170 2024-05-21T03:11:11.897 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-volume-2:19.0.0-3830.g3b748ea0.el9.noarch 37/170 2024-05-21T03:11:11.897 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephadm-2:19.0.0-3830.g3b748ea0.el9.noarch 38/170 2024-05-21T03:11:11.897 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-top-2:19.0.0-3830.g3b748ea0.el9.noarch 39/170 2024-05-21T03:11:11.897 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-21T03:11:11.898 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-21T03:11:11.898 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-21T03:11:11.898 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-21T03:11:11.898 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-21T03:11:11.898 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-21T03:11:11.898 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-21T03:11:11.898 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-21T03:11:11.898 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-21T03:11:11.899 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-21T03:11:11.899 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-21T03:11:11.899 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-21T03:11:11.899 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-21T03:11:11.899 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-21T03:11:11.899 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-21T03:11:11.899 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-21T03:11:11.899 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-21T03:11:11.899 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-21T03:11:11.899 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-21T03:11:11.899 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-21T03:11:11.899 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-21T03:11:11.899 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-21T03:11:11.900 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-21T03:11:11.900 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-21T03:11:11.900 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-21T03:11:11.900 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-21T03:11:11.900 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-21T03:11:11.900 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-21T03:11:11.900 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-21T03:11:11.900 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-21T03:11:11.900 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-21T03:11:11.900 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-21T03:11:11.900 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-21T03:11:11.900 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-21T03:11:11.900 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-21T03:11:11.900 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-21T03:11:11.901 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-21T03:11:11.901 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-21T03:11:11.901 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-21T03:11:11.901 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-21T03:11:11.901 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-21T03:11:11.901 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-21T03:11:11.901 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-21T03:11:11.901 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-21T03:11:11.902 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-21T03:11:11.902 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-21T03:11:11.902 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-21T03:11:11.902 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-21T03:11:11.902 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-21T03:11:11.902 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-21T03:11:11.902 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-21T03:11:11.902 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-21T03:11:11.902 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-21T03:11:11.902 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-21T03:11:11.902 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-21T03:11:11.902 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-21T03:11:11.902 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-21T03:11:11.903 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-21T03:11:11.903 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-21T03:11:11.903 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-21T03:11:11.903 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-21T03:11:11.903 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-21T03:11:11.903 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-21T03:11:11.903 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-21T03:11:11.903 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-21T03:11:11.903 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-21T03:11:11.903 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-21T03:11:11.903 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-21T03:11:11.903 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-21T03:11:11.903 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-21T03:11:11.904 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-21T03:11:11.904 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-21T03:11:11.904 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-21T03:11:11.904 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-21T03:11:11.904 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-21T03:11:11.904 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-21T03:11:11.904 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-21T03:11:11.904 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-21T03:11:11.904 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-21T03:11:11.904 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-21T03:11:11.904 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-21T03:11:11.904 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-21T03:11:11.904 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-21T03:11:11.905 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-21T03:11:11.905 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-21T03:11:11.905 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-21T03:11:11.905 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-21T03:11:11.905 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-21T03:11:11.905 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-21T03:11:11.905 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-21T03:11:11.905 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-21T03:11:11.905 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-21T03:11:11.905 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-21T03:11:11.905 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-21T03:11:11.905 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-21T03:11:11.905 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-21T03:11:11.906 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-21T03:11:11.906 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-21T03:11:11.906 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-21T03:11:11.906 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-21T03:11:11.906 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-21T03:11:11.906 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-21T03:11:11.906 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-21T03:11:11.906 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-21T03:11:11.906 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-21T03:11:11.906 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-21T03:11:11.906 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-21T03:11:11.906 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-21T03:11:11.907 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-21T03:11:11.907 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:19.0.0-3830.g3b748ea0.el9.x86_64 149/170 2024-05-21T03:11:11.907 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-21T03:11:11.907 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:19.0.0-3830.g3b748ea0.el9.x86_64 151/170 2024-05-21T03:11:11.907 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-21T03:11:11.907 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-21T03:11:11.907 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-21T03:11:11.907 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-21T03:11:11.907 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-21T03:11:11.907 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-21T03:11:11.907 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-21T03:11:11.907 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-21T03:11:11.907 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-21T03:11:11.907 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-21T03:11:11.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-21T03:11:11.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-21T03:11:11.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-21T03:11:11.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-21T03:11:11.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-21T03:11:11.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-21T03:11:11.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-21T03:11:11.908 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-21T03:11:14.801 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-21T03:11:14.801 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:11:14.801 INFO:teuthology.orchestra.run.smithi027.stdout:Upgraded: 2024-05-21T03:11:14.801 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-21T03:11:14.802 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-21T03:11:14.802 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-21T03:11:14.802 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-21T03:11:14.802 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-21T03:11:14.802 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-21T03:11:14.802 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-21T03:11:14.802 INFO:teuthology.orchestra.run.smithi027.stdout: librados2-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.802 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.802 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-21T03:11:14.802 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-21T03:11:14.802 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-05-21T03:11:14.802 INFO:teuthology.orchestra.run.smithi027.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-21T03:11:14.803 INFO:teuthology.orchestra.run.smithi027.stdout: automake-1.16.2-8.el9.noarch 2024-05-21T03:11:14.803 INFO:teuthology.orchestra.run.smithi027.stdout: bison-3.7.4-5.el9.x86_64 2024-05-21T03:11:14.803 INFO:teuthology.orchestra.run.smithi027.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-21T03:11:14.803 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.803 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.803 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.803 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.803 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T03:11:14.803 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.803 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.803 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.803 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T03:11:14.803 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T03:11:14.804 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T03:11:14.804 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T03:11:14.804 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T03:11:14.804 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.804 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.804 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T03:11:14.804 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.804 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.804 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.804 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T03:11:14.804 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T03:11:14.804 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.804 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-shell-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.805 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-top-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T03:11:14.805 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-21T03:11:14.805 INFO:teuthology.orchestra.run.smithi027.stdout: flex-2.6.4-9.el9.x86_64 2024-05-21T03:11:14.805 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-21T03:11:14.805 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-21T03:11:14.805 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-21T03:11:14.805 INFO:teuthology.orchestra.run.smithi027.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-21T03:11:14.805 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-21T03:11:14.805 INFO:teuthology.orchestra.run.smithi027.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-21T03:11:14.805 INFO:teuthology.orchestra.run.smithi027.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-21T03:11:14.805 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-21T03:11:14.805 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-21T03:11:14.805 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.806 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.806 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.806 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-21T03:11:14.806 INFO:teuthology.orchestra.run.smithi027.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-21T03:11:14.806 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-21T03:11:14.806 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-21T03:11:14.806 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-21T03:11:14.806 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.806 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.806 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-21T03:11:14.807 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.807 INFO:teuthology.orchestra.run.smithi027.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-21T03:11:14.807 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-21T03:11:14.807 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-21T03:11:14.807 INFO:teuthology.orchestra.run.smithi027.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-21T03:11:14.807 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-21T03:11:14.807 INFO:teuthology.orchestra.run.smithi027.stdout: lua-5.4.4-4.el9.x86_64 2024-05-21T03:11:14.807 INFO:teuthology.orchestra.run.smithi027.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-21T03:11:14.808 INFO:teuthology.orchestra.run.smithi027.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-21T03:11:14.808 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-21T03:11:14.808 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-21T03:11:14.808 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-21T03:11:14.808 INFO:teuthology.orchestra.run.smithi027.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-21T03:11:14.808 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-13.14-1.el9.x86_64 2024-05-21T03:11:14.808 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-21T03:11:14.808 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-21T03:11:14.808 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-21T03:11:14.808 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-21T03:11:14.809 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-21T03:11:14.809 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-21T03:11:14.809 INFO:teuthology.orchestra.run.smithi027.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-21T03:11:14.809 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-21T03:11:14.809 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-21T03:11:14.809 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-21T03:11:14.809 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.809 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.809 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.810 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-21T03:11:14.810 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-21T03:11:14.810 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-21T03:11:14.810 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-21T03:11:14.810 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-21T03:11:14.810 INFO:teuthology.orchestra.run.smithi027.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-21T03:11:14.810 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-21T03:11:14.810 INFO:teuthology.orchestra.run.smithi027.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-21T03:11:14.810 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-21T03:11:14.810 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-21T03:11:14.810 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-21T03:11:14.811 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-21T03:11:14.811 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-21T03:11:14.811 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-21T03:11:14.811 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-21T03:11:14.811 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-21T03:11:14.811 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-21T03:11:14.811 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-21T03:11:14.811 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-21T03:11:14.811 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-21T03:11:14.811 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-21T03:11:14.811 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-21T03:11:14.811 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-21T03:11:14.812 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-21T03:11:14.812 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-21T03:11:14.812 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-21T03:11:14.812 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-21T03:11:14.812 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-21T03:11:14.812 INFO:teuthology.orchestra.run.smithi027.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-21T03:11:14.812 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-21T03:11:14.812 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ply-3.11-14.el9.noarch 2024-05-21T03:11:14.812 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-21T03:11:14.812 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-21T03:11:14.812 INFO:teuthology.orchestra.run.smithi027.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-21T03:11:14.812 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-21T03:11:14.812 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-21T03:11:14.813 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-21T03:11:14.813 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-21T03:11:14.813 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-21T03:11:14.813 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-21T03:11:14.813 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.813 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.813 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-21T03:11:14.813 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-21T03:11:14.814 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-21T03:11:14.814 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.814 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-21T03:11:14.814 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-21T03:11:14.814 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-21T03:11:14.814 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-21T03:11:14.814 INFO:teuthology.orchestra.run.smithi027.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-21T03:11:14.814 INFO:teuthology.orchestra.run.smithi027.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-21T03:11:14.814 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-21T03:11:14.815 INFO:teuthology.orchestra.run.smithi027.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-21T03:11:14.815 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-21T03:11:14.815 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-21T03:11:14.815 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-21T03:11:14.815 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-21T03:11:14.815 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-21T03:11:14.815 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-21T03:11:14.815 INFO:teuthology.orchestra.run.smithi027.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-21T03:11:14.815 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.815 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.815 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T03:11:14.815 INFO:teuthology.orchestra.run.smithi027.stdout: re2-1:20211101-3.el9.x86_64 2024-05-21T03:11:14.815 INFO:teuthology.orchestra.run.smithi027.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-21T03:11:14.815 INFO:teuthology.orchestra.run.smithi027.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-21T03:11:14.816 INFO:teuthology.orchestra.run.smithi027.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-21T03:11:14.816 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-21T03:11:14.816 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-21T03:11:14.816 INFO:teuthology.orchestra.run.smithi027.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-21T03:11:14.816 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:11:14.816 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-05-21T03:11:15.285 DEBUG:teuthology.parallel:result is None 2024-05-21T03:11:15.286 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3b748ea08891006c1152513d25ab4a03f7c9a995 2024-05-21T03:11:15.433 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-21T03:11:15.458 INFO:teuthology.orchestra.run.smithi027.stdout:19.0.0-3830.g3b748ea0.el9 2024-05-21T03:11:15.459 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3830.g3b748ea0.el9 2024-05-21T03:11:15.459 INFO:teuthology.task.install:The correct ceph version 19.0.0-3830.g3b748ea0 is installed. 2024-05-21T03:11:15.461 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3b748ea08891006c1152513d25ab4a03f7c9a995 2024-05-21T03:11:15.604 DEBUG:teuthology.orchestra.run.smithi039:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-21T03:11:15.630 INFO:teuthology.orchestra.run.smithi039.stdout:19.0.0-3830.g3b748ea0.el9 2024-05-21T03:11:15.630 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3830.g3b748ea0.el9 2024-05-21T03:11:15.630 INFO:teuthology.task.install:The correct ceph version 19.0.0-3830.g3b748ea0 is installed. 2024-05-21T03:11:15.632 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3b748ea08891006c1152513d25ab4a03f7c9a995 2024-05-21T03:11:15.773 DEBUG:teuthology.orchestra.run.smithi165:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-21T03:11:15.798 INFO:teuthology.orchestra.run.smithi165.stdout:19.0.0-3830.g3b748ea0.el9 2024-05-21T03:11:15.798 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3830.g3b748ea0.el9 2024-05-21T03:11:15.799 INFO:teuthology.task.install:The correct ceph version 19.0.0-3830.g3b748ea0 is installed. 2024-05-21T03:11:15.800 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-21T03:11:15.801 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-05-21T03:11:15.801 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-21T03:11:15.833 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-21T03:11:15.833 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-21T03:11:15.865 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-05-21T03:11:15.866 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-21T03:11:15.896 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-21T03:11:15.896 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-05-21T03:11:15.896 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/daemon-helper 2024-05-21T03:11:15.925 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-21T03:11:15.993 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-21T03:11:15.993 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/daemon-helper 2024-05-21T03:11:16.023 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-21T03:11:16.092 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-05-21T03:11:16.093 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/daemon-helper 2024-05-21T03:11:16.121 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-21T03:11:16.204 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-21T03:11:16.204 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-05-21T03:11:16.204 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-21T03:11:16.233 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-21T03:11:16.302 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-21T03:11:16.302 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-21T03:11:16.333 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-21T03:11:16.400 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-05-21T03:11:16.400 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-21T03:11:16.424 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-21T03:11:16.487 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-21T03:11:16.487 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-05-21T03:11:16.488 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/stdin-killer 2024-05-21T03:11:16.519 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-21T03:11:16.589 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-21T03:11:16.589 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/stdin-killer 2024-05-21T03:11:16.614 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-21T03:11:16.678 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-05-21T03:11:16.679 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/stdin-killer 2024-05-21T03:11:16.703 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-21T03:11:16.766 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-21T03:11:16.868 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd heartbeat grace': 60, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'SLOW_OPS', 'slow request'], 'sha1': '3b748ea08891006c1152513d25ab4a03f7c9a995', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-21T03:11:16.869 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 2024-05-21T03:11:16.869 INFO:tasks.cephadm:Cluster fsid is c9ca94be-171f-11ef-bc9a-c7b262605968 2024-05-21T03:11:16.869 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-21T03:11:16.870 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.27', 'mon.b': '172.21.15.39', 'mon.c': '172.21.15.165'} 2024-05-21T03:11:16.870 INFO:tasks.cephadm:First mon is mon.a on smithi027 2024-05-21T03:11:16.870 INFO:tasks.cephadm:First mgr is x 2024-05-21T03:11:16.870 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-21T03:11:16.870 DEBUG:teuthology.orchestra.run.smithi027:> sudo hostname $(hostname -s) 2024-05-21T03:11:16.902 DEBUG:teuthology.orchestra.run.smithi039:> sudo hostname $(hostname -s) 2024-05-21T03:11:16.927 DEBUG:teuthology.orchestra.run.smithi165:> sudo hostname $(hostname -s) 2024-05-21T03:11:16.951 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-21T03:11:16.952 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3b748ea08891006c1152513d25ab4a03f7c9a995 2024-05-21T03:11:17.179 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3b748ea08891006c1152513d25ab4a03f7c9a995', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79571/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3830-g3b748ea0', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3830.g3b748ea0'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-20 22:39:51.349239', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiesce-overdrive-export', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-05-21T03:11:17.323 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-lusov-quiesce-overdrive-export, sha1 3b748ea08891006c1152513d25ab4a03f7c9a995 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3b748ea08891006c1152513d25ab4a03f7c9a995 2024-05-21T03:11:17.324 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/x86_64/flavors/default/cephadm 2024-05-21T03:11:17.325 INFO:tasks.cephadm:Downloading cephadm from url: https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/x86_64/flavors/default/cephadm 2024-05-21T03:11:17.325 DEBUG:teuthology.orchestra.run.smithi027:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-21T03:11:18.251 INFO:teuthology.orchestra.run.smithi027.stdout:-rw-r--r--. 1 ubuntu ubuntu 782836 May 21 03:11 /home/ubuntu/cephtest/cephadm 2024-05-21T03:11:18.251 DEBUG:teuthology.orchestra.run.smithi039:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-21T03:11:19.043 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r--. 1 ubuntu ubuntu 782836 May 21 03:11 /home/ubuntu/cephtest/cephadm 2024-05-21T03:11:19.043 DEBUG:teuthology.orchestra.run.smithi165:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-21T03:11:20.125 INFO:teuthology.orchestra.run.smithi165.stdout:-rw-r--r--. 1 ubuntu ubuntu 782836 May 21 03:11 /home/ubuntu/cephtest/cephadm 2024-05-21T03:11:20.126 DEBUG:teuthology.orchestra.run.smithi027:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-21T03:11:20.148 DEBUG:teuthology.orchestra.run.smithi039:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-21T03:11:20.169 DEBUG:teuthology.orchestra.run.smithi165:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-21T03:11:20.203 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 on all hosts... 2024-05-21T03:11:20.203 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 pull 2024-05-21T03:11:20.206 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 pull 2024-05-21T03:11:20.212 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 pull 2024-05-21T03:11:21.089 INFO:teuthology.orchestra.run.smithi039.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995... 2024-05-21T03:11:21.148 INFO:teuthology.orchestra.run.smithi165.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995... 2024-05-21T03:11:21.160 INFO:teuthology.orchestra.run.smithi027.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995... 2024-05-21T03:11:49.721 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-21T03:11:49.722 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_version": "ceph version 19.0.0-3830-g3b748ea0 (3b748ea08891006c1152513d25ab4a03f7c9a995) squid (dev)", 2024-05-21T03:11:49.722 INFO:teuthology.orchestra.run.smithi039.stdout: "image_id": "d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f", 2024-05-21T03:11:49.722 INFO:teuthology.orchestra.run.smithi039.stdout: "repo_digests": [ 2024-05-21T03:11:49.722 INFO:teuthology.orchestra.run.smithi039.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28" 2024-05-21T03:11:49.722 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-05-21T03:11:49.722 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-21T03:11:53.594 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:11:53.594 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 19.0.0-3830-g3b748ea0 (3b748ea08891006c1152513d25ab4a03f7c9a995) squid (dev)", 2024-05-21T03:11:53.594 INFO:teuthology.orchestra.run.smithi027.stdout: "image_id": "d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f", 2024-05-21T03:11:53.594 INFO:teuthology.orchestra.run.smithi027.stdout: "repo_digests": [ 2024-05-21T03:11:53.594 INFO:teuthology.orchestra.run.smithi027.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28" 2024-05-21T03:11:53.594 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-05-21T03:11:53.594 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:11:58.974 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-05-21T03:11:58.974 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_version": "ceph version 19.0.0-3830-g3b748ea0 (3b748ea08891006c1152513d25ab4a03f7c9a995) squid (dev)", 2024-05-21T03:11:58.974 INFO:teuthology.orchestra.run.smithi165.stdout: "image_id": "d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f", 2024-05-21T03:11:58.975 INFO:teuthology.orchestra.run.smithi165.stdout: "repo_digests": [ 2024-05-21T03:11:58.975 INFO:teuthology.orchestra.run.smithi165.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28" 2024-05-21T03:11:58.975 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-05-21T03:11:58.975 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-05-21T03:11:58.999 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /etc/ceph 2024-05-21T03:11:59.033 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /etc/ceph 2024-05-21T03:11:59.066 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /etc/ceph 2024-05-21T03:11:59.093 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 777 /etc/ceph 2024-05-21T03:11:59.123 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 777 /etc/ceph 2024-05-21T03:11:59.152 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 777 /etc/ceph 2024-05-21T03:11:59.175 INFO:tasks.cephadm:Writing seed config... 2024-05-21T03:11:59.176 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-21T03:11:59.176 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-21T03:11:59.176 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-21T03:11:59.176 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-21T03:11:59.177 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-21T03:11:59.177 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-21T03:11:59.177 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-21T03:11:59.177 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-21T03:11:59.177 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-21T03:11:59.177 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-21T03:11:59.177 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-21T03:11:59.177 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-21T03:11:59.177 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-21T03:11:59.177 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-21T03:11:59.177 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-21T03:11:59.177 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-21T03:11:59.177 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-21T03:11:59.177 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-05-21T03:11:59.177 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-21T03:11:59.178 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-21T03:11:59.178 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-21T03:11:59.178 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-21T03:11:59.178 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-05-21T03:11:59.178 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-21T03:11:59.178 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-21T03:11:59.178 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-21T03:11:59.178 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-21T03:11:59.178 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-21T03:11:59.178 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-21T03:11:59.178 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-21T03:11:59.178 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-21T03:11:59.178 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-05-21T03:11:59.179 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-05-21T03:11:59.179 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-05-21T03:11:59.179 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-21T03:11:59.179 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-21T03:11:59.179 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-05-21T03:11:59.179 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-05-21T03:11:59.179 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-21T03:11:59.179 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-21T03:11:59.179 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-05-21T03:11:59.179 INFO:tasks.cephadm: override: [osd] filestore flush min = 0 2024-05-21T03:11:59.179 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-21T03:11:59.179 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-21T03:11:59.179 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-21T03:11:59.179 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-21T03:11:59.180 INFO:tasks.cephadm: override: [osd] osd heartbeat grace = 60 2024-05-21T03:11:59.180 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-21T03:11:59.180 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-21T03:11:59.180 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-21T03:11:59.180 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-05-21T03:11:59.181 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-05-21T03:11:59.181 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-21T03:11:59.199 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = c9ca94be-171f-11ef-bc9a-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 filestore flush min = 0 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd heartbeat grace = 60 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-21T03:11:59.199 DEBUG:teuthology.orchestra.run.smithi027:mon.a> sudo journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mon.a.service 2024-05-21T03:11:59.242 DEBUG:teuthology.orchestra.run.smithi027:mgr.x> sudo journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mgr.x.service 2024-05-21T03:11:59.285 INFO:tasks.cephadm:Bootstrapping... 2024-05-21T03:11:59.285 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 -v bootstrap --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.27 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-21T03:11:59.487 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-05-21T03:11:59.487 INFO:teuthology.orchestra.run.smithi027.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995', '-v', 'bootstrap', '--fsid', 'c9ca94be-171f-11ef-bc9a-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.27', '--skip-admin-label'] 2024-05-21T03:11:59.487 INFO:teuthology.orchestra.run.smithi027.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-21T03:11:59.487 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying podman|docker is present... 2024-05-21T03:11:59.512 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stdout 5.0.2 2024-05-21T03:11:59.513 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying lvm2 is present... 2024-05-21T03:11:59.513 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying time synchronization is in place... 2024-05-21T03:11:59.521 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-21T03:11:59.521 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-21T03:11:59.527 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-21T03:11:59.527 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-05-21T03:11:59.534 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout enabled 2024-05-21T03:11:59.541 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout active 2024-05-21T03:11:59.541 INFO:teuthology.orchestra.run.smithi027.stdout:Unit chronyd.service is enabled and running 2024-05-21T03:11:59.541 INFO:teuthology.orchestra.run.smithi027.stdout:Repeating the final host check... 2024-05-21T03:11:59.566 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stdout 5.0.2 2024-05-21T03:11:59.566 INFO:teuthology.orchestra.run.smithi027.stdout:podman (/bin/podman) version 5.0.2 is present 2024-05-21T03:11:59.566 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl is present 2024-05-21T03:11:59.566 INFO:teuthology.orchestra.run.smithi027.stdout:lvcreate is present 2024-05-21T03:11:59.573 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-21T03:11:59.573 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-21T03:11:59.579 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-21T03:11:59.579 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-05-21T03:11:59.586 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout enabled 2024-05-21T03:11:59.593 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout active 2024-05-21T03:11:59.593 INFO:teuthology.orchestra.run.smithi027.stdout:Unit chronyd.service is enabled and running 2024-05-21T03:11:59.593 INFO:teuthology.orchestra.run.smithi027.stdout:Host looks OK 2024-05-21T03:11:59.593 INFO:teuthology.orchestra.run.smithi027.stdout:Cluster fsid: c9ca94be-171f-11ef-bc9a-c7b262605968 2024-05-21T03:11:59.593 INFO:teuthology.orchestra.run.smithi027.stdout:Acquiring lock 140034638466592 on /run/cephadm/c9ca94be-171f-11ef-bc9a-c7b262605968.lock 2024-05-21T03:11:59.593 INFO:teuthology.orchestra.run.smithi027.stdout:Lock 140034638466592 acquired on /run/cephadm/c9ca94be-171f-11ef-bc9a-c7b262605968.lock 2024-05-21T03:11:59.594 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying IP 172.21.15.27 port 3300 ... 2024-05-21T03:11:59.594 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying IP 172.21.15.27 port 6789 ... 2024-05-21T03:11:59.594 INFO:teuthology.orchestra.run.smithi027.stdout:Base mon IP(s) is [172.21.15.27:3300, 172.21.15.27:6789], mon addrv is [v2:172.21.15.27:3300,v1:172.21.15.27:6789] 2024-05-21T03:11:59.597 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.27 metric 100 2024-05-21T03:11:59.597 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.27 metric 100 2024-05-21T03:11:59.600 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-21T03:11:59.600 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-21T03:11:59.600 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-21T03:11:59.603 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-21T03:11:59.603 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-21T03:11:59.603 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-21T03:11:59.603 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-21T03:11:59.603 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:395a/64 scope link noprefixroute 2024-05-21T03:11:59.603 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-21T03:11:59.604 INFO:teuthology.orchestra.run.smithi027.stdout:Mon IP `172.21.15.27` is in CIDR network `172.21.0.0/20` 2024-05-21T03:11:59.604 INFO:teuthology.orchestra.run.smithi027.stdout:Mon IP `172.21.15.27` is in CIDR network `172.21.0.0/20` 2024-05-21T03:11:59.604 INFO:teuthology.orchestra.run.smithi027.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-21T03:11:59.604 INFO:teuthology.orchestra.run.smithi027.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-21T03:11:59.605 INFO:teuthology.orchestra.run.smithi027.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995... 2024-05-21T03:12:00.087 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stdout d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f 2024-05-21T03:12:00.087 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995... 2024-05-21T03:12:00.087 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Getting image source signatures 2024-05-21T03:12:00.088 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Copying blob sha256:a5d06693b40edf6d7f15546e9c6c501dcf23f4c800cbc84dcc01d75832a05bb1 2024-05-21T03:12:00.088 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Copying blob sha256:d0e808cd455827ca3108db1b0125dcee9ffc148daa1d7dde963f550bc831d501 2024-05-21T03:12:00.088 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Copying config sha256:d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f 2024-05-21T03:12:00.088 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-21T03:12:00.960 INFO:teuthology.orchestra.run.smithi027.stdout:ceph: stdout ceph version 19.0.0-3830-g3b748ea0 (3b748ea08891006c1152513d25ab4a03f7c9a995) squid (dev) 2024-05-21T03:12:00.960 INFO:teuthology.orchestra.run.smithi027.stdout:Ceph version: ceph version 19.0.0-3830-g3b748ea0 (3b748ea08891006c1152513d25ab4a03f7c9a995) squid (dev) 2024-05-21T03:12:00.960 INFO:teuthology.orchestra.run.smithi027.stdout:Extracting ceph user uid/gid from container image... 2024-05-21T03:12:01.959 INFO:teuthology.orchestra.run.smithi027.stdout:stat: stdout 167 167 2024-05-21T03:12:01.959 INFO:teuthology.orchestra.run.smithi027.stdout:Creating initial keys... 2024-05-21T03:12:08.413 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph-authtool: stdout AQACEUxmcZ+kERAAsIDtwlt8KRa6aarVpWNWKQ== 2024-05-21T03:12:12.803 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph-authtool: stdout AQALEUxmwO1XDRAALcGXIi6CcOxHbpYYOY3QsA== 2024-05-21T03:12:14.141 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph-authtool: stdout AQANEUxmeO/aBxAAwtX0M30VxsuXf2JhN+4LWw== 2024-05-21T03:12:14.142 INFO:teuthology.orchestra.run.smithi027.stdout:Creating initial monmap... 2024-05-21T03:12:14.998 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-21T03:12:14.998 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-21T03:12:14.998 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to c9ca94be-171f-11ef-bc9a-c7b262605968 2024-05-21T03:12:14.999 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-21T03:12:14.999 INFO:teuthology.orchestra.run.smithi027.stdout:monmaptool for a [v2:172.21.15.27:3300,v1:172.21.15.27:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-21T03:12:14.999 INFO:teuthology.orchestra.run.smithi027.stdout:setting min_mon_release = quincy 2024-05-21T03:12:14.999 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: set fsid to c9ca94be-171f-11ef-bc9a-c7b262605968 2024-05-21T03:12:14.999 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-21T03:12:14.999 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:12:14.999 INFO:teuthology.orchestra.run.smithi027.stdout:Creating mon... 2024-05-21T03:12:16.179 INFO:teuthology.orchestra.run.smithi027.stdout:create mon.a on 2024-05-21T03:12:16.397 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-21T03:12:16.543 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-21T03:12:16.783 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-c9ca94be-171f-11ef-bc9a-c7b262605968.target → /etc/systemd/system/ceph-c9ca94be-171f-11ef-bc9a-c7b262605968.target. 2024-05-21T03:12:16.783 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-c9ca94be-171f-11ef-bc9a-c7b262605968.target → /etc/systemd/system/ceph-c9ca94be-171f-11ef-bc9a-c7b262605968.target. 2024-05-21T03:12:17.013 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mon.a 2024-05-21T03:12:17.013 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mon.a.service: Unit ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mon.a.service not loaded. 2024-05-21T03:12:17.173 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c9ca94be-171f-11ef-bc9a-c7b262605968.target.wants/ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mon.a.service → /etc/systemd/system/ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@.service. 2024-05-21T03:12:18.276 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-21T03:12:18.276 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2024-05-21T03:12:18.284 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-21T03:12:18.285 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-05-21T03:12:18.285 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2024-05-21T03:12:18.285 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to enable service . firewalld.service is not available 2024-05-21T03:12:18.285 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mon to start... 2024-05-21T03:12:18.285 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mon... 2024-05-21T03:12:18.410 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:18 smithi027 systemd[1]: Started Ceph mon.a for c9ca94be-171f-11ef-bc9a-c7b262605968. 2024-05-21T03:12:19.391 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout cluster: 2024-05-21T03:12:19.392 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout id: c9ca94be-171f-11ef-bc9a-c7b262605968 2024-05-21T03:12:19.392 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-21T03:12:19.392 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-05-21T03:12:19.392 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout services: 2024-05-21T03:12:19.392 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.622289s) 2024-05-21T03:12:19.392 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-21T03:12:19.392 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-21T03:12:19.392 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-05-21T03:12:19.392 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout data: 2024-05-21T03:12:19.392 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-21T03:12:19.393 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-21T03:12:19.393 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-21T03:12:19.393 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout pgs: 2024-05-21T03:12:19.393 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-05-21T03:12:19.393 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.756+0000 7f3a59aee700 1 Processor -- start 2024-05-21T03:12:19.393 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.756+0000 7f3a59aee700 1 -- start start 2024-05-21T03:12:19.393 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.756+0000 7f3a59aee700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a54109c60 0x7f3a5410c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:19.393 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.756+0000 7f3a59aee700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5407ce70 con 0x7f3a54109c60 2024-05-21T03:12:19.393 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.756+0000 7f3a537fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a54109c60 0x7f3a5410c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:19.393 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.756+0000 7f3a537fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a54109c60 0x7f3a5410c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46712/0 (socket says 172.21.15.27:46712) 2024-05-21T03:12:19.393 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.756+0000 7f3a537fe700 1 -- 172.21.15.27:0/1236113044 learned_addr learned my addr 172.21.15.27:0/1236113044 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:19.393 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.757+0000 7f3a537fe700 1 -- 172.21.15.27:0/1236113044 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5410c590 con 0x7f3a54109c60 2024-05-21T03:12:19.394 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.757+0000 7f3a537fe700 1 --2- 172.21.15.27:0/1236113044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a54109c60 0x7f3a5410c050 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3a440098a0 tx=0x7f3a44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2136999c8a9cce7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:19.394 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.757+0000 7f3a527fc700 1 -- 172.21.15.27:0/1236113044 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a44004160 con 0x7f3a54109c60 2024-05-21T03:12:19.394 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.757+0000 7f3a527fc700 1 -- 172.21.15.27:0/1236113044 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a44004300 con 0x7f3a54109c60 2024-05-21T03:12:19.394 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.757+0000 7f3a527fc700 1 -- 172.21.15.27:0/1236113044 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a440036a0 con 0x7f3a54109c60 2024-05-21T03:12:19.394 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.758+0000 7f3a59aee700 1 -- 172.21.15.27:0/1236113044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a54109c60 msgr2=0x7f3a5410c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:19.394 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.758+0000 7f3a59aee700 1 --2- 172.21.15.27:0/1236113044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a54109c60 0x7f3a5410c050 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3a440098a0 tx=0x7f3a44009bb0 comp rx=0 tx=0).stop 2024-05-21T03:12:19.394 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.758+0000 7f3a59aee700 1 -- 172.21.15.27:0/1236113044 shutdown_connections 2024-05-21T03:12:19.394 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.758+0000 7f3a59aee700 1 --2- 172.21.15.27:0/1236113044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a54109c60 0x7f3a5410c050 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:19.394 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.758+0000 7f3a59aee700 1 -- 172.21.15.27:0/1236113044 >> 172.21.15.27:0/1236113044 conn(0x7f3a54103690 msgr2=0x7f3a54105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:19.394 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.758+0000 7f3a59aee700 1 -- 172.21.15.27:0/1236113044 shutdown_connections 2024-05-21T03:12:19.394 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.758+0000 7f3a59aee700 1 -- 172.21.15.27:0/1236113044 wait complete. 2024-05-21T03:12:19.395 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.758+0000 7f3a59aee700 1 Processor -- start 2024-05-21T03:12:19.395 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.758+0000 7f3a59aee700 1 -- start start 2024-05-21T03:12:19.395 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.759+0000 7f3a59aee700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a54109c60 0x7f3a5419c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:19.395 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.759+0000 7f3a59aee700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5410ca00 con 0x7f3a54109c60 2024-05-21T03:12:19.395 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.759+0000 7f3a537fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a54109c60 0x7f3a5419c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:19.395 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.759+0000 7f3a537fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a54109c60 0x7f3a5419c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46726/0 (socket says 172.21.15.27:46726) 2024-05-21T03:12:19.395 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.759+0000 7f3a537fe700 1 -- 172.21.15.27:0/3835667882 learned_addr learned my addr 172.21.15.27:0/3835667882 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:19.395 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.759+0000 7f3a537fe700 1 -- 172.21.15.27:0/3835667882 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5419cc80 con 0x7f3a54109c60 2024-05-21T03:12:19.395 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.759+0000 7f3a537fe700 1 --2- 172.21.15.27:0/3835667882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a54109c60 0x7f3a5419c740 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3a44009fb0 tx=0x7f3a44004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:19.395 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.759+0000 7f3a50ff9700 1 -- 172.21.15.27:0/3835667882 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a44003de0 con 0x7f3a54109c60 2024-05-21T03:12:19.395 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.759+0000 7f3a50ff9700 1 -- 172.21.15.27:0/3835667882 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a44011630 con 0x7f3a54109c60 2024-05-21T03:12:19.395 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.759+0000 7f3a59aee700 1 -- 172.21.15.27:0/3835667882 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a5419cf10 con 0x7f3a54109c60 2024-05-21T03:12:19.396 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.759+0000 7f3a50ff9700 1 -- 172.21.15.27:0/3835667882 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a44011930 con 0x7f3a54109c60 2024-05-21T03:12:19.396 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.760+0000 7f3a59aee700 1 -- 172.21.15.27:0/3835667882 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a5419fc00 con 0x7f3a54109c60 2024-05-21T03:12:19.396 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.760+0000 7f3a50ff9700 1 -- 172.21.15.27:0/3835667882 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3a44024480 con 0x7f3a54109c60 2024-05-21T03:12:19.396 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.760+0000 7f3a50ff9700 1 -- 172.21.15.27:0/3835667882 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3a4401a7e0 con 0x7f3a54109c60 2024-05-21T03:12:19.396 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.760+0000 7f3a59aee700 1 -- 172.21.15.27:0/3835667882 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a54064ba0 con 0x7f3a54109c60 2024-05-21T03:12:19.396 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.763+0000 7f3a50ff9700 1 -- 172.21.15.27:0/3835667882 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f3a4401aa80 con 0x7f3a54109c60 2024-05-21T03:12:19.396 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.837+0000 7f3a59aee700 1 -- 172.21.15.27:0/3835667882 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f3a541a00e0 con 0x7f3a54109c60 2024-05-21T03:12:19.396 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.837+0000 7f3a50ff9700 1 -- 172.21.15.27:0/3835667882 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f3a4401ac60 con 0x7f3a54109c60 2024-05-21T03:12:19.396 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.838+0000 7f3a59aee700 1 -- 172.21.15.27:0/3835667882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a54109c60 msgr2=0x7f3a5419c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:19.396 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.838+0000 7f3a59aee700 1 --2- 172.21.15.27:0/3835667882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a54109c60 0x7f3a5419c740 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3a44009fb0 tx=0x7f3a44004750 comp rx=0 tx=0).stop 2024-05-21T03:12:19.396 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.838+0000 7f3a59aee700 1 -- 172.21.15.27:0/3835667882 shutdown_connections 2024-05-21T03:12:19.396 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.838+0000 7f3a59aee700 1 --2- 172.21.15.27:0/3835667882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a54109c60 0x7f3a5419c740 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:19.396 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.838+0000 7f3a59aee700 1 -- 172.21.15.27:0/3835667882 >> 172.21.15.27:0/3835667882 conn(0x7f3a54103690 msgr2=0x7f3a541043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:19.397 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.838+0000 7f3a59aee700 1 -- 172.21.15.27:0/3835667882 shutdown_connections 2024-05-21T03:12:19.397 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:18.838+0000 7f3a59aee700 1 -- 172.21.15.27:0/3835667882 wait complete. 2024-05-21T03:12:19.397 INFO:teuthology.orchestra.run.smithi027.stdout:mon is available 2024-05-21T03:12:19.397 INFO:teuthology.orchestra.run.smithi027.stdout:Assimilating anything we can from ceph.conf... 2024-05-21T03:12:19.508 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:19 smithi027 ceph-mon[31809]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-21T03:12:19.508 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:19 smithi027 ceph-mon[31809]: monmap epoch 1 2024-05-21T03:12:19.508 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:19 smithi027 ceph-mon[31809]: fsid c9ca94be-171f-11ef-bc9a-c7b262605968 2024-05-21T03:12:19.508 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:19 smithi027 ceph-mon[31809]: last_changed 2024-05-21T03:12:14.443931+0000 2024-05-21T03:12:19.508 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:19 smithi027 ceph-mon[31809]: created 2024-05-21T03:12:14.443931+0000 2024-05-21T03:12:19.509 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:19 smithi027 ceph-mon[31809]: min_mon_release 19 (squid) 2024-05-21T03:12:19.509 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:19 smithi027 ceph-mon[31809]: election_strategy: 1 2024-05-21T03:12:19.509 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:19 smithi027 ceph-mon[31809]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-05-21T03:12:19.509 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:19 smithi027 ceph-mon[31809]: fsmap 2024-05-21T03:12:19.509 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:19 smithi027 ceph-mon[31809]: osdmap e1: 0 total, 0 up, 0 in 2024-05-21T03:12:19.509 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:19 smithi027 ceph-mon[31809]: mgrmap e1: no daemons active 2024-05-21T03:12:19.509 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:19 smithi027 ceph-mon[31809]: from='client.? 172.21.15.27:0/3835667882' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-21T03:12:20.480 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:20 smithi027 ceph-mon[31809]: from='client.? 172.21.15.27:0/4127726061' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-21T03:12:20.480 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:20 smithi027 ceph-mon[31809]: from='client.? 172.21.15.27:0/4127726061' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-21T03:12:20.489 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-05-21T03:12:20.489 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [global] 2024-05-21T03:12:20.489 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout fsid = c9ca94be-171f-11ef-bc9a-c7b262605968 2024-05-21T03:12:20.489 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-21T03:12:20.489 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.27:3300,v1:172.21.15.27:6789] 2024-05-21T03:12:20.489 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-21T03:12:20.490 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-21T03:12:20.490 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-21T03:12:20.490 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-21T03:12:20.490 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-05-21T03:12:20.490 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-21T03:12:20.490 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-21T03:12:20.490 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-05-21T03:12:20.490 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [osd] 2024-05-21T03:12:20.490 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-05-21T03:12:20.490 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-05-21T03:12:20.490 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-21T03:12:20.490 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-21T03:12:20.490 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-21T03:12:20.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-21T03:12:20.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-21T03:12:20.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.830+0000 7fbc1adca700 1 Processor -- start 2024-05-21T03:12:20.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.830+0000 7fbc1adca700 1 -- start start 2024-05-21T03:12:20.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.830+0000 7fbc1adca700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc14108530 0x7fbc14108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:20.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.831+0000 7fbc1adca700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbc14108f20 con 0x7fbc14108530 2024-05-21T03:12:20.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.831+0000 7fbc18b66700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc14108530 0x7fbc14108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:20.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.831+0000 7fbc18b66700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc14108530 0x7fbc14108950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46738/0 (socket says 172.21.15.27:46738) 2024-05-21T03:12:20.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.831+0000 7fbc18b66700 1 -- 172.21.15.27:0/1898319635 learned_addr learned my addr 172.21.15.27:0/1898319635 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:20.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.831+0000 7fbc18b66700 1 -- 172.21.15.27:0/1898319635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc14109250 con 0x7fbc14108530 2024-05-21T03:12:20.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.831+0000 7fbc18b66700 1 --2- 172.21.15.27:0/1898319635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc14108530 0x7fbc14108950 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fbc080098a0 tx=0x7fbc08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93934a07077a302 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:20.491 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.832+0000 7fbc137fe700 1 -- 172.21.15.27:0/1898319635 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc08004160 con 0x7fbc14108530 2024-05-21T03:12:20.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.832+0000 7fbc137fe700 1 -- 172.21.15.27:0/1898319635 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc08004300 con 0x7fbc14108530 2024-05-21T03:12:20.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.832+0000 7fbc137fe700 1 -- 172.21.15.27:0/1898319635 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc080036a0 con 0x7fbc14108530 2024-05-21T03:12:20.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.832+0000 7fbc1adca700 1 -- 172.21.15.27:0/1898319635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc14108530 msgr2=0x7fbc14108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:20.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.832+0000 7fbc1adca700 1 --2- 172.21.15.27:0/1898319635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc14108530 0x7fbc14108950 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fbc080098a0 tx=0x7fbc08009bb0 comp rx=0 tx=0).stop 2024-05-21T03:12:20.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.832+0000 7fbc1adca700 1 -- 172.21.15.27:0/1898319635 shutdown_connections 2024-05-21T03:12:20.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.832+0000 7fbc1adca700 1 --2- 172.21.15.27:0/1898319635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc14108530 0x7fbc14108950 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:20.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.832+0000 7fbc1adca700 1 -- 172.21.15.27:0/1898319635 >> 172.21.15.27:0/1898319635 conn(0x7fbc14102dd0 msgr2=0x7fbc141051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:20.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.832+0000 7fbc1adca700 1 -- 172.21.15.27:0/1898319635 shutdown_connections 2024-05-21T03:12:20.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.832+0000 7fbc1adca700 1 -- 172.21.15.27:0/1898319635 wait complete. 2024-05-21T03:12:20.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.833+0000 7fbc1adca700 1 Processor -- start 2024-05-21T03:12:20.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.833+0000 7fbc1adca700 1 -- start start 2024-05-21T03:12:20.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.833+0000 7fbc1adca700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc14108530 0x7fbc14190c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:20.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.833+0000 7fbc1adca700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbc14109720 con 0x7fbc14108530 2024-05-21T03:12:20.492 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.833+0000 7fbc18b66700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc14108530 0x7fbc14190c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:20.493 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.833+0000 7fbc18b66700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc14108530 0x7fbc14190c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46754/0 (socket says 172.21.15.27:46754) 2024-05-21T03:12:20.493 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.833+0000 7fbc18b66700 1 -- 172.21.15.27:0/4127726061 learned_addr learned my addr 172.21.15.27:0/4127726061 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:20.493 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.833+0000 7fbc18b66700 1 -- 172.21.15.27:0/4127726061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc14191150 con 0x7fbc14108530 2024-05-21T03:12:20.493 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.834+0000 7fbc18b66700 1 --2- 172.21.15.27:0/4127726061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc14108530 0x7fbc14190c10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fbc08000c00 tx=0x7fbc08004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:20.493 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.834+0000 7fbc11ffb700 1 -- 172.21.15.27:0/4127726061 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc08003db0 con 0x7fbc14108530 2024-05-21T03:12:20.493 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.834+0000 7fbc11ffb700 1 -- 172.21.15.27:0/4127726061 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc08011630 con 0x7fbc14108530 2024-05-21T03:12:20.493 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.834+0000 7fbc1adca700 1 -- 172.21.15.27:0/4127726061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc1418f340 con 0x7fbc14108530 2024-05-21T03:12:20.493 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.834+0000 7fbc11ffb700 1 -- 172.21.15.27:0/4127726061 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc08011950 con 0x7fbc14108530 2024-05-21T03:12:20.493 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.834+0000 7fbc1adca700 1 -- 172.21.15.27:0/4127726061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc1418f880 con 0x7fbc14108530 2024-05-21T03:12:20.493 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.834+0000 7fbc11ffb700 1 -- 172.21.15.27:0/4127726061 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbc08011af0 con 0x7fbc14108530 2024-05-21T03:12:20.493 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.834+0000 7fbc11ffb700 1 -- 172.21.15.27:0/4127726061 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbc0801c450 con 0x7fbc14108530 2024-05-21T03:12:20.493 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.835+0000 7fbc1adca700 1 -- 172.21.15.27:0/4127726061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc14064ba0 con 0x7fbc14108530 2024-05-21T03:12:20.493 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.837+0000 7fbc11ffb700 1 -- 172.21.15.27:0/4127726061 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fbc08032020 con 0x7fbc14108530 2024-05-21T03:12:20.494 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.908+0000 7fbc1adca700 1 -- 172.21.15.27:0/4127726061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fbc14195fa0 con 0x7fbc14108530 2024-05-21T03:12:20.494 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.911+0000 7fbc11ffb700 1 -- 172.21.15.27:0/4127726061 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+517 (secure 0 0 0) 0x7fbc08004260 con 0x7fbc14108530 2024-05-21T03:12:20.494 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.912+0000 7fbc11ffb700 1 -- 172.21.15.27:0/4127726061 <== mon.0 v2:172.21.15.27:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbc0801c6a0 con 0x7fbc14108530 2024-05-21T03:12:20.494 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.913+0000 7fbc1adca700 1 -- 172.21.15.27:0/4127726061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc14108530 msgr2=0x7fbc14190c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:20.494 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.913+0000 7fbc1adca700 1 --2- 172.21.15.27:0/4127726061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc14108530 0x7fbc14190c10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fbc08000c00 tx=0x7fbc08004750 comp rx=0 tx=0).stop 2024-05-21T03:12:20.494 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.913+0000 7fbc1adca700 1 -- 172.21.15.27:0/4127726061 shutdown_connections 2024-05-21T03:12:20.494 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.913+0000 7fbc1adca700 1 --2- 172.21.15.27:0/4127726061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc14108530 0x7fbc14190c10 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:20.494 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.913+0000 7fbc1adca700 1 -- 172.21.15.27:0/4127726061 >> 172.21.15.27:0/4127726061 conn(0x7fbc14102dd0 msgr2=0x7fbc141933a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:20.494 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.913+0000 7fbc1adca700 1 -- 172.21.15.27:0/4127726061 shutdown_connections 2024-05-21T03:12:20.494 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:19.913+0000 7fbc1adca700 1 -- 172.21.15.27:0/4127726061 wait complete. 2024-05-21T03:12:20.494 INFO:teuthology.orchestra.run.smithi027.stdout:Generating new minimal ceph.conf... 2024-05-21T03:12:21.520 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:21 smithi027 ceph-mon[31809]: from='client.? 172.21.15.27:0/621215855' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:12:21.527 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.886+0000 7fa8e659c700 1 Processor -- start 2024-05-21T03:12:21.527 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.886+0000 7fa8e659c700 1 -- start start 2024-05-21T03:12:21.527 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.886+0000 7fa8e659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8e007ee40 0x7fa8e007d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:21.527 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.886+0000 7fa8e659c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e007d880 con 0x7fa8e007ee40 2024-05-21T03:12:21.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.886+0000 7fa8dffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8e007ee40 0x7fa8e007d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:21.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.886+0000 7fa8dffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8e007ee40 0x7fa8e007d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46764/0 (socket says 172.21.15.27:46764) 2024-05-21T03:12:21.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.887+0000 7fa8dffff700 1 -- 172.21.15.27:0/3888442421 learned_addr learned my addr 172.21.15.27:0/3888442421 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:21.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.887+0000 7fa8dffff700 1 -- 172.21.15.27:0/3888442421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e007da00 con 0x7fa8e007ee40 2024-05-21T03:12:21.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.887+0000 7fa8dffff700 1 --2- 172.21.15.27:0/3888442421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8e007ee40 0x7fa8e007d340 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa8d00098a0 tx=0x7fa8d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f83ab429cec2ac server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:21.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.887+0000 7fa8deffd700 1 -- 172.21.15.27:0/3888442421 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8d0004160 con 0x7fa8e007ee40 2024-05-21T03:12:21.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.887+0000 7fa8deffd700 1 -- 172.21.15.27:0/3888442421 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa8d0004300 con 0x7fa8e007ee40 2024-05-21T03:12:21.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.887+0000 7fa8deffd700 1 -- 172.21.15.27:0/3888442421 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8d0011af0 con 0x7fa8e007ee40 2024-05-21T03:12:21.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.888+0000 7fa8e659c700 1 -- 172.21.15.27:0/3888442421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8e007ee40 msgr2=0x7fa8e007d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:21.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.888+0000 7fa8e659c700 1 --2- 172.21.15.27:0/3888442421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8e007ee40 0x7fa8e007d340 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa8d00098a0 tx=0x7fa8d0009bb0 comp rx=0 tx=0).stop 2024-05-21T03:12:21.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.888+0000 7fa8e659c700 1 -- 172.21.15.27:0/3888442421 shutdown_connections 2024-05-21T03:12:21.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.888+0000 7fa8e659c700 1 --2- 172.21.15.27:0/3888442421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8e007ee40 0x7fa8e007d340 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:21.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.888+0000 7fa8e659c700 1 -- 172.21.15.27:0/3888442421 >> 172.21.15.27:0/3888442421 conn(0x7fa8e0104840 msgr2=0x7fa8e0106c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:21.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.888+0000 7fa8e659c700 1 -- 172.21.15.27:0/3888442421 shutdown_connections 2024-05-21T03:12:21.529 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.888+0000 7fa8e659c700 1 -- 172.21.15.27:0/3888442421 wait complete. 2024-05-21T03:12:21.529 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.888+0000 7fa8e659c700 1 Processor -- start 2024-05-21T03:12:21.529 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.888+0000 7fa8e659c700 1 -- start start 2024-05-21T03:12:21.529 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.889+0000 7fa8e659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8e007ee40 0x7fa8e019f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:21.529 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.889+0000 7fa8e659c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e010f710 con 0x7fa8e007ee40 2024-05-21T03:12:21.529 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.889+0000 7fa8dffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8e007ee40 0x7fa8e019f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:21.529 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.889+0000 7fa8dffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8e007ee40 0x7fa8e019f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46774/0 (socket says 172.21.15.27:46774) 2024-05-21T03:12:21.529 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.889+0000 7fa8dffff700 1 -- 172.21.15.27:0/621215855 learned_addr learned my addr 172.21.15.27:0/621215855 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:21.529 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.889+0000 7fa8dffff700 1 -- 172.21.15.27:0/621215855 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e019f750 con 0x7fa8e007ee40 2024-05-21T03:12:21.529 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.889+0000 7fa8dffff700 1 --2- 172.21.15.27:0/621215855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8e007ee40 0x7fa8e019f210 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0011ac0 tx=0x7fa8d0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:21.529 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.889+0000 7fa8dd7fa700 1 -- 172.21.15.27:0/621215855 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8d0003970 con 0x7fa8e007ee40 2024-05-21T03:12:21.529 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.890+0000 7fa8e659c700 1 -- 172.21.15.27:0/621215855 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8e019f9e0 con 0x7fa8e007ee40 2024-05-21T03:12:21.529 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.890+0000 7fa8dd7fa700 1 -- 172.21.15.27:0/621215855 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa8d0003b10 con 0x7fa8e007ee40 2024-05-21T03:12:21.529 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.890+0000 7fa8dd7fa700 1 -- 172.21.15.27:0/621215855 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8d0024610 con 0x7fa8e007ee40 2024-05-21T03:12:21.529 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.890+0000 7fa8e659c700 1 -- 172.21.15.27:0/621215855 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8e01a26d0 con 0x7fa8e007ee40 2024-05-21T03:12:21.530 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.890+0000 7fa8dd7fa700 1 -- 172.21.15.27:0/621215855 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa8d0024be0 con 0x7fa8e007ee40 2024-05-21T03:12:21.530 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.890+0000 7fa8dd7fa700 1 -- 172.21.15.27:0/621215855 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa8d0019700 con 0x7fa8e007ee40 2024-05-21T03:12:21.530 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.891+0000 7fa8e659c700 1 -- 172.21.15.27:0/621215855 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8e0064ba0 con 0x7fa8e007ee40 2024-05-21T03:12:21.530 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.893+0000 7fa8dd7fa700 1 -- 172.21.15.27:0/621215855 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa8d0043ec0 con 0x7fa8e007ee40 2024-05-21T03:12:21.530 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.963+0000 7fa8e659c700 1 -- 172.21.15.27:0/621215855 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fa8e01a2c70 con 0x7fa8e007ee40 2024-05-21T03:12:21.530 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.964+0000 7fa8dd7fa700 1 -- 172.21.15.27:0/621215855 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fa8d0032030 con 0x7fa8e007ee40 2024-05-21T03:12:21.530 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.964+0000 7fa8e659c700 1 -- 172.21.15.27:0/621215855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8e007ee40 msgr2=0x7fa8e019f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:21.530 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.964+0000 7fa8e659c700 1 --2- 172.21.15.27:0/621215855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8e007ee40 0x7fa8e019f210 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0011ac0 tx=0x7fa8d0011ba0 comp rx=0 tx=0).stop 2024-05-21T03:12:21.530 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.965+0000 7fa8e659c700 1 -- 172.21.15.27:0/621215855 shutdown_connections 2024-05-21T03:12:21.530 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.965+0000 7fa8e659c700 1 --2- 172.21.15.27:0/621215855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8e007ee40 0x7fa8e019f210 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:21.530 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.965+0000 7fa8e659c700 1 -- 172.21.15.27:0/621215855 >> 172.21.15.27:0/621215855 conn(0x7fa8e0104840 msgr2=0x7fa8e0105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:21.530 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.965+0000 7fa8e659c700 1 -- 172.21.15.27:0/621215855 shutdown_connections 2024-05-21T03:12:21.530 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:20.965+0000 7fa8e659c700 1 -- 172.21.15.27:0/621215855 wait complete. 2024-05-21T03:12:21.530 INFO:teuthology.orchestra.run.smithi027.stdout:Restarting the monitor... 2024-05-21T03:12:21.808 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:21 smithi027 systemd[1]: Stopping Ceph mon.a for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:12:22.173 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:21 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mon-a[31805]: 2024-05-21T03:12:21.806+0000 7f3f5cbbb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T03:12:22.174 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:21 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mon-a[31805]: 2024-05-21T03:12:21.806+0000 7f3f5cbbb700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-21T03:12:22.174 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:21 smithi027 podman[32054]: 2024-05-21 03:12:21.930800966 +0000 UTC m=+0.224871070 container died 89c7929473840873a70dd0306a6624c2e399e110cb8705a407929b0418e5afa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-05-21T03:12:22.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:22 smithi027 podman[32054]: 2024-05-21 03:12:22.257039971 +0000 UTC m=+0.551110070 container cleanup 89c7929473840873a70dd0306a6624c2e399e110cb8705a407929b0418e5afa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mon-a, org.label-schema.build-date=20240514, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-21T03:12:22.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:22 smithi027 bash[32054]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mon-a 2024-05-21T03:12:22.908 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:22 smithi027 podman[32067]: 2024-05-21 03:12:22.458167725 +0000 UTC m=+0.529819104 container remove 89c7929473840873a70dd0306a6624c2e399e110cb8705a407929b0418e5afa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea) 2024-05-21T03:12:22.909 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:22 smithi027 systemd[1]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-21T03:12:22.909 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:22 smithi027 systemd[1]: Stopped Ceph mon.a for c9ca94be-171f-11ef-bc9a-c7b262605968. 2024-05-21T03:12:22.909 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:22 smithi027 systemd[1]: Starting Ceph mon.a for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:12:23.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:22 smithi027 podman[32160]: 2024-05-21 03:12:22.909098951 +0000 UTC m=+0.110096381 container create eb3a7e6bf9c301052ffeb751719914039918507b8941d4bdcfebc68f6689b284 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mon-a, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-05-21T03:12:23.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:22 smithi027 podman[32160]: 2024-05-21 03:12:22.817371294 +0000 UTC m=+0.018368702 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 2024-05-21T03:12:23.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 podman[32160]: 2024-05-21 03:12:23.12008301 +0000 UTC m=+0.321080401 container init eb3a7e6bf9c301052ffeb751719914039918507b8941d4bdcfebc68f6689b284 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mon-a, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, maintainer=Guillaume Abrioux ) 2024-05-21T03:12:23.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 podman[32160]: 2024-05-21 03:12:23.12350616 +0000 UTC m=+0.324503548 container start eb3a7e6bf9c301052ffeb751719914039918507b8941d4bdcfebc68f6689b284 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mon-a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T03:12:23.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: set uid:gid to 167:167 (ceph:ceph) 2024-05-21T03:12:23.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: ceph version 19.0.0-3830-g3b748ea0 (3b748ea08891006c1152513d25ab4a03f7c9a995) squid (dev), process ceph-mon, pid 2 2024-05-21T03:12:23.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: pidfile_write: ignore empty --pid-file 2024-05-21T03:12:23.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: load: jerasure load: lrc 2024-05-21T03:12:23.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: RocksDB version: 7.9.2 2024-05-21T03:12:23.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Git sha 0 2024-05-21T03:12:23.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Compile date 2024-05-20 21:53:18 2024-05-21T03:12:23.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: DB SUMMARY 2024-05-21T03:12:23.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: DB Session ID: F0CML4FBV8G81ZC9LE5S 2024-05-21T03:12:23.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: CURRENT file: CURRENT 2024-05-21T03:12:23.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: IDENTITY file: IDENTITY 2024-05-21T03:12:23.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-21T03:12:23.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-21T03:12:23.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105884 ; 2024-05-21T03:12:23.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.error_if_exists: 0 2024-05-21T03:12:23.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.create_if_missing: 0 2024-05-21T03:12:23.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.paranoid_checks: 1 2024-05-21T03:12:23.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-21T03:12:23.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-21T03:12:23.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-21T03:12:23.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.env: 0x558cac7e9cc0 2024-05-21T03:12:23.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.fs: PosixFileSystem 2024-05-21T03:12:23.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.info_log: 0x558caeb07da0 2024-05-21T03:12:23.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.max_file_opening_threads: 16 2024-05-21T03:12:23.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.statistics: (nil) 2024-05-21T03:12:23.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.use_fsync: 0 2024-05-21T03:12:23.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.max_log_file_size: 0 2024-05-21T03:12:23.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-21T03:12:23.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-21T03:12:23.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.keep_log_file_num: 1000 2024-05-21T03:12:23.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.recycle_log_file_num: 0 2024-05-21T03:12:23.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.allow_fallocate: 1 2024-05-21T03:12:23.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.allow_mmap_reads: 0 2024-05-21T03:12:23.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.allow_mmap_writes: 0 2024-05-21T03:12:23.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.use_direct_reads: 0 2024-05-21T03:12:23.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-21T03:12:23.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.create_missing_column_families: 0 2024-05-21T03:12:23.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.db_log_dir: 2024-05-21T03:12:23.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.wal_dir: 2024-05-21T03:12:23.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-21T03:12:23.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-21T03:12:23.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-21T03:12:23.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-21T03:12:23.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-21T03:12:23.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-21T03:12:23.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.advise_random_on_open: 1 2024-05-21T03:12:23.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.db_write_buffer_size: 0 2024-05-21T03:12:23.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.write_buffer_manager: 0x558caead90e0 2024-05-21T03:12:23.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-21T03:12:23.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-21T03:12:23.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-21T03:12:23.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.rate_limiter: (nil) 2024-05-21T03:12:23.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-21T03:12:23.183 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.wal_recovery_mode: 2 2024-05-21T03:12:23.183 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.enable_thread_tracking: 0 2024-05-21T03:12:23.183 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.enable_pipelined_write: 0 2024-05-21T03:12:23.183 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.unordered_write: 0 2024-05-21T03:12:23.183 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-21T03:12:23.183 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-21T03:12:23.183 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-21T03:12:23.183 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-21T03:12:23.184 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.row_cache: None 2024-05-21T03:12:23.184 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.wal_filter: None 2024-05-21T03:12:23.184 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-21T03:12:23.184 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.allow_ingest_behind: 0 2024-05-21T03:12:23.184 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.two_write_queues: 0 2024-05-21T03:12:23.184 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.manual_wal_flush: 0 2024-05-21T03:12:23.184 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.wal_compression: 0 2024-05-21T03:12:23.184 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.atomic_flush: 0 2024-05-21T03:12:23.184 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-21T03:12:23.184 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-21T03:12:23.184 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-21T03:12:23.184 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.log_readahead_size: 0 2024-05-21T03:12:23.184 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-21T03:12:23.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.best_efforts_recovery: 0 2024-05-21T03:12:23.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-21T03:12:23.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-21T03:12:23.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.allow_data_in_errors: 0 2024-05-21T03:12:23.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.db_host_id: __hostname__ 2024-05-21T03:12:23.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-21T03:12:23.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.max_background_jobs: 2 2024-05-21T03:12:23.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.max_background_compactions: -1 2024-05-21T03:12:23.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.max_subcompactions: 1 2024-05-21T03:12:23.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-21T03:12:23.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-21T03:12:23.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-21T03:12:23.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.max_total_wal_size: 0 2024-05-21T03:12:23.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-21T03:12:23.186 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-21T03:12:23.186 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-21T03:12:23.186 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-21T03:12:23.186 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.max_open_files: -1 2024-05-21T03:12:23.186 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.bytes_per_sync: 0 2024-05-21T03:12:23.186 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-21T03:12:23.186 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-21T03:12:23.186 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.compaction_readahead_size: 0 2024-05-21T03:12:23.186 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.max_background_flushes: -1 2024-05-21T03:12:23.186 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Compression algorithms supported: 2024-05-21T03:12:23.186 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-21T03:12:23.186 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: kZSTD supported: 0 2024-05-21T03:12:23.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: kXpressCompression supported: 0 2024-05-21T03:12:23.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: kLZ4HCCompression supported: 1 2024-05-21T03:12:23.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: kZlibCompression supported: 1 2024-05-21T03:12:23.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: kSnappyCompression supported: 1 2024-05-21T03:12:23.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: kLZ4Compression supported: 1 2024-05-21T03:12:23.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: kBZip2Compression supported: 0 2024-05-21T03:12:23.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-21T03:12:23.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-21T03:12:23.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-21T03:12:23.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-21T03:12:23.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-21T03:12:23.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.merge_operator: 2024-05-21T03:12:23.188 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.compaction_filter: None 2024-05-21T03:12:23.188 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.compaction_filter_factory: None 2024-05-21T03:12:23.188 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.sst_partitioner_factory: None 2024-05-21T03:12:23.188 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-21T03:12:23.188 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-21T03:12:23.188 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558caeb07f60) 2024-05-21T03:12:23.188 INFO:journalctl@ceph.mon.a.smithi027.stdout: cache_index_and_filter_blocks: 1 2024-05-21T03:12:23.188 INFO:journalctl@ceph.mon.a.smithi027.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-21T03:12:23.188 INFO:journalctl@ceph.mon.a.smithi027.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-21T03:12:23.188 INFO:journalctl@ceph.mon.a.smithi027.stdout: pin_top_level_index_and_filter: 1 2024-05-21T03:12:23.189 INFO:journalctl@ceph.mon.a.smithi027.stdout: index_type: 0 2024-05-21T03:12:23.189 INFO:journalctl@ceph.mon.a.smithi027.stdout: data_block_index_type: 0 2024-05-21T03:12:23.189 INFO:journalctl@ceph.mon.a.smithi027.stdout: index_shortening: 1 2024-05-21T03:12:23.189 INFO:journalctl@ceph.mon.a.smithi027.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-21T03:12:23.189 INFO:journalctl@ceph.mon.a.smithi027.stdout: checksum: 4 2024-05-21T03:12:23.189 INFO:journalctl@ceph.mon.a.smithi027.stdout: no_block_cache: 0 2024-05-21T03:12:23.189 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache: 0x558caeb0b1f0 2024-05-21T03:12:23.189 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache_name: BinnedLRUCache 2024-05-21T03:12:23.189 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache_options: 2024-05-21T03:12:23.189 INFO:journalctl@ceph.mon.a.smithi027.stdout: capacity : 536870912 2024-05-21T03:12:23.190 INFO:journalctl@ceph.mon.a.smithi027.stdout: num_shard_bits : 4 2024-05-21T03:12:23.190 INFO:journalctl@ceph.mon.a.smithi027.stdout: strict_capacity_limit : 0 2024-05-21T03:12:23.190 INFO:journalctl@ceph.mon.a.smithi027.stdout: high_pri_pool_ratio: 0.000 2024-05-21T03:12:23.190 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache_compressed: (nil) 2024-05-21T03:12:23.190 INFO:journalctl@ceph.mon.a.smithi027.stdout: persistent_cache: (nil) 2024-05-21T03:12:23.190 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_size: 4096 2024-05-21T03:12:23.190 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_size_deviation: 10 2024-05-21T03:12:23.190 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_restart_interval: 16 2024-05-21T03:12:23.190 INFO:journalctl@ceph.mon.a.smithi027.stdout: index_block_restart_interval: 1 2024-05-21T03:12:23.190 INFO:journalctl@ceph.mon.a.smithi027.stdout: metadata_block_size: 4096 2024-05-21T03:12:23.190 INFO:journalctl@ceph.mon.a.smithi027.stdout: partition_filters: 0 2024-05-21T03:12:23.191 INFO:journalctl@ceph.mon.a.smithi027.stdout: use_delta_encoding: 1 2024-05-21T03:12:23.191 INFO:journalctl@ceph.mon.a.smithi027.stdout: filter_policy: bloomfilter 2024-05-21T03:12:23.191 INFO:journalctl@ceph.mon.a.smithi027.stdout: whole_key_filtering: 1 2024-05-21T03:12:23.191 INFO:journalctl@ceph.mon.a.smithi027.stdout: verify_compression: 0 2024-05-21T03:12:23.191 INFO:journalctl@ceph.mon.a.smithi027.stdout: read_amp_bytes_per_bit: 0 2024-05-21T03:12:23.191 INFO:journalctl@ceph.mon.a.smithi027.stdout: format_version: 5 2024-05-21T03:12:23.191 INFO:journalctl@ceph.mon.a.smithi027.stdout: enable_index_compression: 1 2024-05-21T03:12:23.191 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_align: 0 2024-05-21T03:12:23.191 INFO:journalctl@ceph.mon.a.smithi027.stdout: max_auto_readahead_size: 262144 2024-05-21T03:12:23.191 INFO:journalctl@ceph.mon.a.smithi027.stdout: prepopulate_block_cache: 0 2024-05-21T03:12:23.191 INFO:journalctl@ceph.mon.a.smithi027.stdout: initial_auto_readahead_size: 8192 2024-05-21T03:12:23.191 INFO:journalctl@ceph.mon.a.smithi027.stdout: num_file_reads_for_auto_readahead: 2 2024-05-21T03:12:23.191 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.write_buffer_size: 33554432 2024-05-21T03:12:23.192 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.max_write_buffer_number: 2 2024-05-21T03:12:23.192 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.compression: NoCompression 2024-05-21T03:12:23.192 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.bottommost_compression: Disabled 2024-05-21T03:12:23.192 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.prefix_extractor: nullptr 2024-05-21T03:12:23.192 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-21T03:12:23.192 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.num_levels: 7 2024-05-21T03:12:23.192 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-21T03:12:23.192 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-21T03:12:23.192 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-21T03:12:23.192 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-21T03:12:23.192 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-21T03:12:23.192 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-21T03:12:23.192 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-21T03:12:23.192 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-21T03:12:23.193 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-21T03:12:23.193 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-21T03:12:23.193 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-21T03:12:23.193 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-21T03:12:23.193 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-21T03:12:23.193 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.compression_opts.level: 32767 2024-05-21T03:12:23.193 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.compression_opts.strategy: 0 2024-05-21T03:12:23.193 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-21T03:12:23.193 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-21T03:12:23.193 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-21T03:12:23.193 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-21T03:12:23.193 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.compression_opts.enabled: false 2024-05-21T03:12:23.193 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-21T03:12:23.193 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-21T03:12:23.194 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-21T03:12:23.194 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-21T03:12:23.194 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.target_file_size_base: 67108864 2024-05-21T03:12:23.194 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-21T03:12:23.194 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-21T03:12:23.194 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-21T03:12:23.194 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-21T03:12:23.194 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-21T03:12:23.194 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-21T03:12:23.194 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-21T03:12:23.194 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-21T03:12:23.194 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-21T03:12:23.194 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-21T03:12:23.195 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-21T03:12:23.195 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-21T03:12:23.195 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-21T03:12:23.195 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-21T03:12:23.195 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.arena_block_size: 1048576 2024-05-21T03:12:23.195 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-21T03:12:23.195 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-21T03:12:23.195 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.disable_auto_compactions: 0 2024-05-21T03:12:23.195 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-21T03:12:23.195 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-21T03:12:23.195 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-21T03:12:23.195 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-21T03:12:23.195 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-21T03:12:23.195 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-21T03:12:23.196 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-21T03:12:23.196 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-21T03:12:23.196 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-21T03:12:23.196 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-21T03:12:23.196 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-21T03:12:23.196 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.inplace_update_support: 0 2024-05-21T03:12:23.196 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-21T03:12:23.196 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-21T03:12:23.196 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-21T03:12:23.196 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-21T03:12:23.196 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.bloom_locality: 0 2024-05-21T03:12:23.196 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.max_successive_merges: 0 2024-05-21T03:12:23.196 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-21T03:12:23.196 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.paranoid_file_checks: 0 2024-05-21T03:12:23.197 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.force_consistency_checks: 1 2024-05-21T03:12:23.197 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.report_bg_io_stats: 0 2024-05-21T03:12:23.197 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.ttl: 2592000 2024-05-21T03:12:23.197 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-21T03:12:23.197 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-21T03:12:23.197 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-21T03:12:23.197 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.enable_blob_files: false 2024-05-21T03:12:23.197 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.min_blob_size: 0 2024-05-21T03:12:23.197 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.blob_file_size: 268435456 2024-05-21T03:12:23.197 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-21T03:12:23.197 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-21T03:12:23.197 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-21T03:12:23.197 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-21T03:12:23.197 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-21T03:12:23.198 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.blob_file_starting_level: 0 2024-05-21T03:12:23.198 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-21T03:12:23.198 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-21T03:12:23.198 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-21T03:12:23.198 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 183e3ae3-9bae-4a6a-a7dc-c43e3c68f7e2 2024-05-21T03:12:23.198 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716261143153212, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-21T03:12:23.198 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-21T03:12:23.198 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716261143154317, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100928, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 98816, "index_size": 304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14381, "raw_average_key_size": 49, "raw_value_size": 91028, "raw_average_value_size": 314, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716261143, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "183e3ae3-9bae-4a6a-a7dc-c43e3c68f7e2", "db_session_id": "F0CML4FBV8G81ZC9LE5S", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-21T03:12:23.198 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716261143154408, "job": 1, "event": "recovery_finished"} 2024-05-21T03:12:23.198 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-21T03:12:23.198 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-21T03:12:23.198 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558caec28000 2024-05-21T03:12:23.198 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: DB pointer 0x558caec14000 2024-05-21T03:12:23.198 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-21T03:12:23.199 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-21T03:12:23.199 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** DB Stats ** 2024-05-21T03:12:23.199 INFO:journalctl@ceph.mon.a.smithi027.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-21T03:12:23.199 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-21T03:12:23.199 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-21T03:12:23.199 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-21T03:12:23.199 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-21T03:12:23.199 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-21T03:12:23.199 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-21T03:12:23.199 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-05-21T03:12:23.199 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** Compaction Stats [default] ** 2024-05-21T03:12:23.199 INFO:journalctl@ceph.mon.a.smithi027.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-21T03:12:23.199 INFO:journalctl@ceph.mon.a.smithi027.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-21T03:12:23.199 INFO:journalctl@ceph.mon.a.smithi027.stdout: L0 2/0 100.42 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-21T03:12:23.199 INFO:journalctl@ceph.mon.a.smithi027.stdout: Sum 2/0 100.42 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-21T03:12:23.200 INFO:journalctl@ceph.mon.a.smithi027.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-21T03:12:23.200 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-05-21T03:12:23.200 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** Compaction Stats [default] ** 2024-05-21T03:12:23.200 INFO:journalctl@ceph.mon.a.smithi027.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-21T03:12:23.200 INFO:journalctl@ceph.mon.a.smithi027.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-21T03:12:23.200 INFO:journalctl@ceph.mon.a.smithi027.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 140.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-21T03:12:23.200 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-05-21T03:12:23.200 INFO:journalctl@ceph.mon.a.smithi027.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-21T03:12:23.200 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-05-21T03:12:23.200 INFO:journalctl@ceph.mon.a.smithi027.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-21T03:12:23.200 INFO:journalctl@ceph.mon.a.smithi027.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-21T03:12:23.200 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-21T03:12:23.200 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-21T03:12:23.200 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-21T03:12:23.201 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-21T03:12:23.201 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative compaction: 0.00 GB write, 29.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-21T03:12:23.201 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval compaction: 0.00 GB write, 29.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-21T03:12:23.201 INFO:journalctl@ceph.mon.a.smithi027.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-21T03:12:23.201 INFO:journalctl@ceph.mon.a.smithi027.stdout: Block cache BinnedLRUCache@0x558caeb0b1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-21T03:12:23.201 INFO:journalctl@ceph.mon.a.smithi027.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-21T03:12:23.201 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-05-21T03:12:23.201 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-21T03:12:23.201 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: starting mon.a rank 0 at public addrs [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] at bind addrs [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid c9ca94be-171f-11ef-bc9a-c7b262605968 2024-05-21T03:12:23.201 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: mon.a@-1(???) e1 preinit fsid c9ca94be-171f-11ef-bc9a-c7b262605968 2024-05-21T03:12:23.201 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: mon.a@-1(???).mds e1 new map 2024-05-21T03:12:23.201 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: mon.a@-1(???).mds e1 print_map 2024-05-21T03:12:23.201 INFO:journalctl@ceph.mon.a.smithi027.stdout: e1 2024-05-21T03:12:23.201 INFO:journalctl@ceph.mon.a.smithi027.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-21T03:12:23.202 INFO:journalctl@ceph.mon.a.smithi027.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-21T03:12:23.202 INFO:journalctl@ceph.mon.a.smithi027.stdout: legacy client fscid: -1 2024-05-21T03:12:23.202 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-05-21T03:12:23.202 INFO:journalctl@ceph.mon.a.smithi027.stdout: No filesystems configured 2024-05-21T03:12:23.202 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-21T03:12:23.202 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-21T03:12:23.202 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-21T03:12:23.202 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-21T03:12:23.202 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-21T03:12:23.202 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 bash[32160]: eb3a7e6bf9c301052ffeb751719914039918507b8941d4bdcfebc68f6689b284 2024-05-21T03:12:23.202 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 systemd[1]: Started Ceph mon.a for c9ca94be-171f-11ef-bc9a-c7b262605968. 2024-05-21T03:12:23.202 INFO:teuthology.orchestra.run.smithi027.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-21T03:12:23.603 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-21T03:12:23.603 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: monmap epoch 1 2024-05-21T03:12:23.604 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: fsid c9ca94be-171f-11ef-bc9a-c7b262605968 2024-05-21T03:12:23.604 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: last_changed 2024-05-21T03:12:14.443931+0000 2024-05-21T03:12:23.604 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: created 2024-05-21T03:12:14.443931+0000 2024-05-21T03:12:23.604 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: min_mon_release 19 (squid) 2024-05-21T03:12:23.604 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: election_strategy: 1 2024-05-21T03:12:23.604 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-05-21T03:12:23.604 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: fsmap 2024-05-21T03:12:23.604 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: osdmap e1: 0 total, 0 up, 0 in 2024-05-21T03:12:23.604 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:23 smithi027 ceph-mon[32179]: mgrmap e1: no daemons active 2024-05-21T03:12:24.407 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.830+0000 7ff295120700 1 Processor -- start 2024-05-21T03:12:24.407 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.830+0000 7ff295120700 1 -- start start 2024-05-21T03:12:24.407 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.831+0000 7ff295120700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff290108030 0x7ff290108450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:24.407 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.831+0000 7ff295120700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff290108a20 con 0x7ff290108030 2024-05-21T03:12:24.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.831+0000 7ff28ed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff290108030 0x7ff290108450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:24.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.831+0000 7ff28ed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff290108030 0x7ff290108450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56012/0 (socket says 172.21.15.27:56012) 2024-05-21T03:12:24.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.831+0000 7ff28ed9d700 1 -- 172.21.15.27:0/970452809 learned_addr learned my addr 172.21.15.27:0/970452809 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:24.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.831+0000 7ff28ed9d700 1 -- 172.21.15.27:0/970452809 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff290109230 con 0x7ff290108030 2024-05-21T03:12:24.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.832+0000 7ff28ed9d700 1 --2- 172.21.15.27:0/970452809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff290108030 0x7ff290108450 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff2800098a0 tx=0x7ff280009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a631152f28129c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:24.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.832+0000 7ff28dd9b700 1 -- 172.21.15.27:0/970452809 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff280004160 con 0x7ff290108030 2024-05-21T03:12:24.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.832+0000 7ff28dd9b700 1 -- 172.21.15.27:0/970452809 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff280004300 con 0x7ff290108030 2024-05-21T03:12:24.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.832+0000 7ff28dd9b700 1 -- 172.21.15.27:0/970452809 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff280011af0 con 0x7ff290108030 2024-05-21T03:12:24.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.832+0000 7ff295120700 1 -- 172.21.15.27:0/970452809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff290108030 msgr2=0x7ff290108450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:24.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.832+0000 7ff295120700 1 --2- 172.21.15.27:0/970452809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff290108030 0x7ff290108450 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff2800098a0 tx=0x7ff280009bb0 comp rx=0 tx=0).stop 2024-05-21T03:12:24.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.833+0000 7ff295120700 1 -- 172.21.15.27:0/970452809 shutdown_connections 2024-05-21T03:12:24.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.833+0000 7ff295120700 1 --2- 172.21.15.27:0/970452809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff290108030 0x7ff290108450 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:24.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.833+0000 7ff295120700 1 -- 172.21.15.27:0/970452809 >> 172.21.15.27:0/970452809 conn(0x7ff290103610 msgr2=0x7ff290105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:24.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.833+0000 7ff295120700 1 -- 172.21.15.27:0/970452809 shutdown_connections 2024-05-21T03:12:24.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.833+0000 7ff295120700 1 -- 172.21.15.27:0/970452809 wait complete. 2024-05-21T03:12:24.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.833+0000 7ff295120700 1 Processor -- start 2024-05-21T03:12:24.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.834+0000 7ff295120700 1 -- start start 2024-05-21T03:12:24.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.834+0000 7ff295120700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff290108030 0x7ff29007ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:24.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.834+0000 7ff295120700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff29010c5e0 con 0x7ff290108030 2024-05-21T03:12:24.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.834+0000 7ff28ed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff290108030 0x7ff29007ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:24.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.834+0000 7ff28ed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff290108030 0x7ff29007ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56016/0 (socket says 172.21.15.27:56016) 2024-05-21T03:12:24.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.834+0000 7ff28ed9d700 1 -- 172.21.15.27:0/3953615173 learned_addr learned my addr 172.21.15.27:0/3953615173 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:24.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.834+0000 7ff28ed9d700 1 -- 172.21.15.27:0/3953615173 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff29019fd80 con 0x7ff290108030 2024-05-21T03:12:24.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.834+0000 7ff28ed9d700 1 --2- 172.21.15.27:0/3953615173 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff290108030 0x7ff29007ca70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff280003f40 tx=0x7ff280011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:24.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.835+0000 7ff287fff700 1 -- 172.21.15.27:0/3953615173 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2800038b0 con 0x7ff290108030 2024-05-21T03:12:24.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.835+0000 7ff295120700 1 -- 172.21.15.27:0/3953615173 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff29019ff50 con 0x7ff290108030 2024-05-21T03:12:24.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.835+0000 7ff287fff700 1 -- 172.21.15.27:0/3953615173 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff280003a50 con 0x7ff290108030 2024-05-21T03:12:24.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.835+0000 7ff287fff700 1 -- 172.21.15.27:0/3953615173 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff28001a5c0 con 0x7ff290108030 2024-05-21T03:12:24.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.835+0000 7ff295120700 1 -- 172.21.15.27:0/3953615173 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2901a02f0 con 0x7ff290108030 2024-05-21T03:12:24.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.835+0000 7ff287fff700 1 -- 172.21.15.27:0/3953615173 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff28001ac50 con 0x7ff290108030 2024-05-21T03:12:24.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.835+0000 7ff287fff700 1 -- 172.21.15.27:0/3953615173 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff280019700 con 0x7ff290108030 2024-05-21T03:12:24.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.836+0000 7ff295120700 1 -- 172.21.15.27:0/3953615173 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff290064ba0 con 0x7ff290108030 2024-05-21T03:12:24.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.838+0000 7ff287fff700 1 -- 172.21.15.27:0/3953615173 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7ff280019900 con 0x7ff290108030 2024-05-21T03:12:24.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.909+0000 7ff295120700 1 -- 172.21.15.27:0/3953615173 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7ff2901a06d0 con 0x7ff290108030 2024-05-21T03:12:24.411 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.910+0000 7ff287fff700 1 -- 172.21.15.27:0/3953615173 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7ff280019ae0 con 0x7ff290108030 2024-05-21T03:12:24.411 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.911+0000 7ff295120700 1 -- 172.21.15.27:0/3953615173 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff290108030 msgr2=0x7ff29007ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:24.411 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.911+0000 7ff295120700 1 --2- 172.21.15.27:0/3953615173 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff290108030 0x7ff29007ca70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff280003f40 tx=0x7ff280011890 comp rx=0 tx=0).stop 2024-05-21T03:12:24.411 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.911+0000 7ff295120700 1 -- 172.21.15.27:0/3953615173 shutdown_connections 2024-05-21T03:12:24.411 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.911+0000 7ff295120700 1 --2- 172.21.15.27:0/3953615173 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff290108030 0x7ff29007ca70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:24.411 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.911+0000 7ff295120700 1 -- 172.21.15.27:0/3953615173 >> 172.21.15.27:0/3953615173 conn(0x7ff290103610 msgr2=0x7ff290104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:24.411 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.911+0000 7ff295120700 1 -- 172.21.15.27:0/3953615173 shutdown_connections 2024-05-21T03:12:24.411 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:23.911+0000 7ff295120700 1 -- 172.21.15.27:0/3953615173 wait complete. 2024-05-21T03:12:24.411 INFO:teuthology.orchestra.run.smithi027.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-21T03:12:24.425 INFO:teuthology.orchestra.run.smithi027.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-21T03:12:24.425 INFO:teuthology.orchestra.run.smithi027.stdout:Creating mgr... 2024-05-21T03:12:24.425 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-21T03:12:24.426 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-21T03:12:24.691 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mgr.x 2024-05-21T03:12:24.691 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mgr.x.service: Unit ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mgr.x.service not loaded. 2024-05-21T03:12:24.847 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c9ca94be-171f-11ef-bc9a-c7b262605968.target.wants/ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@.service. 2024-05-21T03:12:25.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:24 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3953615173' entity='client.admin' 2024-05-21T03:12:25.177 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:25 smithi027 podman[32391]: 2024-05-21 03:12:25.10356681 +0000 UTC m=+0.116883923 container create 41efc524711bbeda604cbbf4d11631ed110b75a977140138b00ad7924fc722c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T03:12:25.402 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-21T03:12:25.402 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2024-05-21T03:12:25.410 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-21T03:12:25.410 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-05-21T03:12:25.410 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2024-05-21T03:12:25.410 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to enable service . firewalld.service is not available 2024-05-21T03:12:25.419 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-21T03:12:25.419 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2024-05-21T03:12:25.426 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-21T03:12:25.427 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-05-21T03:12:25.427 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2024-05-21T03:12:25.427 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-21T03:12:25.427 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr to start... 2024-05-21T03:12:25.427 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr... 2024-05-21T03:12:25.476 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:25 smithi027 podman[32391]: 2024-05-21 03:12:25.314185022 +0000 UTC m=+0.327502135 container init 41efc524711bbeda604cbbf4d11631ed110b75a977140138b00ad7924fc722c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea) 2024-05-21T03:12:25.476 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:25 smithi027 podman[32391]: 2024-05-21 03:12:25.31700127 +0000 UTC m=+0.330318390 container start 41efc524711bbeda604cbbf4d11631ed110b75a977140138b00ad7924fc722c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-05-21T03:12:25.476 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:25 smithi027 bash[32391]: 41efc524711bbeda604cbbf4d11631ed110b75a977140138b00ad7924fc722c3 2024-05-21T03:12:25.476 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:25 smithi027 systemd[1]: Started Ceph mgr.x for c9ca94be-171f-11ef-bc9a-c7b262605968. 2024-05-21T03:12:25.476 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:25 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:25.476+0000 7f1dcf6a0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T03:12:25.759 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:25 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:25.674+0000 7f1dcf6a0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T03:12:26.111 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:25 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3749599835' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-21T03:12:26.111 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:25 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:25.781+0000 7f1dcf6a0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T03:12:26.111 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:26 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:26.111+0000 7f1dcf6a0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T03:12:26.426 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:26 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:26.200+0000 7f1dcf6a0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T03:12:26.525 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-05-21T03:12:26.525 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-05-21T03:12:26.526 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "c9ca94be-171f-11ef-bc9a-c7b262605968", 2024-05-21T03:12:26.526 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2024-05-21T03:12:26.526 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-21T03:12:26.526 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-21T03:12:26.526 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-21T03:12:26.526 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-05-21T03:12:26.526 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-21T03:12:26.526 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-21T03:12:26.526 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2024-05-21T03:12:26.527 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-05-21T03:12:26.527 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-21T03:12:26.527 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2024-05-21T03:12:26.527 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-05-21T03:12:26.527 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-21T03:12:26.527 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-21T03:12:26.527 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T03:12:26.527 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-21T03:12:26.527 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-21T03:12:26.527 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-05-21T03:12:26.527 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-21T03:12:26.527 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T03:12:26.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-21T03:12:26.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-21T03:12:26.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-21T03:12:26.536 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-21T03:12:26.536 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-21T03:12:26.536 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-21T03:12:26.536 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-05-21T03:12:26.536 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-21T03:12:26.536 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-21T03:12:26.536 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-21T03:12:26.536 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-21T03:12:26.536 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-21T03:12:26.536 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-21T03:12:26.536 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-21T03:12:26.537 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-21T03:12:26.537 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-21T03:12:26.537 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-05-21T03:12:26.537 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-21T03:12:26.537 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T03:12:26.537 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-21T03:12:26.537 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-21T03:12:26.537 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-05-21T03:12:26.537 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-21T03:12:26.537 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-21T03:12:26.537 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-21T03:12:26.537 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-21T03:12:26.538 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-21T03:12:26.538 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-21T03:12:26.538 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2024-05-21T03:12:26.538 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-05-21T03:12:26.538 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T03:12:26.538 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-05-21T03:12:26.538 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-21T03:12:26.538 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T03:12:26.538 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2024-05-21T03:12:18.217740+0000", 2024-05-21T03:12:26.538 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T03:12:26.538 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-05-21T03:12:26.538 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-21T03:12:26.539 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-05-21T03:12:26.539 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.863+0000 7efd604aa700 1 Processor -- start 2024-05-21T03:12:26.539 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.863+0000 7efd604aa700 1 -- start start 2024-05-21T03:12:26.539 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.863+0000 7efd604aa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd5807edf0 0x7efd5807d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:26.539 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.863+0000 7efd604aa700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efd5807d8a0 con 0x7efd5807edf0 2024-05-21T03:12:26.539 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.864+0000 7efd5e246700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd5807edf0 0x7efd5807d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:26.539 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.864+0000 7efd5e246700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd5807edf0 0x7efd5807d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56040/0 (socket says 172.21.15.27:56040) 2024-05-21T03:12:26.539 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.864+0000 7efd5e246700 1 -- 172.21.15.27:0/1884500068 learned_addr learned my addr 172.21.15.27:0/1884500068 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:26.539 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.864+0000 7efd5e246700 1 -- 172.21.15.27:0/1884500068 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd5807da20 con 0x7efd5807edf0 2024-05-21T03:12:26.539 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.864+0000 7efd5e246700 1 --2- 172.21.15.27:0/1884500068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd5807edf0 0x7efd5807d360 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efd4c0098a0 tx=0x7efd4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e55892d89ee9f41 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:26.539 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.864+0000 7efd5d244700 1 -- 172.21.15.27:0/1884500068 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd4c004160 con 0x7efd5807edf0 2024-05-21T03:12:26.539 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.864+0000 7efd5d244700 1 -- 172.21.15.27:0/1884500068 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efd4c004300 con 0x7efd5807edf0 2024-05-21T03:12:26.540 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.864+0000 7efd5d244700 1 -- 172.21.15.27:0/1884500068 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd4c011af0 con 0x7efd5807edf0 2024-05-21T03:12:26.540 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.865+0000 7efd604aa700 1 -- 172.21.15.27:0/1884500068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd5807edf0 msgr2=0x7efd5807d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:26.540 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.865+0000 7efd604aa700 1 --2- 172.21.15.27:0/1884500068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd5807edf0 0x7efd5807d360 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efd4c0098a0 tx=0x7efd4c009bb0 comp rx=0 tx=0).stop 2024-05-21T03:12:26.540 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.865+0000 7efd604aa700 1 -- 172.21.15.27:0/1884500068 shutdown_connections 2024-05-21T03:12:26.540 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.865+0000 7efd604aa700 1 --2- 172.21.15.27:0/1884500068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd5807edf0 0x7efd5807d360 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:26.540 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.865+0000 7efd604aa700 1 -- 172.21.15.27:0/1884500068 >> 172.21.15.27:0/1884500068 conn(0x7efd581047f0 msgr2=0x7efd58106c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:26.540 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.865+0000 7efd604aa700 1 -- 172.21.15.27:0/1884500068 shutdown_connections 2024-05-21T03:12:26.540 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.865+0000 7efd604aa700 1 -- 172.21.15.27:0/1884500068 wait complete. 2024-05-21T03:12:26.540 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.866+0000 7efd604aa700 1 Processor -- start 2024-05-21T03:12:26.540 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.866+0000 7efd604aa700 1 -- start start 2024-05-21T03:12:26.540 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.866+0000 7efd604aa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd5807edf0 0x7efd5819f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:26.540 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.866+0000 7efd604aa700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efd5810f620 con 0x7efd5807edf0 2024-05-21T03:12:26.540 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.866+0000 7efd5e246700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd5807edf0 0x7efd5819f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:26.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.866+0000 7efd5e246700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd5807edf0 0x7efd5819f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56042/0 (socket says 172.21.15.27:56042) 2024-05-21T03:12:26.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.866+0000 7efd5e246700 1 -- 172.21.15.27:0/3749599835 learned_addr learned my addr 172.21.15.27:0/3749599835 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:26.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.866+0000 7efd5e246700 1 -- 172.21.15.27:0/3749599835 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd5819f6f0 con 0x7efd5807edf0 2024-05-21T03:12:26.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.867+0000 7efd5e246700 1 --2- 172.21.15.27:0/3749599835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd5807edf0 0x7efd5819f1b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efd4c011ac0 tx=0x7efd4c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:26.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.867+0000 7efd4b7fe700 1 -- 172.21.15.27:0/3749599835 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd4c003b20 con 0x7efd5807edf0 2024-05-21T03:12:26.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.867+0000 7efd4b7fe700 1 -- 172.21.15.27:0/3749599835 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efd4c01a8c0 con 0x7efd5807edf0 2024-05-21T03:12:26.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.867+0000 7efd4b7fe700 1 -- 172.21.15.27:0/3749599835 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd4c0248d0 con 0x7efd5807edf0 2024-05-21T03:12:26.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.867+0000 7efd604aa700 1 -- 172.21.15.27:0/3749599835 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd5819f980 con 0x7efd5807edf0 2024-05-21T03:12:26.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.867+0000 7efd604aa700 1 -- 172.21.15.27:0/3749599835 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd581a2670 con 0x7efd5807edf0 2024-05-21T03:12:26.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.867+0000 7efd4b7fe700 1 -- 172.21.15.27:0/3749599835 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efd4c022020 con 0x7efd5807edf0 2024-05-21T03:12:26.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.867+0000 7efd4b7fe700 1 -- 172.21.15.27:0/3749599835 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efd4c0193c0 con 0x7efd5807edf0 2024-05-21T03:12:26.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.868+0000 7efd604aa700 1 -- 172.21.15.27:0/3749599835 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd58064ba0 con 0x7efd5807edf0 2024-05-21T03:12:26.541 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.870+0000 7efd4b7fe700 1 -- 172.21.15.27:0/3749599835 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7efd4c042ec0 con 0x7efd5807edf0 2024-05-21T03:12:26.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.946+0000 7efd604aa700 1 -- 172.21.15.27:0/3749599835 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7efd581a2bb0 con 0x7efd5807edf0 2024-05-21T03:12:26.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.946+0000 7efd4b7fe700 1 -- 172.21.15.27:0/3749599835 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7efd4c031020 con 0x7efd5807edf0 2024-05-21T03:12:26.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.947+0000 7efd604aa700 1 -- 172.21.15.27:0/3749599835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd5807edf0 msgr2=0x7efd5819f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:26.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.947+0000 7efd604aa700 1 --2- 172.21.15.27:0/3749599835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd5807edf0 0x7efd5819f1b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efd4c011ac0 tx=0x7efd4c011ba0 comp rx=0 tx=0).stop 2024-05-21T03:12:26.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.947+0000 7efd604aa700 1 -- 172.21.15.27:0/3749599835 shutdown_connections 2024-05-21T03:12:26.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.947+0000 7efd604aa700 1 --2- 172.21.15.27:0/3749599835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd5807edf0 0x7efd5819f1b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:26.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.947+0000 7efd604aa700 1 -- 172.21.15.27:0/3749599835 >> 172.21.15.27:0/3749599835 conn(0x7efd581047f0 msgr2=0x7efd581052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:26.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.947+0000 7efd604aa700 1 -- 172.21.15.27:0/3749599835 shutdown_connections 2024-05-21T03:12:26.542 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:25.947+0000 7efd604aa700 1 -- 172.21.15.27:0/3749599835 wait complete. 2024-05-21T03:12:26.542 INFO:teuthology.orchestra.run.smithi027.stdout:mgr not available, waiting (1/15)... 2024-05-21T03:12:27.826 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:27 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:27.475+0000 7f1dcf6a0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T03:12:27.826 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:27 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:27.650+0000 7f1dcf6a0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T03:12:27.826 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:27 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:27.826+0000 7f1dcf6a0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T03:12:28.426 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:28 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:28.128+0000 7f1dcf6a0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T03:12:28.427 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:28 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:28.223+0000 7f1dcf6a0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T03:12:28.763 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:28 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:28.435+0000 7f1dcf6a0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T03:12:28.763 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:28 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:28.645+0000 7f1dcf6a0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T03:12:29.396 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:29 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/407284618' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-21T03:12:29.396 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:29 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:29.320+0000 7f1dcf6a0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T03:12:29.657 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:29 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:29.559+0000 7f1dcf6a0200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-21T03:12:29.760 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-05-21T03:12:29.761 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-05-21T03:12:29.761 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "c9ca94be-171f-11ef-bc9a-c7b262605968", 2024-05-21T03:12:29.761 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2024-05-21T03:12:29.761 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-21T03:12:29.761 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-21T03:12:29.761 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-21T03:12:29.762 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-05-21T03:12:29.762 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-21T03:12:29.762 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-21T03:12:29.762 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2024-05-21T03:12:29.762 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-05-21T03:12:29.762 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-21T03:12:29.763 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2024-05-21T03:12:29.763 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-05-21T03:12:29.763 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-21T03:12:29.763 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-21T03:12:29.763 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T03:12:29.763 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-21T03:12:29.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-21T03:12:29.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-05-21T03:12:29.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-21T03:12:29.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T03:12:29.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-21T03:12:29.764 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-21T03:12:29.780 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-21T03:12:29.780 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-21T03:12:29.780 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-21T03:12:29.781 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-21T03:12:29.781 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-05-21T03:12:29.781 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-21T03:12:29.781 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-21T03:12:29.781 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-21T03:12:29.782 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-21T03:12:29.782 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-21T03:12:29.782 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-21T03:12:29.782 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-21T03:12:29.782 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-21T03:12:29.783 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-21T03:12:29.783 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-05-21T03:12:29.783 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-21T03:12:29.783 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T03:12:29.783 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-21T03:12:29.784 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-21T03:12:29.784 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-05-21T03:12:29.784 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-21T03:12:29.784 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-21T03:12:29.784 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-21T03:12:29.785 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-21T03:12:29.785 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-21T03:12:29.785 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-21T03:12:29.785 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2024-05-21T03:12:29.785 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-05-21T03:12:29.786 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T03:12:29.786 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-05-21T03:12:29.786 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-21T03:12:29.786 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T03:12:29.787 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2024-05-21T03:12:18.217740+0000", 2024-05-21T03:12:29.787 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T03:12:29.787 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-05-21T03:12:29.787 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-21T03:12:29.787 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-05-21T03:12:29.788 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.968+0000 7f059cdb3700 1 Processor -- start 2024-05-21T03:12:29.788 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.968+0000 7f059cdb3700 1 -- start start 2024-05-21T03:12:29.788 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.968+0000 7f059cdb3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0598109be0 0x7f059810bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:29.788 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.969+0000 7f059cdb3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f059807ce90 con 0x7f0598109be0 2024-05-21T03:12:29.789 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.969+0000 7f059659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0598109be0 0x7f059810bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:29.789 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.969+0000 7f059659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0598109be0 0x7f059810bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56056/0 (socket says 172.21.15.27:56056) 2024-05-21T03:12:29.789 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.969+0000 7f059659c700 1 -- 172.21.15.27:0/96209718 learned_addr learned my addr 172.21.15.27:0/96209718 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:29.789 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.969+0000 7f059659c700 1 -- 172.21.15.27:0/96209718 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f059810c510 con 0x7f0598109be0 2024-05-21T03:12:29.789 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.969+0000 7f059659c700 1 --2- 172.21.15.27:0/96209718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0598109be0 0x7f059810bfd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f05880098a0 tx=0x7f0588009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea615c7e927d3a9e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:29.790 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.969+0000 7f059559a700 1 -- 172.21.15.27:0/96209718 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0588004160 con 0x7f0598109be0 2024-05-21T03:12:29.790 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.969+0000 7f059559a700 1 -- 172.21.15.27:0/96209718 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0588004300 con 0x7f0598109be0 2024-05-21T03:12:29.790 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.969+0000 7f059559a700 1 -- 172.21.15.27:0/96209718 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0588011af0 con 0x7f0598109be0 2024-05-21T03:12:29.790 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.970+0000 7f059cdb3700 1 -- 172.21.15.27:0/96209718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0598109be0 msgr2=0x7f059810bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:29.791 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.970+0000 7f059cdb3700 1 --2- 172.21.15.27:0/96209718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0598109be0 0x7f059810bfd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f05880098a0 tx=0x7f0588009bb0 comp rx=0 tx=0).stop 2024-05-21T03:12:29.791 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.970+0000 7f059cdb3700 1 -- 172.21.15.27:0/96209718 shutdown_connections 2024-05-21T03:12:29.791 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.970+0000 7f059cdb3700 1 --2- 172.21.15.27:0/96209718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0598109be0 0x7f059810bfd0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:29.791 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.970+0000 7f059cdb3700 1 -- 172.21.15.27:0/96209718 >> 172.21.15.27:0/96209718 conn(0x7f0598103610 msgr2=0x7f0598105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:29.791 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.970+0000 7f059cdb3700 1 -- 172.21.15.27:0/96209718 shutdown_connections 2024-05-21T03:12:29.792 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.970+0000 7f059cdb3700 1 -- 172.21.15.27:0/96209718 wait complete. 2024-05-21T03:12:29.792 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.971+0000 7f059cdb3700 1 Processor -- start 2024-05-21T03:12:29.792 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.971+0000 7f059cdb3700 1 -- start start 2024-05-21T03:12:29.792 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.971+0000 7f059cdb3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0598109be0 0x7f059819cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:29.792 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.971+0000 7f059cdb3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f059810d4d0 con 0x7f0598109be0 2024-05-21T03:12:29.793 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.972+0000 7f059659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0598109be0 0x7f059819cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:29.793 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.972+0000 7f059659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0598109be0 0x7f059819cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56070/0 (socket says 172.21.15.27:56070) 2024-05-21T03:12:29.793 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.972+0000 7f059659c700 1 -- 172.21.15.27:0/407284618 learned_addr learned my addr 172.21.15.27:0/407284618 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:29.793 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.972+0000 7f059659c700 1 -- 172.21.15.27:0/407284618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f059819d500 con 0x7f0598109be0 2024-05-21T03:12:29.793 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.972+0000 7f059659c700 1 --2- 172.21.15.27:0/407284618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0598109be0 0x7f059819cfc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0588003f40 tx=0x7f0588011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:29.794 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.972+0000 7f05877fe700 1 -- 172.21.15.27:0/407284618 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05880038b0 con 0x7f0598109be0 2024-05-21T03:12:29.794 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.972+0000 7f05877fe700 1 -- 172.21.15.27:0/407284618 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0588003a50 con 0x7f0598109be0 2024-05-21T03:12:29.794 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.972+0000 7f059cdb3700 1 -- 172.21.15.27:0/407284618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f059819d790 con 0x7f0598109be0 2024-05-21T03:12:29.794 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.972+0000 7f05877fe700 1 -- 172.21.15.27:0/407284618 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f058801a5c0 con 0x7f0598109be0 2024-05-21T03:12:29.795 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.972+0000 7f059cdb3700 1 -- 172.21.15.27:0/407284618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05981a0480 con 0x7f0598109be0 2024-05-21T03:12:29.795 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.973+0000 7f05877fe700 1 -- 172.21.15.27:0/407284618 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f058801ac50 con 0x7f0598109be0 2024-05-21T03:12:29.795 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.973+0000 7f05877fe700 1 -- 172.21.15.27:0/407284618 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0588019700 con 0x7f0598109be0 2024-05-21T03:12:29.795 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.973+0000 7f059cdb3700 1 -- 172.21.15.27:0/407284618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0598064ba0 con 0x7f0598109be0 2024-05-21T03:12:29.795 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:28.975+0000 7f05877fe700 1 -- 172.21.15.27:0/407284618 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f0588019900 con 0x7f0598109be0 2024-05-21T03:12:29.796 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:29.091+0000 7f059cdb3700 1 -- 172.21.15.27:0/407284618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f05981a0650 con 0x7f0598109be0 2024-05-21T03:12:29.796 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:29.092+0000 7f05877fe700 1 -- 172.21.15.27:0/407284618 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f0588019ae0 con 0x7f0598109be0 2024-05-21T03:12:29.796 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:29.093+0000 7f059cdb3700 1 -- 172.21.15.27:0/407284618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0598109be0 msgr2=0x7f059819cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:29.796 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:29.093+0000 7f059cdb3700 1 --2- 172.21.15.27:0/407284618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0598109be0 0x7f059819cfc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0588003f40 tx=0x7f0588011890 comp rx=0 tx=0).stop 2024-05-21T03:12:29.796 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:29.093+0000 7f059cdb3700 1 -- 172.21.15.27:0/407284618 shutdown_connections 2024-05-21T03:12:29.797 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:29.093+0000 7f059cdb3700 1 --2- 172.21.15.27:0/407284618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0598109be0 0x7f059819cfc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:29.797 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:29.093+0000 7f059cdb3700 1 -- 172.21.15.27:0/407284618 >> 172.21.15.27:0/407284618 conn(0x7f0598103610 msgr2=0x7f05981040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:29.797 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:29.093+0000 7f059cdb3700 1 -- 172.21.15.27:0/407284618 shutdown_connections 2024-05-21T03:12:29.797 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:29.093+0000 7f059cdb3700 1 -- 172.21.15.27:0/407284618 wait complete. 2024-05-21T03:12:29.797 INFO:teuthology.orchestra.run.smithi027.stdout:mgr not available, waiting (2/15)... 2024-05-21T03:12:30.387 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:30 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:30.009+0000 7f1dcf6a0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T03:12:30.387 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:30 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:30.110+0000 7f1dcf6a0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T03:12:30.676 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:30 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:30.519+0000 7f1dcf6a0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T03:12:31.176 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:30 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:30.891+0000 7f1dcf6a0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T03:12:31.177 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:30 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:30.986+0000 7f1dcf6a0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T03:12:31.676 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:31 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:31.241+0000 7f1dcf6a0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T03:12:31.677 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:31 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:31.380+0000 7f1dcf6a0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T03:12:31.677 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:31 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:31.480+0000 7f1dcf6a0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T03:12:32.130 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:31 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:31.828+0000 7f1dcf6a0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T03:12:32.130 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:31 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:31.924+0000 7f1dcf6a0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T03:12:32.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:32 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2027252544' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-21T03:12:32.927 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:32 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:32.573+0000 7f1dcf6a0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T03:12:32.927 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:32 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:32.800+0000 7f1dcf6a0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T03:12:32.930 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-05-21T03:12:32.940 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-05-21T03:12:32.940 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "c9ca94be-171f-11ef-bc9a-c7b262605968", 2024-05-21T03:12:32.940 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2024-05-21T03:12:32.940 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-21T03:12:32.940 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-21T03:12:32.940 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-21T03:12:32.941 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-05-21T03:12:32.941 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-21T03:12:32.941 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-21T03:12:32.941 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2024-05-21T03:12:32.941 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-05-21T03:12:32.941 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-21T03:12:32.941 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2024-05-21T03:12:32.941 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-05-21T03:12:32.941 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-05-21T03:12:32.941 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-21T03:12:32.941 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T03:12:32.941 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-21T03:12:32.941 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-21T03:12:32.942 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-05-21T03:12:32.942 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-21T03:12:32.942 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T03:12:32.942 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-21T03:12:32.942 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-21T03:12:32.942 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-21T03:12:32.942 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-21T03:12:32.942 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-21T03:12:32.942 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-21T03:12:32.942 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-05-21T03:12:32.942 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-21T03:12:32.942 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-21T03:12:32.942 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-21T03:12:32.943 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-21T03:12:32.943 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-21T03:12:32.943 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-21T03:12:32.943 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-21T03:12:32.943 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-21T03:12:32.943 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-21T03:12:32.943 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-05-21T03:12:32.943 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-21T03:12:32.943 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T03:12:32.943 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-21T03:12:32.943 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-21T03:12:32.943 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-05-21T03:12:32.943 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-21T03:12:32.943 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-21T03:12:32.943 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-21T03:12:32.944 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-21T03:12:32.944 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-21T03:12:32.944 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-21T03:12:32.944 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2024-05-21T03:12:32.944 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-05-21T03:12:32.944 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T03:12:32.944 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-05-21T03:12:32.944 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-21T03:12:32.944 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T03:12:32.944 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2024-05-21T03:12:18.217740+0000", 2024-05-21T03:12:32.944 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T03:12:32.944 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-05-21T03:12:32.944 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-21T03:12:32.944 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-05-21T03:12:32.945 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.235+0000 7fbd1d520700 1 Processor -- start 2024-05-21T03:12:32.945 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.235+0000 7fbd1d520700 1 -- start start 2024-05-21T03:12:32.945 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.236+0000 7fbd1d520700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd18107a70 0x7fbd18107e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:32.945 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.236+0000 7fbd1d520700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1807ce50 con 0x7fbd18107a70 2024-05-21T03:12:32.945 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.236+0000 7fbd16ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd18107a70 0x7fbd18107e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:32.945 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.236+0000 7fbd16ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd18107a70 0x7fbd18107e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56080/0 (socket says 172.21.15.27:56080) 2024-05-21T03:12:32.945 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.236+0000 7fbd16ffd700 1 -- 172.21.15.27:0/499941560 learned_addr learned my addr 172.21.15.27:0/499941560 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:32.945 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.236+0000 7fbd16ffd700 1 -- 172.21.15.27:0/499941560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd181083d0 con 0x7fbd18107a70 2024-05-21T03:12:32.945 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.236+0000 7fbd16ffd700 1 --2- 172.21.15.27:0/499941560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd18107a70 0x7fbd18107e90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbd080098a0 tx=0x7fbd08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36697dc0b5d8d7e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:32.945 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.236+0000 7fbd15ffb700 1 -- 172.21.15.27:0/499941560 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd08004160 con 0x7fbd18107a70 2024-05-21T03:12:32.945 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.237+0000 7fbd15ffb700 1 -- 172.21.15.27:0/499941560 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbd08004300 con 0x7fbd18107a70 2024-05-21T03:12:32.945 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.237+0000 7fbd15ffb700 1 -- 172.21.15.27:0/499941560 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd08011af0 con 0x7fbd18107a70 2024-05-21T03:12:32.945 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.237+0000 7fbd1d520700 1 -- 172.21.15.27:0/499941560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd18107a70 msgr2=0x7fbd18107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:32.946 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.237+0000 7fbd1d520700 1 --2- 172.21.15.27:0/499941560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd18107a70 0x7fbd18107e90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbd080098a0 tx=0x7fbd08009bb0 comp rx=0 tx=0).stop 2024-05-21T03:12:32.946 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.237+0000 7fbd1d520700 1 -- 172.21.15.27:0/499941560 shutdown_connections 2024-05-21T03:12:32.946 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.237+0000 7fbd1d520700 1 --2- 172.21.15.27:0/499941560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd18107a70 0x7fbd18107e90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:32.946 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.237+0000 7fbd1d520700 1 -- 172.21.15.27:0/499941560 >> 172.21.15.27:0/499941560 conn(0x7fbd18103690 msgr2=0x7fbd18105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:32.946 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.237+0000 7fbd1d520700 1 -- 172.21.15.27:0/499941560 shutdown_connections 2024-05-21T03:12:32.946 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.237+0000 7fbd1d520700 1 -- 172.21.15.27:0/499941560 wait complete. 2024-05-21T03:12:32.946 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.238+0000 7fbd1d520700 1 Processor -- start 2024-05-21T03:12:32.946 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.238+0000 7fbd1d520700 1 -- start start 2024-05-21T03:12:32.946 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.239+0000 7fbd1d520700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd18107a70 0x7fbd1819f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:32.946 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.239+0000 7fbd1d520700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbd18109400 con 0x7fbd18107a70 2024-05-21T03:12:32.946 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.239+0000 7fbd16ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd18107a70 0x7fbd1819f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:32.946 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.239+0000 7fbd16ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd18107a70 0x7fbd1819f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56082/0 (socket says 172.21.15.27:56082) 2024-05-21T03:12:32.946 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.239+0000 7fbd16ffd700 1 -- 172.21.15.27:0/2027252544 learned_addr learned my addr 172.21.15.27:0/2027252544 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:32.947 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.239+0000 7fbd16ffd700 1 -- 172.21.15.27:0/2027252544 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd1819f780 con 0x7fbd18107a70 2024-05-21T03:12:32.947 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.239+0000 7fbd16ffd700 1 --2- 172.21.15.27:0/2027252544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd18107a70 0x7fbd1819f240 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbd08011ac0 tx=0x7fbd08011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:32.947 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.239+0000 7fbcfffff700 1 -- 172.21.15.27:0/2027252544 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd08003970 con 0x7fbd18107a70 2024-05-21T03:12:32.947 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.239+0000 7fbcfffff700 1 -- 172.21.15.27:0/2027252544 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbd08003b10 con 0x7fbd18107a70 2024-05-21T03:12:32.947 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.239+0000 7fbcfffff700 1 -- 172.21.15.27:0/2027252544 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd08024610 con 0x7fbd18107a70 2024-05-21T03:12:32.947 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.239+0000 7fbd1d520700 1 -- 172.21.15.27:0/2027252544 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd1819fa10 con 0x7fbd18107a70 2024-05-21T03:12:32.947 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.240+0000 7fbd1d520700 1 -- 172.21.15.27:0/2027252544 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd181a2700 con 0x7fbd18107a70 2024-05-21T03:12:32.947 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.240+0000 7fbcfffff700 1 -- 172.21.15.27:0/2027252544 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbd08024840 con 0x7fbd18107a70 2024-05-21T03:12:32.947 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.240+0000 7fbd1d520700 1 -- 172.21.15.27:0/2027252544 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd18064ba0 con 0x7fbd18107a70 2024-05-21T03:12:32.947 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.240+0000 7fbcfffff700 1 -- 172.21.15.27:0/2027252544 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbd0802dcd0 con 0x7fbd18107a70 2024-05-21T03:12:32.947 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.243+0000 7fbcfffff700 1 -- 172.21.15.27:0/2027252544 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fbd08043ec0 con 0x7fbd18107a70 2024-05-21T03:12:32.947 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.320+0000 7fbd1d520700 1 -- 172.21.15.27:0/2027252544 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fbd180008d0 con 0x7fbd18107a70 2024-05-21T03:12:32.947 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.321+0000 7fbcfffff700 1 -- 172.21.15.27:0/2027252544 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fbd08032030 con 0x7fbd18107a70 2024-05-21T03:12:32.947 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.322+0000 7fbd1d520700 1 -- 172.21.15.27:0/2027252544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd18107a70 msgr2=0x7fbd1819f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:32.948 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.322+0000 7fbd1d520700 1 --2- 172.21.15.27:0/2027252544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd18107a70 0x7fbd1819f240 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbd08011ac0 tx=0x7fbd08011ba0 comp rx=0 tx=0).stop 2024-05-21T03:12:32.948 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.322+0000 7fbd1d520700 1 -- 172.21.15.27:0/2027252544 shutdown_connections 2024-05-21T03:12:32.948 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.322+0000 7fbd1d520700 1 --2- 172.21.15.27:0/2027252544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd18107a70 0x7fbd1819f240 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:32.948 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.322+0000 7fbd1d520700 1 -- 172.21.15.27:0/2027252544 >> 172.21.15.27:0/2027252544 conn(0x7fbd18103690 msgr2=0x7fbd18195ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:32.948 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.322+0000 7fbd1d520700 1 -- 172.21.15.27:0/2027252544 shutdown_connections 2024-05-21T03:12:32.948 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:32.322+0000 7fbd1d520700 1 -- 172.21.15.27:0/2027252544 wait complete. 2024-05-21T03:12:32.948 INFO:teuthology.orchestra.run.smithi027.stdout:mgr not available, waiting (3/15)... 2024-05-21T03:12:33.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:33 smithi027 ceph-mon[32179]: Activating manager daemon x 2024-05-21T03:12:33.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:33 smithi027 ceph-mon[32179]: mgrmap e2: x(active, starting, since 0.00319547s) 2024-05-21T03:12:33.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:33 smithi027 ceph-mon[32179]: from='mgr.14100 172.21.15.27:0/3167079051' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-21T03:12:33.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:33 smithi027 ceph-mon[32179]: from='mgr.14100 172.21.15.27:0/3167079051' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-21T03:12:33.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:33 smithi027 ceph-mon[32179]: from='mgr.14100 172.21.15.27:0/3167079051' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-21T03:12:33.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:33 smithi027 ceph-mon[32179]: from='mgr.14100 172.21.15.27:0/3167079051' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T03:12:33.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:33 smithi027 ceph-mon[32179]: from='mgr.14100 172.21.15.27:0/3167079051' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-21T03:12:33.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:33 smithi027 ceph-mon[32179]: Manager daemon x is now available 2024-05-21T03:12:33.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:33 smithi027 ceph-mon[32179]: from='mgr.14100 172.21.15.27:0/3167079051' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-21T03:12:33.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:33 smithi027 ceph-mon[32179]: from='mgr.14100 172.21.15.27:0/3167079051' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-21T03:12:33.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:33 smithi027 ceph-mon[32179]: from='mgr.14100 172.21.15.27:0/3167079051' entity='mgr.x' 2024-05-21T03:12:33.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:33 smithi027 ceph-mon[32179]: from='mgr.14100 172.21.15.27:0/3167079051' entity='mgr.x' 2024-05-21T03:12:33.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:33 smithi027 ceph-mon[32179]: from='mgr.14100 172.21.15.27:0/3167079051' entity='mgr.x' 2024-05-21T03:12:35.062 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:34 smithi027 ceph-mon[32179]: mgrmap e3: x(active, since 1.0074s) 2024-05-21T03:12:36.000 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:35 smithi027 ceph-mon[32179]: mgrmap e4: x(active, since 2s) 2024-05-21T03:12:36.000 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:35 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2316161805' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-21T03:12:36.289 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-05-21T03:12:36.289 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-05-21T03:12:36.290 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "c9ca94be-171f-11ef-bc9a-c7b262605968", 2024-05-21T03:12:36.290 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2024-05-21T03:12:36.290 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-21T03:12:36.290 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-21T03:12:36.290 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-21T03:12:36.290 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-05-21T03:12:36.290 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-21T03:12:36.290 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-21T03:12:36.290 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2024-05-21T03:12:36.290 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-05-21T03:12:36.291 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-21T03:12:36.291 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2024-05-21T03:12:36.291 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-05-21T03:12:36.291 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-05-21T03:12:36.291 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-21T03:12:36.291 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T03:12:36.291 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-21T03:12:36.291 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-21T03:12:36.291 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-05-21T03:12:36.291 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-21T03:12:36.291 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T03:12:36.291 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-21T03:12:36.291 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-21T03:12:36.292 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-21T03:12:36.301 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-21T03:12:36.301 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-21T03:12:36.301 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-21T03:12:36.301 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-05-21T03:12:36.301 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-21T03:12:36.301 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-21T03:12:36.301 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-21T03:12:36.301 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-21T03:12:36.301 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-21T03:12:36.301 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-21T03:12:36.301 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-21T03:12:36.301 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-21T03:12:36.302 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-21T03:12:36.302 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-05-21T03:12:36.302 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-21T03:12:36.302 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T03:12:36.302 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-21T03:12:36.302 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-21T03:12:36.302 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-05-21T03:12:36.302 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-21T03:12:36.302 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-21T03:12:36.302 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-21T03:12:36.302 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-21T03:12:36.302 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-21T03:12:36.302 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-21T03:12:36.303 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2024-05-21T03:12:36.303 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-05-21T03:12:36.303 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T03:12:36.303 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-05-21T03:12:36.303 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-21T03:12:36.303 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T03:12:36.303 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2024-05-21T03:12:18.217740+0000", 2024-05-21T03:12:36.303 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T03:12:36.303 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-05-21T03:12:36.303 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-21T03:12:36.303 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-05-21T03:12:36.303 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.375+0000 7f139c3d2700 1 Processor -- start 2024-05-21T03:12:36.303 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.375+0000 7f139c3d2700 1 -- start start 2024-05-21T03:12:36.304 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.376+0000 7f139c3d2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1394108080 0x7f13941084a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:36.304 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.376+0000 7f139c3d2700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1394108a70 con 0x7f1394108080 2024-05-21T03:12:36.304 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.376+0000 7f139a16e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1394108080 0x7f13941084a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:36.304 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.376+0000 7f139a16e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1394108080 0x7f13941084a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44764/0 (socket says 172.21.15.27:44764) 2024-05-21T03:12:36.304 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.376+0000 7f139a16e700 1 -- 172.21.15.27:0/2488762039 learned_addr learned my addr 172.21.15.27:0/2488762039 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:36.304 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.376+0000 7f139a16e700 1 -- 172.21.15.27:0/2488762039 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1394109280 con 0x7f1394108080 2024-05-21T03:12:36.304 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.376+0000 7f139a16e700 1 --2- 172.21.15.27:0/2488762039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1394108080 0x7f13941084a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f13880098a0 tx=0x7f1388009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48969a470c7c3e94 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:36.304 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.377+0000 7f139916c700 1 -- 172.21.15.27:0/2488762039 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1388004160 con 0x7f1394108080 2024-05-21T03:12:36.304 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.377+0000 7f139916c700 1 -- 172.21.15.27:0/2488762039 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1388004300 con 0x7f1394108080 2024-05-21T03:12:36.304 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.377+0000 7f139916c700 1 -- 172.21.15.27:0/2488762039 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1388011af0 con 0x7f1394108080 2024-05-21T03:12:36.305 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.377+0000 7f139c3d2700 1 -- 172.21.15.27:0/2488762039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1394108080 msgr2=0x7f13941084a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:36.305 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.377+0000 7f139c3d2700 1 --2- 172.21.15.27:0/2488762039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1394108080 0x7f13941084a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f13880098a0 tx=0x7f1388009bb0 comp rx=0 tx=0).stop 2024-05-21T03:12:36.305 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.377+0000 7f139c3d2700 1 -- 172.21.15.27:0/2488762039 shutdown_connections 2024-05-21T03:12:36.305 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.377+0000 7f139c3d2700 1 --2- 172.21.15.27:0/2488762039 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1394108080 0x7f13941084a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:36.305 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.377+0000 7f139c3d2700 1 -- 172.21.15.27:0/2488762039 >> 172.21.15.27:0/2488762039 conn(0x7f1394103620 msgr2=0x7f1394105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:36.305 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.377+0000 7f139c3d2700 1 -- 172.21.15.27:0/2488762039 shutdown_connections 2024-05-21T03:12:36.305 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.377+0000 7f139c3d2700 1 -- 172.21.15.27:0/2488762039 wait complete. 2024-05-21T03:12:36.305 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.378+0000 7f139c3d2700 1 Processor -- start 2024-05-21T03:12:36.305 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.378+0000 7f139c3d2700 1 -- start start 2024-05-21T03:12:36.305 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.378+0000 7f139c3d2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1394108080 0x7f139419d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:36.305 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.378+0000 7f139c3d2700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f139410c630 con 0x7f1394108080 2024-05-21T03:12:36.305 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.378+0000 7f139a16e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1394108080 0x7f139419d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:36.305 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.379+0000 7f139a16e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1394108080 0x7f139419d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44766/0 (socket says 172.21.15.27:44766) 2024-05-21T03:12:36.306 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.379+0000 7f139a16e700 1 -- 172.21.15.27:0/2316161805 learned_addr learned my addr 172.21.15.27:0/2316161805 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:36.306 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.379+0000 7f139a16e700 1 -- 172.21.15.27:0/2316161805 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f139419d5c0 con 0x7f1394108080 2024-05-21T03:12:36.306 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.379+0000 7f139a16e700 1 --2- 172.21.15.27:0/2316161805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1394108080 0x7f139419d080 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1388009fb0 tx=0x7f1388011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:36.306 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.379+0000 7f13877fe700 1 -- 172.21.15.27:0/2316161805 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13880036a0 con 0x7f1394108080 2024-05-21T03:12:36.306 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.379+0000 7f139c3d2700 1 -- 172.21.15.27:0/2316161805 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f139419d850 con 0x7f1394108080 2024-05-21T03:12:36.306 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.379+0000 7f13877fe700 1 -- 172.21.15.27:0/2316161805 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f138801a430 con 0x7f1394108080 2024-05-21T03:12:36.306 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.379+0000 7f13877fe700 1 -- 172.21.15.27:0/2316161805 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1388019550 con 0x7f1394108080 2024-05-21T03:12:36.306 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.379+0000 7f139c3d2700 1 -- 172.21.15.27:0/2316161805 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13941a0540 con 0x7f1394108080 2024-05-21T03:12:36.306 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.380+0000 7f13877fe700 1 -- 172.21.15.27:0/2316161805 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f138801a8e0 con 0x7f1394108080 2024-05-21T03:12:36.306 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.380+0000 7f139c3d2700 1 -- 172.21.15.27:0/2316161805 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1394064ba0 con 0x7f1394108080 2024-05-21T03:12:36.306 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.380+0000 7f13877fe700 1 --2- 172.21.15.27:0/2316161805 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7f138003e490 0x7f1380040950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:36.306 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.381+0000 7f13877fe700 1 -- 172.21.15.27:0/2316161805 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f13880537c0 con 0x7f1394108080 2024-05-21T03:12:36.307 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.381+0000 7f139996d700 1 --2- 172.21.15.27:0/2316161805 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7f138003e490 0x7f1380040950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:36.307 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.381+0000 7f139996d700 1 --2- 172.21.15.27:0/2316161805 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7f138003e490 0x7f1380040950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1390006fd0 tx=0x7f1390006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:36.307 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.386+0000 7f13877fe700 1 -- 172.21.15.27:0/2316161805 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f138800b3e0 con 0x7f1394108080 2024-05-21T03:12:36.307 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.695+0000 7f139c3d2700 1 -- 172.21.15.27:0/2316161805 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f13941a0710 con 0x7f1394108080 2024-05-21T03:12:36.307 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.695+0000 7f13877fe700 1 -- 172.21.15.27:0/2316161805 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f13941a0710 con 0x7f1394108080 2024-05-21T03:12:36.307 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.698+0000 7f139c3d2700 1 -- 172.21.15.27:0/2316161805 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7f138003e490 msgr2=0x7f1380040950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:36.308 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.698+0000 7f139c3d2700 1 --2- 172.21.15.27:0/2316161805 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7f138003e490 0x7f1380040950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1390006fd0 tx=0x7f1390006e40 comp rx=0 tx=0).stop 2024-05-21T03:12:36.308 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.698+0000 7f139c3d2700 1 -- 172.21.15.27:0/2316161805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1394108080 msgr2=0x7f139419d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:36.308 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.698+0000 7f139c3d2700 1 --2- 172.21.15.27:0/2316161805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1394108080 0x7f139419d080 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1388009fb0 tx=0x7f1388011940 comp rx=0 tx=0).stop 2024-05-21T03:12:36.308 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.698+0000 7f139c3d2700 1 -- 172.21.15.27:0/2316161805 shutdown_connections 2024-05-21T03:12:36.308 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.698+0000 7f139c3d2700 1 --2- 172.21.15.27:0/2316161805 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7f138003e490 0x7f1380040950 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:36.308 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.698+0000 7f139c3d2700 1 --2- 172.21.15.27:0/2316161805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1394108080 0x7f139419d080 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:36.308 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.698+0000 7f139c3d2700 1 -- 172.21.15.27:0/2316161805 >> 172.21.15.27:0/2316161805 conn(0x7f1394103620 msgr2=0x7f1394104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:36.308 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.698+0000 7f139c3d2700 1 -- 172.21.15.27:0/2316161805 shutdown_connections 2024-05-21T03:12:36.308 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:35.698+0000 7f139c3d2700 1 -- 172.21.15.27:0/2316161805 wait complete. 2024-05-21T03:12:36.308 INFO:teuthology.orchestra.run.smithi027.stdout:mgr is available 2024-05-21T03:12:37.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:36 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3688745955' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-21T03:12:37.525 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-05-21T03:12:37.525 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [global] 2024-05-21T03:12:37.525 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout fsid = c9ca94be-171f-11ef-bc9a-c7b262605968 2024-05-21T03:12:37.525 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-21T03:12:37.525 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-21T03:12:37.525 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-21T03:12:37.526 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-21T03:12:37.526 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-21T03:12:37.526 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-05-21T03:12:37.526 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-21T03:12:37.526 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-21T03:12:37.526 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-05-21T03:12:37.526 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [osd] 2024-05-21T03:12:37.526 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-05-21T03:12:37.526 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-05-21T03:12:37.526 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-21T03:12:37.526 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-21T03:12:37.526 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-21T03:12:37.526 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-21T03:12:37.526 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-21T03:12:37.527 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.710+0000 7f830b4e4700 1 Processor -- start 2024-05-21T03:12:37.527 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.710+0000 7f830b4e4700 1 -- start start 2024-05-21T03:12:37.527 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.710+0000 7f830b4e4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f830407ee00 0x7f830407d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:37.527 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.710+0000 7f830b4e4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f830407d840 con 0x7f830407ee00 2024-05-21T03:12:37.527 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.710+0000 7f8309280700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f830407ee00 0x7f830407d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:37.527 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.710+0000 7f8309280700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f830407ee00 0x7f830407d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44780/0 (socket says 172.21.15.27:44780) 2024-05-21T03:12:37.527 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.710+0000 7f8309280700 1 -- 172.21.15.27:0/3505070150 learned_addr learned my addr 172.21.15.27:0/3505070150 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:37.527 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.710+0000 7f8309280700 1 -- 172.21.15.27:0/3505070150 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830407d9c0 con 0x7f830407ee00 2024-05-21T03:12:37.527 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.711+0000 7f8309280700 1 --2- 172.21.15.27:0/3505070150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f830407ee00 0x7f830407d300 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f83000098a0 tx=0x7f8300009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d86df3eed3b1b55 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:37.527 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.711+0000 7f82f7fff700 1 -- 172.21.15.27:0/3505070150 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8300004160 con 0x7f830407ee00 2024-05-21T03:12:37.527 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.711+0000 7f82f7fff700 1 -- 172.21.15.27:0/3505070150 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8300004300 con 0x7f830407ee00 2024-05-21T03:12:37.527 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.711+0000 7f82f7fff700 1 -- 172.21.15.27:0/3505070150 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8300011af0 con 0x7f830407ee00 2024-05-21T03:12:37.527 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.711+0000 7f830b4e4700 1 -- 172.21.15.27:0/3505070150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f830407ee00 msgr2=0x7f830407d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:37.527 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.711+0000 7f830b4e4700 1 --2- 172.21.15.27:0/3505070150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f830407ee00 0x7f830407d300 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f83000098a0 tx=0x7f8300009bb0 comp rx=0 tx=0).stop 2024-05-21T03:12:37.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.711+0000 7f830b4e4700 1 -- 172.21.15.27:0/3505070150 shutdown_connections 2024-05-21T03:12:37.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.711+0000 7f830b4e4700 1 --2- 172.21.15.27:0/3505070150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f830407ee00 0x7f830407d300 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:37.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.711+0000 7f830b4e4700 1 -- 172.21.15.27:0/3505070150 >> 172.21.15.27:0/3505070150 conn(0x7f8304104840 msgr2=0x7f8304106c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:37.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.711+0000 7f830b4e4700 1 -- 172.21.15.27:0/3505070150 shutdown_connections 2024-05-21T03:12:37.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.712+0000 7f830b4e4700 1 -- 172.21.15.27:0/3505070150 wait complete. 2024-05-21T03:12:37.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.712+0000 7f830b4e4700 1 Processor -- start 2024-05-21T03:12:37.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.712+0000 7f830b4e4700 1 -- start start 2024-05-21T03:12:37.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.712+0000 7f830b4e4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f830407ee00 0x7f830419f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:37.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.712+0000 7f830b4e4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f830410f6f0 con 0x7f830407ee00 2024-05-21T03:12:37.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.712+0000 7f8309280700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f830407ee00 0x7f830419f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:37.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.712+0000 7f8309280700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f830407ee00 0x7f830419f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44790/0 (socket says 172.21.15.27:44790) 2024-05-21T03:12:37.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.712+0000 7f8309280700 1 -- 172.21.15.27:0/3688745955 learned_addr learned my addr 172.21.15.27:0/3688745955 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:37.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.713+0000 7f8309280700 1 -- 172.21.15.27:0/3688745955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830419f730 con 0x7f830407ee00 2024-05-21T03:12:37.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.713+0000 7f8309280700 1 --2- 172.21.15.27:0/3688745955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f830407ee00 0x7f830419f1f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8300009fb0 tx=0x7f8300011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:37.528 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.713+0000 7f82f67fc700 1 -- 172.21.15.27:0/3688745955 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83000036a0 con 0x7f830407ee00 2024-05-21T03:12:37.529 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.713+0000 7f82f67fc700 1 -- 172.21.15.27:0/3688745955 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f830001a430 con 0x7f830407ee00 2024-05-21T03:12:37.529 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.713+0000 7f830b4e4700 1 -- 172.21.15.27:0/3688745955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f830419f9c0 con 0x7f830407ee00 2024-05-21T03:12:37.529 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.713+0000 7f82f67fc700 1 -- 172.21.15.27:0/3688745955 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8300019550 con 0x7f830407ee00 2024-05-21T03:12:37.529 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.713+0000 7f830b4e4700 1 -- 172.21.15.27:0/3688745955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83041a26b0 con 0x7f830407ee00 2024-05-21T03:12:37.529 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.714+0000 7f82f67fc700 1 -- 172.21.15.27:0/3688745955 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f8300019850 con 0x7f830407ee00 2024-05-21T03:12:37.529 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.714+0000 7f82f67fc700 1 --2- 172.21.15.27:0/3688745955 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7f82f003e490 0x7f82f0040950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:37.529 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.714+0000 7f82f67fc700 1 -- 172.21.15.27:0/3688745955 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f83000537e0 con 0x7f830407ee00 2024-05-21T03:12:37.529 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.714+0000 7f830b4e4700 1 -- 172.21.15.27:0/3688745955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8304064ba0 con 0x7f830407ee00 2024-05-21T03:12:37.529 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.714+0000 7f8308a7f700 1 --2- 172.21.15.27:0/3688745955 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7f82f003e490 0x7f82f0040950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:37.529 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.715+0000 7f8308a7f700 1 --2- 172.21.15.27:0/3688745955 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7f82f003e490 0x7f82f0040950 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f82f8006fd0 tx=0x7f82f8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:37.529 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.719+0000 7f82f67fc700 1 -- 172.21.15.27:0/3688745955 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8300017e90 con 0x7f830407ee00 2024-05-21T03:12:37.529 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.935+0000 7f830b4e4700 1 -- 172.21.15.27:0/3688745955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f83041a2880 con 0x7f830407ee00 2024-05-21T03:12:37.529 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.936+0000 7f82f67fc700 1 -- 172.21.15.27:0/3688745955 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+461 (secure 0 0 0) 0x7f8300053200 con 0x7f830407ee00 2024-05-21T03:12:37.529 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.938+0000 7f830b4e4700 1 -- 172.21.15.27:0/3688745955 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7f82f003e490 msgr2=0x7f82f0040950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:37.530 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.938+0000 7f830b4e4700 1 --2- 172.21.15.27:0/3688745955 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7f82f003e490 0x7f82f0040950 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f82f8006fd0 tx=0x7f82f8006e40 comp rx=0 tx=0).stop 2024-05-21T03:12:37.530 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.938+0000 7f830b4e4700 1 -- 172.21.15.27:0/3688745955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f830407ee00 msgr2=0x7f830419f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:37.530 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.938+0000 7f830b4e4700 1 --2- 172.21.15.27:0/3688745955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f830407ee00 0x7f830419f1f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8300009fb0 tx=0x7f8300011940 comp rx=0 tx=0).stop 2024-05-21T03:12:37.530 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.938+0000 7f830b4e4700 1 -- 172.21.15.27:0/3688745955 shutdown_connections 2024-05-21T03:12:37.530 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.938+0000 7f830b4e4700 1 --2- 172.21.15.27:0/3688745955 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7f82f003e490 0x7f82f0040950 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:37.530 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.938+0000 7f830b4e4700 1 --2- 172.21.15.27:0/3688745955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f830407ee00 0x7f830419f1f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:37.530 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.938+0000 7f830b4e4700 1 -- 172.21.15.27:0/3688745955 >> 172.21.15.27:0/3688745955 conn(0x7f8304104840 msgr2=0x7f8304105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:37.530 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.938+0000 7f830b4e4700 1 -- 172.21.15.27:0/3688745955 shutdown_connections 2024-05-21T03:12:37.530 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:36.938+0000 7f830b4e4700 1 -- 172.21.15.27:0/3688745955 wait complete. 2024-05-21T03:12:37.530 INFO:teuthology.orchestra.run.smithi027.stdout:Enabling cephadm module... 2024-05-21T03:12:38.639 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:38 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/756995972' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-21T03:12:38.639 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:38 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: ignoring --setuser ceph since I am not root 2024-05-21T03:12:38.639 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:38 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: ignoring --setgroup ceph since I am not root 2024-05-21T03:12:38.639 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:38 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:38.483+0000 7f530434a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T03:12:38.922 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:38 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:38.678+0000 7f530434a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T03:12:38.923 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:38 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:38.784+0000 7f530434a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T03:12:38.931 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.966+0000 7f8322837700 1 Processor -- start 2024-05-21T03:12:38.931 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.966+0000 7f8322837700 1 -- start start 2024-05-21T03:12:38.931 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.966+0000 7f8322837700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f831c1091e0 0x7f831c109600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:38.931 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.966+0000 7f8322837700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f831c109bd0 con 0x7f831c1091e0 2024-05-21T03:12:38.931 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.966+0000 7f831bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f831c1091e0 0x7f831c109600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:38.931 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.967+0000 7f831bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f831c1091e0 0x7f831c109600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44796/0 (socket says 172.21.15.27:44796) 2024-05-21T03:12:38.931 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.967+0000 7f831bfff700 1 -- 172.21.15.27:0/4078807714 learned_addr learned my addr 172.21.15.27:0/4078807714 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:38.932 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.967+0000 7f831bfff700 1 -- 172.21.15.27:0/4078807714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831c10a340 con 0x7f831c1091e0 2024-05-21T03:12:38.932 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.967+0000 7f831bfff700 1 --2- 172.21.15.27:0/4078807714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f831c1091e0 0x7f831c109600 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f830c0098a0 tx=0x7f830c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf8debaefed01f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:38.932 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.967+0000 7f831affd700 1 -- 172.21.15.27:0/4078807714 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f830c004160 con 0x7f831c1091e0 2024-05-21T03:12:38.932 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.967+0000 7f831affd700 1 -- 172.21.15.27:0/4078807714 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f830c004300 con 0x7f831c1091e0 2024-05-21T03:12:38.932 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.967+0000 7f831affd700 1 -- 172.21.15.27:0/4078807714 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f830c011af0 con 0x7f831c1091e0 2024-05-21T03:12:38.932 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.968+0000 7f8322837700 1 -- 172.21.15.27:0/4078807714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f831c1091e0 msgr2=0x7f831c109600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:38.932 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.968+0000 7f8322837700 1 --2- 172.21.15.27:0/4078807714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f831c1091e0 0x7f831c109600 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f830c0098a0 tx=0x7f830c009bb0 comp rx=0 tx=0).stop 2024-05-21T03:12:38.932 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.968+0000 7f8322837700 1 -- 172.21.15.27:0/4078807714 shutdown_connections 2024-05-21T03:12:38.932 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.968+0000 7f8322837700 1 --2- 172.21.15.27:0/4078807714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f831c1091e0 0x7f831c109600 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:38.932 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.968+0000 7f8322837700 1 -- 172.21.15.27:0/4078807714 >> 172.21.15.27:0/4078807714 conn(0x7f831c104800 msgr2=0x7f831c106c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:38.933 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.968+0000 7f8322837700 1 -- 172.21.15.27:0/4078807714 shutdown_connections 2024-05-21T03:12:38.933 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.968+0000 7f8322837700 1 -- 172.21.15.27:0/4078807714 wait complete. 2024-05-21T03:12:38.933 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.968+0000 7f8322837700 1 Processor -- start 2024-05-21T03:12:38.933 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.969+0000 7f8322837700 1 -- start start 2024-05-21T03:12:38.933 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.969+0000 7f8322837700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f831c1091e0 0x7f831c07eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:38.933 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.969+0000 7f8322837700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f831c10d6f0 con 0x7f831c1091e0 2024-05-21T03:12:38.933 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.969+0000 7f831bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f831c1091e0 0x7f831c07eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:38.933 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.969+0000 7f831bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f831c1091e0 0x7f831c07eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44806/0 (socket says 172.21.15.27:44806) 2024-05-21T03:12:38.933 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.969+0000 7f831bfff700 1 -- 172.21.15.27:0/756995972 learned_addr learned my addr 172.21.15.27:0/756995972 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:38.933 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.969+0000 7f831bfff700 1 -- 172.21.15.27:0/756995972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831c07f0d0 con 0x7f831c1091e0 2024-05-21T03:12:38.934 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.969+0000 7f831bfff700 1 --2- 172.21.15.27:0/756995972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f831c1091e0 0x7f831c07eb90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f830c0117b0 tx=0x7f830c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:38.934 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.970+0000 7f83197fa700 1 -- 172.21.15.27:0/756995972 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f830c003af0 con 0x7f831c1091e0 2024-05-21T03:12:38.934 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.970+0000 7f83197fa700 1 -- 172.21.15.27:0/756995972 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f830c020070 con 0x7f831c1091e0 2024-05-21T03:12:38.934 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.970+0000 7f8322837700 1 -- 172.21.15.27:0/756995972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f831c07d2c0 con 0x7f831c1091e0 2024-05-21T03:12:38.934 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.970+0000 7f83197fa700 1 -- 172.21.15.27:0/756995972 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f830c019630 con 0x7f831c1091e0 2024-05-21T03:12:38.934 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.970+0000 7f8322837700 1 -- 172.21.15.27:0/756995972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f831c07d800 con 0x7f831c1091e0 2024-05-21T03:12:38.934 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.971+0000 7f83197fa700 1 -- 172.21.15.27:0/756995972 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f830c0198a0 con 0x7f831c1091e0 2024-05-21T03:12:38.934 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.971+0000 7f8322837700 1 -- 172.21.15.27:0/756995972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f831c064ba0 con 0x7f831c1091e0 2024-05-21T03:12:38.934 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.971+0000 7f83197fa700 1 --2- 172.21.15.27:0/756995972 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7f830403e580 0x7f8304040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:38.934 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.971+0000 7f83197fa700 1 -- 172.21.15.27:0/756995972 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f830c053b30 con 0x7f831c1091e0 2024-05-21T03:12:38.935 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.971+0000 7f831b7fe700 1 --2- 172.21.15.27:0/756995972 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7f830403e580 0x7f8304040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:38.935 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.971+0000 7f831b7fe700 1 --2- 172.21.15.27:0/756995972 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7f830403e580 0x7f8304040a40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8310006fd0 tx=0x7f8310006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:38.935 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:37.976+0000 7f83197fa700 1 -- 172.21.15.27:0/756995972 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f830c0042a0 con 0x7f831c1091e0 2024-05-21T03:12:38.935 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:38.253+0000 7f8322837700 1 -- 172.21.15.27:0/756995972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f831c07da80 con 0x7f831c1091e0 2024-05-21T03:12:38.935 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:38.304+0000 7f83197fa700 1 -- 172.21.15.27:0/756995972 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f830c053550 con 0x7f831c1091e0 2024-05-21T03:12:38.935 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:38.306+0000 7f83197fa700 1 -- 172.21.15.27:0/756995972 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f830c019b90 con 0x7f831c1091e0 2024-05-21T03:12:38.935 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:38.307+0000 7f8322837700 1 -- 172.21.15.27:0/756995972 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7f830403e580 msgr2=0x7f8304040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:38.935 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:38.307+0000 7f8322837700 1 --2- 172.21.15.27:0/756995972 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7f830403e580 0x7f8304040a40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8310006fd0 tx=0x7f8310006e40 comp rx=0 tx=0).stop 2024-05-21T03:12:38.935 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:38.307+0000 7f8322837700 1 -- 172.21.15.27:0/756995972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f831c1091e0 msgr2=0x7f831c07eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:38.935 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:38.307+0000 7f8322837700 1 --2- 172.21.15.27:0/756995972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f831c1091e0 0x7f831c07eb90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f830c0117b0 tx=0x7f830c011b70 comp rx=0 tx=0).stop 2024-05-21T03:12:38.935 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:38.307+0000 7f8322837700 1 -- 172.21.15.27:0/756995972 shutdown_connections 2024-05-21T03:12:38.935 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:38.307+0000 7f8322837700 1 --2- 172.21.15.27:0/756995972 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7f830403e580 0x7f8304040a40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:38.936 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:38.307+0000 7f8322837700 1 --2- 172.21.15.27:0/756995972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f831c1091e0 0x7f831c07eb90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:38.936 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:38.307+0000 7f8322837700 1 -- 172.21.15.27:0/756995972 >> 172.21.15.27:0/756995972 conn(0x7f831c104800 msgr2=0x7f831c1052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:38.936 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:38.307+0000 7f8322837700 1 -- 172.21.15.27:0/756995972 shutdown_connections 2024-05-21T03:12:38.936 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:38.307+0000 7f8322837700 1 -- 172.21.15.27:0/756995972 wait complete. 2024-05-21T03:12:39.176 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:39 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:39.124+0000 7f530434a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T03:12:39.595 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:39 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/756995972' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-21T03:12:39.595 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:39 smithi027 ceph-mon[32179]: mgrmap e5: x(active, since 5s) 2024-05-21T03:12:39.595 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:39 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:39.218+0000 7f530434a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T03:12:40.347 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-05-21T03:12:40.347 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-21T03:12:40.347 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-21T03:12:40.347 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-21T03:12:40.347 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-21T03:12:40.347 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-05-21T03:12:40.347 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.388+0000 7ff8262f4700 1 Processor -- start 2024-05-21T03:12:40.347 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.388+0000 7ff8262f4700 1 -- start start 2024-05-21T03:12:40.347 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.389+0000 7ff8262f4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff820109ba0 0x7ff82010bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:40.348 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.389+0000 7ff8262f4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff82007ced0 con 0x7ff820109ba0 2024-05-21T03:12:40.348 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.389+0000 7ff81ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff820109ba0 0x7ff82010bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:40.348 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.389+0000 7ff81ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff820109ba0 0x7ff82010bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44828/0 (socket says 172.21.15.27:44828) 2024-05-21T03:12:40.348 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.389+0000 7ff81ffff700 1 -- 172.21.15.27:0/3646259235 learned_addr learned my addr 172.21.15.27:0/3646259235 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:40.348 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.389+0000 7ff81ffff700 1 -- 172.21.15.27:0/3646259235 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82010c4d0 con 0x7ff820109ba0 2024-05-21T03:12:40.348 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.389+0000 7ff81ffff700 1 --2- 172.21.15.27:0/3646259235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff820109ba0 0x7ff82010bf90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff8100098a0 tx=0x7ff810009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ebcd17682021c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:40.348 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.389+0000 7ff81effd700 1 -- 172.21.15.27:0/3646259235 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff810004160 con 0x7ff820109ba0 2024-05-21T03:12:40.348 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.390+0000 7ff81effd700 1 -- 172.21.15.27:0/3646259235 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff810004300 con 0x7ff820109ba0 2024-05-21T03:12:40.348 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.390+0000 7ff81effd700 1 -- 172.21.15.27:0/3646259235 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff810011af0 con 0x7ff820109ba0 2024-05-21T03:12:40.348 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.390+0000 7ff8262f4700 1 -- 172.21.15.27:0/3646259235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff820109ba0 msgr2=0x7ff82010bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:40.348 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.390+0000 7ff8262f4700 1 --2- 172.21.15.27:0/3646259235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff820109ba0 0x7ff82010bf90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff8100098a0 tx=0x7ff810009bb0 comp rx=0 tx=0).stop 2024-05-21T03:12:40.348 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.390+0000 7ff8262f4700 1 -- 172.21.15.27:0/3646259235 shutdown_connections 2024-05-21T03:12:40.348 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.390+0000 7ff8262f4700 1 --2- 172.21.15.27:0/3646259235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff820109ba0 0x7ff82010bf90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:40.348 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.390+0000 7ff8262f4700 1 -- 172.21.15.27:0/3646259235 >> 172.21.15.27:0/3646259235 conn(0x7ff820103610 msgr2=0x7ff820105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:40.349 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.391+0000 7ff8262f4700 1 -- 172.21.15.27:0/3646259235 shutdown_connections 2024-05-21T03:12:40.349 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.391+0000 7ff8262f4700 1 -- 172.21.15.27:0/3646259235 wait complete. 2024-05-21T03:12:40.349 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.391+0000 7ff8262f4700 1 Processor -- start 2024-05-21T03:12:40.349 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.391+0000 7ff8262f4700 1 -- start start 2024-05-21T03:12:40.349 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.392+0000 7ff8262f4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff820109ba0 0x7ff82019cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:40.349 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.392+0000 7ff8262f4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff82010d4a0 con 0x7ff820109ba0 2024-05-21T03:12:40.349 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.392+0000 7ff81ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff820109ba0 0x7ff82019cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:40.349 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.392+0000 7ff81ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff820109ba0 0x7ff82019cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44832/0 (socket says 172.21.15.27:44832) 2024-05-21T03:12:40.349 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.392+0000 7ff81ffff700 1 -- 172.21.15.27:0/51427507 learned_addr learned my addr 172.21.15.27:0/51427507 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:40.349 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.392+0000 7ff81ffff700 1 -- 172.21.15.27:0/51427507 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82019d4f0 con 0x7ff820109ba0 2024-05-21T03:12:40.349 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.392+0000 7ff81ffff700 1 --2- 172.21.15.27:0/51427507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff820109ba0 0x7ff82019cfb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff810011ac0 tx=0x7ff810011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:40.349 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.392+0000 7ff81d7fa700 1 -- 172.21.15.27:0/51427507 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff810003b20 con 0x7ff820109ba0 2024-05-21T03:12:40.349 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.392+0000 7ff81d7fa700 1 -- 172.21.15.27:0/51427507 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff81001a8c0 con 0x7ff820109ba0 2024-05-21T03:12:40.350 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.392+0000 7ff8262f4700 1 -- 172.21.15.27:0/51427507 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff82019d780 con 0x7ff820109ba0 2024-05-21T03:12:40.350 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.392+0000 7ff81d7fa700 1 -- 172.21.15.27:0/51427507 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8100248d0 con 0x7ff820109ba0 2024-05-21T03:12:40.350 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.393+0000 7ff8262f4700 1 -- 172.21.15.27:0/51427507 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8201a0470 con 0x7ff820109ba0 2024-05-21T03:12:40.350 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.393+0000 7ff81d7fa700 1 -- 172.21.15.27:0/51427507 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7ff810022020 con 0x7ff820109ba0 2024-05-21T03:12:40.350 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.393+0000 7ff8262f4700 1 -- 172.21.15.27:0/51427507 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff820064ba0 con 0x7ff820109ba0 2024-05-21T03:12:40.350 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.394+0000 7ff81d7fa700 1 --2- 172.21.15.27:0/51427507 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7ff80803e580 0x7ff808040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:40.350 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.394+0000 7ff81d7fa700 1 -- 172.21.15.27:0/51427507 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff810053490 con 0x7ff820109ba0 2024-05-21T03:12:40.350 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.394+0000 7ff81f7fe700 1 -- 172.21.15.27:0/51427507 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7ff80803e580 msgr2=0x7ff808040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/175229478 2024-05-21T03:12:40.350 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.394+0000 7ff81f7fe700 1 --2- 172.21.15.27:0/51427507 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7ff80803e580 0x7ff808040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T03:12:40.350 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.399+0000 7ff81d7fa700 1 -- 172.21.15.27:0/51427507 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff81000e260 con 0x7ff820109ba0 2024-05-21T03:12:40.350 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.595+0000 7ff81f7fe700 1 -- 172.21.15.27:0/51427507 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7ff80803e580 msgr2=0x7ff808040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/175229478 2024-05-21T03:12:40.350 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.595+0000 7ff81f7fe700 1 --2- 172.21.15.27:0/51427507 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7ff80803e580 0x7ff808040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T03:12:40.350 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.720+0000 7ff8262f4700 1 -- 172.21.15.27:0/51427507 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7ff8201a0640 con 0x7ff820109ba0 2024-05-21T03:12:40.351 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.721+0000 7ff81d7fa700 1 -- 172.21.15.27:0/51427507 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7ff8201a0640 con 0x7ff820109ba0 2024-05-21T03:12:40.351 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.723+0000 7ff8262f4700 1 -- 172.21.15.27:0/51427507 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7ff80803e580 msgr2=0x7ff808040a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T03:12:40.351 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.723+0000 7ff8262f4700 1 --2- 172.21.15.27:0/51427507 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7ff80803e580 0x7ff808040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:40.351 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.723+0000 7ff8262f4700 1 -- 172.21.15.27:0/51427507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff820109ba0 msgr2=0x7ff82019cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:40.351 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.723+0000 7ff8262f4700 1 --2- 172.21.15.27:0/51427507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff820109ba0 0x7ff82019cfb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff810011ac0 tx=0x7ff810011ba0 comp rx=0 tx=0).stop 2024-05-21T03:12:40.351 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.723+0000 7ff8262f4700 1 -- 172.21.15.27:0/51427507 shutdown_connections 2024-05-21T03:12:40.351 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.723+0000 7ff8262f4700 1 --2- 172.21.15.27:0/51427507 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7ff80803e580 0x7ff808040a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:40.351 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.723+0000 7ff8262f4700 1 --2- 172.21.15.27:0/51427507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff820109ba0 0x7ff82019cfb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:40.351 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.723+0000 7ff8262f4700 1 -- 172.21.15.27:0/51427507 >> 172.21.15.27:0/51427507 conn(0x7ff820103610 msgr2=0x7ff820104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:40.351 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.724+0000 7ff8262f4700 1 -- 172.21.15.27:0/51427507 shutdown_connections 2024-05-21T03:12:40.351 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:39.724+0000 7ff8262f4700 1 -- 172.21.15.27:0/51427507 wait complete. 2024-05-21T03:12:40.351 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for the mgr to restart... 2024-05-21T03:12:40.351 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr epoch 5... 2024-05-21T03:12:40.604 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:40 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/51427507' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-21T03:12:40.604 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:40 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:40.505+0000 7f530434a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T03:12:40.869 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:40 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:40.687+0000 7f530434a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T03:12:40.869 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:40 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:40.868+0000 7f530434a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T03:12:41.426 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:41 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:41.171+0000 7f530434a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T03:12:41.427 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:41 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:41.267+0000 7f530434a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T03:12:41.681 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:41 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:41.479+0000 7f530434a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T03:12:42.176 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:41 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:41.681+0000 7f530434a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T03:12:42.544 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:42 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:42.314+0000 7f530434a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T03:12:42.545 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:42 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:42.544+0000 7f530434a200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-21T03:12:43.427 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:42 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:42.989+0000 7f530434a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T03:12:43.427 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:43 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:43.089+0000 7f530434a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T03:12:43.831 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:43 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:43.474+0000 7f530434a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T03:12:44.176 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:43 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:43.848+0000 7f530434a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T03:12:44.177 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:43 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:43.946+0000 7f530434a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T03:12:44.677 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:44 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:44.201+0000 7f530434a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T03:12:44.677 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:44 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:44.341+0000 7f530434a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T03:12:44.677 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:44 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:44.441+0000 7f530434a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T03:12:45.177 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:44 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:44.780+0000 7f530434a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T03:12:45.177 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:44 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:44.876+0000 7f530434a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T03:12:45.790 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:45 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:45.509+0000 7f530434a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T03:12:45.790 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:45 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:12:45.736+0000 7f530434a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T03:12:46.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:45 smithi027 ceph-mon[32179]: Active manager daemon x restarted 2024-05-21T03:12:46.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:45 smithi027 ceph-mon[32179]: Activating manager daemon x 2024-05-21T03:12:46.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:45 smithi027 ceph-mon[32179]: osdmap e2: 0 total, 0 up, 0 in 2024-05-21T03:12:46.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:45 smithi027 ceph-mon[32179]: mgrmap e6: x(active, starting, since 0.00369207s) 2024-05-21T03:12:46.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:45 smithi027 ceph-mon[32179]: from='mgr.14120 172.21.15.27:0/3461495745' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T03:12:46.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:45 smithi027 ceph-mon[32179]: from='mgr.14120 172.21.15.27:0/3461495745' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-21T03:12:46.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:45 smithi027 ceph-mon[32179]: from='mgr.14120 172.21.15.27:0/3461495745' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-21T03:12:46.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:45 smithi027 ceph-mon[32179]: from='mgr.14120 172.21.15.27:0/3461495745' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-21T03:12:46.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:45 smithi027 ceph-mon[32179]: from='mgr.14120 172.21.15.27:0/3461495745' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-21T03:12:46.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:45 smithi027 ceph-mon[32179]: Manager daemon x is now available 2024-05-21T03:12:46.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:45 smithi027 ceph-mon[32179]: from='mgr.14120 172.21.15.27:0/3461495745' entity='mgr.x' 2024-05-21T03:12:46.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:45 smithi027 ceph-mon[32179]: from='mgr.14120 172.21.15.27:0/3461495745' entity='mgr.x' 2024-05-21T03:12:46.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:45 smithi027 ceph-mon[32179]: from='mgr.14120 172.21.15.27:0/3461495745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:12:46.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:45 smithi027 ceph-mon[32179]: from='mgr.14120 172.21.15.27:0/3461495745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:12:47.119 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:46 smithi027 ceph-mon[32179]: Found migration_current of "None". Setting to last migration. 2024-05-21T03:12:47.119 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:46 smithi027 ceph-mon[32179]: from='mgr.14120 172.21.15.27:0/3461495745' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-21T03:12:47.119 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:46 smithi027 ceph-mon[32179]: from='mgr.14120 172.21.15.27:0/3461495745' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-21T03:12:47.119 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:46 smithi027 ceph-mon[32179]: mgrmap e7: x(active, since 1.00583s) 2024-05-21T03:12:47.396 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-05-21T03:12:47.397 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-21T03:12:47.397 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-21T03:12:47.397 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-05-21T03:12:47.397 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.821+0000 7fd6cb30a700 1 Processor -- start 2024-05-21T03:12:47.397 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.821+0000 7fd6cb30a700 1 -- start start 2024-05-21T03:12:47.397 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.822+0000 7fd6cb30a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6c41091f0 0x7fd6c4109610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:47.397 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.822+0000 7fd6cb30a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c4109be0 con 0x7fd6c41091f0 2024-05-21T03:12:47.397 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.822+0000 7fd6c90a6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6c41091f0 0x7fd6c4109610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:47.397 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.822+0000 7fd6c90a6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6c41091f0 0x7fd6c4109610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44846/0 (socket says 172.21.15.27:44846) 2024-05-21T03:12:47.397 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.822+0000 7fd6c90a6700 1 -- 172.21.15.27:0/3414883065 learned_addr learned my addr 172.21.15.27:0/3414883065 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:47.397 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.822+0000 7fd6c90a6700 1 -- 172.21.15.27:0/3414883065 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c410a350 con 0x7fd6c41091f0 2024-05-21T03:12:47.397 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.822+0000 7fd6c90a6700 1 --2- 172.21.15.27:0/3414883065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6c41091f0 0x7fd6c4109610 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00098a0 tx=0x7fd6c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7c2ca632be1e2e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:47.398 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.822+0000 7fd6bbfff700 1 -- 172.21.15.27:0/3414883065 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6c0004160 con 0x7fd6c41091f0 2024-05-21T03:12:47.398 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.822+0000 7fd6bbfff700 1 -- 172.21.15.27:0/3414883065 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd6c0004300 con 0x7fd6c41091f0 2024-05-21T03:12:47.398 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.823+0000 7fd6bbfff700 1 -- 172.21.15.27:0/3414883065 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6c0011af0 con 0x7fd6c41091f0 2024-05-21T03:12:47.398 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.823+0000 7fd6cb30a700 1 -- 172.21.15.27:0/3414883065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6c41091f0 msgr2=0x7fd6c4109610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:47.398 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.823+0000 7fd6cb30a700 1 --2- 172.21.15.27:0/3414883065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6c41091f0 0x7fd6c4109610 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00098a0 tx=0x7fd6c0009bb0 comp rx=0 tx=0).stop 2024-05-21T03:12:47.398 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.823+0000 7fd6cb30a700 1 -- 172.21.15.27:0/3414883065 shutdown_connections 2024-05-21T03:12:47.398 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.823+0000 7fd6cb30a700 1 --2- 172.21.15.27:0/3414883065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6c41091f0 0x7fd6c4109610 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:47.398 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.823+0000 7fd6cb30a700 1 -- 172.21.15.27:0/3414883065 >> 172.21.15.27:0/3414883065 conn(0x7fd6c41047f0 msgr2=0x7fd6c4106c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:47.398 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.823+0000 7fd6cb30a700 1 -- 172.21.15.27:0/3414883065 shutdown_connections 2024-05-21T03:12:47.398 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.823+0000 7fd6cb30a700 1 -- 172.21.15.27:0/3414883065 wait complete. 2024-05-21T03:12:47.398 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.824+0000 7fd6cb30a700 1 Processor -- start 2024-05-21T03:12:47.398 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.824+0000 7fd6cb30a700 1 -- start start 2024-05-21T03:12:47.399 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.824+0000 7fd6cb30a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6c41091f0 0x7fd6c419f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:47.399 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.824+0000 7fd6cb30a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c410d700 con 0x7fd6c41091f0 2024-05-21T03:12:47.399 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.824+0000 7fd6c90a6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6c41091f0 0x7fd6c419f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:47.399 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.824+0000 7fd6c90a6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6c41091f0 0x7fd6c419f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44862/0 (socket says 172.21.15.27:44862) 2024-05-21T03:12:47.399 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.824+0000 7fd6c90a6700 1 -- 172.21.15.27:0/1386711133 learned_addr learned my addr 172.21.15.27:0/1386711133 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:47.399 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.825+0000 7fd6c90a6700 1 -- 172.21.15.27:0/1386711133 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c419f780 con 0x7fd6c41091f0 2024-05-21T03:12:47.399 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.825+0000 7fd6c90a6700 1 --2- 172.21.15.27:0/1386711133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6c41091f0 0x7fd6c419f240 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0003f40 tx=0x7fd6c0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:47.399 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.825+0000 7fd6ba7fc700 1 -- 172.21.15.27:0/1386711133 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6c00038b0 con 0x7fd6c41091f0 2024-05-21T03:12:47.399 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.825+0000 7fd6ba7fc700 1 -- 172.21.15.27:0/1386711133 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd6c0003a50 con 0x7fd6c41091f0 2024-05-21T03:12:47.399 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.825+0000 7fd6cb30a700 1 -- 172.21.15.27:0/1386711133 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6c419fa10 con 0x7fd6c41091f0 2024-05-21T03:12:47.399 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.825+0000 7fd6ba7fc700 1 -- 172.21.15.27:0/1386711133 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6c001a5c0 con 0x7fd6c41091f0 2024-05-21T03:12:47.399 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.825+0000 7fd6cb30a700 1 -- 172.21.15.27:0/1386711133 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6c41a2700 con 0x7fd6c41091f0 2024-05-21T03:12:47.399 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.826+0000 7fd6ba7fc700 1 -- 172.21.15.27:0/1386711133 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fd6c001a870 con 0x7fd6c41091f0 2024-05-21T03:12:47.400 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.826+0000 7fd6ba7fc700 1 --2- 172.21.15.27:0/1386711133 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7fd6b003e550 0x7fd6b0040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:47.400 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.826+0000 7fd6ba7fc700 1 -- 172.21.15.27:0/1386711133 --> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd6b0041120 con 0x7fd6b003e550 2024-05-21T03:12:47.400 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.826+0000 7fd6ba7fc700 1 -- 172.21.15.27:0/1386711133 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd6c0053b20 con 0x7fd6c41091f0 2024-05-21T03:12:47.400 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.826+0000 7fd6c88a5700 1 -- 172.21.15.27:0/1386711133 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7fd6b003e550 msgr2=0x7fd6b0040a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/175229478 2024-05-21T03:12:47.400 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:40.826+0000 7fd6c88a5700 1 --2- 172.21.15.27:0/1386711133 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7fd6b003e550 0x7fd6b0040a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T03:12:47.400 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:41.027+0000 7fd6c88a5700 1 -- 172.21.15.27:0/1386711133 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7fd6b003e550 msgr2=0x7fd6b0040a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/175229478 2024-05-21T03:12:47.400 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:41.027+0000 7fd6c88a5700 1 --2- 172.21.15.27:0/1386711133 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7fd6b003e550 0x7fd6b0040a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T03:12:47.400 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:41.428+0000 7fd6c88a5700 1 -- 172.21.15.27:0/1386711133 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7fd6b003e550 msgr2=0x7fd6b0040a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/175229478 2024-05-21T03:12:47.400 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:41.428+0000 7fd6c88a5700 1 --2- 172.21.15.27:0/1386711133 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7fd6b003e550 0x7fd6b0040a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-21T03:12:47.400 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:42.229+0000 7fd6c88a5700 1 -- 172.21.15.27:0/1386711133 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7fd6b003e550 msgr2=0x7fd6b0040a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/175229478 2024-05-21T03:12:47.400 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:42.229+0000 7fd6c88a5700 1 --2- 172.21.15.27:0/1386711133 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7fd6b003e550 0x7fd6b0040a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-21T03:12:47.401 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:43.830+0000 7fd6c88a5700 1 -- 172.21.15.27:0/1386711133 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7fd6b003e550 msgr2=0x7fd6b0040a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/175229478 2024-05-21T03:12:47.401 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:43.830+0000 7fd6c88a5700 1 --2- 172.21.15.27:0/1386711133 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7fd6b003e550 0x7fd6b0040a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-21T03:12:47.401 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:45.740+0000 7fd6ba7fc700 1 -- 172.21.15.27:0/1386711133 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7fd6c001abb0 con 0x7fd6c41091f0 2024-05-21T03:12:47.401 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:45.740+0000 7fd6ba7fc700 1 -- 172.21.15.27:0/1386711133 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7fd6b003e550 msgr2=0x7fd6b0040a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T03:12:47.401 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:45.740+0000 7fd6ba7fc700 1 --2- 172.21.15.27:0/1386711133 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7fd6b003e550 0x7fd6b0040a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:47.401 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:46.741+0000 7fd6ba7fc700 1 -- 172.21.15.27:0/1386711133 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7fd6c0055530 con 0x7fd6c41091f0 2024-05-21T03:12:47.401 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:46.741+0000 7fd6ba7fc700 1 --2- 172.21.15.27:0/1386711133 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fd6b0041e90 0x7fd6b0044280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:47.401 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:46.741+0000 7fd6ba7fc700 1 -- 172.21.15.27:0/1386711133 --> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd6b0041120 con 0x7fd6b0041e90 2024-05-21T03:12:47.401 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:46.741+0000 7fd6c88a5700 1 --2- 172.21.15.27:0/1386711133 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fd6b0041e90 0x7fd6b0044280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:47.401 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:46.742+0000 7fd6c88a5700 1 --2- 172.21.15.27:0/1386711133 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fd6b0041e90 0x7fd6b0044280 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4003a10 tx=0x7fd6b40092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:47.401 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:46.743+0000 7fd6ba7fc700 1 -- 172.21.15.27:0/1386711133 <== mgr.14120 v2:172.21.15.27:6800/1185518061 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fd6b0041120 con 0x7fd6b0041e90 2024-05-21T03:12:47.401 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:46.751+0000 7fd6cb30a700 1 -- 172.21.15.27:0/1386711133 --> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fd6c4064ba0 con 0x7fd6b0041e90 2024-05-21T03:12:47.401 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:46.751+0000 7fd6ba7fc700 1 -- 172.21.15.27:0/1386711133 <== mgr.14120 v2:172.21.15.27:6800/1185518061 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fd6c4064ba0 con 0x7fd6b0041e90 2024-05-21T03:12:47.402 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:46.752+0000 7fd6cb30a700 1 -- 172.21.15.27:0/1386711133 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fd6b0041e90 msgr2=0x7fd6b0044280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:47.402 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:46.752+0000 7fd6cb30a700 1 --2- 172.21.15.27:0/1386711133 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fd6b0041e90 0x7fd6b0044280 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4003a10 tx=0x7fd6b40092b0 comp rx=0 tx=0).stop 2024-05-21T03:12:47.402 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:46.752+0000 7fd6cb30a700 1 -- 172.21.15.27:0/1386711133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6c41091f0 msgr2=0x7fd6c419f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:47.402 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:46.752+0000 7fd6cb30a700 1 --2- 172.21.15.27:0/1386711133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6c41091f0 0x7fd6c419f240 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0003f40 tx=0x7fd6c0011890 comp rx=0 tx=0).stop 2024-05-21T03:12:47.402 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:46.752+0000 7fd6cb30a700 1 -- 172.21.15.27:0/1386711133 shutdown_connections 2024-05-21T03:12:47.402 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:46.752+0000 7fd6cb30a700 1 --2- 172.21.15.27:0/1386711133 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fd6b0041e90 0x7fd6b0044280 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:47.402 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:46.752+0000 7fd6cb30a700 1 --2- 172.21.15.27:0/1386711133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6c41091f0 0x7fd6c419f240 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:47.402 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:46.752+0000 7fd6cb30a700 1 --2- 172.21.15.27:0/1386711133 >> [v2:172.21.15.27:6800/175229478,v1:172.21.15.27:6801/175229478] conn(0x7fd6b003e550 0x7fd6b0040a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:47.402 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:46.752+0000 7fd6cb30a700 1 -- 172.21.15.27:0/1386711133 >> 172.21.15.27:0/1386711133 conn(0x7fd6c41047f0 msgr2=0x7fd6c41051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:47.402 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:46.752+0000 7fd6cb30a700 1 -- 172.21.15.27:0/1386711133 shutdown_connections 2024-05-21T03:12:47.402 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:46.752+0000 7fd6cb30a700 1 -- 172.21.15.27:0/1386711133 wait complete. 2024-05-21T03:12:47.402 INFO:teuthology.orchestra.run.smithi027.stdout:mgr epoch 5 is available 2024-05-21T03:12:47.403 INFO:teuthology.orchestra.run.smithi027.stdout:Setting orchestrator backend to cephadm... 2024-05-21T03:12:48.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:48 smithi027 ceph-mon[32179]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-21T03:12:48.368 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:48 smithi027 ceph-mon[32179]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-21T03:12:48.368 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:48 smithi027 ceph-mon[32179]: from='mgr.14120 172.21.15.27:0/3461495745' entity='mgr.x' 2024-05-21T03:12:48.368 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:48 smithi027 ceph-mon[32179]: from='mgr.14120 172.21.15.27:0/3461495745' entity='mgr.x' 2024-05-21T03:12:48.368 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:48 smithi027 ceph-mon[32179]: from='mgr.14120 172.21.15.27:0/3461495745' entity='mgr.x' 2024-05-21T03:12:48.626 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.800+0000 7f7e8db64700 1 Processor -- start 2024-05-21T03:12:48.626 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.800+0000 7f7e8db64700 1 -- start start 2024-05-21T03:12:48.626 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.801+0000 7f7e8db64700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e88109bf0 0x7f7e8810bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:48.626 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.801+0000 7f7e8db64700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8807ce70 con 0x7f7e88109bf0 2024-05-21T03:12:48.626 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.801+0000 7f7e877fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e88109bf0 0x7f7e8810bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:48.626 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.801+0000 7f7e877fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e88109bf0 0x7f7e8810bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46614/0 (socket says 172.21.15.27:46614) 2024-05-21T03:12:48.626 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.801+0000 7f7e877fe700 1 -- 172.21.15.27:0/4176626889 learned_addr learned my addr 172.21.15.27:0/4176626889 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:48.626 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.801+0000 7f7e877fe700 1 -- 172.21.15.27:0/4176626889 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8810c520 con 0x7f7e88109bf0 2024-05-21T03:12:48.626 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.801+0000 7f7e877fe700 1 --2- 172.21.15.27:0/4176626889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e88109bf0 0x7f7e8810bfe0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7e780098a0 tx=0x7f7e78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2da52102b1d5165 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:48.627 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.801+0000 7f7e867fc700 1 -- 172.21.15.27:0/4176626889 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e78004160 con 0x7f7e88109bf0 2024-05-21T03:12:48.627 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.801+0000 7f7e867fc700 1 -- 172.21.15.27:0/4176626889 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7e78004300 con 0x7f7e88109bf0 2024-05-21T03:12:48.627 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.802+0000 7f7e867fc700 1 -- 172.21.15.27:0/4176626889 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e78011af0 con 0x7f7e88109bf0 2024-05-21T03:12:48.627 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.802+0000 7f7e8db64700 1 -- 172.21.15.27:0/4176626889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e88109bf0 msgr2=0x7f7e8810bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:48.627 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.802+0000 7f7e8db64700 1 --2- 172.21.15.27:0/4176626889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e88109bf0 0x7f7e8810bfe0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7e780098a0 tx=0x7f7e78009bb0 comp rx=0 tx=0).stop 2024-05-21T03:12:48.627 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.802+0000 7f7e8db64700 1 -- 172.21.15.27:0/4176626889 shutdown_connections 2024-05-21T03:12:48.627 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.802+0000 7f7e8db64700 1 --2- 172.21.15.27:0/4176626889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e88109bf0 0x7f7e8810bfe0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:48.627 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.802+0000 7f7e8db64700 1 -- 172.21.15.27:0/4176626889 >> 172.21.15.27:0/4176626889 conn(0x7f7e88103640 msgr2=0x7f7e88105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:48.627 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.802+0000 7f7e8db64700 1 -- 172.21.15.27:0/4176626889 shutdown_connections 2024-05-21T03:12:48.627 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.802+0000 7f7e8db64700 1 -- 172.21.15.27:0/4176626889 wait complete. 2024-05-21T03:12:48.627 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.803+0000 7f7e8db64700 1 Processor -- start 2024-05-21T03:12:48.627 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.803+0000 7f7e8db64700 1 -- start start 2024-05-21T03:12:48.627 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.804+0000 7f7e8db64700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e88109bf0 0x7f7e88109470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:48.627 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.804+0000 7f7e8db64700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8810d490 con 0x7f7e88109bf0 2024-05-21T03:12:48.628 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.804+0000 7f7e877fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e88109bf0 0x7f7e88109470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:48.628 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.804+0000 7f7e877fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e88109bf0 0x7f7e88109470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46626/0 (socket says 172.21.15.27:46626) 2024-05-21T03:12:48.628 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.804+0000 7f7e877fe700 1 -- 172.21.15.27:0/2424214821 learned_addr learned my addr 172.21.15.27:0/2424214821 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:48.628 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.804+0000 7f7e877fe700 1 -- 172.21.15.27:0/2424214821 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e881099b0 con 0x7f7e88109bf0 2024-05-21T03:12:48.628 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.804+0000 7f7e877fe700 1 --2- 172.21.15.27:0/2424214821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e88109bf0 0x7f7e88109470 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7e78011ac0 tx=0x7f7e78011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:48.628 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.804+0000 7f7e84ff9700 1 -- 172.21.15.27:0/2424214821 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e78003b20 con 0x7f7e88109bf0 2024-05-21T03:12:48.628 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.804+0000 7f7e8db64700 1 -- 172.21.15.27:0/2424214821 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e88107ba0 con 0x7f7e88109bf0 2024-05-21T03:12:48.628 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.804+0000 7f7e84ff9700 1 -- 172.21.15.27:0/2424214821 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7e7801a8c0 con 0x7f7e88109bf0 2024-05-21T03:12:48.628 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.804+0000 7f7e84ff9700 1 -- 172.21.15.27:0/2424214821 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e780248d0 con 0x7f7e88109bf0 2024-05-21T03:12:48.628 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.804+0000 7f7e8db64700 1 -- 172.21.15.27:0/2424214821 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e881080e0 con 0x7f7e88109bf0 2024-05-21T03:12:48.628 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.806+0000 7f7e8db64700 1 -- 172.21.15.27:0/2424214821 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e88064ba0 con 0x7f7e88109bf0 2024-05-21T03:12:48.628 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.806+0000 7f7e84ff9700 1 -- 172.21.15.27:0/2424214821 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f7e78022020 con 0x7f7e88109bf0 2024-05-21T03:12:48.628 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.806+0000 7f7e84ff9700 1 --2- 172.21.15.27:0/2424214821 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f7e7003e460 0x7f7e70040920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:48.628 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.806+0000 7f7e84ff9700 1 -- 172.21.15.27:0/2424214821 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7e780535e0 con 0x7f7e88109bf0 2024-05-21T03:12:48.629 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.806+0000 7f7e86ffd700 1 --2- 172.21.15.27:0/2424214821 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f7e7003e460 0x7f7e70040920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:48.629 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.807+0000 7f7e86ffd700 1 --2- 172.21.15.27:0/2424214821 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f7e7003e460 0x7f7e70040920 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c006fd0 tx=0x7f7e7c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:48.629 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:47.811+0000 7f7e84ff9700 1 -- 172.21.15.27:0/2424214821 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7e78029de0 con 0x7f7e88109bf0 2024-05-21T03:12:48.629 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:48.043+0000 7f7e8db64700 1 -- 172.21.15.27:0/2424214821 --> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f7e88108940 con 0x7f7e7003e460 2024-05-21T03:12:48.629 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:48.050+0000 7f7e84ff9700 1 -- 172.21.15.27:0/2424214821 <== mgr.14120 v2:172.21.15.27:6800/1185518061 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7e88108940 con 0x7f7e7003e460 2024-05-21T03:12:48.629 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:48.052+0000 7f7e8db64700 1 -- 172.21.15.27:0/2424214821 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f7e7003e460 msgr2=0x7f7e70040920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:48.629 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:48.052+0000 7f7e8db64700 1 --2- 172.21.15.27:0/2424214821 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f7e7003e460 0x7f7e70040920 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c006fd0 tx=0x7f7e7c006e40 comp rx=0 tx=0).stop 2024-05-21T03:12:48.629 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:48.052+0000 7f7e8db64700 1 -- 172.21.15.27:0/2424214821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e88109bf0 msgr2=0x7f7e88109470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:48.629 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:48.052+0000 7f7e8db64700 1 --2- 172.21.15.27:0/2424214821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e88109bf0 0x7f7e88109470 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7e78011ac0 tx=0x7f7e78011ba0 comp rx=0 tx=0).stop 2024-05-21T03:12:48.629 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:48.052+0000 7f7e8db64700 1 -- 172.21.15.27:0/2424214821 shutdown_connections 2024-05-21T03:12:48.629 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:48.052+0000 7f7e8db64700 1 --2- 172.21.15.27:0/2424214821 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f7e7003e460 0x7f7e70040920 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:48.629 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:48.052+0000 7f7e8db64700 1 --2- 172.21.15.27:0/2424214821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e88109bf0 0x7f7e88109470 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:48.629 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:48.052+0000 7f7e8db64700 1 -- 172.21.15.27:0/2424214821 >> 172.21.15.27:0/2424214821 conn(0x7f7e88103640 msgr2=0x7f7e88104110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:48.630 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:48.052+0000 7f7e8db64700 1 -- 172.21.15.27:0/2424214821 shutdown_connections 2024-05-21T03:12:48.630 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:48.053+0000 7f7e8db64700 1 -- 172.21.15.27:0/2424214821 wait complete. 2024-05-21T03:12:49.295 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:49 smithi027 ceph-mon[32179]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:12:49.295 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:49 smithi027 ceph-mon[32179]: from='mgr.14120 172.21.15.27:0/3461495745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:12:49.295 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:49 smithi027 ceph-mon[32179]: mgrmap e8: x(active, since 2s) 2024-05-21T03:12:49.295 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:49 smithi027 ceph-mon[32179]: [21/May/2024:03:12:48] ENGINE Bus STARTING 2024-05-21T03:12:49.295 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:49 smithi027 ceph-mon[32179]: [21/May/2024:03:12:48] ENGINE Serving on https://172.21.15.27:7150 2024-05-21T03:12:49.296 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:49 smithi027 ceph-mon[32179]: from='mgr.14120 172.21.15.27:0/3461495745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:12:49.882 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-21T03:12:49.882 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.071+0000 7f9f618b8700 1 Processor -- start 2024-05-21T03:12:49.882 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.071+0000 7f9f618b8700 1 -- start start 2024-05-21T03:12:49.882 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.071+0000 7f9f618b8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c107a70 0x7f9f5c107e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:49.882 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.071+0000 7f9f618b8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c07ce50 con 0x7f9f5c107a70 2024-05-21T03:12:49.882 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.071+0000 7f9f5affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c107a70 0x7f9f5c107e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:49.882 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.071+0000 7f9f5affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c107a70 0x7f9f5c107e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46638/0 (socket says 172.21.15.27:46638) 2024-05-21T03:12:49.883 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.071+0000 7f9f5affd700 1 -- 172.21.15.27:0/2706057602 learned_addr learned my addr 172.21.15.27:0/2706057602 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:49.883 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.071+0000 7f9f5affd700 1 -- 172.21.15.27:0/2706057602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5c1083d0 con 0x7f9f5c107a70 2024-05-21T03:12:49.883 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.072+0000 7f9f5affd700 1 --2- 172.21.15.27:0/2706057602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c107a70 0x7f9f5c107e90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0098a0 tx=0x7f9f4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1ca93632dfd9ae3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:49.883 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.072+0000 7f9f59ffb700 1 -- 172.21.15.27:0/2706057602 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f4c004160 con 0x7f9f5c107a70 2024-05-21T03:12:49.883 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.072+0000 7f9f59ffb700 1 -- 172.21.15.27:0/2706057602 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9f4c004300 con 0x7f9f5c107a70 2024-05-21T03:12:49.883 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.072+0000 7f9f59ffb700 1 -- 172.21.15.27:0/2706057602 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f4c011af0 con 0x7f9f5c107a70 2024-05-21T03:12:49.883 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.072+0000 7f9f618b8700 1 -- 172.21.15.27:0/2706057602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c107a70 msgr2=0x7f9f5c107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:49.883 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.072+0000 7f9f618b8700 1 --2- 172.21.15.27:0/2706057602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c107a70 0x7f9f5c107e90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0098a0 tx=0x7f9f4c009bb0 comp rx=0 tx=0).stop 2024-05-21T03:12:49.883 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.072+0000 7f9f618b8700 1 -- 172.21.15.27:0/2706057602 shutdown_connections 2024-05-21T03:12:49.883 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.072+0000 7f9f618b8700 1 --2- 172.21.15.27:0/2706057602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c107a70 0x7f9f5c107e90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:49.883 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.072+0000 7f9f618b8700 1 -- 172.21.15.27:0/2706057602 >> 172.21.15.27:0/2706057602 conn(0x7f9f5c103690 msgr2=0x7f9f5c105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:49.883 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.073+0000 7f9f618b8700 1 -- 172.21.15.27:0/2706057602 shutdown_connections 2024-05-21T03:12:49.883 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.073+0000 7f9f618b8700 1 -- 172.21.15.27:0/2706057602 wait complete. 2024-05-21T03:12:49.884 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.073+0000 7f9f618b8700 1 Processor -- start 2024-05-21T03:12:49.884 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.073+0000 7f9f618b8700 1 -- start start 2024-05-21T03:12:49.884 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.073+0000 7f9f618b8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c107a70 0x7f9f5c19f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:49.884 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.073+0000 7f9f618b8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c109400 con 0x7f9f5c107a70 2024-05-21T03:12:49.884 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.074+0000 7f9f5affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c107a70 0x7f9f5c19f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:49.884 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.074+0000 7f9f5affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c107a70 0x7f9f5c19f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46654/0 (socket says 172.21.15.27:46654) 2024-05-21T03:12:49.884 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.074+0000 7f9f5affd700 1 -- 172.21.15.27:0/1360648719 learned_addr learned my addr 172.21.15.27:0/1360648719 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:49.884 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.074+0000 7f9f5affd700 1 -- 172.21.15.27:0/1360648719 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5c19f780 con 0x7f9f5c107a70 2024-05-21T03:12:49.884 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.074+0000 7f9f5affd700 1 --2- 172.21.15.27:0/1360648719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c107a70 0x7f9f5c19f240 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c011ac0 tx=0x7f9f4c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:49.884 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.074+0000 7f9f608b6700 1 -- 172.21.15.27:0/1360648719 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f4c003970 con 0x7f9f5c107a70 2024-05-21T03:12:49.884 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.074+0000 7f9f608b6700 1 -- 172.21.15.27:0/1360648719 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9f4c003b10 con 0x7f9f5c107a70 2024-05-21T03:12:49.884 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.074+0000 7f9f618b8700 1 -- 172.21.15.27:0/1360648719 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f5c19fa10 con 0x7f9f5c107a70 2024-05-21T03:12:49.884 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.074+0000 7f9f608b6700 1 -- 172.21.15.27:0/1360648719 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f4c024610 con 0x7f9f5c107a70 2024-05-21T03:12:49.885 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.074+0000 7f9f618b8700 1 -- 172.21.15.27:0/1360648719 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f5c1a2700 con 0x7f9f5c107a70 2024-05-21T03:12:49.885 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.075+0000 7f9f618b8700 1 -- 172.21.15.27:0/1360648719 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f5c064ba0 con 0x7f9f5c107a70 2024-05-21T03:12:49.885 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.075+0000 7f9f608b6700 1 -- 172.21.15.27:0/1360648719 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f9f4c024840 con 0x7f9f5c107a70 2024-05-21T03:12:49.885 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.076+0000 7f9f608b6700 1 --2- 172.21.15.27:0/1360648719 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f9f4403e5d0 0x7f9f44040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:49.885 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.076+0000 7f9f608b6700 1 -- 172.21.15.27:0/1360648719 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9f4c053980 con 0x7f9f5c107a70 2024-05-21T03:12:49.885 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.076+0000 7f9f5a7fc700 1 --2- 172.21.15.27:0/1360648719 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f9f4403e5d0 0x7f9f44040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:49.885 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.076+0000 7f9f5a7fc700 1 --2- 172.21.15.27:0/1360648719 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f9f4403e5d0 0x7f9f44040a90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9f50006fd0 tx=0x7f9f50006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:49.885 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.081+0000 7f9f608b6700 1 -- 172.21.15.27:0/1360648719 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9f4c02c350 con 0x7f9f5c107a70 2024-05-21T03:12:49.885 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.294+0000 7f9f618b8700 1 -- 172.21.15.27:0/1360648719 --> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f9f5c0008d0 con 0x7f9f4403e5d0 2024-05-21T03:12:49.885 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.295+0000 7f9f608b6700 1 -- 172.21.15.27:0/1360648719 <== mgr.14120 v2:172.21.15.27:6800/1185518061 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f9f5c0008d0 con 0x7f9f4403e5d0 2024-05-21T03:12:49.885 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.297+0000 7f9f618b8700 1 -- 172.21.15.27:0/1360648719 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f9f4403e5d0 msgr2=0x7f9f44040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:49.885 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.297+0000 7f9f618b8700 1 --2- 172.21.15.27:0/1360648719 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f9f4403e5d0 0x7f9f44040a90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9f50006fd0 tx=0x7f9f50006e40 comp rx=0 tx=0).stop 2024-05-21T03:12:49.885 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.297+0000 7f9f618b8700 1 -- 172.21.15.27:0/1360648719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c107a70 msgr2=0x7f9f5c19f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:49.885 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.297+0000 7f9f618b8700 1 --2- 172.21.15.27:0/1360648719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c107a70 0x7f9f5c19f240 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c011ac0 tx=0x7f9f4c011ba0 comp rx=0 tx=0).stop 2024-05-21T03:12:49.886 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.297+0000 7f9f618b8700 1 -- 172.21.15.27:0/1360648719 shutdown_connections 2024-05-21T03:12:49.886 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.297+0000 7f9f618b8700 1 --2- 172.21.15.27:0/1360648719 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f9f4403e5d0 0x7f9f44040a90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:49.886 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.297+0000 7f9f618b8700 1 --2- 172.21.15.27:0/1360648719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c107a70 0x7f9f5c19f240 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:49.886 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.297+0000 7f9f618b8700 1 -- 172.21.15.27:0/1360648719 >> 172.21.15.27:0/1360648719 conn(0x7f9f5c103690 msgr2=0x7f9f5c195ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:49.886 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.298+0000 7f9f618b8700 1 -- 172.21.15.27:0/1360648719 shutdown_connections 2024-05-21T03:12:49.886 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:49.298+0000 7f9f618b8700 1 -- 172.21.15.27:0/1360648719 wait complete. 2024-05-21T03:12:49.886 INFO:teuthology.orchestra.run.smithi027.stdout:Generating ssh key... 2024-05-21T03:12:50.126 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:50 smithi027 ceph-mon[32179]: [21/May/2024:03:12:48] ENGINE Serving on http://172.21.15.27:8765 2024-05-21T03:12:50.126 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:50 smithi027 ceph-mon[32179]: [21/May/2024:03:12:48] ENGINE Bus STARTED 2024-05-21T03:12:50.126 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:50 smithi027 ceph-mon[32179]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:12:50.927 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:50 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: Generating public/private rsa key pair. 2024-05-21T03:12:50.928 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:50 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: Your identification has been saved in /tmp/tmp32ejrpj2/key. 2024-05-21T03:12:50.928 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:50 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: Your public key has been saved in /tmp/tmp32ejrpj2/key.pub. 2024-05-21T03:12:50.928 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:50 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: The key fingerprint is: 2024-05-21T03:12:50.928 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:50 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: SHA256:2ROeygNWnfZfieMCIXUGvsZPHRwFPFP/eZET1nPu60c ceph-c9ca94be-171f-11ef-bc9a-c7b262605968 2024-05-21T03:12:50.928 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:50 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: The key's randomart image is: 2024-05-21T03:12:50.928 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:50 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: +---[RSA 3072]----+ 2024-05-21T03:12:50.928 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:50 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: | o.o.o+=.| 2024-05-21T03:12:50.928 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:50 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: | o.o..+o.*| 2024-05-21T03:12:50.928 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:50 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: | ..o= oo=+| 2024-05-21T03:12:50.928 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:50 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: | .o=o+. o B| 2024-05-21T03:12:50.928 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:50 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: | o S==..+ ++| 2024-05-21T03:12:50.928 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:50 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: | . o..+.....E| 2024-05-21T03:12:50.929 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:50 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: | + o ....| 2024-05-21T03:12:50.929 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:50 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: | . . ..| 2024-05-21T03:12:50.929 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:50 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: | ...| 2024-05-21T03:12:50.929 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:12:50 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: +----[SHA256]-----+ 2024-05-21T03:12:51.306 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.325+0000 7f47c5b48700 1 Processor -- start 2024-05-21T03:12:51.306 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.325+0000 7f47c5b48700 1 -- start start 2024-05-21T03:12:51.307 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.325+0000 7f47c5b48700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47c0109c60 0x7f47c010c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:51.307 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.325+0000 7f47c5b48700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f47c007ce70 con 0x7f47c0109c60 2024-05-21T03:12:51.307 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.325+0000 7f47bf7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47c0109c60 0x7f47c010c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:51.307 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.325+0000 7f47bf7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47c0109c60 0x7f47c010c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46664/0 (socket says 172.21.15.27:46664) 2024-05-21T03:12:51.307 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.325+0000 7f47bf7fe700 1 -- 172.21.15.27:0/28548175 learned_addr learned my addr 172.21.15.27:0/28548175 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:51.307 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.326+0000 7f47bf7fe700 1 -- 172.21.15.27:0/28548175 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c010c590 con 0x7f47c0109c60 2024-05-21T03:12:51.307 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.326+0000 7f47bf7fe700 1 --2- 172.21.15.27:0/28548175 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47c0109c60 0x7f47c010c050 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f47b00098a0 tx=0x7f47b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1beee1b552196f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:51.307 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.326+0000 7f47be7fc700 1 -- 172.21.15.27:0/28548175 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47b0004160 con 0x7f47c0109c60 2024-05-21T03:12:51.307 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.326+0000 7f47be7fc700 1 -- 172.21.15.27:0/28548175 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f47b0004300 con 0x7f47c0109c60 2024-05-21T03:12:51.307 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.326+0000 7f47be7fc700 1 -- 172.21.15.27:0/28548175 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47b0011af0 con 0x7f47c0109c60 2024-05-21T03:12:51.307 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.327+0000 7f47c5b48700 1 -- 172.21.15.27:0/28548175 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47c0109c60 msgr2=0x7f47c010c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:51.308 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.327+0000 7f47c5b48700 1 --2- 172.21.15.27:0/28548175 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47c0109c60 0x7f47c010c050 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f47b00098a0 tx=0x7f47b0009bb0 comp rx=0 tx=0).stop 2024-05-21T03:12:51.308 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.327+0000 7f47c5b48700 1 -- 172.21.15.27:0/28548175 shutdown_connections 2024-05-21T03:12:51.308 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.327+0000 7f47c5b48700 1 --2- 172.21.15.27:0/28548175 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47c0109c60 0x7f47c010c050 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:51.308 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.327+0000 7f47c5b48700 1 -- 172.21.15.27:0/28548175 >> 172.21.15.27:0/28548175 conn(0x7f47c0103690 msgr2=0x7f47c0105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:51.308 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.327+0000 7f47c5b48700 1 -- 172.21.15.27:0/28548175 shutdown_connections 2024-05-21T03:12:51.308 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.327+0000 7f47c5b48700 1 -- 172.21.15.27:0/28548175 wait complete. 2024-05-21T03:12:51.308 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.327+0000 7f47c5b48700 1 Processor -- start 2024-05-21T03:12:51.308 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.328+0000 7f47c5b48700 1 -- start start 2024-05-21T03:12:51.308 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.328+0000 7f47c5b48700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47c0109c60 0x7f47c019d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:51.308 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.328+0000 7f47c5b48700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f47c010d500 con 0x7f47c0109c60 2024-05-21T03:12:51.308 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.328+0000 7f47bf7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47c0109c60 0x7f47c019d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:51.309 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.328+0000 7f47bf7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47c0109c60 0x7f47c019d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46674/0 (socket says 172.21.15.27:46674) 2024-05-21T03:12:51.309 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.328+0000 7f47bf7fe700 1 -- 172.21.15.27:0/2986034540 learned_addr learned my addr 172.21.15.27:0/2986034540 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:51.309 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.328+0000 7f47bf7fe700 1 -- 172.21.15.27:0/2986034540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c019d580 con 0x7f47c0109c60 2024-05-21T03:12:51.309 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.328+0000 7f47bf7fe700 1 --2- 172.21.15.27:0/2986034540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47c0109c60 0x7f47c019d040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009fb0 tx=0x7f47b0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:51.309 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.329+0000 7f47bcff9700 1 -- 172.21.15.27:0/2986034540 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47b00036a0 con 0x7f47c0109c60 2024-05-21T03:12:51.309 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.329+0000 7f47bcff9700 1 -- 172.21.15.27:0/2986034540 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f47b001a430 con 0x7f47c0109c60 2024-05-21T03:12:51.309 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.329+0000 7f47bcff9700 1 -- 172.21.15.27:0/2986034540 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47b0019550 con 0x7f47c0109c60 2024-05-21T03:12:51.309 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.329+0000 7f47c5b48700 1 -- 172.21.15.27:0/2986034540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c019d810 con 0x7f47c0109c60 2024-05-21T03:12:51.309 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.329+0000 7f47c5b48700 1 -- 172.21.15.27:0/2986034540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c01a0500 con 0x7f47c0109c60 2024-05-21T03:12:51.309 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.330+0000 7f47bcff9700 1 -- 172.21.15.27:0/2986034540 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f47b0003840 con 0x7f47c0109c60 2024-05-21T03:12:51.309 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.330+0000 7f47c5b48700 1 -- 172.21.15.27:0/2986034540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47c0064ba0 con 0x7f47c0109c60 2024-05-21T03:12:51.309 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.330+0000 7f47bcff9700 1 --2- 172.21.15.27:0/2986034540 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f47a803e4e0 0x7f47a80409a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:51.310 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.330+0000 7f47bcff9700 1 -- 172.21.15.27:0/2986034540 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f47b00538a0 con 0x7f47c0109c60 2024-05-21T03:12:51.310 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.330+0000 7f47beffd700 1 --2- 172.21.15.27:0/2986034540 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f47a803e4e0 0x7f47a80409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:51.310 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.330+0000 7f47beffd700 1 --2- 172.21.15.27:0/2986034540 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f47a803e4e0 0x7f47a80409a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f47b4006fd0 tx=0x7f47b4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:51.310 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.335+0000 7f47bcff9700 1 -- 172.21.15.27:0/2986034540 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f47b00172b0 con 0x7f47c0109c60 2024-05-21T03:12:51.310 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.555+0000 7f47c5b48700 1 -- 172.21.15.27:0/2986034540 --> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f47c0104810 con 0x7f47a803e4e0 2024-05-21T03:12:51.310 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.735+0000 7f47bcff9700 1 -- 172.21.15.27:0/2986034540 <== mgr.14120 v2:172.21.15.27:6800/1185518061 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f47c0104810 con 0x7f47a803e4e0 2024-05-21T03:12:51.310 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.737+0000 7f47c5b48700 1 -- 172.21.15.27:0/2986034540 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f47a803e4e0 msgr2=0x7f47a80409a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:51.310 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.737+0000 7f47c5b48700 1 --2- 172.21.15.27:0/2986034540 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f47a803e4e0 0x7f47a80409a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f47b4006fd0 tx=0x7f47b4006e40 comp rx=0 tx=0).stop 2024-05-21T03:12:51.310 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.737+0000 7f47c5b48700 1 -- 172.21.15.27:0/2986034540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47c0109c60 msgr2=0x7f47c019d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:51.310 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.737+0000 7f47c5b48700 1 --2- 172.21.15.27:0/2986034540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47c0109c60 0x7f47c019d040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009fb0 tx=0x7f47b0011940 comp rx=0 tx=0).stop 2024-05-21T03:12:51.310 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.737+0000 7f47c5b48700 1 -- 172.21.15.27:0/2986034540 shutdown_connections 2024-05-21T03:12:51.311 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.737+0000 7f47c5b48700 1 --2- 172.21.15.27:0/2986034540 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f47a803e4e0 0x7f47a80409a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:51.311 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.737+0000 7f47c5b48700 1 --2- 172.21.15.27:0/2986034540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47c0109c60 0x7f47c019d040 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:51.311 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.737+0000 7f47c5b48700 1 -- 172.21.15.27:0/2986034540 >> 172.21.15.27:0/2986034540 conn(0x7f47c0103690 msgr2=0x7f47c0104100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:51.311 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.738+0000 7f47c5b48700 1 -- 172.21.15.27:0/2986034540 shutdown_connections 2024-05-21T03:12:51.311 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:50.738+0000 7f47c5b48700 1 -- 172.21.15.27:0/2986034540 wait complete. 2024-05-21T03:12:51.979 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:51 smithi027 ceph-mon[32179]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:12:51.979 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:51 smithi027 ceph-mon[32179]: Generating ssh key... 2024-05-21T03:12:51.979 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:51 smithi027 ceph-mon[32179]: from='mgr.14120 172.21.15.27:0/3461495745' entity='mgr.x' 2024-05-21T03:12:51.979 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:51 smithi027 ceph-mon[32179]: from='mgr.14120 172.21.15.27:0/3461495745' entity='mgr.x' 2024-05-21T03:12:52.562 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRXneae/LCKt5IPkYIMkCI1LKoJecyDq09qcZ8b7G1KAQ9u28IcR13W1wFNA/KFnWjSv6cWa0MChMu/owZQ+cBalVPyOPb5JPuxWS5lXumhgci1yockXWkonaoOTAd4x/3htk7VFBZuY/+vzZIGhN3DeBNUuBAQvTGg+UjURd3yZzDGzJEtem6YPWh8mDgFhWfKviqactMRpOMTd9cNgYzq0zaKrla4vTKLnJyGGekQElhw5BKQU0wgYNUrD37g4Y6X/MnLY+iTaggZHOIKrI5CgbfNoytnTupLjt6nv+gHxFlWQpdS5hkBA2yIaWum0rmnxUVOwoMzeJKsB4MR54uWV8v5zkwnEsG/ozgBEyK5p95EVuJn0vMCh/vpkyQeK+LtJwrYSoK7IZmV5CBlcWo9j/UpT1kNfnDecbkB/b6uzjAtgRlt1scZT6Awr6KzfTtCJj5cJ1D+xIFGW0u/zZAR5/xFHZB2iyi4Gg/TjxoCYp0StZhmhxCe0nqLGmTKy0= ceph-c9ca94be-171f-11ef-bc9a-c7b262605968 2024-05-21T03:12:52.562 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.752+0000 7f2e09a34700 1 Processor -- start 2024-05-21T03:12:52.563 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.752+0000 7f2e09a34700 1 -- start start 2024-05-21T03:12:52.563 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.752+0000 7f2e09a34700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e041091e0 0x7f2e04109600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:52.563 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.752+0000 7f2e09a34700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2e04109bd0 con 0x7f2e041091e0 2024-05-21T03:12:52.563 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.752+0000 7f2e02ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e041091e0 0x7f2e04109600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:52.563 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.752+0000 7f2e02ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e041091e0 0x7f2e04109600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46680/0 (socket says 172.21.15.27:46680) 2024-05-21T03:12:52.563 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.752+0000 7f2e02ffd700 1 -- 172.21.15.27:0/1983311525 learned_addr learned my addr 172.21.15.27:0/1983311525 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:52.563 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.752+0000 7f2e02ffd700 1 -- 172.21.15.27:0/1983311525 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0410a340 con 0x7f2e041091e0 2024-05-21T03:12:52.563 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.753+0000 7f2e02ffd700 1 --2- 172.21.15.27:0/1983311525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e041091e0 0x7f2e04109600 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2df40098a0 tx=0x7f2df4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6e0685e325dd38 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:52.563 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.753+0000 7f2e01ffb700 1 -- 172.21.15.27:0/1983311525 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2df4004160 con 0x7f2e041091e0 2024-05-21T03:12:52.563 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.753+0000 7f2e01ffb700 1 -- 172.21.15.27:0/1983311525 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2df4004300 con 0x7f2e041091e0 2024-05-21T03:12:52.563 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.753+0000 7f2e01ffb700 1 -- 172.21.15.27:0/1983311525 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2df4011af0 con 0x7f2e041091e0 2024-05-21T03:12:52.563 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.753+0000 7f2e09a34700 1 -- 172.21.15.27:0/1983311525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e041091e0 msgr2=0x7f2e04109600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:52.563 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.753+0000 7f2e09a34700 1 --2- 172.21.15.27:0/1983311525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e041091e0 0x7f2e04109600 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2df40098a0 tx=0x7f2df4009bb0 comp rx=0 tx=0).stop 2024-05-21T03:12:52.563 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.753+0000 7f2e09a34700 1 -- 172.21.15.27:0/1983311525 shutdown_connections 2024-05-21T03:12:52.564 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.753+0000 7f2e09a34700 1 --2- 172.21.15.27:0/1983311525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e041091e0 0x7f2e04109600 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:52.564 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.753+0000 7f2e09a34700 1 -- 172.21.15.27:0/1983311525 >> 172.21.15.27:0/1983311525 conn(0x7f2e04104840 msgr2=0x7f2e04106c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:52.564 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.753+0000 7f2e09a34700 1 -- 172.21.15.27:0/1983311525 shutdown_connections 2024-05-21T03:12:52.564 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.753+0000 7f2e09a34700 1 -- 172.21.15.27:0/1983311525 wait complete. 2024-05-21T03:12:52.564 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.754+0000 7f2e09a34700 1 Processor -- start 2024-05-21T03:12:52.564 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.754+0000 7f2e09a34700 1 -- start start 2024-05-21T03:12:52.564 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.754+0000 7f2e09a34700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e041091e0 0x7f2e0419d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:52.564 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.754+0000 7f2e09a34700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0410d6f0 con 0x7f2e041091e0 2024-05-21T03:12:52.564 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.754+0000 7f2e02ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e041091e0 0x7f2e0419d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:52.564 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.755+0000 7f2e02ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e041091e0 0x7f2e0419d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46696/0 (socket says 172.21.15.27:46696) 2024-05-21T03:12:52.564 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.755+0000 7f2e02ffd700 1 -- 172.21.15.27:0/606177057 learned_addr learned my addr 172.21.15.27:0/606177057 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:52.564 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.755+0000 7f2e02ffd700 1 -- 172.21.15.27:0/606177057 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0419d540 con 0x7f2e041091e0 2024-05-21T03:12:52.564 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.755+0000 7f2e02ffd700 1 --2- 172.21.15.27:0/606177057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e041091e0 0x7f2e0419d000 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2df40117b0 tx=0x7f2df4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:52.564 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.755+0000 7f2e08a32700 1 -- 172.21.15.27:0/606177057 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2df4003af0 con 0x7f2e041091e0 2024-05-21T03:12:52.565 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.755+0000 7f2e08a32700 1 -- 172.21.15.27:0/606177057 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2df4020070 con 0x7f2e041091e0 2024-05-21T03:12:52.565 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.755+0000 7f2e09a34700 1 -- 172.21.15.27:0/606177057 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e0419d7d0 con 0x7f2e041091e0 2024-05-21T03:12:52.565 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.755+0000 7f2e08a32700 1 -- 172.21.15.27:0/606177057 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2df4019630 con 0x7f2e041091e0 2024-05-21T03:12:52.565 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.755+0000 7f2e09a34700 1 -- 172.21.15.27:0/606177057 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e041a04c0 con 0x7f2e041091e0 2024-05-21T03:12:52.565 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.756+0000 7f2e08a32700 1 -- 172.21.15.27:0/606177057 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f2df40198a0 con 0x7f2e041091e0 2024-05-21T03:12:52.565 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.756+0000 7f2e08a32700 1 --2- 172.21.15.27:0/606177057 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f2df003e5d0 0x7f2df0040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:52.565 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.756+0000 7f2e09a34700 1 -- 172.21.15.27:0/606177057 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e04064ba0 con 0x7f2e041091e0 2024-05-21T03:12:52.565 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.756+0000 7f2e08a32700 1 -- 172.21.15.27:0/606177057 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2df4053af0 con 0x7f2e041091e0 2024-05-21T03:12:52.565 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.756+0000 7f2e027fc700 1 --2- 172.21.15.27:0/606177057 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f2df003e5d0 0x7f2df0040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:52.565 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.757+0000 7f2e027fc700 1 --2- 172.21.15.27:0/606177057 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f2df003e5d0 0x7f2df0040a90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2dec006fd0 tx=0x7f2dec006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:52.565 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.762+0000 7f2e08a32700 1 -- 172.21.15.27:0/606177057 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2df40042a0 con 0x7f2e041091e0 2024-05-21T03:12:52.565 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.979+0000 7f2e09a34700 1 -- 172.21.15.27:0/606177057 --> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f2e041059a0 con 0x7f2df003e5d0 2024-05-21T03:12:52.565 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.979+0000 7f2e08a32700 1 -- 172.21.15.27:0/606177057 <== mgr.14120 v2:172.21.15.27:6800/1185518061 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f2e041059a0 con 0x7f2df003e5d0 2024-05-21T03:12:52.565 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.982+0000 7f2e09a34700 1 -- 172.21.15.27:0/606177057 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f2df003e5d0 msgr2=0x7f2df0040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:52.566 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.982+0000 7f2e09a34700 1 --2- 172.21.15.27:0/606177057 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f2df003e5d0 0x7f2df0040a90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2dec006fd0 tx=0x7f2dec006e40 comp rx=0 tx=0).stop 2024-05-21T03:12:52.566 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.982+0000 7f2e09a34700 1 -- 172.21.15.27:0/606177057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e041091e0 msgr2=0x7f2e0419d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:52.566 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.982+0000 7f2e09a34700 1 --2- 172.21.15.27:0/606177057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e041091e0 0x7f2e0419d000 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2df40117b0 tx=0x7f2df4011b70 comp rx=0 tx=0).stop 2024-05-21T03:12:52.566 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.982+0000 7f2e09a34700 1 -- 172.21.15.27:0/606177057 shutdown_connections 2024-05-21T03:12:52.566 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.982+0000 7f2e09a34700 1 --2- 172.21.15.27:0/606177057 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f2df003e5d0 0x7f2df0040a90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:52.566 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.982+0000 7f2e09a34700 1 --2- 172.21.15.27:0/606177057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e041091e0 0x7f2e0419d000 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:52.566 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.982+0000 7f2e09a34700 1 -- 172.21.15.27:0/606177057 >> 172.21.15.27:0/606177057 conn(0x7f2e04104840 msgr2=0x7f2e04105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:52.566 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.983+0000 7f2e09a34700 1 -- 172.21.15.27:0/606177057 shutdown_connections 2024-05-21T03:12:52.566 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:51.983+0000 7f2e09a34700 1 -- 172.21.15.27:0/606177057 wait complete. 2024-05-21T03:12:52.566 INFO:teuthology.orchestra.run.smithi027.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-21T03:12:52.566 INFO:teuthology.orchestra.run.smithi027.stdout:Adding key to root@localhost authorized_keys... 2024-05-21T03:12:52.566 INFO:teuthology.orchestra.run.smithi027.stdout:Adding host smithi027... 2024-05-21T03:12:52.804 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:52 smithi027 ceph-mon[32179]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:12:53.885 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:53 smithi027 ceph-mon[32179]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi027", "addr": "172.21.15.27", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:12:55.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:54 smithi027 ceph-mon[32179]: Deploying cephadm binary to smithi027 2024-05-21T03:12:58.111 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout Added host 'smithi027' with addr '172.21.15.27' 2024-05-21T03:12:58.111 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.020+0000 7f3b37f95700 1 Processor -- start 2024-05-21T03:12:58.111 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.020+0000 7f3b37f95700 1 -- start start 2024-05-21T03:12:58.111 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.021+0000 7f3b37f95700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b30108030 0x7f3b30108450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:58.111 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.021+0000 7f3b37f95700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3b30108a20 con 0x7f3b30108030 2024-05-21T03:12:58.111 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.021+0000 7f3b35d31700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b30108030 0x7f3b30108450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:58.111 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.021+0000 7f3b35d31700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b30108030 0x7f3b30108450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50216/0 (socket says 172.21.15.27:50216) 2024-05-21T03:12:58.112 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.021+0000 7f3b35d31700 1 -- 172.21.15.27:0/3191904863 learned_addr learned my addr 172.21.15.27:0/3191904863 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:58.112 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.021+0000 7f3b35d31700 1 -- 172.21.15.27:0/3191904863 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b30109230 con 0x7f3b30108030 2024-05-21T03:12:58.112 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.021+0000 7f3b35d31700 1 --2- 172.21.15.27:0/3191904863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b30108030 0x7f3b30108450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c00bb40 tx=0x7f3b2c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483339ea1572cc4c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:58.112 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.022+0000 7f3b34d2f700 1 -- 172.21.15.27:0/3191904863 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b2c00dbe0 con 0x7f3b30108030 2024-05-21T03:12:58.112 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.022+0000 7f3b34d2f700 1 -- 172.21.15.27:0/3191904863 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3b2c0044c0 con 0x7f3b30108030 2024-05-21T03:12:58.112 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.022+0000 7f3b34d2f700 1 -- 172.21.15.27:0/3191904863 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b2c010470 con 0x7f3b30108030 2024-05-21T03:12:58.112 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.022+0000 7f3b37f95700 1 -- 172.21.15.27:0/3191904863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b30108030 msgr2=0x7f3b30108450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:58.112 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.022+0000 7f3b37f95700 1 --2- 172.21.15.27:0/3191904863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b30108030 0x7f3b30108450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c00bb40 tx=0x7f3b2c00be50 comp rx=0 tx=0).stop 2024-05-21T03:12:58.112 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.022+0000 7f3b37f95700 1 -- 172.21.15.27:0/3191904863 shutdown_connections 2024-05-21T03:12:58.112 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.022+0000 7f3b37f95700 1 --2- 172.21.15.27:0/3191904863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b30108030 0x7f3b30108450 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:58.112 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.022+0000 7f3b37f95700 1 -- 172.21.15.27:0/3191904863 >> 172.21.15.27:0/3191904863 conn(0x7f3b30103610 msgr2=0x7f3b30105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:58.112 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.022+0000 7f3b37f95700 1 -- 172.21.15.27:0/3191904863 shutdown_connections 2024-05-21T03:12:58.112 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.022+0000 7f3b37f95700 1 -- 172.21.15.27:0/3191904863 wait complete. 2024-05-21T03:12:58.112 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.023+0000 7f3b37f95700 1 Processor -- start 2024-05-21T03:12:58.113 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.023+0000 7f3b37f95700 1 -- start start 2024-05-21T03:12:58.113 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.023+0000 7f3b37f95700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b30108030 0x7f3b3019ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:58.113 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.023+0000 7f3b37f95700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3007bff0 con 0x7f3b30108030 2024-05-21T03:12:58.113 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.023+0000 7f3b35d31700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b30108030 0x7f3b3019ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:58.113 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.023+0000 7f3b35d31700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b30108030 0x7f3b3019ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50218/0 (socket says 172.21.15.27:50218) 2024-05-21T03:12:58.113 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.023+0000 7f3b35d31700 1 -- 172.21.15.27:0/190972637 learned_addr learned my addr 172.21.15.27:0/190972637 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:58.113 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.024+0000 7f3b35d31700 1 -- 172.21.15.27:0/190972637 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b3019b2d0 con 0x7f3b30108030 2024-05-21T03:12:58.113 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.024+0000 7f3b35d31700 1 --2- 172.21.15.27:0/190972637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b30108030 0x7f3b3019ad90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c00b340 tx=0x7f3b2c003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:58.113 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.024+0000 7f3b26ffd700 1 -- 172.21.15.27:0/190972637 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b2c027070 con 0x7f3b30108030 2024-05-21T03:12:58.113 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.024+0000 7f3b26ffd700 1 -- 172.21.15.27:0/190972637 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3b2c010bb0 con 0x7f3b30108030 2024-05-21T03:12:58.113 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.024+0000 7f3b37f95700 1 -- 172.21.15.27:0/190972637 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b3019b560 con 0x7f3b30108030 2024-05-21T03:12:58.113 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.024+0000 7f3b26ffd700 1 -- 172.21.15.27:0/190972637 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b2c022ac0 con 0x7f3b30108030 2024-05-21T03:12:58.113 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.024+0000 7f3b37f95700 1 -- 172.21.15.27:0/190972637 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b3019e200 con 0x7f3b30108030 2024-05-21T03:12:58.113 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.025+0000 7f3b26ffd700 1 -- 172.21.15.27:0/190972637 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f3b2c02a070 con 0x7f3b30108030 2024-05-21T03:12:58.114 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.025+0000 7f3b37f95700 1 -- 172.21.15.27:0/190972637 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b30064ba0 con 0x7f3b30108030 2024-05-21T03:12:58.114 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.025+0000 7f3b26ffd700 1 --2- 172.21.15.27:0/190972637 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f3b1c03e5d0 0x7f3b1c040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:58.114 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.025+0000 7f3b26ffd700 1 -- 172.21.15.27:0/190972637 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3b2c01f070 con 0x7f3b30108030 2024-05-21T03:12:58.114 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.025+0000 7f3b35530700 1 --2- 172.21.15.27:0/190972637 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f3b1c03e5d0 0x7f3b1c040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:58.114 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.026+0000 7f3b35530700 1 --2- 172.21.15.27:0/190972637 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f3b1c03e5d0 0x7f3b1c040a90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3b20006fd0 tx=0x7f3b20006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:58.114 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.030+0000 7f3b26ffd700 1 -- 172.21.15.27:0/190972637 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3b2c015070 con 0x7f3b30108030 2024-05-21T03:12:58.114 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:53.250+0000 7f3b37f95700 1 -- 172.21.15.27:0/190972637 --> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi027", "addr": "172.21.15.27", "target": ["mon-mgr", ""]}) -- 0x7f3b30063980 con 0x7f3b1c03e5d0 2024-05-21T03:12:58.114 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:57.460+0000 7f3b26ffd700 1 -- 172.21.15.27:0/190972637 <== mgr.14120 v2:172.21.15.27:6800/1185518061 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f3b30063980 con 0x7f3b1c03e5d0 2024-05-21T03:12:58.114 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:57.463+0000 7f3b37f95700 1 -- 172.21.15.27:0/190972637 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f3b1c03e5d0 msgr2=0x7f3b1c040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:58.114 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:57.463+0000 7f3b37f95700 1 --2- 172.21.15.27:0/190972637 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f3b1c03e5d0 0x7f3b1c040a90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3b20006fd0 tx=0x7f3b20006e40 comp rx=0 tx=0).stop 2024-05-21T03:12:58.114 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:57.463+0000 7f3b37f95700 1 -- 172.21.15.27:0/190972637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b30108030 msgr2=0x7f3b3019ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:58.114 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:57.463+0000 7f3b37f95700 1 --2- 172.21.15.27:0/190972637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b30108030 0x7f3b3019ad90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c00b340 tx=0x7f3b2c003dc0 comp rx=0 tx=0).stop 2024-05-21T03:12:58.115 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:57.463+0000 7f3b37f95700 1 -- 172.21.15.27:0/190972637 shutdown_connections 2024-05-21T03:12:58.115 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:57.463+0000 7f3b37f95700 1 --2- 172.21.15.27:0/190972637 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f3b1c03e5d0 0x7f3b1c040a90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:58.115 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:57.463+0000 7f3b37f95700 1 --2- 172.21.15.27:0/190972637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b30108030 0x7f3b3019ad90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:58.115 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:57.463+0000 7f3b37f95700 1 -- 172.21.15.27:0/190972637 >> 172.21.15.27:0/190972637 conn(0x7f3b30103610 msgr2=0x7f3b301913b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:58.115 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:57.464+0000 7f3b37f95700 1 -- 172.21.15.27:0/190972637 shutdown_connections 2024-05-21T03:12:58.115 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:57.464+0000 7f3b37f95700 1 -- 172.21.15.27:0/190972637 wait complete. 2024-05-21T03:12:58.115 INFO:teuthology.orchestra.run.smithi027.stdout:Deploying unmanaged mon service... 2024-05-21T03:12:58.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:58 smithi027 ceph-mon[32179]: from='mgr.14120 172.21.15.27:0/3461495745' entity='mgr.x' 2024-05-21T03:12:58.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:58 smithi027 ceph-mon[32179]: Added host smithi027 2024-05-21T03:12:58.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:58 smithi027 ceph-mon[32179]: from='mgr.14120 172.21.15.27:0/3461495745' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:12:59.546 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-21T03:12:59.546 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.530+0000 7fbd396e9700 1 Processor -- start 2024-05-21T03:12:59.546 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.530+0000 7fbd396e9700 1 -- start start 2024-05-21T03:12:59.546 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.530+0000 7fbd396e9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd34109190 0x7fbd341095b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:59.546 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.530+0000 7fbd396e9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbd34109af0 con 0x7fbd34109190 2024-05-21T03:12:59.546 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.531+0000 7fbd32ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd34109190 0x7fbd341095b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:59.546 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.531+0000 7fbd32ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd34109190 0x7fbd341095b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50224/0 (socket says 172.21.15.27:50224) 2024-05-21T03:12:59.547 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.531+0000 7fbd32ffd700 1 -- 172.21.15.27:0/557791549 learned_addr learned my addr 172.21.15.27:0/557791549 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:59.547 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.531+0000 7fbd32ffd700 1 -- 172.21.15.27:0/557791549 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd34109c70 con 0x7fbd34109190 2024-05-21T03:12:59.547 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.531+0000 7fbd32ffd700 1 --2- 172.21.15.27:0/557791549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd34109190 0x7fbd341095b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbd240098a0 tx=0x7fbd24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0846e8e2add50e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:59.547 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.531+0000 7fbd327fc700 1 -- 172.21.15.27:0/557791549 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd24004160 con 0x7fbd34109190 2024-05-21T03:12:59.547 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.531+0000 7fbd327fc700 1 -- 172.21.15.27:0/557791549 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbd24004300 con 0x7fbd34109190 2024-05-21T03:12:59.547 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.531+0000 7fbd327fc700 1 -- 172.21.15.27:0/557791549 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd24011af0 con 0x7fbd34109190 2024-05-21T03:12:59.547 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.532+0000 7fbd396e9700 1 -- 172.21.15.27:0/557791549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd34109190 msgr2=0x7fbd341095b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:59.547 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.532+0000 7fbd396e9700 1 --2- 172.21.15.27:0/557791549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd34109190 0x7fbd341095b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbd240098a0 tx=0x7fbd24009bb0 comp rx=0 tx=0).stop 2024-05-21T03:12:59.547 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.532+0000 7fbd396e9700 1 -- 172.21.15.27:0/557791549 shutdown_connections 2024-05-21T03:12:59.547 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.532+0000 7fbd396e9700 1 --2- 172.21.15.27:0/557791549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd34109190 0x7fbd341095b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:59.547 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.532+0000 7fbd396e9700 1 -- 172.21.15.27:0/557791549 >> 172.21.15.27:0/557791549 conn(0x7fbd341047f0 msgr2=0x7fbd34106c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:59.547 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.532+0000 7fbd396e9700 1 -- 172.21.15.27:0/557791549 shutdown_connections 2024-05-21T03:12:59.547 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.532+0000 7fbd396e9700 1 -- 172.21.15.27:0/557791549 wait complete. 2024-05-21T03:12:59.548 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.532+0000 7fbd396e9700 1 Processor -- start 2024-05-21T03:12:59.548 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.533+0000 7fbd396e9700 1 -- start start 2024-05-21T03:12:59.548 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.533+0000 7fbd396e9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd34109190 0x7fbd3407eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:59.548 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.533+0000 7fbd396e9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3410b190 con 0x7fbd34109190 2024-05-21T03:12:59.548 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.533+0000 7fbd32ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd34109190 0x7fbd3407eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:59.548 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.533+0000 7fbd32ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd34109190 0x7fbd3407eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50228/0 (socket says 172.21.15.27:50228) 2024-05-21T03:12:59.548 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.533+0000 7fbd32ffd700 1 -- 172.21.15.27:0/3888178950 learned_addr learned my addr 172.21.15.27:0/3888178950 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:12:59.548 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.533+0000 7fbd32ffd700 1 -- 172.21.15.27:0/3888178950 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3407f0d0 con 0x7fbd34109190 2024-05-21T03:12:59.548 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.533+0000 7fbd32ffd700 1 --2- 172.21.15.27:0/3888178950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd34109190 0x7fbd3407eb90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbd24011ac0 tx=0x7fbd24011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:59.548 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.534+0000 7fbd30ff9700 1 -- 172.21.15.27:0/3888178950 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd24003b20 con 0x7fbd34109190 2024-05-21T03:12:59.548 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.534+0000 7fbd30ff9700 1 -- 172.21.15.27:0/3888178950 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbd2401a8c0 con 0x7fbd34109190 2024-05-21T03:12:59.548 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.534+0000 7fbd396e9700 1 -- 172.21.15.27:0/3888178950 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd3407d2c0 con 0x7fbd34109190 2024-05-21T03:12:59.549 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.534+0000 7fbd30ff9700 1 -- 172.21.15.27:0/3888178950 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd240248d0 con 0x7fbd34109190 2024-05-21T03:12:59.549 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.534+0000 7fbd396e9700 1 -- 172.21.15.27:0/3888178950 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd3407d800 con 0x7fbd34109190 2024-05-21T03:12:59.549 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.534+0000 7fbd30ff9700 1 -- 172.21.15.27:0/3888178950 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fbd24022020 con 0x7fbd34109190 2024-05-21T03:12:59.549 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.535+0000 7fbd30ff9700 1 --2- 172.21.15.27:0/3888178950 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fbd2003e5d0 0x7fbd20040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:12:59.549 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.535+0000 7fbd30ff9700 1 -- 172.21.15.27:0/3888178950 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbd240537b0 con 0x7fbd34109190 2024-05-21T03:12:59.549 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.535+0000 7fbd396e9700 1 -- 172.21.15.27:0/3888178950 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd34064ba0 con 0x7fbd34109190 2024-05-21T03:12:59.549 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.535+0000 7fbd2a7fc700 1 --2- 172.21.15.27:0/3888178950 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fbd2003e5d0 0x7fbd20040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:12:59.549 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.535+0000 7fbd2a7fc700 1 --2- 172.21.15.27:0/3888178950 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fbd2003e5d0 0x7fbd20040a90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c006fd0 tx=0x7fbd1c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:12:59.549 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.540+0000 7fbd30ff9700 1 -- 172.21.15.27:0/3888178950 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbd2400e260 con 0x7fbd34109190 2024-05-21T03:12:59.549 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.805+0000 7fbd396e9700 1 -- 172.21.15.27:0/3888178950 --> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fbd3407e170 con 0x7fbd2003e5d0 2024-05-21T03:12:59.549 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.813+0000 7fbd30ff9700 1 -- 172.21.15.27:0/3888178950 <== mgr.14120 v2:172.21.15.27:6800/1185518061 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fbd1c00a040 con 0x7fbd2003e5d0 2024-05-21T03:12:59.549 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.815+0000 7fbd396e9700 1 -- 172.21.15.27:0/3888178950 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fbd2003e5d0 msgr2=0x7fbd20040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:59.549 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.815+0000 7fbd396e9700 1 --2- 172.21.15.27:0/3888178950 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fbd2003e5d0 0x7fbd20040a90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c006fd0 tx=0x7fbd1c006e40 comp rx=0 tx=0).stop 2024-05-21T03:12:59.550 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.815+0000 7fbd396e9700 1 -- 172.21.15.27:0/3888178950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd34109190 msgr2=0x7fbd3407eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:12:59.550 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.815+0000 7fbd396e9700 1 --2- 172.21.15.27:0/3888178950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd34109190 0x7fbd3407eb90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbd24011ac0 tx=0x7fbd24011ba0 comp rx=0 tx=0).stop 2024-05-21T03:12:59.550 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.815+0000 7fbd396e9700 1 -- 172.21.15.27:0/3888178950 shutdown_connections 2024-05-21T03:12:59.550 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.815+0000 7fbd396e9700 1 --2- 172.21.15.27:0/3888178950 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fbd2003e5d0 0x7fbd20040a90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:59.550 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.815+0000 7fbd396e9700 1 --2- 172.21.15.27:0/3888178950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd34109190 0x7fbd3407eb90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:12:59.550 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.815+0000 7fbd396e9700 1 -- 172.21.15.27:0/3888178950 >> 172.21.15.27:0/3888178950 conn(0x7fbd341047f0 msgr2=0x7fbd34105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:12:59.550 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.815+0000 7fbd396e9700 1 -- 172.21.15.27:0/3888178950 shutdown_connections 2024-05-21T03:12:59.550 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:12:58.815+0000 7fbd396e9700 1 -- 172.21.15.27:0/3888178950 wait complete. 2024-05-21T03:12:59.550 INFO:teuthology.orchestra.run.smithi027.stdout:Deploying unmanaged mgr service... 2024-05-21T03:13:00.145 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:59 smithi027 ceph-mon[32179]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:13:00.146 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:59 smithi027 ceph-mon[32179]: Saving service mon spec with placement count:5 2024-05-21T03:13:00.146 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:12:59 smithi027 ceph-mon[32179]: from='mgr.14120 172.21.15.27:0/3461495745' entity='mgr.x' 2024-05-21T03:13:01.540 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:01 smithi027 ceph-mon[32179]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:13:01.541 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:01 smithi027 ceph-mon[32179]: Saving service mgr spec with placement count:2 2024-05-21T03:13:01.541 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:01 smithi027 ceph-mon[32179]: from='mgr.14120 172.21.15.27:0/3461495745' entity='mgr.x' 2024-05-21T03:13:01.541 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:01 smithi027 ceph-mon[32179]: from='mgr.14120 172.21.15.27:0/3461495745' entity='mgr.x' 2024-05-21T03:13:01.584 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-21T03:13:01.584 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.006+0000 7f422faed700 1 Processor -- start 2024-05-21T03:13:01.584 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.007+0000 7f422faed700 1 -- start start 2024-05-21T03:13:01.584 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.007+0000 7f422faed700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f422807ed60 0x7f422807d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:01.584 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.007+0000 7f422faed700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f422807f180 con 0x7f422807ed60 2024-05-21T03:13:01.584 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.007+0000 7f422d889700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f422807ed60 0x7f422807d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:01.585 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.007+0000 7f422d889700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f422807ed60 0x7f422807d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50242/0 (socket says 172.21.15.27:50242) 2024-05-21T03:13:01.585 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.007+0000 7f422d889700 1 -- 172.21.15.27:0/2427270482 learned_addr learned my addr 172.21.15.27:0/2427270482 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:13:01.585 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.007+0000 7f422d889700 1 -- 172.21.15.27:0/2427270482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f422807d860 con 0x7f422807ed60 2024-05-21T03:13:01.585 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.008+0000 7f422d889700 1 --2- 172.21.15.27:0/2427270482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f422807ed60 0x7f422807d320 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f42240098a0 tx=0x7f4224009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf5ce8707be1d90 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:01.585 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.008+0000 7f422c887700 1 -- 172.21.15.27:0/2427270482 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4224004160 con 0x7f422807ed60 2024-05-21T03:13:01.585 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.008+0000 7f422c887700 1 -- 172.21.15.27:0/2427270482 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4224004300 con 0x7f422807ed60 2024-05-21T03:13:01.585 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.008+0000 7f422c887700 1 -- 172.21.15.27:0/2427270482 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4224011af0 con 0x7f422807ed60 2024-05-21T03:13:01.585 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.008+0000 7f422faed700 1 -- 172.21.15.27:0/2427270482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f422807ed60 msgr2=0x7f422807d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:01.585 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.008+0000 7f422faed700 1 --2- 172.21.15.27:0/2427270482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f422807ed60 0x7f422807d320 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f42240098a0 tx=0x7f4224009bb0 comp rx=0 tx=0).stop 2024-05-21T03:13:01.585 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.008+0000 7f422faed700 1 -- 172.21.15.27:0/2427270482 shutdown_connections 2024-05-21T03:13:01.585 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.008+0000 7f422faed700 1 --2- 172.21.15.27:0/2427270482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f422807ed60 0x7f422807d320 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:01.586 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.008+0000 7f422faed700 1 -- 172.21.15.27:0/2427270482 >> 172.21.15.27:0/2427270482 conn(0x7f4228104870 msgr2=0x7f4228106c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:01.586 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.008+0000 7f422faed700 1 -- 172.21.15.27:0/2427270482 shutdown_connections 2024-05-21T03:13:01.586 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.009+0000 7f422faed700 1 -- 172.21.15.27:0/2427270482 wait complete. 2024-05-21T03:13:01.586 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.009+0000 7f422faed700 1 Processor -- start 2024-05-21T03:13:01.586 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.009+0000 7f422faed700 1 -- start start 2024-05-21T03:13:01.586 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.009+0000 7f422faed700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f422807ed60 0x7f422819f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:01.586 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.009+0000 7f422faed700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f422810f5e0 con 0x7f422807ed60 2024-05-21T03:13:01.586 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.009+0000 7f422d889700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f422807ed60 0x7f422819f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:01.586 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.009+0000 7f422d889700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f422807ed60 0x7f422819f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50254/0 (socket says 172.21.15.27:50254) 2024-05-21T03:13:01.586 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.009+0000 7f422d889700 1 -- 172.21.15.27:0/632315947 learned_addr learned my addr 172.21.15.27:0/632315947 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:13:01.586 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.010+0000 7f422d889700 1 -- 172.21.15.27:0/632315947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f422819f750 con 0x7f422807ed60 2024-05-21T03:13:01.587 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.010+0000 7f422d889700 1 --2- 172.21.15.27:0/632315947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f422807ed60 0x7f422819f210 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4224003f40 tx=0x7f4224011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:01.587 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.010+0000 7f421effd700 1 -- 172.21.15.27:0/632315947 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42240038b0 con 0x7f422807ed60 2024-05-21T03:13:01.587 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.010+0000 7f421effd700 1 -- 172.21.15.27:0/632315947 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4224003a50 con 0x7f422807ed60 2024-05-21T03:13:01.587 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.010+0000 7f422faed700 1 -- 172.21.15.27:0/632315947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f422819f9e0 con 0x7f422807ed60 2024-05-21T03:13:01.587 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.010+0000 7f421effd700 1 -- 172.21.15.27:0/632315947 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f422401a5c0 con 0x7f422807ed60 2024-05-21T03:13:01.587 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.010+0000 7f422faed700 1 -- 172.21.15.27:0/632315947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42281a26d0 con 0x7f422807ed60 2024-05-21T03:13:01.587 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.011+0000 7f422faed700 1 -- 172.21.15.27:0/632315947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4228064ba0 con 0x7f422807ed60 2024-05-21T03:13:01.587 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.011+0000 7f421effd700 1 -- 172.21.15.27:0/632315947 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f4224024430 con 0x7f422807ed60 2024-05-21T03:13:01.587 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.011+0000 7f421effd700 1 --2- 172.21.15.27:0/632315947 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f421403e550 0x7f4214040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:01.587 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.011+0000 7f421effd700 1 -- 172.21.15.27:0/632315947 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4224053a40 con 0x7f422807ed60 2024-05-21T03:13:01.587 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.012+0000 7f422d088700 1 --2- 172.21.15.27:0/632315947 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f421403e550 0x7f4214040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:01.588 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.012+0000 7f422d088700 1 --2- 172.21.15.27:0/632315947 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f421403e550 0x7f4214040a10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4218006fd0 tx=0x7f4218006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:01.588 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.017+0000 7f421effd700 1 -- 172.21.15.27:0/632315947 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4224055e20 con 0x7f422807ed60 2024-05-21T03:13:01.588 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.252+0000 7f422faed700 1 -- 172.21.15.27:0/632315947 --> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f4228105ab0 con 0x7f421403e550 2024-05-21T03:13:01.588 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.259+0000 7f421effd700 1 -- 172.21.15.27:0/632315947 <== mgr.14120 v2:172.21.15.27:6800/1185518061 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4228105ab0 con 0x7f421403e550 2024-05-21T03:13:01.588 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.261+0000 7f422faed700 1 -- 172.21.15.27:0/632315947 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f421403e550 msgr2=0x7f4214040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:01.588 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.261+0000 7f422faed700 1 --2- 172.21.15.27:0/632315947 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f421403e550 0x7f4214040a10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4218006fd0 tx=0x7f4218006e40 comp rx=0 tx=0).stop 2024-05-21T03:13:01.588 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.261+0000 7f422faed700 1 -- 172.21.15.27:0/632315947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f422807ed60 msgr2=0x7f422819f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:01.588 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.261+0000 7f422faed700 1 --2- 172.21.15.27:0/632315947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f422807ed60 0x7f422819f210 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4224003f40 tx=0x7f4224011890 comp rx=0 tx=0).stop 2024-05-21T03:13:01.588 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.261+0000 7f422faed700 1 -- 172.21.15.27:0/632315947 shutdown_connections 2024-05-21T03:13:01.588 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.261+0000 7f422faed700 1 --2- 172.21.15.27:0/632315947 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f421403e550 0x7f4214040a10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:01.588 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.261+0000 7f422faed700 1 --2- 172.21.15.27:0/632315947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f422807ed60 0x7f422819f210 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:01.589 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.261+0000 7f422faed700 1 -- 172.21.15.27:0/632315947 >> 172.21.15.27:0/632315947 conn(0x7f4228104870 msgr2=0x7f42281053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:01.589 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.261+0000 7f422faed700 1 -- 172.21.15.27:0/632315947 shutdown_connections 2024-05-21T03:13:01.589 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:00.261+0000 7f422faed700 1 -- 172.21.15.27:0/632315947 wait complete. 2024-05-21T03:13:02.832 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.033+0000 7fa4c5700700 1 Processor -- start 2024-05-21T03:13:02.832 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.033+0000 7fa4c5700700 1 -- start start 2024-05-21T03:13:02.832 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.033+0000 7fa4c5700700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4c0108080 0x7fa4c01084a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:02.832 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.033+0000 7fa4c5700700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c0108a70 con 0x7fa4c0108080 2024-05-21T03:13:02.832 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.033+0000 7fa4beffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4c0108080 0x7fa4c01084a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:02.832 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.033+0000 7fa4beffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4c0108080 0x7fa4c01084a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50260/0 (socket says 172.21.15.27:50260) 2024-05-21T03:13:02.832 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.033+0000 7fa4beffd700 1 -- 172.21.15.27:0/2084662003 learned_addr learned my addr 172.21.15.27:0/2084662003 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:13:02.833 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.033+0000 7fa4beffd700 1 -- 172.21.15.27:0/2084662003 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c0109280 con 0x7fa4c0108080 2024-05-21T03:13:02.833 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.034+0000 7fa4beffd700 1 --2- 172.21.15.27:0/2084662003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4c0108080 0x7fa4c01084a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000bb40 tx=0x7fa4b000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c0602786d35b33 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:02.833 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.034+0000 7fa4bdffb700 1 -- 172.21.15.27:0/2084662003 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4b000dbe0 con 0x7fa4c0108080 2024-05-21T03:13:02.833 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.034+0000 7fa4bdffb700 1 -- 172.21.15.27:0/2084662003 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4b00044e0 con 0x7fa4c0108080 2024-05-21T03:13:02.833 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.034+0000 7fa4bdffb700 1 -- 172.21.15.27:0/2084662003 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4b0010470 con 0x7fa4c0108080 2024-05-21T03:13:02.833 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.035+0000 7fa4c5700700 1 -- 172.21.15.27:0/2084662003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4c0108080 msgr2=0x7fa4c01084a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:02.833 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.035+0000 7fa4c5700700 1 --2- 172.21.15.27:0/2084662003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4c0108080 0x7fa4c01084a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000bb40 tx=0x7fa4b000be50 comp rx=0 tx=0).stop 2024-05-21T03:13:02.833 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.035+0000 7fa4c5700700 1 -- 172.21.15.27:0/2084662003 shutdown_connections 2024-05-21T03:13:02.833 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.035+0000 7fa4c5700700 1 --2- 172.21.15.27:0/2084662003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4c0108080 0x7fa4c01084a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:02.833 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.035+0000 7fa4c5700700 1 -- 172.21.15.27:0/2084662003 >> 172.21.15.27:0/2084662003 conn(0x7fa4c0103620 msgr2=0x7fa4c0105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:02.833 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.035+0000 7fa4c5700700 1 -- 172.21.15.27:0/2084662003 shutdown_connections 2024-05-21T03:13:02.833 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.035+0000 7fa4c5700700 1 -- 172.21.15.27:0/2084662003 wait complete. 2024-05-21T03:13:02.833 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.035+0000 7fa4c5700700 1 Processor -- start 2024-05-21T03:13:02.833 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.036+0000 7fa4c5700700 1 -- start start 2024-05-21T03:13:02.834 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.036+0000 7fa4c5700700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4c0108080 0x7fa4c019f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:02.834 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.036+0000 7fa4c5700700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c007bff0 con 0x7fa4c0108080 2024-05-21T03:13:02.834 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.036+0000 7fa4beffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4c0108080 0x7fa4c019f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:02.834 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.036+0000 7fa4beffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4c0108080 0x7fa4c019f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50268/0 (socket says 172.21.15.27:50268) 2024-05-21T03:13:02.834 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.036+0000 7fa4beffd700 1 -- 172.21.15.27:0/2004348885 learned_addr learned my addr 172.21.15.27:0/2004348885 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:13:02.834 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.036+0000 7fa4beffd700 1 -- 172.21.15.27:0/2004348885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c019f720 con 0x7fa4c0108080 2024-05-21T03:13:02.834 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.036+0000 7fa4beffd700 1 --2- 172.21.15.27:0/2004348885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4c0108080 0x7fa4c019f1e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000b340 tx=0x7fa4b0003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:02.834 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.036+0000 7fa4a7fff700 1 -- 172.21.15.27:0/2004348885 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4b0010770 con 0x7fa4c0108080 2024-05-21T03:13:02.834 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.036+0000 7fa4a7fff700 1 -- 172.21.15.27:0/2004348885 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4b001d070 con 0x7fa4c0108080 2024-05-21T03:13:02.834 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.036+0000 7fa4c5700700 1 -- 172.21.15.27:0/2004348885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4c019f9b0 con 0x7fa4c0108080 2024-05-21T03:13:02.834 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.037+0000 7fa4a7fff700 1 -- 172.21.15.27:0/2004348885 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4b0021bb0 con 0x7fa4c0108080 2024-05-21T03:13:02.834 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.037+0000 7fa4c5700700 1 -- 172.21.15.27:0/2004348885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4c01a26a0 con 0x7fa4c0108080 2024-05-21T03:13:02.834 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.038+0000 7fa4a7fff700 1 -- 172.21.15.27:0/2004348885 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fa4b001f030 con 0x7fa4c0108080 2024-05-21T03:13:02.835 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.038+0000 7fa4c5700700 1 -- 172.21.15.27:0/2004348885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4c0064ba0 con 0x7fa4c0108080 2024-05-21T03:13:02.835 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.038+0000 7fa4a7fff700 1 --2- 172.21.15.27:0/2004348885 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fa4a803e550 0x7fa4a8040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:02.835 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.038+0000 7fa4a7fff700 1 -- 172.21.15.27:0/2004348885 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa4b0053d40 con 0x7fa4c0108080 2024-05-21T03:13:02.835 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.038+0000 7fa4be7fc700 1 --2- 172.21.15.27:0/2004348885 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fa4a803e550 0x7fa4a8040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:02.835 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.038+0000 7fa4be7fc700 1 --2- 172.21.15.27:0/2004348885 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fa4a803e550 0x7fa4a8040a10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4006fd0 tx=0x7fa4b4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:02.835 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.043+0000 7fa4a7fff700 1 -- 172.21.15.27:0/2004348885 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa4b0015070 con 0x7fa4c0108080 2024-05-21T03:13:02.835 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.258+0000 7fa4c5700700 1 -- 172.21.15.27:0/2004348885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fa4c0194360 con 0x7fa4c0108080 2024-05-21T03:13:02.835 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.260+0000 7fa4a7fff700 1 -- 172.21.15.27:0/2004348885 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fa4c0194360 con 0x7fa4c0108080 2024-05-21T03:13:02.835 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.262+0000 7fa4c5700700 1 -- 172.21.15.27:0/2004348885 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fa4a803e550 msgr2=0x7fa4a8040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:02.835 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.262+0000 7fa4c5700700 1 --2- 172.21.15.27:0/2004348885 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fa4a803e550 0x7fa4a8040a10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4006fd0 tx=0x7fa4b4006e40 comp rx=0 tx=0).stop 2024-05-21T03:13:02.835 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.262+0000 7fa4c5700700 1 -- 172.21.15.27:0/2004348885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4c0108080 msgr2=0x7fa4c019f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:02.835 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.263+0000 7fa4c5700700 1 --2- 172.21.15.27:0/2004348885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4c0108080 0x7fa4c019f1e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000b340 tx=0x7fa4b0003dc0 comp rx=0 tx=0).stop 2024-05-21T03:13:02.835 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.263+0000 7fa4c5700700 1 -- 172.21.15.27:0/2004348885 shutdown_connections 2024-05-21T03:13:02.836 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.263+0000 7fa4c5700700 1 --2- 172.21.15.27:0/2004348885 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fa4a803e550 0x7fa4a8040a10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:02.836 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.263+0000 7fa4c5700700 1 --2- 172.21.15.27:0/2004348885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4c0108080 0x7fa4c019f1e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:02.836 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.263+0000 7fa4c5700700 1 -- 172.21.15.27:0/2004348885 >> 172.21.15.27:0/2004348885 conn(0x7fa4c0103620 msgr2=0x7fa4c0191440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:02.836 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.263+0000 7fa4c5700700 1 -- 172.21.15.27:0/2004348885 shutdown_connections 2024-05-21T03:13:02.836 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:02.263+0000 7fa4c5700700 1 -- 172.21.15.27:0/2004348885 wait complete. 2024-05-21T03:13:03.088 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:02 smithi027 ceph-mon[32179]: from='mgr.14120 172.21.15.27:0/3461495745' entity='mgr.x' 2024-05-21T03:13:03.088 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:02 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2004348885' entity='client.admin' 2024-05-21T03:13:04.407 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.308+0000 7f5a4a1e9700 1 Processor -- start 2024-05-21T03:13:04.407 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.308+0000 7f5a4a1e9700 1 -- start start 2024-05-21T03:13:04.407 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.308+0000 7f5a4a1e9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a44107a30 0x7f5a44107e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:04.407 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.308+0000 7f5a4a1e9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4407ce50 con 0x7f5a44107a30 2024-05-21T03:13:04.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.308+0000 7f5a437fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a44107a30 0x7f5a44107e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:04.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.308+0000 7f5a437fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a44107a30 0x7f5a44107e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48848/0 (socket says 172.21.15.27:48848) 2024-05-21T03:13:04.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.308+0000 7f5a437fe700 1 -- 172.21.15.27:0/3751343287 learned_addr learned my addr 172.21.15.27:0/3751343287 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:13:04.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.309+0000 7f5a437fe700 1 -- 172.21.15.27:0/3751343287 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a44108390 con 0x7f5a44107a30 2024-05-21T03:13:04.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.309+0000 7f5a437fe700 1 --2- 172.21.15.27:0/3751343287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a44107a30 0x7f5a44107e50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5a340098a0 tx=0x7f5a34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c80733d581c23f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:04.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.309+0000 7f5a427fc700 1 -- 172.21.15.27:0/3751343287 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a34004160 con 0x7f5a44107a30 2024-05-21T03:13:04.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.309+0000 7f5a427fc700 1 -- 172.21.15.27:0/3751343287 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a34004300 con 0x7f5a44107a30 2024-05-21T03:13:04.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.309+0000 7f5a427fc700 1 -- 172.21.15.27:0/3751343287 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a34011a20 con 0x7f5a44107a30 2024-05-21T03:13:04.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.310+0000 7f5a4a1e9700 1 -- 172.21.15.27:0/3751343287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a44107a30 msgr2=0x7f5a44107e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:04.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.310+0000 7f5a4a1e9700 1 --2- 172.21.15.27:0/3751343287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a44107a30 0x7f5a44107e50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5a340098a0 tx=0x7f5a34009bb0 comp rx=0 tx=0).stop 2024-05-21T03:13:04.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.310+0000 7f5a4a1e9700 1 -- 172.21.15.27:0/3751343287 shutdown_connections 2024-05-21T03:13:04.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.310+0000 7f5a4a1e9700 1 --2- 172.21.15.27:0/3751343287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a44107a30 0x7f5a44107e50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:04.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.310+0000 7f5a4a1e9700 1 -- 172.21.15.27:0/3751343287 >> 172.21.15.27:0/3751343287 conn(0x7f5a44103630 msgr2=0x7f5a44105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:04.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.310+0000 7f5a4a1e9700 1 -- 172.21.15.27:0/3751343287 shutdown_connections 2024-05-21T03:13:04.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.310+0000 7f5a4a1e9700 1 -- 172.21.15.27:0/3751343287 wait complete. 2024-05-21T03:13:04.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.310+0000 7f5a4a1e9700 1 Processor -- start 2024-05-21T03:13:04.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.311+0000 7f5a4a1e9700 1 -- start start 2024-05-21T03:13:04.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.311+0000 7f5a4a1e9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a44107a30 0x7f5a4419f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:04.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.311+0000 7f5a4a1e9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5a44109400 con 0x7f5a44107a30 2024-05-21T03:13:04.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.311+0000 7f5a437fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a44107a30 0x7f5a4419f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:04.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.311+0000 7f5a437fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a44107a30 0x7f5a4419f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48852/0 (socket says 172.21.15.27:48852) 2024-05-21T03:13:04.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.311+0000 7f5a437fe700 1 -- 172.21.15.27:0/2180530670 learned_addr learned my addr 172.21.15.27:0/2180530670 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:13:04.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.311+0000 7f5a437fe700 1 -- 172.21.15.27:0/2180530670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4419f6b0 con 0x7f5a44107a30 2024-05-21T03:13:04.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.311+0000 7f5a437fe700 1 --2- 172.21.15.27:0/2180530670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a44107a30 0x7f5a4419f170 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5a34000c00 tx=0x7f5a34011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:04.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.312+0000 7f5a40ff9700 1 -- 172.21.15.27:0/2180530670 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a34003aa0 con 0x7f5a44107a30 2024-05-21T03:13:04.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.312+0000 7f5a40ff9700 1 -- 172.21.15.27:0/2180530670 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a3401b8f0 con 0x7f5a44107a30 2024-05-21T03:13:04.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.312+0000 7f5a4a1e9700 1 -- 172.21.15.27:0/2180530670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4419f940 con 0x7f5a44107a30 2024-05-21T03:13:04.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.312+0000 7f5a40ff9700 1 -- 172.21.15.27:0/2180530670 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a34023870 con 0x7f5a44107a30 2024-05-21T03:13:04.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.312+0000 7f5a4a1e9700 1 -- 172.21.15.27:0/2180530670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a441a2630 con 0x7f5a44107a30 2024-05-21T03:13:04.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.313+0000 7f5a40ff9700 1 -- 172.21.15.27:0/2180530670 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f5a34023b20 con 0x7f5a44107a30 2024-05-21T03:13:04.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.313+0000 7f5a4a1e9700 1 -- 172.21.15.27:0/2180530670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a44064ba0 con 0x7f5a44107a30 2024-05-21T03:13:04.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.313+0000 7f5a40ff9700 1 --2- 172.21.15.27:0/2180530670 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f5a2c03e5d0 0x7f5a2c040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:04.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.313+0000 7f5a40ff9700 1 -- 172.21.15.27:0/2180530670 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5a34053d40 con 0x7f5a44107a30 2024-05-21T03:13:04.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.313+0000 7f5a42ffd700 1 --2- 172.21.15.27:0/2180530670 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f5a2c03e5d0 0x7f5a2c040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:04.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.314+0000 7f5a42ffd700 1 --2- 172.21.15.27:0/2180530670 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f5a2c03e5d0 0x7f5a2c040a90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5a38006fd0 tx=0x7f5a38006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:04.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.318+0000 7f5a40ff9700 1 -- 172.21.15.27:0/2180530670 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5a3401f030 con 0x7f5a44107a30 2024-05-21T03:13:04.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.549+0000 7f5a4a1e9700 1 -- 172.21.15.27:0/2180530670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f5a4402e840 con 0x7f5a44107a30 2024-05-21T03:13:04.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.552+0000 7f5a40ff9700 1 -- 172.21.15.27:0/2180530670 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f5a3400b3e0 con 0x7f5a44107a30 2024-05-21T03:13:04.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.554+0000 7f5a4a1e9700 1 -- 172.21.15.27:0/2180530670 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f5a2c03e5d0 msgr2=0x7f5a2c040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:04.411 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.554+0000 7f5a4a1e9700 1 --2- 172.21.15.27:0/2180530670 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f5a2c03e5d0 0x7f5a2c040a90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5a38006fd0 tx=0x7f5a38006e40 comp rx=0 tx=0).stop 2024-05-21T03:13:04.411 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.554+0000 7f5a4a1e9700 1 -- 172.21.15.27:0/2180530670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a44107a30 msgr2=0x7f5a4419f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:04.411 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.554+0000 7f5a4a1e9700 1 --2- 172.21.15.27:0/2180530670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a44107a30 0x7f5a4419f170 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5a34000c00 tx=0x7f5a34011aa0 comp rx=0 tx=0).stop 2024-05-21T03:13:04.411 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.554+0000 7f5a4a1e9700 1 -- 172.21.15.27:0/2180530670 shutdown_connections 2024-05-21T03:13:04.411 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.554+0000 7f5a4a1e9700 1 --2- 172.21.15.27:0/2180530670 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f5a2c03e5d0 0x7f5a2c040a90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:04.411 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.554+0000 7f5a4a1e9700 1 --2- 172.21.15.27:0/2180530670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a44107a30 0x7f5a4419f170 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:04.411 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.554+0000 7f5a4a1e9700 1 -- 172.21.15.27:0/2180530670 >> 172.21.15.27:0/2180530670 conn(0x7f5a44103630 msgr2=0x7f5a44195e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:04.411 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.555+0000 7f5a4a1e9700 1 -- 172.21.15.27:0/2180530670 shutdown_connections 2024-05-21T03:13:04.411 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:03.555+0000 7f5a4a1e9700 1 -- 172.21.15.27:0/2180530670 wait complete. 2024-05-21T03:13:04.411 INFO:teuthology.orchestra.run.smithi027.stdout:Enabling the dashboard module... 2024-05-21T03:13:04.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:04 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2180530670' entity='client.admin' 2024-05-21T03:13:06.108 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:05 smithi027 ceph-mon[32179]: from='mgr.14120 172.21.15.27:0/3461495745' entity='mgr.x' 2024-05-21T03:13:06.108 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:05 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3061328636' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-21T03:13:06.108 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:13:05 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: ignoring --setuser ceph since I am not root 2024-05-21T03:13:06.108 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:13:05 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: ignoring --setgroup ceph since I am not root 2024-05-21T03:13:06.108 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:13:05 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:13:05.980+0000 7f5b549e0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T03:13:06.400 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:13:06 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:13:06.175+0000 7f5b549e0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T03:13:06.400 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:13:06 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:13:06.281+0000 7f5b549e0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T03:13:06.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.237+0000 7f93eae38700 1 Processor -- start 2024-05-21T03:13:06.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.237+0000 7f93eae38700 1 -- start start 2024-05-21T03:13:06.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.237+0000 7f93eae38700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93e41095e0 0x7f93e4109a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:06.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.237+0000 7f93eae38700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f93e4109fd0 con 0x7f93e41095e0 2024-05-21T03:13:06.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.238+0000 7f93e8bd4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93e41095e0 0x7f93e4109a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:06.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.238+0000 7f93e8bd4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93e41095e0 0x7f93e4109a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48866/0 (socket says 172.21.15.27:48866) 2024-05-21T03:13:06.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.238+0000 7f93e8bd4700 1 -- 172.21.15.27:0/1966129752 learned_addr learned my addr 172.21.15.27:0/1966129752 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:13:06.408 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.238+0000 7f93e8bd4700 1 -- 172.21.15.27:0/1966129752 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e410a150 con 0x7f93e41095e0 2024-05-21T03:13:06.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.238+0000 7f93e8bd4700 1 --2- 172.21.15.27:0/1966129752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93e41095e0 0x7f93e4109a00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f93d80098a0 tx=0x7f93d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3eb2ae538f1511e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:06.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.238+0000 7f93e37fe700 1 -- 172.21.15.27:0/1966129752 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93d8004160 con 0x7f93e41095e0 2024-05-21T03:13:06.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.238+0000 7f93e37fe700 1 -- 172.21.15.27:0/1966129752 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93d8004300 con 0x7f93e41095e0 2024-05-21T03:13:06.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.238+0000 7f93e37fe700 1 -- 172.21.15.27:0/1966129752 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93d8011a20 con 0x7f93e41095e0 2024-05-21T03:13:06.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.239+0000 7f93eae38700 1 -- 172.21.15.27:0/1966129752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93e41095e0 msgr2=0x7f93e4109a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:06.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.239+0000 7f93eae38700 1 --2- 172.21.15.27:0/1966129752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93e41095e0 0x7f93e4109a00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f93d80098a0 tx=0x7f93d8009bb0 comp rx=0 tx=0).stop 2024-05-21T03:13:06.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.239+0000 7f93eae38700 1 -- 172.21.15.27:0/1966129752 shutdown_connections 2024-05-21T03:13:06.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.239+0000 7f93eae38700 1 --2- 172.21.15.27:0/1966129752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93e41095e0 0x7f93e4109a00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:06.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.239+0000 7f93eae38700 1 -- 172.21.15.27:0/1966129752 >> 172.21.15.27:0/1966129752 conn(0x7f93e4102dd0 msgr2=0x7f93e41051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:06.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.239+0000 7f93eae38700 1 -- 172.21.15.27:0/1966129752 shutdown_connections 2024-05-21T03:13:06.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.239+0000 7f93eae38700 1 -- 172.21.15.27:0/1966129752 wait complete. 2024-05-21T03:13:06.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.240+0000 7f93eae38700 1 Processor -- start 2024-05-21T03:13:06.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.240+0000 7f93eae38700 1 -- start start 2024-05-21T03:13:06.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.240+0000 7f93eae38700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93e41095e0 0x7f93e419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:06.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.240+0000 7f93eae38700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f93e4107fa0 con 0x7f93e41095e0 2024-05-21T03:13:06.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.240+0000 7f93e8bd4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93e41095e0 0x7f93e419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:06.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.240+0000 7f93e8bd4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93e41095e0 0x7f93e419f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48882/0 (socket says 172.21.15.27:48882) 2024-05-21T03:13:06.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.240+0000 7f93e8bd4700 1 -- 172.21.15.27:0/3061328636 learned_addr learned my addr 172.21.15.27:0/3061328636 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:13:06.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.241+0000 7f93e8bd4700 1 -- 172.21.15.27:0/3061328636 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e419f5a0 con 0x7f93e41095e0 2024-05-21T03:13:06.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.241+0000 7f93e8bd4700 1 --2- 172.21.15.27:0/3061328636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93e41095e0 0x7f93e419f060 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f93d8000c00 tx=0x7f93d8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:06.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.241+0000 7f93e1ffb700 1 -- 172.21.15.27:0/3061328636 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93d8003aa0 con 0x7f93e41095e0 2024-05-21T03:13:06.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.241+0000 7f93eae38700 1 -- 172.21.15.27:0/3061328636 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e419f890 con 0x7f93e41095e0 2024-05-21T03:13:06.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.241+0000 7f93eae38700 1 -- 172.21.15.27:0/3061328636 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e41a2520 con 0x7f93e41095e0 2024-05-21T03:13:06.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.242+0000 7f93e1ffb700 1 -- 172.21.15.27:0/3061328636 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93d801b8f0 con 0x7f93e41095e0 2024-05-21T03:13:06.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.242+0000 7f93e1ffb700 1 -- 172.21.15.27:0/3061328636 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93d8023800 con 0x7f93e41095e0 2024-05-21T03:13:06.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.242+0000 7f93eae38700 1 -- 172.21.15.27:0/3061328636 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93c8005360 con 0x7f93e41095e0 2024-05-21T03:13:06.410 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.243+0000 7f93e1ffb700 1 -- 172.21.15.27:0/3061328636 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f93d80239a0 con 0x7f93e41095e0 2024-05-21T03:13:06.411 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.243+0000 7f93e1ffb700 1 --2- 172.21.15.27:0/3061328636 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f93d403e5d0 0x7f93d4040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:06.411 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.243+0000 7f93e1ffb700 1 -- 172.21.15.27:0/3061328636 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f93d8053c80 con 0x7f93e41095e0 2024-05-21T03:13:06.411 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.243+0000 7f93e3fff700 1 --2- 172.21.15.27:0/3061328636 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f93d403e5d0 0x7f93d4040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:06.411 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.244+0000 7f93e3fff700 1 --2- 172.21.15.27:0/3061328636 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f93d403e5d0 0x7f93d4040a90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f93d0006fd0 tx=0x7f93d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:06.411 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.249+0000 7f93e1ffb700 1 -- 172.21.15.27:0/3061328636 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f93d805a080 con 0x7f93e41095e0 2024-05-21T03:13:06.411 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.528+0000 7f93eae38700 1 -- 172.21.15.27:0/3061328636 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f93c8005650 con 0x7f93e41095e0 2024-05-21T03:13:06.411 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.782+0000 7f93e1ffb700 1 -- 172.21.15.27:0/3061328636 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f93d801f030 con 0x7f93e41095e0 2024-05-21T03:13:06.411 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.784+0000 7f93e1ffb700 1 -- 172.21.15.27:0/3061328636 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f93d802c910 con 0x7f93e41095e0 2024-05-21T03:13:06.411 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.786+0000 7f93eae38700 1 -- 172.21.15.27:0/3061328636 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f93d403e5d0 msgr2=0x7f93d4040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:06.411 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.786+0000 7f93eae38700 1 --2- 172.21.15.27:0/3061328636 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f93d403e5d0 0x7f93d4040a90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f93d0006fd0 tx=0x7f93d0006e40 comp rx=0 tx=0).stop 2024-05-21T03:13:06.411 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.786+0000 7f93eae38700 1 -- 172.21.15.27:0/3061328636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93e41095e0 msgr2=0x7f93e419f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:06.411 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.786+0000 7f93eae38700 1 --2- 172.21.15.27:0/3061328636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93e41095e0 0x7f93e419f060 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f93d8000c00 tx=0x7f93d8011aa0 comp rx=0 tx=0).stop 2024-05-21T03:13:06.411 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.786+0000 7f93eae38700 1 -- 172.21.15.27:0/3061328636 shutdown_connections 2024-05-21T03:13:06.412 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.786+0000 7f93eae38700 1 --2- 172.21.15.27:0/3061328636 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f93d403e5d0 0x7f93d4040a90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:06.412 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.786+0000 7f93eae38700 1 --2- 172.21.15.27:0/3061328636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93e41095e0 0x7f93e419f060 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:06.412 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.786+0000 7f93eae38700 1 -- 172.21.15.27:0/3061328636 >> 172.21.15.27:0/3061328636 conn(0x7f93e4102dd0 msgr2=0x7f93e4195680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:06.412 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.786+0000 7f93eae38700 1 -- 172.21.15.27:0/3061328636 shutdown_connections 2024-05-21T03:13:06.412 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:05.786+0000 7f93eae38700 1 -- 172.21.15.27:0/3061328636 wait complete. 2024-05-21T03:13:06.677 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:13:06 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:13:06.608+0000 7f5b549e0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T03:13:07.049 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:13:06 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:13:06.701+0000 7f5b549e0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T03:13:07.050 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:06 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3061328636' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-21T03:13:07.050 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:06 smithi027 ceph-mon[32179]: mgrmap e9: x(active, since 20s) 2024-05-21T03:13:07.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:07 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2589327485' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-21T03:13:08.073 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-05-21T03:13:08.073 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-21T03:13:08.073 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-21T03:13:08.073 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-21T03:13:08.073 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-21T03:13:08.073 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-05-21T03:13:08.073 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.843+0000 7f5f088d5700 1 Processor -- start 2024-05-21T03:13:08.074 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.843+0000 7f5f088d5700 1 -- start start 2024-05-21T03:13:08.074 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.843+0000 7f5f088d5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f0407ed70 0x7f5f0407d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:08.074 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.843+0000 7f5f088d5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0407f190 con 0x7f5f0407ed70 2024-05-21T03:13:08.074 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.844+0000 7f5f0259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f0407ed70 0x7f5f0407d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:08.074 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.844+0000 7f5f0259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f0407ed70 0x7f5f0407d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48892/0 (socket says 172.21.15.27:48892) 2024-05-21T03:13:08.074 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.844+0000 7f5f0259c700 1 -- 172.21.15.27:0/3699441718 learned_addr learned my addr 172.21.15.27:0/3699441718 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:13:08.074 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.844+0000 7f5f0259c700 1 -- 172.21.15.27:0/3699441718 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f0407d820 con 0x7f5f0407ed70 2024-05-21T03:13:08.074 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.844+0000 7f5f0259c700 1 --2- 172.21.15.27:0/3699441718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f0407ed70 0x7f5f0407d2e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5ef40098a0 tx=0x7f5ef4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510ea4c5ec8e4cfa server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:08.074 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.844+0000 7f5f0159a700 1 -- 172.21.15.27:0/3699441718 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ef4004160 con 0x7f5f0407ed70 2024-05-21T03:13:08.074 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.844+0000 7f5f0159a700 1 -- 172.21.15.27:0/3699441718 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ef4004300 con 0x7f5f0407ed70 2024-05-21T03:13:08.074 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.844+0000 7f5f0159a700 1 -- 172.21.15.27:0/3699441718 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ef4011a20 con 0x7f5f0407ed70 2024-05-21T03:13:08.074 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.845+0000 7f5f088d5700 1 -- 172.21.15.27:0/3699441718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f0407ed70 msgr2=0x7f5f0407d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:08.075 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.845+0000 7f5f088d5700 1 --2- 172.21.15.27:0/3699441718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f0407ed70 0x7f5f0407d2e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5ef40098a0 tx=0x7f5ef4009bb0 comp rx=0 tx=0).stop 2024-05-21T03:13:08.075 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.845+0000 7f5f088d5700 1 -- 172.21.15.27:0/3699441718 shutdown_connections 2024-05-21T03:13:08.075 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.845+0000 7f5f088d5700 1 --2- 172.21.15.27:0/3699441718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f0407ed70 0x7f5f0407d2e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:08.075 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.845+0000 7f5f088d5700 1 -- 172.21.15.27:0/3699441718 >> 172.21.15.27:0/3699441718 conn(0x7f5f041047f0 msgr2=0x7f5f04106c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:08.075 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.845+0000 7f5f088d5700 1 -- 172.21.15.27:0/3699441718 shutdown_connections 2024-05-21T03:13:08.075 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.845+0000 7f5f088d5700 1 -- 172.21.15.27:0/3699441718 wait complete. 2024-05-21T03:13:08.075 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.846+0000 7f5f088d5700 1 Processor -- start 2024-05-21T03:13:08.075 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.846+0000 7f5f088d5700 1 -- start start 2024-05-21T03:13:08.075 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.846+0000 7f5f088d5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f0407ed70 0x7f5f0419f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:08.075 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.846+0000 7f5f088d5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0410f620 con 0x7f5f0407ed70 2024-05-21T03:13:08.075 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.846+0000 7f5f0259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f0407ed70 0x7f5f0419f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:08.075 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.846+0000 7f5f0259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f0407ed70 0x7f5f0419f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48906/0 (socket says 172.21.15.27:48906) 2024-05-21T03:13:08.075 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.846+0000 7f5f0259c700 1 -- 172.21.15.27:0/2589327485 learned_addr learned my addr 172.21.15.27:0/2589327485 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:13:08.075 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.847+0000 7f5f0259c700 1 -- 172.21.15.27:0/2589327485 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f0419f6a0 con 0x7f5f0407ed70 2024-05-21T03:13:08.076 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.847+0000 7f5f0259c700 1 --2- 172.21.15.27:0/2589327485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f0407ed70 0x7f5f0419f160 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4000c00 tx=0x7f5ef4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:08.076 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.847+0000 7f5ef37fe700 1 -- 172.21.15.27:0/2589327485 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ef4003aa0 con 0x7f5f0407ed70 2024-05-21T03:13:08.076 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.847+0000 7f5ef37fe700 1 -- 172.21.15.27:0/2589327485 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ef401b8f0 con 0x7f5f0407ed70 2024-05-21T03:13:08.076 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.847+0000 7f5f088d5700 1 -- 172.21.15.27:0/2589327485 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f0419f930 con 0x7f5f0407ed70 2024-05-21T03:13:08.076 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.847+0000 7f5ef37fe700 1 -- 172.21.15.27:0/2589327485 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ef4023870 con 0x7f5f0407ed70 2024-05-21T03:13:08.076 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.847+0000 7f5f088d5700 1 -- 172.21.15.27:0/2589327485 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f041a2620 con 0x7f5f0407ed70 2024-05-21T03:13:08.076 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.848+0000 7f5f088d5700 1 -- 172.21.15.27:0/2589327485 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f04064ba0 con 0x7f5f0407ed70 2024-05-21T03:13:08.076 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.848+0000 7f5ef37fe700 1 -- 172.21.15.27:0/2589327485 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f5ef4023b30 con 0x7f5f0407ed70 2024-05-21T03:13:08.076 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.849+0000 7f5ef37fe700 1 --2- 172.21.15.27:0/2589327485 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f5eec03e5d0 0x7f5eec040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:08.076 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.849+0000 7f5ef37fe700 1 -- 172.21.15.27:0/2589327485 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5ef4054040 con 0x7f5f0407ed70 2024-05-21T03:13:08.076 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.849+0000 7f5f01d9b700 1 -- 172.21.15.27:0/2589327485 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f5eec03e5d0 msgr2=0x7f5eec040a90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/1185518061 2024-05-21T03:13:08.076 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.849+0000 7f5f01d9b700 1 --2- 172.21.15.27:0/2589327485 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f5eec03e5d0 0x7f5eec040a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T03:13:08.076 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:06.855+0000 7f5ef37fe700 1 -- 172.21.15.27:0/2589327485 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5ef401f030 con 0x7f5f0407ed70 2024-05-21T03:13:08.076 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:07.050+0000 7f5f01d9b700 1 -- 172.21.15.27:0/2589327485 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f5eec03e5d0 msgr2=0x7f5eec040a90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/1185518061 2024-05-21T03:13:08.076 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:07.050+0000 7f5f01d9b700 1 --2- 172.21.15.27:0/2589327485 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f5eec03e5d0 0x7f5eec040a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T03:13:08.077 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:07.177+0000 7f5f088d5700 1 -- 172.21.15.27:0/2589327485 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f5f041a2c10 con 0x7f5f0407ed70 2024-05-21T03:13:08.077 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:07.178+0000 7f5ef37fe700 1 -- 172.21.15.27:0/2589327485 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f5ef400b3e0 con 0x7f5f0407ed70 2024-05-21T03:13:08.077 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:07.180+0000 7f5f088d5700 1 -- 172.21.15.27:0/2589327485 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f5eec03e5d0 msgr2=0x7f5eec040a90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T03:13:08.077 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:07.180+0000 7f5f088d5700 1 --2- 172.21.15.27:0/2589327485 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f5eec03e5d0 0x7f5eec040a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:08.077 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:07.180+0000 7f5f088d5700 1 -- 172.21.15.27:0/2589327485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f0407ed70 msgr2=0x7f5f0419f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:08.077 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:07.180+0000 7f5f088d5700 1 --2- 172.21.15.27:0/2589327485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f0407ed70 0x7f5f0419f160 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4000c00 tx=0x7f5ef4011aa0 comp rx=0 tx=0).stop 2024-05-21T03:13:08.077 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:07.180+0000 7f5f088d5700 1 -- 172.21.15.27:0/2589327485 shutdown_connections 2024-05-21T03:13:08.077 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:07.180+0000 7f5f088d5700 1 --2- 172.21.15.27:0/2589327485 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7f5eec03e5d0 0x7f5eec040a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:08.077 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:07.180+0000 7f5f088d5700 1 --2- 172.21.15.27:0/2589327485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f0407ed70 0x7f5f0419f160 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:08.077 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:07.180+0000 7f5f088d5700 1 -- 172.21.15.27:0/2589327485 >> 172.21.15.27:0/2589327485 conn(0x7f5f041047f0 msgr2=0x7f5f041051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:08.077 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:07.180+0000 7f5f088d5700 1 -- 172.21.15.27:0/2589327485 shutdown_connections 2024-05-21T03:13:08.077 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:07.181+0000 7f5f088d5700 1 -- 172.21.15.27:0/2589327485 wait complete. 2024-05-21T03:13:08.077 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for the mgr to restart... 2024-05-21T03:13:08.078 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr epoch 9... 2024-05-21T03:13:08.270 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:13:07 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:13:07.970+0000 7f5b549e0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T03:13:08.271 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:13:08 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:13:08.151+0000 7f5b549e0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T03:13:08.640 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:13:08 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:13:08.333+0000 7f5b549e0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T03:13:08.640 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:13:08 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:13:08.641+0000 7f5b549e0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T03:13:08.927 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:13:08 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:13:08.737+0000 7f5b549e0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T03:13:09.426 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:13:08 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:13:08.949+0000 7f5b549e0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T03:13:09.427 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:13:09 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:13:09.152+0000 7f5b549e0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T03:13:10.177 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:13:09 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:13:09.795+0000 7f5b549e0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T03:13:10.177 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:13:10 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:13:10.028+0000 7f5b549e0200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-21T03:13:10.926 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:13:10 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:13:10.476+0000 7f5b549e0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T03:13:10.927 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:13:10 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:13:10.577+0000 7f5b549e0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T03:13:11.343 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:13:10 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:13:10.963+0000 7f5b549e0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T03:13:11.343 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:13:11 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:13:11.343+0000 7f5b549e0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T03:13:11.612 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:13:11 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:13:11.442+0000 7f5b549e0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T03:13:11.926 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:13:11 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:13:11.704+0000 7f5b549e0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T03:13:11.927 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:13:11 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:13:11.846+0000 7f5b549e0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T03:13:12.285 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:13:11 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:13:11.947+0000 7f5b549e0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T03:13:12.285 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:13:12 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:13:12.285+0000 7f5b549e0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T03:13:12.676 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:13:12 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:13:12.383+0000 7f5b549e0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T03:13:13.300 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:13:13 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:13:13.018+0000 7f5b549e0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T03:13:13.301 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:13:13 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:13:13.247+0000 7f5b549e0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T03:13:13.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:13 smithi027 ceph-mon[32179]: Active manager daemon x restarted 2024-05-21T03:13:13.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:13 smithi027 ceph-mon[32179]: Activating manager daemon x 2024-05-21T03:13:13.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:13 smithi027 ceph-mon[32179]: osdmap e3: 0 total, 0 up, 0 in 2024-05-21T03:13:13.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:13 smithi027 ceph-mon[32179]: mgrmap e10: x(active, starting, since 0.00403884s) 2024-05-21T03:13:13.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:13 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T03:13:13.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:13 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-21T03:13:13.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:13 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-21T03:13:13.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:13 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-21T03:13:13.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:13 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-21T03:13:13.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:13 smithi027 ceph-mon[32179]: Manager daemon x is now available 2024-05-21T03:13:13.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:13 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:13:14.604 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:14 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-21T03:13:14.604 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:14 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-21T03:13:14.604 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:14 smithi027 ceph-mon[32179]: mgrmap e11: x(active, since 1.00623s) 2024-05-21T03:13:15.077 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-05-21T03:13:15.077 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-21T03:13:15.077 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-21T03:13:15.077 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-05-21T03:13:15.077 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.466+0000 7fdf0af94700 1 Processor -- start 2024-05-21T03:13:15.077 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.466+0000 7fdf0af94700 1 -- start start 2024-05-21T03:13:15.078 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.466+0000 7fdf0af94700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf04109c50 0x7fdf0410c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:15.078 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.466+0000 7fdf0af94700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0407ce10 con 0x7fdf04109c50 2024-05-21T03:13:15.078 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.466+0000 7fdf08d30700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf04109c50 0x7fdf0410c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:15.078 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.466+0000 7fdf08d30700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf04109c50 0x7fdf0410c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48912/0 (socket says 172.21.15.27:48912) 2024-05-21T03:13:15.078 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.466+0000 7fdf08d30700 1 -- 172.21.15.27:0/4210681581 learned_addr learned my addr 172.21.15.27:0/4210681581 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:13:15.078 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.467+0000 7fdf08d30700 1 -- 172.21.15.27:0/4210681581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf0410c580 con 0x7fdf04109c50 2024-05-21T03:13:15.078 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.467+0000 7fdf08d30700 1 --2- 172.21.15.27:0/4210681581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf04109c50 0x7fdf0410c040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdef80098a0 tx=0x7fdef8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc3b60c80bdd4f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:15.078 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.467+0000 7fdf037fe700 1 -- 172.21.15.27:0/4210681581 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdef8004160 con 0x7fdf04109c50 2024-05-21T03:13:15.078 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.467+0000 7fdf037fe700 1 -- 172.21.15.27:0/4210681581 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdef8004300 con 0x7fdf04109c50 2024-05-21T03:13:15.078 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.467+0000 7fdf037fe700 1 -- 172.21.15.27:0/4210681581 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdef8011a20 con 0x7fdf04109c50 2024-05-21T03:13:15.078 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.468+0000 7fdf0af94700 1 -- 172.21.15.27:0/4210681581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf04109c50 msgr2=0x7fdf0410c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:15.078 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.468+0000 7fdf0af94700 1 --2- 172.21.15.27:0/4210681581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf04109c50 0x7fdf0410c040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdef80098a0 tx=0x7fdef8009bb0 comp rx=0 tx=0).stop 2024-05-21T03:13:15.078 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.468+0000 7fdf0af94700 1 -- 172.21.15.27:0/4210681581 shutdown_connections 2024-05-21T03:13:15.079 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.468+0000 7fdf0af94700 1 --2- 172.21.15.27:0/4210681581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf04109c50 0x7fdf0410c040 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:15.079 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.468+0000 7fdf0af94700 1 -- 172.21.15.27:0/4210681581 >> 172.21.15.27:0/4210681581 conn(0x7fdf04103640 msgr2=0x7fdf04105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:15.079 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.468+0000 7fdf0af94700 1 -- 172.21.15.27:0/4210681581 shutdown_connections 2024-05-21T03:13:15.079 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.468+0000 7fdf0af94700 1 -- 172.21.15.27:0/4210681581 wait complete. 2024-05-21T03:13:15.079 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.469+0000 7fdf0af94700 1 Processor -- start 2024-05-21T03:13:15.079 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.469+0000 7fdf0af94700 1 -- start start 2024-05-21T03:13:15.079 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.469+0000 7fdf0af94700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf04109c50 0x7fdf0419cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:15.079 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.469+0000 7fdf0af94700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0410d4d0 con 0x7fdf04109c50 2024-05-21T03:13:15.079 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.469+0000 7fdf08d30700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf04109c50 0x7fdf0419cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:15.079 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.469+0000 7fdf08d30700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf04109c50 0x7fdf0419cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48916/0 (socket says 172.21.15.27:48916) 2024-05-21T03:13:15.079 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.469+0000 7fdf08d30700 1 -- 172.21.15.27:0/2444841838 learned_addr learned my addr 172.21.15.27:0/2444841838 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:13:15.079 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.469+0000 7fdf08d30700 1 -- 172.21.15.27:0/2444841838 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf0419d490 con 0x7fdf04109c50 2024-05-21T03:13:15.079 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.469+0000 7fdf08d30700 1 --2- 172.21.15.27:0/2444841838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf04109c50 0x7fdf0419cf50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdef80116e0 tx=0x7fdef8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:15.079 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.470+0000 7fdf01ffb700 1 -- 172.21.15.27:0/2444841838 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdef8003a60 con 0x7fdf04109c50 2024-05-21T03:13:15.080 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.470+0000 7fdf01ffb700 1 -- 172.21.15.27:0/2444841838 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdef801b8f0 con 0x7fdf04109c50 2024-05-21T03:13:15.080 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.470+0000 7fdf0af94700 1 -- 172.21.15.27:0/2444841838 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf0419d720 con 0x7fdf04109c50 2024-05-21T03:13:15.080 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.470+0000 7fdf01ffb700 1 -- 172.21.15.27:0/2444841838 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdef8019860 con 0x7fdf04109c50 2024-05-21T03:13:15.080 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.470+0000 7fdf0af94700 1 -- 172.21.15.27:0/2444841838 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf041a0410 con 0x7fdf04109c50 2024-05-21T03:13:15.080 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.471+0000 7fdf01ffb700 1 -- 172.21.15.27:0/2444841838 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fdef8019b10 con 0x7fdf04109c50 2024-05-21T03:13:15.080 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.471+0000 7fdf01ffb700 1 --2- 172.21.15.27:0/2444841838 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fdef403e5a0 0x7fdef4040a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:15.080 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.471+0000 7fdf01ffb700 1 -- 172.21.15.27:0/2444841838 --> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdef4041170 con 0x7fdef403e5a0 2024-05-21T03:13:15.080 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.471+0000 7fdf03fff700 1 -- 172.21.15.27:0/2444841838 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fdef403e5a0 msgr2=0x7fdef4040a60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/1185518061 2024-05-21T03:13:15.080 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.471+0000 7fdf01ffb700 1 -- 172.21.15.27:0/2444841838 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdef8053da0 con 0x7fdf04109c50 2024-05-21T03:13:15.080 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.471+0000 7fdf03fff700 1 --2- 172.21.15.27:0/2444841838 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fdef403e5a0 0x7fdef4040a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T03:13:15.080 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.672+0000 7fdf03fff700 1 -- 172.21.15.27:0/2444841838 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fdef403e5a0 msgr2=0x7fdef4040a60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/1185518061 2024-05-21T03:13:15.080 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:08.672+0000 7fdf03fff700 1 --2- 172.21.15.27:0/2444841838 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fdef403e5a0 0x7fdef4040a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T03:13:15.081 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:09.073+0000 7fdf03fff700 1 -- 172.21.15.27:0/2444841838 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fdef403e5a0 msgr2=0x7fdef4040a60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/1185518061 2024-05-21T03:13:15.081 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:09.073+0000 7fdf03fff700 1 --2- 172.21.15.27:0/2444841838 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fdef403e5a0 0x7fdef4040a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-21T03:13:15.081 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:09.874+0000 7fdf03fff700 1 -- 172.21.15.27:0/2444841838 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fdef403e5a0 msgr2=0x7fdef4040a60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/1185518061 2024-05-21T03:13:15.081 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:09.874+0000 7fdf03fff700 1 --2- 172.21.15.27:0/2444841838 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fdef403e5a0 0x7fdef4040a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-21T03:13:15.081 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:11.475+0000 7fdf03fff700 1 -- 172.21.15.27:0/2444841838 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fdef403e5a0 msgr2=0x7fdef4040a60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/1185518061 2024-05-21T03:13:15.081 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:11.475+0000 7fdf03fff700 1 --2- 172.21.15.27:0/2444841838 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fdef403e5a0 0x7fdef4040a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-21T03:13:15.081 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:13.251+0000 7fdf01ffb700 1 -- 172.21.15.27:0/2444841838 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7fdef802bd00 con 0x7fdf04109c50 2024-05-21T03:13:15.081 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:13.252+0000 7fdf01ffb700 1 -- 172.21.15.27:0/2444841838 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fdef403e5a0 msgr2=0x7fdef4040a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T03:13:15.081 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:13.252+0000 7fdf01ffb700 1 --2- 172.21.15.27:0/2444841838 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fdef403e5a0 0x7fdef4040a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:15.081 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:14.252+0000 7fdf01ffb700 1 -- 172.21.15.27:0/2444841838 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7fdef8055830 con 0x7fdf04109c50 2024-05-21T03:13:15.081 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:14.252+0000 7fdf01ffb700 1 --2- 172.21.15.27:0/2444841838 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fdef4041fd0 0x7fdef40443c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:15.081 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:14.253+0000 7fdf01ffb700 1 -- 172.21.15.27:0/2444841838 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdef8020e80 con 0x7fdef4041fd0 2024-05-21T03:13:15.081 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:14.253+0000 7fdf03fff700 1 --2- 172.21.15.27:0/2444841838 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fdef4041fd0 0x7fdef40443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:15.082 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:14.254+0000 7fdf03fff700 1 --2- 172.21.15.27:0/2444841838 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fdef4041fd0 0x7fdef40443c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdef0003a10 tx=0x7fdef0009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:15.082 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:14.254+0000 7fdf01ffb700 1 -- 172.21.15.27:0/2444841838 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fdef8020e80 con 0x7fdef4041fd0 2024-05-21T03:13:15.082 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:14.263+0000 7fdf0af94700 1 -- 172.21.15.27:0/2444841838 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fdf04064ba0 con 0x7fdef4041fd0 2024-05-21T03:13:15.082 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:14.263+0000 7fdf01ffb700 1 -- 172.21.15.27:0/2444841838 <== mgr.14152 v2:172.21.15.27:6800/3683408243 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fdf04064ba0 con 0x7fdef4041fd0 2024-05-21T03:13:15.082 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:14.264+0000 7fdf0af94700 1 -- 172.21.15.27:0/2444841838 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fdef4041fd0 msgr2=0x7fdef40443c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:15.082 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:14.264+0000 7fdf0af94700 1 --2- 172.21.15.27:0/2444841838 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fdef4041fd0 0x7fdef40443c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdef0003a10 tx=0x7fdef0009270 comp rx=0 tx=0).stop 2024-05-21T03:13:15.082 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:14.264+0000 7fdf0af94700 1 -- 172.21.15.27:0/2444841838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf04109c50 msgr2=0x7fdf0419cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:15.082 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:14.264+0000 7fdf0af94700 1 --2- 172.21.15.27:0/2444841838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf04109c50 0x7fdf0419cf50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdef80116e0 tx=0x7fdef8011aa0 comp rx=0 tx=0).stop 2024-05-21T03:13:15.082 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:14.264+0000 7fdf0af94700 1 -- 172.21.15.27:0/2444841838 shutdown_connections 2024-05-21T03:13:15.082 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:14.264+0000 7fdf0af94700 1 --2- 172.21.15.27:0/2444841838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf04109c50 0x7fdf0419cf50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:15.082 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:14.264+0000 7fdf0af94700 1 --2- 172.21.15.27:0/2444841838 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fdef4041fd0 0x7fdef40443c0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:15.082 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:14.264+0000 7fdf0af94700 1 --2- 172.21.15.27:0/2444841838 >> [v2:172.21.15.27:6800/1185518061,v1:172.21.15.27:6801/1185518061] conn(0x7fdef403e5a0 0x7fdef4040a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:15.083 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:14.264+0000 7fdf0af94700 1 -- 172.21.15.27:0/2444841838 >> 172.21.15.27:0/2444841838 conn(0x7fdf04103640 msgr2=0x7fdf04104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:15.083 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:14.264+0000 7fdf0af94700 1 -- 172.21.15.27:0/2444841838 shutdown_connections 2024-05-21T03:13:15.083 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:14.264+0000 7fdf0af94700 1 -- 172.21.15.27:0/2444841838 wait complete. 2024-05-21T03:13:15.083 INFO:teuthology.orchestra.run.smithi027.stdout:mgr epoch 9 is available 2024-05-21T03:13:15.083 INFO:teuthology.orchestra.run.smithi027.stdout:Generating a dashboard self-signed certificate... 2024-05-21T03:13:15.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:15 smithi027 ceph-mon[32179]: [21/May/2024:03:13:13] ENGINE Bus STARTING 2024-05-21T03:13:15.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:15 smithi027 ceph-mon[32179]: [21/May/2024:03:13:14] ENGINE Serving on http://172.21.15.27:8765 2024-05-21T03:13:15.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:15 smithi027 ceph-mon[32179]: [21/May/2024:03:13:14] ENGINE Serving on https://172.21.15.27:7150 2024-05-21T03:13:15.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:15 smithi027 ceph-mon[32179]: [21/May/2024:03:13:14] ENGINE Bus STARTED 2024-05-21T03:13:15.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:15 smithi027 ceph-mon[32179]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-21T03:13:15.318 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:15 smithi027 ceph-mon[32179]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-21T03:13:16.447 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-21T03:13:16.447 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.517+0000 7f21cfacd700 1 Processor -- start 2024-05-21T03:13:16.447 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.517+0000 7f21cfacd700 1 -- start start 2024-05-21T03:13:16.447 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.517+0000 7f21cfacd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c8107a70 0x7f21c8107e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:16.447 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.518+0000 7f21cfacd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f21c807ce50 con 0x7f21c8107a70 2024-05-21T03:13:16.447 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.518+0000 7f21cd869700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c8107a70 0x7f21c8107e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:16.447 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.518+0000 7f21cd869700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c8107a70 0x7f21c8107e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58164/0 (socket says 172.21.15.27:58164) 2024-05-21T03:13:16.447 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.518+0000 7f21cd869700 1 -- 172.21.15.27:0/562672298 learned_addr learned my addr 172.21.15.27:0/562672298 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:13:16.447 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.518+0000 7f21cd869700 1 -- 172.21.15.27:0/562672298 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c81083d0 con 0x7f21c8107a70 2024-05-21T03:13:16.447 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.518+0000 7f21cd869700 1 --2- 172.21.15.27:0/562672298 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c8107a70 0x7f21c8107e90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f21b8009e80 tx=0x7f21b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d4e7f88e4290b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:16.447 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.518+0000 7f21cc867700 1 -- 172.21.15.27:0/562672298 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21b8017070 con 0x7f21c8107a70 2024-05-21T03:13:16.448 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.519+0000 7f21cc867700 1 -- 172.21.15.27:0/562672298 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21b800b8c0 con 0x7f21c8107a70 2024-05-21T03:13:16.448 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.519+0000 7f21cc867700 1 -- 172.21.15.27:0/562672298 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21b8004490 con 0x7f21c8107a70 2024-05-21T03:13:16.448 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.519+0000 7f21cfacd700 1 -- 172.21.15.27:0/562672298 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c8107a70 msgr2=0x7f21c8107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:16.448 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.519+0000 7f21cfacd700 1 --2- 172.21.15.27:0/562672298 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c8107a70 0x7f21c8107e90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f21b8009e80 tx=0x7f21b800b330 comp rx=0 tx=0).stop 2024-05-21T03:13:16.448 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.519+0000 7f21cfacd700 1 -- 172.21.15.27:0/562672298 shutdown_connections 2024-05-21T03:13:16.448 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.519+0000 7f21cfacd700 1 --2- 172.21.15.27:0/562672298 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c8107a70 0x7f21c8107e90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:16.448 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.519+0000 7f21cfacd700 1 -- 172.21.15.27:0/562672298 >> 172.21.15.27:0/562672298 conn(0x7f21c8103690 msgr2=0x7f21c8105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:16.448 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.519+0000 7f21cfacd700 1 -- 172.21.15.27:0/562672298 shutdown_connections 2024-05-21T03:13:16.448 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.519+0000 7f21cfacd700 1 -- 172.21.15.27:0/562672298 wait complete. 2024-05-21T03:13:16.448 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.520+0000 7f21cfacd700 1 Processor -- start 2024-05-21T03:13:16.448 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.520+0000 7f21cfacd700 1 -- start start 2024-05-21T03:13:16.448 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.520+0000 7f21cfacd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c8107a70 0x7f21c819f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:16.448 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.520+0000 7f21cfacd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f21c8109440 con 0x7f21c8107a70 2024-05-21T03:13:16.448 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.520+0000 7f21cd869700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c8107a70 0x7f21c819f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:16.449 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.520+0000 7f21cd869700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c8107a70 0x7f21c819f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58176/0 (socket says 172.21.15.27:58176) 2024-05-21T03:13:16.449 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.520+0000 7f21cd869700 1 -- 172.21.15.27:0/1811374127 learned_addr learned my addr 172.21.15.27:0/1811374127 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:13:16.449 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.521+0000 7f21cd869700 1 -- 172.21.15.27:0/1811374127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c819f710 con 0x7f21c8107a70 2024-05-21T03:13:16.449 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.521+0000 7f21cd869700 1 --2- 172.21.15.27:0/1811374127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c8107a70 0x7f21c819f1d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f21b8009e50 tx=0x7f21b800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:16.449 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.521+0000 7f21c6ffd700 1 -- 172.21.15.27:0/1811374127 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21b8020020 con 0x7f21c8107a70 2024-05-21T03:13:16.449 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.521+0000 7f21c6ffd700 1 -- 172.21.15.27:0/1811374127 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21b8003b50 con 0x7f21c8107a70 2024-05-21T03:13:16.449 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.521+0000 7f21cfacd700 1 -- 172.21.15.27:0/1811374127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21c819f9a0 con 0x7f21c8107a70 2024-05-21T03:13:16.449 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.521+0000 7f21c6ffd700 1 -- 172.21.15.27:0/1811374127 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21b8017040 con 0x7f21c8107a70 2024-05-21T03:13:16.449 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.521+0000 7f21cfacd700 1 -- 172.21.15.27:0/1811374127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21c81a2690 con 0x7f21c8107a70 2024-05-21T03:13:16.449 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.522+0000 7f21c6ffd700 1 -- 172.21.15.27:0/1811374127 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f21b8023020 con 0x7f21c8107a70 2024-05-21T03:13:16.449 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.522+0000 7f21cfacd700 1 -- 172.21.15.27:0/1811374127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21c8064ba0 con 0x7f21c8107a70 2024-05-21T03:13:16.449 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.522+0000 7f21c6ffd700 1 --2- 172.21.15.27:0/1811374127 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f21b003e580 0x7f21b0040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:16.449 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.522+0000 7f21c6ffd700 1 -- 172.21.15.27:0/1811374127 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f21b800f050 con 0x7f21c8107a70 2024-05-21T03:13:16.449 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.522+0000 7f21cd068700 1 --2- 172.21.15.27:0/1811374127 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f21b003e580 0x7f21b0040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:16.450 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.523+0000 7f21cd068700 1 --2- 172.21.15.27:0/1811374127 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f21b003e580 0x7f21b0040a40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f21bc006fd0 tx=0x7f21bc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:16.450 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.527+0000 7f21c6ffd700 1 -- 172.21.15.27:0/1811374127 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f21b802a020 con 0x7f21c8107a70 2024-05-21T03:13:16.450 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.754+0000 7f21cfacd700 1 -- 172.21.15.27:0/1811374127 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f21c802e820 con 0x7f21b003e580 2024-05-21T03:13:16.450 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.868+0000 7f21c6ffd700 1 -- 172.21.15.27:0/1811374127 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f21c802e820 con 0x7f21b003e580 2024-05-21T03:13:16.450 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.870+0000 7f21cfacd700 1 -- 172.21.15.27:0/1811374127 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f21b003e580 msgr2=0x7f21b0040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:16.450 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.870+0000 7f21cfacd700 1 --2- 172.21.15.27:0/1811374127 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f21b003e580 0x7f21b0040a40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f21bc006fd0 tx=0x7f21bc006e40 comp rx=0 tx=0).stop 2024-05-21T03:13:16.450 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.870+0000 7f21cfacd700 1 -- 172.21.15.27:0/1811374127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c8107a70 msgr2=0x7f21c819f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:16.450 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.870+0000 7f21cfacd700 1 --2- 172.21.15.27:0/1811374127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c8107a70 0x7f21c819f1d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f21b8009e50 tx=0x7f21b800bd50 comp rx=0 tx=0).stop 2024-05-21T03:13:16.450 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.870+0000 7f21cfacd700 1 -- 172.21.15.27:0/1811374127 shutdown_connections 2024-05-21T03:13:16.450 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.870+0000 7f21cfacd700 1 --2- 172.21.15.27:0/1811374127 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f21b003e580 0x7f21b0040a40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:16.450 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.870+0000 7f21cfacd700 1 --2- 172.21.15.27:0/1811374127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c8107a70 0x7f21c819f1d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:16.450 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.870+0000 7f21cfacd700 1 -- 172.21.15.27:0/1811374127 >> 172.21.15.27:0/1811374127 conn(0x7f21c8103690 msgr2=0x7f21c8195e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:16.450 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.870+0000 7f21cfacd700 1 -- 172.21.15.27:0/1811374127 shutdown_connections 2024-05-21T03:13:16.451 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:15.870+0000 7f21cfacd700 1 -- 172.21.15.27:0/1811374127 wait complete. 2024-05-21T03:13:16.451 INFO:teuthology.orchestra.run.smithi027.stdout:Creating initial admin user... 2024-05-21T03:13:16.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:16 smithi027 ceph-mon[32179]: mgrmap e12: x(active, since 2s) 2024-05-21T03:13:16.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:16 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:16.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:16 smithi027 ceph-mon[32179]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:13:16.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:16 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:16.567 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:16 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:18.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:17 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:18.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:17 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:18.844 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$LHQZJMs5Oajlu1nG6qdKw.qMFP.M6wsz3CUKGEoEFcWapy/WzHnF6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1716261197, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-21T03:13:18.844 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.923+0000 7fc89b59e700 1 Processor -- start 2024-05-21T03:13:18.844 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.923+0000 7fc89b59e700 1 -- start start 2024-05-21T03:13:18.844 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.923+0000 7fc89b59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc89c109a40 0x7fc89c10be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:18.844 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.923+0000 7fc89b59e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c07ce50 con 0x7fc89c109a40 2024-05-21T03:13:18.845 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.923+0000 7fc89a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc89c109a40 0x7fc89c10be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:18.845 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.923+0000 7fc89a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc89c109a40 0x7fc89c10be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58180/0 (socket says 172.21.15.27:58180) 2024-05-21T03:13:18.845 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.923+0000 7fc89a59c700 1 -- 172.21.15.27:0/3317368561 learned_addr learned my addr 172.21.15.27:0/3317368561 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:13:18.845 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.924+0000 7fc89a59c700 1 -- 172.21.15.27:0/3317368561 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc89c10c370 con 0x7fc89c109a40 2024-05-21T03:13:18.845 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.924+0000 7fc89a59c700 1 --2- 172.21.15.27:0/3317368561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc89c109a40 0x7fc89c10be30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0098a0 tx=0x7fc88c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff015d2fe9185795 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:18.845 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.924+0000 7fc89959a700 1 -- 172.21.15.27:0/3317368561 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc88c004160 con 0x7fc89c109a40 2024-05-21T03:13:18.845 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.924+0000 7fc89959a700 1 -- 172.21.15.27:0/3317368561 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc88c004300 con 0x7fc89c109a40 2024-05-21T03:13:18.845 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.924+0000 7fc89959a700 1 -- 172.21.15.27:0/3317368561 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc88c011a20 con 0x7fc89c109a40 2024-05-21T03:13:18.845 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.925+0000 7fc89b59e700 1 -- 172.21.15.27:0/3317368561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc89c109a40 msgr2=0x7fc89c10be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:18.845 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.925+0000 7fc89b59e700 1 --2- 172.21.15.27:0/3317368561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc89c109a40 0x7fc89c10be30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0098a0 tx=0x7fc88c009bb0 comp rx=0 tx=0).stop 2024-05-21T03:13:18.845 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.925+0000 7fc89b59e700 1 -- 172.21.15.27:0/3317368561 shutdown_connections 2024-05-21T03:13:18.845 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.925+0000 7fc89b59e700 1 --2- 172.21.15.27:0/3317368561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc89c109a40 0x7fc89c10be30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:18.846 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.925+0000 7fc89b59e700 1 -- 172.21.15.27:0/3317368561 >> 172.21.15.27:0/3317368561 conn(0x7fc89c1033b0 msgr2=0x7fc89c105830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:18.846 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.925+0000 7fc89b59e700 1 -- 172.21.15.27:0/3317368561 shutdown_connections 2024-05-21T03:13:18.846 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.925+0000 7fc89b59e700 1 -- 172.21.15.27:0/3317368561 wait complete. 2024-05-21T03:13:18.846 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.926+0000 7fc89b59e700 1 Processor -- start 2024-05-21T03:13:18.846 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.926+0000 7fc89b59e700 1 -- start start 2024-05-21T03:13:18.846 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.927+0000 7fc89b59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc89c109a40 0x7fc89c19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:18.846 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.927+0000 7fc89b59e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c10d310 con 0x7fc89c109a40 2024-05-21T03:13:18.846 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.927+0000 7fc89a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc89c109a40 0x7fc89c19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:18.846 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.927+0000 7fc89a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc89c109a40 0x7fc89c19cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58188/0 (socket says 172.21.15.27:58188) 2024-05-21T03:13:18.846 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.927+0000 7fc89a59c700 1 -- 172.21.15.27:0/182195703 learned_addr learned my addr 172.21.15.27:0/182195703 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:13:18.846 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.927+0000 7fc89a59c700 1 -- 172.21.15.27:0/182195703 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc89c19d310 con 0x7fc89c109a40 2024-05-21T03:13:18.846 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.927+0000 7fc89a59c700 1 --2- 172.21.15.27:0/182195703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc89c109a40 0x7fc89c19cdd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0116e0 tx=0x7fc88c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:18.847 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.927+0000 7fc88b7fe700 1 -- 172.21.15.27:0/182195703 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc88c003b20 con 0x7fc89c109a40 2024-05-21T03:13:18.847 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.927+0000 7fc88b7fe700 1 -- 172.21.15.27:0/182195703 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc88c01b8f0 con 0x7fc89c109a40 2024-05-21T03:13:18.847 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.927+0000 7fc89b59e700 1 -- 172.21.15.27:0/182195703 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc89c19d5a0 con 0x7fc89c109a40 2024-05-21T03:13:18.847 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.927+0000 7fc88b7fe700 1 -- 172.21.15.27:0/182195703 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc88c0238c0 con 0x7fc89c109a40 2024-05-21T03:13:18.847 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.927+0000 7fc89b59e700 1 -- 172.21.15.27:0/182195703 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc89c1a0290 con 0x7fc89c109a40 2024-05-21T03:13:18.847 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.928+0000 7fc89b59e700 1 -- 172.21.15.27:0/182195703 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc89c064ba0 con 0x7fc89c109a40 2024-05-21T03:13:18.847 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.928+0000 7fc88b7fe700 1 -- 172.21.15.27:0/182195703 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fc88c023a60 con 0x7fc89c109a40 2024-05-21T03:13:18.847 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.929+0000 7fc88b7fe700 1 --2- 172.21.15.27:0/182195703 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fc88403e5a0 0x7fc884040a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:18.847 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.929+0000 7fc88b7fe700 1 -- 172.21.15.27:0/182195703 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc88c0541d0 con 0x7fc89c109a40 2024-05-21T03:13:18.847 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.929+0000 7fc899d9b700 1 --2- 172.21.15.27:0/182195703 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fc88403e5a0 0x7fc884040a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:18.847 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.929+0000 7fc899d9b700 1 --2- 172.21.15.27:0/182195703 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fc88403e5a0 0x7fc884040a60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc890006fd0 tx=0x7fc890006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:18.847 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:16.934+0000 7fc88b7fe700 1 -- 172.21.15.27:0/182195703 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc88c01f030 con 0x7fc89c109a40 2024-05-21T03:13:18.847 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:17.166+0000 7fc89b59e700 1 -- 172.21.15.27:0/182195703 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fc89c1a0520 con 0x7fc88403e5a0 2024-05-21T03:13:18.848 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:17.413+0000 7fc88b7fe700 1 -- 172.21.15.27:0/182195703 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fc89c1a0520 con 0x7fc88403e5a0 2024-05-21T03:13:18.848 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:17.415+0000 7fc89b59e700 1 -- 172.21.15.27:0/182195703 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fc88403e5a0 msgr2=0x7fc884040a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:18.848 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:17.415+0000 7fc89b59e700 1 --2- 172.21.15.27:0/182195703 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fc88403e5a0 0x7fc884040a60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc890006fd0 tx=0x7fc890006e40 comp rx=0 tx=0).stop 2024-05-21T03:13:18.848 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:17.415+0000 7fc89b59e700 1 -- 172.21.15.27:0/182195703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc89c109a40 msgr2=0x7fc89c19cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:18.848 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:17.415+0000 7fc89b59e700 1 --2- 172.21.15.27:0/182195703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc89c109a40 0x7fc89c19cdd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0116e0 tx=0x7fc88c011aa0 comp rx=0 tx=0).stop 2024-05-21T03:13:18.848 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:17.415+0000 7fc89b59e700 1 -- 172.21.15.27:0/182195703 shutdown_connections 2024-05-21T03:13:18.848 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:17.415+0000 7fc89b59e700 1 --2- 172.21.15.27:0/182195703 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fc88403e5a0 0x7fc884040a60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:18.848 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:17.415+0000 7fc89b59e700 1 --2- 172.21.15.27:0/182195703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc89c109a40 0x7fc89c19cdd0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:18.848 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:17.415+0000 7fc89b59e700 1 -- 172.21.15.27:0/182195703 >> 172.21.15.27:0/182195703 conn(0x7fc89c1033b0 msgr2=0x7fc89c103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:18.848 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:17.415+0000 7fc89b59e700 1 -- 172.21.15.27:0/182195703 shutdown_connections 2024-05-21T03:13:18.848 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:17.415+0000 7fc89b59e700 1 -- 172.21.15.27:0/182195703 wait complete. 2024-05-21T03:13:18.848 INFO:teuthology.orchestra.run.smithi027.stdout:Fetching dashboard port number... 2024-05-21T03:13:19.031 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:18 smithi027 ceph-mon[32179]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:13:20.026 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:19 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/825276130' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-21T03:13:20.034 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 8443 2024-05-21T03:13:20.034 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.209+0000 7fb152a49700 1 Processor -- start 2024-05-21T03:13:20.034 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.209+0000 7fb152a49700 1 -- start start 2024-05-21T03:13:20.034 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.210+0000 7fb152a49700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb14c1091e0 0x7fb14c109600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:20.034 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.210+0000 7fb152a49700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c109bd0 con 0x7fb14c1091e0 2024-05-21T03:13:20.034 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.210+0000 7fb14bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb14c1091e0 0x7fb14c109600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:20.034 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.210+0000 7fb14bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb14c1091e0 0x7fb14c109600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58198/0 (socket says 172.21.15.27:58198) 2024-05-21T03:13:20.034 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.210+0000 7fb14bfff700 1 -- 172.21.15.27:0/4148531996 learned_addr learned my addr 172.21.15.27:0/4148531996 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:13:20.035 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.210+0000 7fb14bfff700 1 -- 172.21.15.27:0/4148531996 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14c10a340 con 0x7fb14c1091e0 2024-05-21T03:13:20.035 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.210+0000 7fb14bfff700 1 --2- 172.21.15.27:0/4148531996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb14c1091e0 0x7fb14c109600 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb13c0098a0 tx=0x7fb13c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c066b86aae132421 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:20.035 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.210+0000 7fb14b7fe700 1 -- 172.21.15.27:0/4148531996 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb13c004160 con 0x7fb14c1091e0 2024-05-21T03:13:20.035 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.210+0000 7fb14b7fe700 1 -- 172.21.15.27:0/4148531996 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb13c004300 con 0x7fb14c1091e0 2024-05-21T03:13:20.035 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.210+0000 7fb14b7fe700 1 -- 172.21.15.27:0/4148531996 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb13c011a20 con 0x7fb14c1091e0 2024-05-21T03:13:20.035 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.211+0000 7fb152a49700 1 -- 172.21.15.27:0/4148531996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb14c1091e0 msgr2=0x7fb14c109600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:20.035 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.211+0000 7fb152a49700 1 --2- 172.21.15.27:0/4148531996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb14c1091e0 0x7fb14c109600 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb13c0098a0 tx=0x7fb13c009bb0 comp rx=0 tx=0).stop 2024-05-21T03:13:20.035 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.211+0000 7fb152a49700 1 -- 172.21.15.27:0/4148531996 shutdown_connections 2024-05-21T03:13:20.035 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.211+0000 7fb152a49700 1 --2- 172.21.15.27:0/4148531996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb14c1091e0 0x7fb14c109600 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:20.035 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.211+0000 7fb152a49700 1 -- 172.21.15.27:0/4148531996 >> 172.21.15.27:0/4148531996 conn(0x7fb14c104840 msgr2=0x7fb14c106c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:20.035 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.211+0000 7fb152a49700 1 -- 172.21.15.27:0/4148531996 shutdown_connections 2024-05-21T03:13:20.035 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.211+0000 7fb152a49700 1 -- 172.21.15.27:0/4148531996 wait complete. 2024-05-21T03:13:20.036 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.212+0000 7fb152a49700 1 Processor -- start 2024-05-21T03:13:20.036 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.212+0000 7fb152a49700 1 -- start start 2024-05-21T03:13:20.036 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.212+0000 7fb152a49700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb14c1091e0 0x7fb14c07eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:20.036 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.212+0000 7fb152a49700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c10d6f0 con 0x7fb14c1091e0 2024-05-21T03:13:20.036 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.212+0000 7fb14bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb14c1091e0 0x7fb14c07eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:20.036 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.212+0000 7fb14bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb14c1091e0 0x7fb14c07eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58200/0 (socket says 172.21.15.27:58200) 2024-05-21T03:13:20.036 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.212+0000 7fb14bfff700 1 -- 172.21.15.27:0/825276130 learned_addr learned my addr 172.21.15.27:0/825276130 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:13:20.036 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.212+0000 7fb14bfff700 1 -- 172.21.15.27:0/825276130 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14c07f0d0 con 0x7fb14c1091e0 2024-05-21T03:13:20.036 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.213+0000 7fb14bfff700 1 --2- 172.21.15.27:0/825276130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb14c1091e0 0x7fb14c07eb90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb13c0119f0 tx=0x7fb13c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:20.036 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.213+0000 7fb149ffb700 1 -- 172.21.15.27:0/825276130 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb13c0039e0 con 0x7fb14c1091e0 2024-05-21T03:13:20.036 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.213+0000 7fb149ffb700 1 -- 172.21.15.27:0/825276130 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb13c01b8f0 con 0x7fb14c1091e0 2024-05-21T03:13:20.036 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.213+0000 7fb149ffb700 1 -- 172.21.15.27:0/825276130 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb13c0237a0 con 0x7fb14c1091e0 2024-05-21T03:13:20.037 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.213+0000 7fb152a49700 1 -- 172.21.15.27:0/825276130 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb14c07d2c0 con 0x7fb14c1091e0 2024-05-21T03:13:20.037 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.213+0000 7fb152a49700 1 -- 172.21.15.27:0/825276130 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb14c07d800 con 0x7fb14c1091e0 2024-05-21T03:13:20.037 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.214+0000 7fb149ffb700 1 -- 172.21.15.27:0/825276130 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fb13c01b440 con 0x7fb14c1091e0 2024-05-21T03:13:20.037 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.214+0000 7fb152a49700 1 -- 172.21.15.27:0/825276130 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb14c064ba0 con 0x7fb14c1091e0 2024-05-21T03:13:20.037 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.214+0000 7fb149ffb700 1 --2- 172.21.15.27:0/825276130 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb13803e5a0 0x7fb138040a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:20.037 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.214+0000 7fb149ffb700 1 -- 172.21.15.27:0/825276130 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb13c053ec0 con 0x7fb14c1091e0 2024-05-21T03:13:20.037 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.214+0000 7fb143fff700 1 --2- 172.21.15.27:0/825276130 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb13803e5a0 0x7fb138040a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:20.037 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.215+0000 7fb143fff700 1 --2- 172.21.15.27:0/825276130 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb13803e5a0 0x7fb138040a60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb134006fd0 tx=0x7fb134006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:20.037 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.219+0000 7fb149ffb700 1 -- 172.21.15.27:0/825276130 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb13c01f030 con 0x7fb14c1091e0 2024-05-21T03:13:20.037 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.435+0000 7fb152a49700 1 -- 172.21.15.27:0/825276130 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fb14c07da80 con 0x7fb14c1091e0 2024-05-21T03:13:20.037 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.435+0000 7fb149ffb700 1 -- 172.21.15.27:0/825276130 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fb13c0202b0 con 0x7fb14c1091e0 2024-05-21T03:13:20.037 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.437+0000 7fb152a49700 1 -- 172.21.15.27:0/825276130 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb13803e5a0 msgr2=0x7fb138040a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:20.037 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.437+0000 7fb152a49700 1 --2- 172.21.15.27:0/825276130 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb13803e5a0 0x7fb138040a60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb134006fd0 tx=0x7fb134006e40 comp rx=0 tx=0).stop 2024-05-21T03:13:20.038 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.437+0000 7fb152a49700 1 -- 172.21.15.27:0/825276130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb14c1091e0 msgr2=0x7fb14c07eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:20.038 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.437+0000 7fb152a49700 1 --2- 172.21.15.27:0/825276130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb14c1091e0 0x7fb14c07eb90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb13c0119f0 tx=0x7fb13c011ad0 comp rx=0 tx=0).stop 2024-05-21T03:13:20.038 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.438+0000 7fb152a49700 1 -- 172.21.15.27:0/825276130 shutdown_connections 2024-05-21T03:13:20.038 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.438+0000 7fb152a49700 1 --2- 172.21.15.27:0/825276130 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb13803e5a0 0x7fb138040a60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:20.038 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.438+0000 7fb152a49700 1 --2- 172.21.15.27:0/825276130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb14c1091e0 0x7fb14c07eb90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:20.038 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.438+0000 7fb152a49700 1 -- 172.21.15.27:0/825276130 >> 172.21.15.27:0/825276130 conn(0x7fb14c104840 msgr2=0x7fb14c105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:20.038 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.438+0000 7fb152a49700 1 -- 172.21.15.27:0/825276130 shutdown_connections 2024-05-21T03:13:20.038 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:19.438+0000 7fb152a49700 1 -- 172.21.15.27:0/825276130 wait complete. 2024-05-21T03:13:20.041 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-21T03:13:20.041 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2024-05-21T03:13:20.049 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-21T03:13:20.049 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-05-21T03:13:20.049 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2024-05-21T03:13:20.049 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-21T03:13:20.051 INFO:teuthology.orchestra.run.smithi027.stdout:Ceph Dashboard is now available at: 2024-05-21T03:13:20.051 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:13:20.051 INFO:teuthology.orchestra.run.smithi027.stdout: URL: https://smithi027.front.sepia.ceph.com:8443/ 2024-05-21T03:13:20.051 INFO:teuthology.orchestra.run.smithi027.stdout: User: admin 2024-05-21T03:13:20.051 INFO:teuthology.orchestra.run.smithi027.stdout: Password: ynnzlbktbg 2024-05-21T03:13:20.051 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:13:20.052 INFO:teuthology.orchestra.run.smithi027.stdout:Saving cluster configuration to /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config directory 2024-05-21T03:13:20.052 INFO:teuthology.orchestra.run.smithi027.stdout:Enabling autotune for osd_memory_target 2024-05-21T03:13:20.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:20 smithi027 ceph-mon[32179]: mgrmap e13: x(active, since 6s) 2024-05-21T03:13:21.185 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.444+0000 7fa12cc85700 1 Processor -- start 2024-05-21T03:13:21.185 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.444+0000 7fa12cc85700 1 -- start start 2024-05-21T03:13:21.185 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.444+0000 7fa12cc85700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa128108030 0x7fa128108450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:21.185 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.444+0000 7fa12cc85700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa128108a20 con 0x7fa128108030 2024-05-21T03:13:21.185 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.445+0000 7fa12659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa128108030 0x7fa128108450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:21.185 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.445+0000 7fa12659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa128108030 0x7fa128108450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58206/0 (socket says 172.21.15.27:58206) 2024-05-21T03:13:21.185 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.445+0000 7fa12659c700 1 -- 172.21.15.27:0/2665686840 learned_addr learned my addr 172.21.15.27:0/2665686840 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:13:21.185 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.445+0000 7fa12659c700 1 -- 172.21.15.27:0/2665686840 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa128109230 con 0x7fa128108030 2024-05-21T03:13:21.185 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.445+0000 7fa12659c700 1 --2- 172.21.15.27:0/2665686840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa128108030 0x7fa128108450 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa1180098a0 tx=0x7fa118009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2992916c84c93cce server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:21.185 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.445+0000 7fa12559a700 1 -- 172.21.15.27:0/2665686840 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa118004160 con 0x7fa128108030 2024-05-21T03:13:21.186 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.445+0000 7fa12559a700 1 -- 172.21.15.27:0/2665686840 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa118004300 con 0x7fa128108030 2024-05-21T03:13:21.186 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.445+0000 7fa12559a700 1 -- 172.21.15.27:0/2665686840 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa118011a20 con 0x7fa128108030 2024-05-21T03:13:21.186 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.446+0000 7fa12cc85700 1 -- 172.21.15.27:0/2665686840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa128108030 msgr2=0x7fa128108450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:21.186 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.446+0000 7fa12cc85700 1 --2- 172.21.15.27:0/2665686840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa128108030 0x7fa128108450 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa1180098a0 tx=0x7fa118009bb0 comp rx=0 tx=0).stop 2024-05-21T03:13:21.186 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.446+0000 7fa12cc85700 1 -- 172.21.15.27:0/2665686840 shutdown_connections 2024-05-21T03:13:21.186 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.446+0000 7fa12cc85700 1 --2- 172.21.15.27:0/2665686840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa128108030 0x7fa128108450 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:21.186 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.446+0000 7fa12cc85700 1 -- 172.21.15.27:0/2665686840 >> 172.21.15.27:0/2665686840 conn(0x7fa128103610 msgr2=0x7fa128105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:21.186 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.446+0000 7fa12cc85700 1 -- 172.21.15.27:0/2665686840 shutdown_connections 2024-05-21T03:13:21.186 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.446+0000 7fa12cc85700 1 -- 172.21.15.27:0/2665686840 wait complete. 2024-05-21T03:13:21.186 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.447+0000 7fa12cc85700 1 Processor -- start 2024-05-21T03:13:21.186 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.447+0000 7fa12cc85700 1 -- start start 2024-05-21T03:13:21.186 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.447+0000 7fa12cc85700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa128108030 0x7fa1281931a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:21.186 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.447+0000 7fa12cc85700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa12810c5e0 con 0x7fa128108030 2024-05-21T03:13:21.187 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.447+0000 7fa12659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa128108030 0x7fa1281931a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:21.187 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.447+0000 7fa12659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa128108030 0x7fa1281931a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58210/0 (socket says 172.21.15.27:58210) 2024-05-21T03:13:21.187 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.447+0000 7fa12659c700 1 -- 172.21.15.27:0/687243597 learned_addr learned my addr 172.21.15.27:0/687243597 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:13:21.187 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.447+0000 7fa12659c700 1 -- 172.21.15.27:0/687243597 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1281936e0 con 0x7fa128108030 2024-05-21T03:13:21.187 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.448+0000 7fa12659c700 1 --2- 172.21.15.27:0/687243597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa128108030 0x7fa1281931a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa1180116e0 tx=0x7fa118011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:21.187 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.448+0000 7fa1177fe700 1 -- 172.21.15.27:0/687243597 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1180039e0 con 0x7fa128108030 2024-05-21T03:13:21.187 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.448+0000 7fa1177fe700 1 -- 172.21.15.27:0/687243597 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa11801b8f0 con 0x7fa128108030 2024-05-21T03:13:21.187 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.448+0000 7fa12cc85700 1 -- 172.21.15.27:0/687243597 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1281918d0 con 0x7fa128108030 2024-05-21T03:13:21.187 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.448+0000 7fa1177fe700 1 -- 172.21.15.27:0/687243597 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa118023760 con 0x7fa128108030 2024-05-21T03:13:21.187 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.448+0000 7fa12cc85700 1 -- 172.21.15.27:0/687243597 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa128191e10 con 0x7fa128108030 2024-05-21T03:13:21.187 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.449+0000 7fa1177fe700 1 -- 172.21.15.27:0/687243597 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fa1180239f0 con 0x7fa128108030 2024-05-21T03:13:21.187 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.449+0000 7fa12cc85700 1 -- 172.21.15.27:0/687243597 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa128064ba0 con 0x7fa128108030 2024-05-21T03:13:21.187 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.449+0000 7fa1177fe700 1 --2- 172.21.15.27:0/687243597 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fa11003e5f0 0x7fa110040ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:21.187 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.449+0000 7fa1177fe700 1 -- 172.21.15.27:0/687243597 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa118054200 con 0x7fa128108030 2024-05-21T03:13:21.188 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.449+0000 7fa125d9b700 1 --2- 172.21.15.27:0/687243597 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fa11003e5f0 0x7fa110040ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:21.188 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.450+0000 7fa125d9b700 1 --2- 172.21.15.27:0/687243597 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fa11003e5f0 0x7fa110040ab0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa11c006fd0 tx=0x7fa11c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:21.188 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.454+0000 7fa1177fe700 1 -- 172.21.15.27:0/687243597 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa11801f030 con 0x7fa128108030 2024-05-21T03:13:21.188 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.666+0000 7fa12cc85700 1 -- 172.21.15.27:0/687243597 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fa128196f20 con 0x7fa128108030 2024-05-21T03:13:21.188 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.666+0000 7fa1177fe700 1 -- 172.21.15.27:0/687243597 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fa11800b3e0 con 0x7fa128108030 2024-05-21T03:13:21.188 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.669+0000 7fa12cc85700 1 -- 172.21.15.27:0/687243597 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fa11003e5f0 msgr2=0x7fa110040ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:21.188 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.669+0000 7fa12cc85700 1 --2- 172.21.15.27:0/687243597 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fa11003e5f0 0x7fa110040ab0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa11c006fd0 tx=0x7fa11c006e40 comp rx=0 tx=0).stop 2024-05-21T03:13:21.188 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.669+0000 7fa12cc85700 1 -- 172.21.15.27:0/687243597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa128108030 msgr2=0x7fa1281931a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:21.188 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.669+0000 7fa12cc85700 1 --2- 172.21.15.27:0/687243597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa128108030 0x7fa1281931a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa1180116e0 tx=0x7fa118011aa0 comp rx=0 tx=0).stop 2024-05-21T03:13:21.189 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.669+0000 7fa12cc85700 1 -- 172.21.15.27:0/687243597 shutdown_connections 2024-05-21T03:13:21.189 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.669+0000 7fa12cc85700 1 --2- 172.21.15.27:0/687243597 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fa11003e5f0 0x7fa110040ab0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:21.189 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.669+0000 7fa12cc85700 1 --2- 172.21.15.27:0/687243597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa128108030 0x7fa1281931a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:21.189 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.669+0000 7fa12cc85700 1 -- 172.21.15.27:0/687243597 >> 172.21.15.27:0/687243597 conn(0x7fa128103610 msgr2=0x7fa1281043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:21.189 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.670+0000 7fa12cc85700 1 -- 172.21.15.27:0/687243597 shutdown_connections 2024-05-21T03:13:21.189 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:20.670+0000 7fa12cc85700 1 -- 172.21.15.27:0/687243597 wait complete. 2024-05-21T03:13:22.472 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.603+0000 7f4c92c97700 1 Processor -- start 2024-05-21T03:13:22.472 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.604+0000 7f4c92c97700 1 -- start start 2024-05-21T03:13:22.473 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.604+0000 7f4c92c97700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c8c109bc0 0x7f4c8c10bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:22.473 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.604+0000 7f4c92c97700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c07ced0 con 0x7f4c8c109bc0 2024-05-21T03:13:22.473 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.604+0000 7f4c90a33700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c8c109bc0 0x7f4c8c10bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:22.473 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.604+0000 7f4c90a33700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c8c109bc0 0x7f4c8c10bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58222/0 (socket says 172.21.15.27:58222) 2024-05-21T03:13:22.473 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.604+0000 7f4c90a33700 1 -- 172.21.15.27:0/2014310013 learned_addr learned my addr 172.21.15.27:0/2014310013 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:13:22.473 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.604+0000 7f4c90a33700 1 -- 172.21.15.27:0/2014310013 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c8c10c4f0 con 0x7f4c8c109bc0 2024-05-21T03:13:22.473 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.604+0000 7f4c90a33700 1 --2- 172.21.15.27:0/2014310013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c8c109bc0 0x7f4c8c10bfb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4c800098a0 tx=0x7f4c80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba03f28bca448c77 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:22.473 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.605+0000 7f4c8b7fe700 1 -- 172.21.15.27:0/2014310013 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c80004160 con 0x7f4c8c109bc0 2024-05-21T03:13:22.473 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.605+0000 7f4c8b7fe700 1 -- 172.21.15.27:0/2014310013 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c80004300 con 0x7f4c8c109bc0 2024-05-21T03:13:22.473 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.605+0000 7f4c8b7fe700 1 -- 172.21.15.27:0/2014310013 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c80011a20 con 0x7f4c8c109bc0 2024-05-21T03:13:22.473 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.605+0000 7f4c92c97700 1 -- 172.21.15.27:0/2014310013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c8c109bc0 msgr2=0x7f4c8c10bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:22.473 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.605+0000 7f4c92c97700 1 --2- 172.21.15.27:0/2014310013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c8c109bc0 0x7f4c8c10bfb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4c800098a0 tx=0x7f4c80009bb0 comp rx=0 tx=0).stop 2024-05-21T03:13:22.474 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.605+0000 7f4c92c97700 1 -- 172.21.15.27:0/2014310013 shutdown_connections 2024-05-21T03:13:22.474 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.605+0000 7f4c92c97700 1 --2- 172.21.15.27:0/2014310013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c8c109bc0 0x7f4c8c10bfb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:22.474 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.605+0000 7f4c92c97700 1 -- 172.21.15.27:0/2014310013 >> 172.21.15.27:0/2014310013 conn(0x7f4c8c103630 msgr2=0x7f4c8c105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:22.474 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.605+0000 7f4c92c97700 1 -- 172.21.15.27:0/2014310013 shutdown_connections 2024-05-21T03:13:22.474 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.605+0000 7f4c92c97700 1 -- 172.21.15.27:0/2014310013 wait complete. 2024-05-21T03:13:22.474 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.606+0000 7f4c92c97700 1 Processor -- start 2024-05-21T03:13:22.474 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.606+0000 7f4c92c97700 1 -- start start 2024-05-21T03:13:22.474 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.606+0000 7f4c92c97700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c8c109bc0 0x7f4c8c19cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:22.474 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.606+0000 7f4c92c97700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c10d500 con 0x7f4c8c109bc0 2024-05-21T03:13:22.474 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.606+0000 7f4c90a33700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c8c109bc0 0x7f4c8c19cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:22.474 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.606+0000 7f4c90a33700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c8c109bc0 0x7f4c8c19cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58228/0 (socket says 172.21.15.27:58228) 2024-05-21T03:13:22.474 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.606+0000 7f4c90a33700 1 -- 172.21.15.27:0/2270407884 learned_addr learned my addr 172.21.15.27:0/2270407884 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:13:22.474 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.607+0000 7f4c90a33700 1 -- 172.21.15.27:0/2270407884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c8c19d4c0 con 0x7f4c8c109bc0 2024-05-21T03:13:22.475 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.607+0000 7f4c90a33700 1 --2- 172.21.15.27:0/2270407884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c8c109bc0 0x7f4c8c19cf80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4c800116e0 tx=0x7f4c80011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:22.475 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.607+0000 7f4c89ffb700 1 -- 172.21.15.27:0/2270407884 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c80003a60 con 0x7f4c8c109bc0 2024-05-21T03:13:22.475 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.607+0000 7f4c89ffb700 1 -- 172.21.15.27:0/2270407884 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c8001b8f0 con 0x7f4c8c109bc0 2024-05-21T03:13:22.475 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.607+0000 7f4c92c97700 1 -- 172.21.15.27:0/2270407884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c8c19d750 con 0x7f4c8c109bc0 2024-05-21T03:13:22.475 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.607+0000 7f4c89ffb700 1 -- 172.21.15.27:0/2270407884 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c80019860 con 0x7f4c8c109bc0 2024-05-21T03:13:22.475 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.607+0000 7f4c92c97700 1 -- 172.21.15.27:0/2270407884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c8c1a0440 con 0x7f4c8c109bc0 2024-05-21T03:13:22.475 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.608+0000 7f4c92c97700 1 -- 172.21.15.27:0/2270407884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c8c064ba0 con 0x7f4c8c109bc0 2024-05-21T03:13:22.475 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.608+0000 7f4c89ffb700 1 -- 172.21.15.27:0/2270407884 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f4c8001b440 con 0x7f4c8c109bc0 2024-05-21T03:13:22.475 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.609+0000 7f4c89ffb700 1 --2- 172.21.15.27:0/2270407884 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f4c7c03e640 0x7f4c7c040b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:22.475 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.609+0000 7f4c89ffb700 1 -- 172.21.15.27:0/2270407884 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4c80053e10 con 0x7f4c8c109bc0 2024-05-21T03:13:22.475 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.609+0000 7f4c8bfff700 1 --2- 172.21.15.27:0/2270407884 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f4c7c03e640 0x7f4c7c040b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:22.475 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.609+0000 7f4c8bfff700 1 --2- 172.21.15.27:0/2270407884 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f4c7c03e640 0x7f4c7c040b00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4c78006fd0 tx=0x7f4c78006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:22.475 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.614+0000 7f4c89ffb700 1 -- 172.21.15.27:0/2270407884 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4c8001f030 con 0x7f4c8c109bc0 2024-05-21T03:13:22.476 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.949+0000 7f4c92c97700 1 -- 172.21.15.27:0/2270407884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f4c8c1a09f0 con 0x7f4c8c109bc0 2024-05-21T03:13:22.476 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.951+0000 7f4c89ffb700 1 -- 172.21.15.27:0/2270407884 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f4c800223b0 con 0x7f4c8c109bc0 2024-05-21T03:13:22.476 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.953+0000 7f4c92c97700 1 -- 172.21.15.27:0/2270407884 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f4c7c03e640 msgr2=0x7f4c7c040b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:22.476 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.953+0000 7f4c92c97700 1 --2- 172.21.15.27:0/2270407884 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f4c7c03e640 0x7f4c7c040b00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4c78006fd0 tx=0x7f4c78006e40 comp rx=0 tx=0).stop 2024-05-21T03:13:22.476 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.953+0000 7f4c92c97700 1 -- 172.21.15.27:0/2270407884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c8c109bc0 msgr2=0x7f4c8c19cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:22.476 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.953+0000 7f4c92c97700 1 --2- 172.21.15.27:0/2270407884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c8c109bc0 0x7f4c8c19cf80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4c800116e0 tx=0x7f4c80011aa0 comp rx=0 tx=0).stop 2024-05-21T03:13:22.476 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.953+0000 7f4c92c97700 1 -- 172.21.15.27:0/2270407884 shutdown_connections 2024-05-21T03:13:22.476 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.953+0000 7f4c92c97700 1 --2- 172.21.15.27:0/2270407884 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f4c7c03e640 0x7f4c7c040b00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:22.476 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.953+0000 7f4c92c97700 1 --2- 172.21.15.27:0/2270407884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c8c109bc0 0x7f4c8c19cf80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:22.476 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.953+0000 7f4c92c97700 1 -- 172.21.15.27:0/2270407884 >> 172.21.15.27:0/2270407884 conn(0x7f4c8c103630 msgr2=0x7f4c8c104010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:22.476 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.954+0000 7f4c92c97700 1 -- 172.21.15.27:0/2270407884 shutdown_connections 2024-05-21T03:13:22.476 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr 2024-05-21T03:13:21.954+0000 7f4c92c97700 1 -- 172.21.15.27:0/2270407884 wait complete. 2024-05-21T03:13:22.477 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-21T03:13:22.477 INFO:teuthology.orchestra.run.smithi027.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-21T03:13:22.477 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:13:22.477 INFO:teuthology.orchestra.run.smithi027.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-21T03:13:22.477 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:13:22.477 INFO:teuthology.orchestra.run.smithi027.stdout:Or, if you are only running a single cluster on this host: 2024-05-21T03:13:22.477 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:13:22.477 INFO:teuthology.orchestra.run.smithi027.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-21T03:13:22.477 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:13:22.477 INFO:teuthology.orchestra.run.smithi027.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-21T03:13:22.477 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:13:22.477 INFO:teuthology.orchestra.run.smithi027.stdout: ceph telemetry on 2024-05-21T03:13:22.477 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:13:22.478 INFO:teuthology.orchestra.run.smithi027.stdout:For more information see: 2024-05-21T03:13:22.478 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:13:22.478 INFO:teuthology.orchestra.run.smithi027.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-21T03:13:22.478 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:13:22.478 INFO:teuthology.orchestra.run.smithi027.stdout:Bootstrap complete. 2024-05-21T03:13:22.510 INFO:tasks.cephadm:Fetching config... 2024-05-21T03:13:22.511 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-05-21T03:13:22.511 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-21T03:13:22.528 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-21T03:13:22.528 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-05-21T03:13:22.528 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-21T03:13:22.585 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-21T03:13:22.585 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-05-21T03:13:22.585 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-21T03:13:22.651 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-21T03:13:22.651 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-05-21T03:13:22.651 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-21T03:13:22.707 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-21T03:13:22.707 DEBUG:teuthology.orchestra.run.smithi027:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRXneae/LCKt5IPkYIMkCI1LKoJecyDq09qcZ8b7G1KAQ9u28IcR13W1wFNA/KFnWjSv6cWa0MChMu/owZQ+cBalVPyOPb5JPuxWS5lXumhgci1yockXWkonaoOTAd4x/3htk7VFBZuY/+vzZIGhN3DeBNUuBAQvTGg+UjURd3yZzDGzJEtem6YPWh8mDgFhWfKviqactMRpOMTd9cNgYzq0zaKrla4vTKLnJyGGekQElhw5BKQU0wgYNUrD37g4Y6X/MnLY+iTaggZHOIKrI5CgbfNoytnTupLjt6nv+gHxFlWQpdS5hkBA2yIaWum0rmnxUVOwoMzeJKsB4MR54uWV8v5zkwnEsG/ozgBEyK5p95EVuJn0vMCh/vpkyQeK+LtJwrYSoK7IZmV5CBlcWo9j/UpT1kNfnDecbkB/b6uzjAtgRlt1scZT6Awr6KzfTtCJj5cJ1D+xIFGW0u/zZAR5/xFHZB2iyi4Gg/TjxoCYp0StZhmhxCe0nqLGmTKy0= ceph-c9ca94be-171f-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-21T03:13:22.785 INFO:teuthology.orchestra.run.smithi027.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRXneae/LCKt5IPkYIMkCI1LKoJecyDq09qcZ8b7G1KAQ9u28IcR13W1wFNA/KFnWjSv6cWa0MChMu/owZQ+cBalVPyOPb5JPuxWS5lXumhgci1yockXWkonaoOTAd4x/3htk7VFBZuY/+vzZIGhN3DeBNUuBAQvTGg+UjURd3yZzDGzJEtem6YPWh8mDgFhWfKviqactMRpOMTd9cNgYzq0zaKrla4vTKLnJyGGekQElhw5BKQU0wgYNUrD37g4Y6X/MnLY+iTaggZHOIKrI5CgbfNoytnTupLjt6nv+gHxFlWQpdS5hkBA2yIaWum0rmnxUVOwoMzeJKsB4MR54uWV8v5zkwnEsG/ozgBEyK5p95EVuJn0vMCh/vpkyQeK+LtJwrYSoK7IZmV5CBlcWo9j/UpT1kNfnDecbkB/b6uzjAtgRlt1scZT6Awr6KzfTtCJj5cJ1D+xIFGW0u/zZAR5/xFHZB2iyi4Gg/TjxoCYp0StZhmhxCe0nqLGmTKy0= ceph-c9ca94be-171f-11ef-bc9a-c7b262605968 2024-05-21T03:13:22.797 DEBUG:teuthology.orchestra.run.smithi039:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRXneae/LCKt5IPkYIMkCI1LKoJecyDq09qcZ8b7G1KAQ9u28IcR13W1wFNA/KFnWjSv6cWa0MChMu/owZQ+cBalVPyOPb5JPuxWS5lXumhgci1yockXWkonaoOTAd4x/3htk7VFBZuY/+vzZIGhN3DeBNUuBAQvTGg+UjURd3yZzDGzJEtem6YPWh8mDgFhWfKviqactMRpOMTd9cNgYzq0zaKrla4vTKLnJyGGekQElhw5BKQU0wgYNUrD37g4Y6X/MnLY+iTaggZHOIKrI5CgbfNoytnTupLjt6nv+gHxFlWQpdS5hkBA2yIaWum0rmnxUVOwoMzeJKsB4MR54uWV8v5zkwnEsG/ozgBEyK5p95EVuJn0vMCh/vpkyQeK+LtJwrYSoK7IZmV5CBlcWo9j/UpT1kNfnDecbkB/b6uzjAtgRlt1scZT6Awr6KzfTtCJj5cJ1D+xIFGW0u/zZAR5/xFHZB2iyi4Gg/TjxoCYp0StZhmhxCe0nqLGmTKy0= ceph-c9ca94be-171f-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-21T03:13:22.840 INFO:teuthology.orchestra.run.smithi039.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRXneae/LCKt5IPkYIMkCI1LKoJecyDq09qcZ8b7G1KAQ9u28IcR13W1wFNA/KFnWjSv6cWa0MChMu/owZQ+cBalVPyOPb5JPuxWS5lXumhgci1yockXWkonaoOTAd4x/3htk7VFBZuY/+vzZIGhN3DeBNUuBAQvTGg+UjURd3yZzDGzJEtem6YPWh8mDgFhWfKviqactMRpOMTd9cNgYzq0zaKrla4vTKLnJyGGekQElhw5BKQU0wgYNUrD37g4Y6X/MnLY+iTaggZHOIKrI5CgbfNoytnTupLjt6nv+gHxFlWQpdS5hkBA2yIaWum0rmnxUVOwoMzeJKsB4MR54uWV8v5zkwnEsG/ozgBEyK5p95EVuJn0vMCh/vpkyQeK+LtJwrYSoK7IZmV5CBlcWo9j/UpT1kNfnDecbkB/b6uzjAtgRlt1scZT6Awr6KzfTtCJj5cJ1D+xIFGW0u/zZAR5/xFHZB2iyi4Gg/TjxoCYp0StZhmhxCe0nqLGmTKy0= ceph-c9ca94be-171f-11ef-bc9a-c7b262605968 2024-05-21T03:13:22.853 DEBUG:teuthology.orchestra.run.smithi165:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRXneae/LCKt5IPkYIMkCI1LKoJecyDq09qcZ8b7G1KAQ9u28IcR13W1wFNA/KFnWjSv6cWa0MChMu/owZQ+cBalVPyOPb5JPuxWS5lXumhgci1yockXWkonaoOTAd4x/3htk7VFBZuY/+vzZIGhN3DeBNUuBAQvTGg+UjURd3yZzDGzJEtem6YPWh8mDgFhWfKviqactMRpOMTd9cNgYzq0zaKrla4vTKLnJyGGekQElhw5BKQU0wgYNUrD37g4Y6X/MnLY+iTaggZHOIKrI5CgbfNoytnTupLjt6nv+gHxFlWQpdS5hkBA2yIaWum0rmnxUVOwoMzeJKsB4MR54uWV8v5zkwnEsG/ozgBEyK5p95EVuJn0vMCh/vpkyQeK+LtJwrYSoK7IZmV5CBlcWo9j/UpT1kNfnDecbkB/b6uzjAtgRlt1scZT6Awr6KzfTtCJj5cJ1D+xIFGW0u/zZAR5/xFHZB2iyi4Gg/TjxoCYp0StZhmhxCe0nqLGmTKy0= ceph-c9ca94be-171f-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-21T03:13:22.891 INFO:teuthology.orchestra.run.smithi165.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRXneae/LCKt5IPkYIMkCI1LKoJecyDq09qcZ8b7G1KAQ9u28IcR13W1wFNA/KFnWjSv6cWa0MChMu/owZQ+cBalVPyOPb5JPuxWS5lXumhgci1yockXWkonaoOTAd4x/3htk7VFBZuY/+vzZIGhN3DeBNUuBAQvTGg+UjURd3yZzDGzJEtem6YPWh8mDgFhWfKviqactMRpOMTd9cNgYzq0zaKrla4vTKLnJyGGekQElhw5BKQU0wgYNUrD37g4Y6X/MnLY+iTaggZHOIKrI5CgbfNoytnTupLjt6nv+gHxFlWQpdS5hkBA2yIaWum0rmnxUVOwoMzeJKsB4MR54uWV8v5zkwnEsG/ozgBEyK5p95EVuJn0vMCh/vpkyQeK+LtJwrYSoK7IZmV5CBlcWo9j/UpT1kNfnDecbkB/b6uzjAtgRlt1scZT6Awr6KzfTtCJj5cJ1D+xIFGW0u/zZAR5/xFHZB2iyi4Gg/TjxoCYp0StZhmhxCe0nqLGmTKy0= ceph-c9ca94be-171f-11ef-bc9a-c7b262605968 2024-05-21T03:13:22.903 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-21T03:13:23.149 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:13:23.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:22 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2270407884' entity='client.admin' 2024-05-21T03:13:25.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.365+0000 7f979f0f0700 1 -- 172.21.15.27:0/115482218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97981026f0 msgr2=0x7f9798104ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:25.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.365+0000 7f979f0f0700 1 --2- 172.21.15.27:0/115482218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97981026f0 0x7f9798104ae0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f978c009910 tx=0x7f978c009c20 comp rx=0 tx=0).stop 2024-05-21T03:13:25.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.366+0000 7f979f0f0700 1 -- 172.21.15.27:0/115482218 shutdown_connections 2024-05-21T03:13:25.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.366+0000 7f979f0f0700 1 --2- 172.21.15.27:0/115482218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97981026f0 0x7f9798104ae0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:25.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.366+0000 7f979f0f0700 1 -- 172.21.15.27:0/115482218 >> 172.21.15.27:0/115482218 conn(0x7f97980fc780 msgr2=0x7f97980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:25.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.366+0000 7f979f0f0700 1 -- 172.21.15.27:0/115482218 shutdown_connections 2024-05-21T03:13:25.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.366+0000 7f979f0f0700 1 -- 172.21.15.27:0/115482218 wait complete. 2024-05-21T03:13:25.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.366+0000 7f979f0f0700 1 Processor -- start 2024-05-21T03:13:25.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.366+0000 7f979f0f0700 1 -- start start 2024-05-21T03:13:25.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.366+0000 7f979f0f0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97981026f0 0x7f97981985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:25.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.366+0000 7f979f0f0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9798107df0 con 0x7f97981026f0 2024-05-21T03:13:25.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.367+0000 7f979ce8c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97981026f0 0x7f97981985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:25.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.367+0000 7f979ce8c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97981026f0 0x7f97981985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60774/0 (socket says 172.21.15.27:60774) 2024-05-21T03:13:25.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.367+0000 7f979ce8c700 1 -- 172.21.15.27:0/4211671905 learned_addr learned my addr 172.21.15.27:0/4211671905 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:13:25.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.367+0000 7f979ce8c700 1 -- 172.21.15.27:0/4211671905 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9798198af0 con 0x7f97981026f0 2024-05-21T03:13:25.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.367+0000 7f979ce8c700 1 --2- 172.21.15.27:0/4211671905 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97981026f0 0x7f97981985b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f978c005190 tx=0x7f978c009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:25.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.367+0000 7f9795ffb700 1 -- 172.21.15.27:0/4211671905 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f978c027070 con 0x7f97981026f0 2024-05-21T03:13:25.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.367+0000 7f9795ffb700 1 -- 172.21.15.27:0/4211671905 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f978c022470 con 0x7f97981026f0 2024-05-21T03:13:25.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.367+0000 7f979f0f0700 1 -- 172.21.15.27:0/4211671905 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9798198d80 con 0x7f97981026f0 2024-05-21T03:13:25.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.367+0000 7f9795ffb700 1 -- 172.21.15.27:0/4211671905 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f978c00f5f0 con 0x7f97981026f0 2024-05-21T03:13:25.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.367+0000 7f979f0f0700 1 -- 172.21.15.27:0/4211671905 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f979819ba70 con 0x7f97981026f0 2024-05-21T03:13:25.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.368+0000 7f9795ffb700 1 -- 172.21.15.27:0/4211671905 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f978c02a070 con 0x7f97981026f0 2024-05-21T03:13:25.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.368+0000 7f979f0f0700 1 -- 172.21.15.27:0/4211671905 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9798108570 con 0x7f97981026f0 2024-05-21T03:13:25.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.369+0000 7f9795ffb700 1 --2- 172.21.15.27:0/4211671905 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f978803e570 0x7f9788040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:25.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.369+0000 7f9795ffb700 1 -- 172.21.15.27:0/4211671905 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f978c01e030 con 0x7f97981026f0 2024-05-21T03:13:25.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.369+0000 7f9797fff700 1 --2- 172.21.15.27:0/4211671905 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f978803e570 0x7f9788040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:25.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.369+0000 7f9797fff700 1 --2- 172.21.15.27:0/4211671905 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f978803e570 0x7f9788040a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9784006fd0 tx=0x7f9784006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:25.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.374+0000 7f9795ffb700 1 -- 172.21.15.27:0/4211671905 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f978c013070 con 0x7f97981026f0 2024-05-21T03:13:25.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.589+0000 7f979f0f0700 1 -- 172.21.15.27:0/4211671905 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f979802e670 con 0x7f97981026f0 2024-05-21T03:13:25.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.594+0000 7f9795ffb700 1 -- 172.21.15.27:0/4211671905 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f978c031020 con 0x7f97981026f0 2024-05-21T03:13:25.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.597+0000 7f979f0f0700 1 -- 172.21.15.27:0/4211671905 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f978803e570 msgr2=0x7f9788040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:25.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.597+0000 7f979f0f0700 1 --2- 172.21.15.27:0/4211671905 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f978803e570 0x7f9788040a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9784006fd0 tx=0x7f9784006e40 comp rx=0 tx=0).stop 2024-05-21T03:13:25.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.597+0000 7f979f0f0700 1 -- 172.21.15.27:0/4211671905 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97981026f0 msgr2=0x7f97981985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:25.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.597+0000 7f979f0f0700 1 --2- 172.21.15.27:0/4211671905 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97981026f0 0x7f97981985b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f978c005190 tx=0x7f978c009ea0 comp rx=0 tx=0).stop 2024-05-21T03:13:25.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.597+0000 7f979f0f0700 1 -- 172.21.15.27:0/4211671905 shutdown_connections 2024-05-21T03:13:25.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.597+0000 7f979f0f0700 1 --2- 172.21.15.27:0/4211671905 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f978803e570 0x7f9788040a30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:25.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.597+0000 7f979f0f0700 1 --2- 172.21.15.27:0/4211671905 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97981026f0 0x7f97981985b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:25.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.597+0000 7f979f0f0700 1 -- 172.21.15.27:0/4211671905 >> 172.21.15.27:0/4211671905 conn(0x7f97980fc780 msgr2=0x7f9798101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:25.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.597+0000 7f979f0f0700 1 -- 172.21.15.27:0/4211671905 shutdown_connections 2024-05-21T03:13:25.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:25.597+0000 7f979f0f0700 1 -- 172.21.15.27:0/4211671905 wait complete. 2024-05-21T03:13:25.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:25 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:25.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:25 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:25.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:25 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:13:25.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:25 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:25.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:25 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:13:25.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:25 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:26.234 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-21T03:13:26.234 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-21T03:13:26.480 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:13:26.889 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:26 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/4211671905' entity='client.admin' 2024-05-21T03:13:26.889 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:26 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:13:26.889 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:26 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:27.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:27.930+0000 7ff0805a5700 1 -- 172.21.15.27:0/1774409512 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff078101ea0 msgr2=0x7ff0781022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:27.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:27.930+0000 7ff0805a5700 1 --2- 172.21.15.27:0/1774409512 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff078101ea0 0x7ff0781022c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff074009910 tx=0x7ff074009c20 comp rx=0 tx=0).stop 2024-05-21T03:13:27.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:27.930+0000 7ff0805a5700 1 -- 172.21.15.27:0/1774409512 shutdown_connections 2024-05-21T03:13:27.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:27.930+0000 7ff0805a5700 1 --2- 172.21.15.27:0/1774409512 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff078101ea0 0x7ff0781022c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:27.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:27.930+0000 7ff0805a5700 1 -- 172.21.15.27:0/1774409512 >> 172.21.15.27:0/1774409512 conn(0x7ff0780fd450 msgr2=0x7ff0780ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:27.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:27.930+0000 7ff0805a5700 1 -- 172.21.15.27:0/1774409512 shutdown_connections 2024-05-21T03:13:27.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:27.930+0000 7ff0805a5700 1 -- 172.21.15.27:0/1774409512 wait complete. 2024-05-21T03:13:27.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:27.931+0000 7ff0805a5700 1 Processor -- start 2024-05-21T03:13:27.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:27.931+0000 7ff0805a5700 1 -- start start 2024-05-21T03:13:27.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:27.931+0000 7ff0805a5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff078101ea0 0x7ff078198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:27.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:27.931+0000 7ff0805a5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff078107da0 con 0x7ff078101ea0 2024-05-21T03:13:27.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:27.931+0000 7ff07e341700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff078101ea0 0x7ff078198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:27.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:27.931+0000 7ff07e341700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff078101ea0 0x7ff078198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60794/0 (socket says 172.21.15.27:60794) 2024-05-21T03:13:27.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:27.931+0000 7ff07e341700 1 -- 172.21.15.27:0/1461536262 learned_addr learned my addr 172.21.15.27:0/1461536262 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:13:27.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:27.931+0000 7ff07e341700 1 -- 172.21.15.27:0/1461536262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff078198b40 con 0x7ff078101ea0 2024-05-21T03:13:27.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:27.932+0000 7ff07e341700 1 --2- 172.21.15.27:0/1461536262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff078101ea0 0x7ff078198600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff074005800 tx=0x7ff074009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:27.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:27.932+0000 7ff06b7fe700 1 -- 172.21.15.27:0/1461536262 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff074027070 con 0x7ff078101ea0 2024-05-21T03:13:27.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:27.932+0000 7ff06b7fe700 1 -- 172.21.15.27:0/1461536262 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff07400be10 con 0x7ff078101ea0 2024-05-21T03:13:27.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:27.932+0000 7ff0805a5700 1 -- 172.21.15.27:0/1461536262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff078198dd0 con 0x7ff078101ea0 2024-05-21T03:13:27.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:27.932+0000 7ff06b7fe700 1 -- 172.21.15.27:0/1461536262 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff07400f460 con 0x7ff078101ea0 2024-05-21T03:13:27.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:27.932+0000 7ff0805a5700 1 -- 172.21.15.27:0/1461536262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff07819bac0 con 0x7ff078101ea0 2024-05-21T03:13:27.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:27.933+0000 7ff06b7fe700 1 -- 172.21.15.27:0/1461536262 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7ff07402a070 con 0x7ff078101ea0 2024-05-21T03:13:27.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:27.933+0000 7ff06b7fe700 1 --2- 172.21.15.27:0/1461536262 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7ff06403e510 0x7ff0640409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:27.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:27.933+0000 7ff0805a5700 1 -- 172.21.15.27:0/1461536262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff078108520 con 0x7ff078101ea0 2024-05-21T03:13:27.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:27.933+0000 7ff07db40700 1 --2- 172.21.15.27:0/1461536262 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7ff06403e510 0x7ff0640409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:27.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:27.933+0000 7ff06b7fe700 1 -- 172.21.15.27:0/1461536262 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff07401e030 con 0x7ff078101ea0 2024-05-21T03:13:27.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:27.933+0000 7ff07db40700 1 --2- 172.21.15.27:0/1461536262 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7ff06403e510 0x7ff0640409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff06c006fd0 tx=0x7ff06c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:27.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:27.938+0000 7ff06b7fe700 1 -- 172.21.15.27:0/1461536262 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff074013070 con 0x7ff078101ea0 2024-05-21T03:13:28.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:28.158+0000 7ff0805a5700 1 -- 172.21.15.27:0/1461536262 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7ff07802e640 con 0x7ff06403e510 2024-05-21T03:13:28.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:28.162+0000 7ff06b7fe700 1 -- 172.21.15.27:0/1461536262 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff07802e640 con 0x7ff06403e510 2024-05-21T03:13:28.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:28.165+0000 7ff0805a5700 1 -- 172.21.15.27:0/1461536262 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7ff06403e510 msgr2=0x7ff0640409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:28.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:28.165+0000 7ff0805a5700 1 --2- 172.21.15.27:0/1461536262 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7ff06403e510 0x7ff0640409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff06c006fd0 tx=0x7ff06c006e40 comp rx=0 tx=0).stop 2024-05-21T03:13:28.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:28.165+0000 7ff0805a5700 1 -- 172.21.15.27:0/1461536262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff078101ea0 msgr2=0x7ff078198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:28.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:28.165+0000 7ff0805a5700 1 --2- 172.21.15.27:0/1461536262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff078101ea0 0x7ff078198600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff074005800 tx=0x7ff074009f60 comp rx=0 tx=0).stop 2024-05-21T03:13:28.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:28.165+0000 7ff0805a5700 1 -- 172.21.15.27:0/1461536262 shutdown_connections 2024-05-21T03:13:28.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:28.165+0000 7ff0805a5700 1 --2- 172.21.15.27:0/1461536262 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7ff06403e510 0x7ff0640409d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:28.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:28.165+0000 7ff0805a5700 1 --2- 172.21.15.27:0/1461536262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff078101ea0 0x7ff078198600 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:28.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:28.165+0000 7ff0805a5700 1 -- 172.21.15.27:0/1461536262 >> 172.21.15.27:0/1461536262 conn(0x7ff0780fd450 msgr2=0x7ff07806b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:28.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:28.165+0000 7ff0805a5700 1 -- 172.21.15.27:0/1461536262 shutdown_connections 2024-05-21T03:13:28.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:28.165+0000 7ff0805a5700 1 -- 172.21.15.27:0/1461536262 wait complete. 2024-05-21T03:13:29.006 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi039 2024-05-21T03:13:29.006 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-21T03:13:29.006 DEBUG:teuthology.orchestra.run.smithi039:> dd of=/etc/ceph/ceph.conf 2024-05-21T03:13:29.025 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-21T03:13:29.025 DEBUG:teuthology.orchestra.run.smithi039:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-21T03:13:29.083 INFO:tasks.cephadm:Adding host smithi039 to orchestrator... 2024-05-21T03:13:29.083 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph orch host add smithi039 2024-05-21T03:13:29.342 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:13:29.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:29 smithi027 ceph-mon[32179]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:13:29.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:29 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:29.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:29 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:13:29.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:29 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:13:29.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:29 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:13:29.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:29 smithi027 ceph-mon[32179]: Updating smithi027:/etc/ceph/ceph.conf 2024-05-21T03:13:29.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:29 smithi027 ceph-mon[32179]: Updating smithi027:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T03:13:29.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:29 smithi027 ceph-mon[32179]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-05-21T03:13:30.481 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:30 smithi027 ceph-mon[32179]: Updating smithi027:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T03:13:30.481 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:30 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:30.481 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:30 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:30.481 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:30 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:30.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:30.808+0000 7f788e397700 1 -- 172.21.15.27:0/2412443834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7888104150 msgr2=0x7f7888104530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:30.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:30.808+0000 7f788e397700 1 --2- 172.21.15.27:0/2412443834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7888104150 0x7f7888104530 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7878009910 tx=0x7f7878009c20 comp rx=0 tx=0).stop 2024-05-21T03:13:30.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:30.808+0000 7f788e397700 1 -- 172.21.15.27:0/2412443834 shutdown_connections 2024-05-21T03:13:30.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:30.808+0000 7f788e397700 1 --2- 172.21.15.27:0/2412443834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7888104150 0x7f7888104530 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:30.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:30.808+0000 7f788e397700 1 -- 172.21.15.27:0/2412443834 >> 172.21.15.27:0/2412443834 conn(0x7f78880fd9a0 msgr2=0x7f78880ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:30.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:30.808+0000 7f788e397700 1 -- 172.21.15.27:0/2412443834 shutdown_connections 2024-05-21T03:13:30.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:30.808+0000 7f788e397700 1 -- 172.21.15.27:0/2412443834 wait complete. 2024-05-21T03:13:30.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:30.809+0000 7f788e397700 1 Processor -- start 2024-05-21T03:13:30.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:30.809+0000 7f788e397700 1 -- start start 2024-05-21T03:13:30.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:30.809+0000 7f788e397700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7888104150 0x7f788819e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:30.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:30.809+0000 7f788e397700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f788810d4d0 con 0x7f7888104150 2024-05-21T03:13:30.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:30.809+0000 7f7887fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7888104150 0x7f788819e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:30.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:30.809+0000 7f7887fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7888104150 0x7f788819e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60818/0 (socket says 172.21.15.27:60818) 2024-05-21T03:13:30.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:30.809+0000 7f7887fff700 1 -- 172.21.15.27:0/2481058108 learned_addr learned my addr 172.21.15.27:0/2481058108 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:13:30.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:30.810+0000 7f7887fff700 1 -- 172.21.15.27:0/2481058108 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f788819ec20 con 0x7f7888104150 2024-05-21T03:13:30.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:30.810+0000 7f7887fff700 1 --2- 172.21.15.27:0/2481058108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7888104150 0x7f788819e6e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7878006010 tx=0x7f7878005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:30.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:30.810+0000 7f7885ffb700 1 -- 172.21.15.27:0/2481058108 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7878026070 con 0x7f7888104150 2024-05-21T03:13:30.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:30.810+0000 7f7885ffb700 1 -- 172.21.15.27:0/2481058108 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f787800bc00 con 0x7f7888104150 2024-05-21T03:13:30.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:30.810+0000 7f788e397700 1 -- 172.21.15.27:0/2481058108 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f788819eeb0 con 0x7f7888104150 2024-05-21T03:13:30.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:30.810+0000 7f7885ffb700 1 -- 172.21.15.27:0/2481058108 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7878003910 con 0x7f7888104150 2024-05-21T03:13:30.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:30.810+0000 7f788e397700 1 -- 172.21.15.27:0/2481058108 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78881988a0 con 0x7f7888104150 2024-05-21T03:13:30.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:30.811+0000 7f7885ffb700 1 -- 172.21.15.27:0/2481058108 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f7878029070 con 0x7f7888104150 2024-05-21T03:13:30.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:30.811+0000 7f788e397700 1 -- 172.21.15.27:0/2481058108 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f788806bc90 con 0x7f7888104150 2024-05-21T03:13:30.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:30.811+0000 7f7885ffb700 1 --2- 172.21.15.27:0/2481058108 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f787403e510 0x7f78740409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:30.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:30.811+0000 7f7885ffb700 1 -- 172.21.15.27:0/2481058108 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f787801e070 con 0x7f7888104150 2024-05-21T03:13:30.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:30.811+0000 7f787ffff700 1 --2- 172.21.15.27:0/2481058108 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f787403e510 0x7f78740409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:30.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:30.812+0000 7f787ffff700 1 --2- 172.21.15.27:0/2481058108 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f787403e510 0x7f78740409d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7870006fd0 tx=0x7f7870006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:30.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:30.817+0000 7f7885ffb700 1 -- 172.21.15.27:0/2481058108 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7878013070 con 0x7f7888104150 2024-05-21T03:13:31.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:31.034+0000 7f788e397700 1 -- 172.21.15.27:0/2481058108 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi039", "target": ["mon-mgr", ""]}) -- 0x7f7888199dd0 con 0x7f787403e510 2024-05-21T03:13:32.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:31 smithi027 ceph-mon[32179]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi039", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:13:32.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:31 smithi027 ceph-mon[32179]: Deploying cephadm binary to smithi039 2024-05-21T03:13:34.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:34 smithi027 ceph-mon[32179]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:13:35.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:35.316+0000 7f7885ffb700 1 -- 172.21.15.27:0/2481058108 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f7888199dd0 con 0x7f787403e510 2024-05-21T03:13:35.316 INFO:teuthology.orchestra.run.smithi027.stdout:Added host 'smithi039' with addr '172.21.15.39' 2024-05-21T03:13:35.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:35.319+0000 7f788e397700 1 -- 172.21.15.27:0/2481058108 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f787403e510 msgr2=0x7f78740409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:35.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:35.319+0000 7f788e397700 1 --2- 172.21.15.27:0/2481058108 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f787403e510 0x7f78740409d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7870006fd0 tx=0x7f7870006e40 comp rx=0 tx=0).stop 2024-05-21T03:13:35.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:35.319+0000 7f788e397700 1 -- 172.21.15.27:0/2481058108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7888104150 msgr2=0x7f788819e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:35.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:35.319+0000 7f788e397700 1 --2- 172.21.15.27:0/2481058108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7888104150 0x7f788819e6e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7878006010 tx=0x7f7878005780 comp rx=0 tx=0).stop 2024-05-21T03:13:35.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:35.319+0000 7f788e397700 1 -- 172.21.15.27:0/2481058108 shutdown_connections 2024-05-21T03:13:35.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:35.319+0000 7f788e397700 1 --2- 172.21.15.27:0/2481058108 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f787403e510 0x7f78740409d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:35.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:35.319+0000 7f788e397700 1 --2- 172.21.15.27:0/2481058108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7888104150 0x7f788819e6e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:35.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:35.319+0000 7f788e397700 1 -- 172.21.15.27:0/2481058108 >> 172.21.15.27:0/2481058108 conn(0x7f78880fd9a0 msgr2=0x7f7888104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:35.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:35.319+0000 7f788e397700 1 -- 172.21.15.27:0/2481058108 shutdown_connections 2024-05-21T03:13:35.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:35.319+0000 7f788e397700 1 -- 172.21.15.27:0/2481058108 wait complete. 2024-05-21T03:13:35.859 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph orch host ls --format=json 2024-05-21T03:13:36.112 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:13:36.535 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:36 smithi027 ceph-mon[32179]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:13:36.535 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:36 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:36.535 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:36 smithi027 ceph-mon[32179]: Added host smithi039 2024-05-21T03:13:36.536 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:36 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:13:36.536 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:36 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:37.558 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:37 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:37.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.588+0000 7fb092fdb700 1 -- 172.21.15.27:0/719069446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb08c1020b0 msgr2=0x7fb08c102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:37.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.588+0000 7fb092fdb700 1 --2- 172.21.15.27:0/719069446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb08c1020b0 0x7fb08c102490 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb07c009910 tx=0x7fb07c009c20 comp rx=0 tx=0).stop 2024-05-21T03:13:37.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.588+0000 7fb092fdb700 1 -- 172.21.15.27:0/719069446 shutdown_connections 2024-05-21T03:13:37.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.588+0000 7fb092fdb700 1 --2- 172.21.15.27:0/719069446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb08c1020b0 0x7fb08c102490 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:37.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.588+0000 7fb092fdb700 1 -- 172.21.15.27:0/719069446 >> 172.21.15.27:0/719069446 conn(0x7fb08c0fd940 msgr2=0x7fb08c0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:37.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.589+0000 7fb092fdb700 1 -- 172.21.15.27:0/719069446 shutdown_connections 2024-05-21T03:13:37.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.589+0000 7fb092fdb700 1 -- 172.21.15.27:0/719069446 wait complete. 2024-05-21T03:13:37.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.589+0000 7fb092fdb700 1 Processor -- start 2024-05-21T03:13:37.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.589+0000 7fb092fdb700 1 -- start start 2024-05-21T03:13:37.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.589+0000 7fb092fdb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb08c1020b0 0x7fb08c103b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:37.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.589+0000 7fb092fdb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c10c330 con 0x7fb08c1020b0 2024-05-21T03:13:37.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.589+0000 7fb090d77700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb08c1020b0 0x7fb08c103b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:37.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.590+0000 7fb090d77700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb08c1020b0 0x7fb08c103b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39624/0 (socket says 172.21.15.27:39624) 2024-05-21T03:13:37.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.590+0000 7fb090d77700 1 -- 172.21.15.27:0/4110763461 learned_addr learned my addr 172.21.15.27:0/4110763461 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:13:37.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.590+0000 7fb090d77700 1 -- 172.21.15.27:0/4110763461 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb08c105a40 con 0x7fb08c1020b0 2024-05-21T03:13:37.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.590+0000 7fb090d77700 1 --2- 172.21.15.27:0/4110763461 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb08c1020b0 0x7fb08c103b50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0056d0 tx=0x7fb07c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:37.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.590+0000 7fb089ffb700 1 -- 172.21.15.27:0/4110763461 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb07c026070 con 0x7fb08c1020b0 2024-05-21T03:13:37.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.590+0000 7fb089ffb700 1 -- 172.21.15.27:0/4110763461 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb07c00bc00 con 0x7fb08c1020b0 2024-05-21T03:13:37.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.590+0000 7fb092fdb700 1 -- 172.21.15.27:0/4110763461 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb08c1040f0 con 0x7fb08c1020b0 2024-05-21T03:13:37.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.590+0000 7fb089ffb700 1 -- 172.21.15.27:0/4110763461 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb07c003920 con 0x7fb08c1020b0 2024-05-21T03:13:37.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.590+0000 7fb092fdb700 1 -- 172.21.15.27:0/4110763461 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb08c1045b0 con 0x7fb08c1020b0 2024-05-21T03:13:37.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.591+0000 7fb089ffb700 1 -- 172.21.15.27:0/4110763461 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fb07c029070 con 0x7fb08c1020b0 2024-05-21T03:13:37.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.591+0000 7fb092fdb700 1 -- 172.21.15.27:0/4110763461 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb08c103590 con 0x7fb08c1020b0 2024-05-21T03:13:37.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.591+0000 7fb089ffb700 1 --2- 172.21.15.27:0/4110763461 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb07803e570 0x7fb078040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:37.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.592+0000 7fb089ffb700 1 -- 172.21.15.27:0/4110763461 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb07c01e070 con 0x7fb08c1020b0 2024-05-21T03:13:37.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.592+0000 7fb08bfff700 1 --2- 172.21.15.27:0/4110763461 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb07803e570 0x7fb078040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:37.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.592+0000 7fb08bfff700 1 --2- 172.21.15.27:0/4110763461 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb07803e570 0x7fb078040a30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb074006fd0 tx=0x7fb074006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:37.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.597+0000 7fb089ffb700 1 -- 172.21.15.27:0/4110763461 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb07c013070 con 0x7fb08c1020b0 2024-05-21T03:13:37.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.814+0000 7fb092fdb700 1 -- 172.21.15.27:0/4110763461 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb08c1050f0 con 0x7fb07803e570 2024-05-21T03:13:37.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.815+0000 7fb089ffb700 1 -- 172.21.15.27:0/4110763461 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fb08c1050f0 con 0x7fb07803e570 2024-05-21T03:13:37.815 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:13:37.815 INFO:teuthology.orchestra.run.smithi027.stdout:[{"addr": "172.21.15.27", "hostname": "smithi027", "labels": [], "status": ""}, {"addr": "172.21.15.39", "hostname": "smithi039", "labels": [], "status": ""}] 2024-05-21T03:13:37.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.817+0000 7fb092fdb700 1 -- 172.21.15.27:0/4110763461 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb07803e570 msgr2=0x7fb078040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:37.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.817+0000 7fb092fdb700 1 --2- 172.21.15.27:0/4110763461 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb07803e570 0x7fb078040a30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb074006fd0 tx=0x7fb074006e40 comp rx=0 tx=0).stop 2024-05-21T03:13:37.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.818+0000 7fb092fdb700 1 -- 172.21.15.27:0/4110763461 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb08c1020b0 msgr2=0x7fb08c103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:37.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.818+0000 7fb092fdb700 1 --2- 172.21.15.27:0/4110763461 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb08c1020b0 0x7fb08c103b50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0056d0 tx=0x7fb07c0057b0 comp rx=0 tx=0).stop 2024-05-21T03:13:37.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.818+0000 7fb092fdb700 1 -- 172.21.15.27:0/4110763461 shutdown_connections 2024-05-21T03:13:37.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.818+0000 7fb092fdb700 1 --2- 172.21.15.27:0/4110763461 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb07803e570 0x7fb078040a30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:37.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.818+0000 7fb092fdb700 1 --2- 172.21.15.27:0/4110763461 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb08c1020b0 0x7fb08c103b50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:37.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.818+0000 7fb092fdb700 1 -- 172.21.15.27:0/4110763461 >> 172.21.15.27:0/4110763461 conn(0x7fb08c0fd940 msgr2=0x7fb08c06b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:37.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.818+0000 7fb092fdb700 1 -- 172.21.15.27:0/4110763461 shutdown_connections 2024-05-21T03:13:37.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:37.818+0000 7fb092fdb700 1 -- 172.21.15.27:0/4110763461 wait complete. 2024-05-21T03:13:38.458 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi165 2024-05-21T03:13:38.458 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-05-21T03:13:38.458 DEBUG:teuthology.orchestra.run.smithi165:> dd of=/etc/ceph/ceph.conf 2024-05-21T03:13:38.477 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-05-21T03:13:38.477 DEBUG:teuthology.orchestra.run.smithi165:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-21T03:13:38.536 INFO:tasks.cephadm:Adding host smithi165 to orchestrator... 2024-05-21T03:13:38.536 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph orch host add smithi165 2024-05-21T03:13:38.781 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:13:38.903 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:38 smithi027 ceph-mon[32179]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:13:38.903 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:38 smithi027 ceph-mon[32179]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:13:38.904 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:38 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:40.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:40.325+0000 7f854b674700 1 -- 172.21.15.27:0/223704307 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8544104110 msgr2=0x7f85441044f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:40.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:40.325+0000 7f854b674700 1 --2- 172.21.15.27:0/223704307 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8544104110 0x7f85441044f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8540009910 tx=0x7f8540009c20 comp rx=0 tx=0).stop 2024-05-21T03:13:40.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:40.325+0000 7f854b674700 1 -- 172.21.15.27:0/223704307 shutdown_connections 2024-05-21T03:13:40.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:40.325+0000 7f854b674700 1 --2- 172.21.15.27:0/223704307 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8544104110 0x7f85441044f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:40.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:40.325+0000 7f854b674700 1 -- 172.21.15.27:0/223704307 >> 172.21.15.27:0/223704307 conn(0x7f85440fd960 msgr2=0x7f85440ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:40.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:40.326+0000 7f854b674700 1 -- 172.21.15.27:0/223704307 shutdown_connections 2024-05-21T03:13:40.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:40.326+0000 7f854b674700 1 -- 172.21.15.27:0/223704307 wait complete. 2024-05-21T03:13:40.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:40.326+0000 7f854b674700 1 Processor -- start 2024-05-21T03:13:40.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:40.326+0000 7f854b674700 1 -- start start 2024-05-21T03:13:40.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:40.327+0000 7f854b674700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8544104110 0x7f854419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:40.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:40.327+0000 7f854b674700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f854410d4b0 con 0x7f8544104110 2024-05-21T03:13:40.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:40.327+0000 7f8549410700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8544104110 0x7f854419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:40.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:40.327+0000 7f8549410700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8544104110 0x7f854419e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39636/0 (socket says 172.21.15.27:39636) 2024-05-21T03:13:40.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:40.327+0000 7f8549410700 1 -- 172.21.15.27:0/309844183 learned_addr learned my addr 172.21.15.27:0/309844183 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:13:40.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:40.327+0000 7f8549410700 1 -- 172.21.15.27:0/309844183 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854419ec10 con 0x7f8544104110 2024-05-21T03:13:40.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:40.327+0000 7f8549410700 1 --2- 172.21.15.27:0/309844183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8544104110 0x7f854419e6d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8540006010 tx=0x7f8540005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:40.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:40.327+0000 7f853a7fc700 1 -- 172.21.15.27:0/309844183 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8540026070 con 0x7f8544104110 2024-05-21T03:13:40.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:40.327+0000 7f853a7fc700 1 -- 172.21.15.27:0/309844183 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f854000bc00 con 0x7f8544104110 2024-05-21T03:13:40.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:40.327+0000 7f854b674700 1 -- 172.21.15.27:0/309844183 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f854419eea0 con 0x7f8544104110 2024-05-21T03:13:40.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:40.327+0000 7f853a7fc700 1 -- 172.21.15.27:0/309844183 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8540003910 con 0x7f8544104110 2024-05-21T03:13:40.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:40.328+0000 7f854b674700 1 -- 172.21.15.27:0/309844183 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8544198850 con 0x7f8544104110 2024-05-21T03:13:40.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:40.328+0000 7f854b674700 1 -- 172.21.15.27:0/309844183 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f854406bcd0 con 0x7f8544104110 2024-05-21T03:13:40.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:40.328+0000 7f853a7fc700 1 -- 172.21.15.27:0/309844183 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f8540029070 con 0x7f8544104110 2024-05-21T03:13:40.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:40.329+0000 7f853a7fc700 1 --2- 172.21.15.27:0/309844183 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f853003e510 0x7f85300409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:40.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:40.329+0000 7f853a7fc700 1 -- 172.21.15.27:0/309844183 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f854001e070 con 0x7f8544104110 2024-05-21T03:13:40.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:40.329+0000 7f8548c0f700 1 --2- 172.21.15.27:0/309844183 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f853003e510 0x7f85300409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:40.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:40.329+0000 7f8548c0f700 1 --2- 172.21.15.27:0/309844183 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f853003e510 0x7f85300409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8534006fd0 tx=0x7f8534006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:40.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:40.335+0000 7f853a7fc700 1 -- 172.21.15.27:0/309844183 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8540013070 con 0x7f8544104110 2024-05-21T03:13:40.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:40.553+0000 7f854b674700 1 -- 172.21.15.27:0/309844183 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi165", "target": ["mon-mgr", ""]}) -- 0x7f8544199d80 con 0x7f853003e510 2024-05-21T03:13:40.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:40 smithi027 ceph-mon[32179]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:13:41.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:41 smithi027 ceph-mon[32179]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi165", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:13:41.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:41 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:41.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:41 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:41.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:41 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:41.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:41 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:41.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:41 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi039", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:13:41.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:41 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:13:41.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:41 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:13:42.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:42 smithi027 ceph-mon[32179]: Updating smithi039:/etc/ceph/ceph.conf 2024-05-21T03:13:42.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:42 smithi027 ceph-mon[32179]: Deploying cephadm binary to smithi165 2024-05-21T03:13:42.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:42 smithi027 ceph-mon[32179]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:13:42.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:42 smithi027 ceph-mon[32179]: Updating smithi039:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T03:13:42.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:42 smithi027 ceph-mon[32179]: Updating smithi039:/etc/ceph/ceph.client.admin.keyring 2024-05-21T03:13:44.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:44.767+0000 7f853a7fc700 1 -- 172.21.15.27:0/309844183 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f8544199d80 con 0x7f853003e510 2024-05-21T03:13:44.767 INFO:teuthology.orchestra.run.smithi027.stdout:Added host 'smithi165' with addr '172.21.15.165' 2024-05-21T03:13:44.767 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:44 smithi027 ceph-mon[32179]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:13:44.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:44.770+0000 7f854b674700 1 -- 172.21.15.27:0/309844183 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f853003e510 msgr2=0x7f85300409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:44.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:44.770+0000 7f854b674700 1 --2- 172.21.15.27:0/309844183 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f853003e510 0x7f85300409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8534006fd0 tx=0x7f8534006e40 comp rx=0 tx=0).stop 2024-05-21T03:13:44.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:44.770+0000 7f854b674700 1 -- 172.21.15.27:0/309844183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8544104110 msgr2=0x7f854419e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:44.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:44.770+0000 7f854b674700 1 --2- 172.21.15.27:0/309844183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8544104110 0x7f854419e6d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8540006010 tx=0x7f8540005780 comp rx=0 tx=0).stop 2024-05-21T03:13:44.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:44.770+0000 7f854b674700 1 -- 172.21.15.27:0/309844183 shutdown_connections 2024-05-21T03:13:44.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:44.770+0000 7f854b674700 1 --2- 172.21.15.27:0/309844183 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f853003e510 0x7f85300409d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:44.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:44.770+0000 7f854b674700 1 --2- 172.21.15.27:0/309844183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8544104110 0x7f854419e6d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:44.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:44.770+0000 7f854b674700 1 -- 172.21.15.27:0/309844183 >> 172.21.15.27:0/309844183 conn(0x7f85440fd960 msgr2=0x7f8544104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:44.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:44.770+0000 7f854b674700 1 -- 172.21.15.27:0/309844183 shutdown_connections 2024-05-21T03:13:44.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:44.770+0000 7f854b674700 1 -- 172.21.15.27:0/309844183 wait complete. 2024-05-21T03:13:45.312 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph orch host ls --format=json 2024-05-21T03:13:45.557 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:13:45.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:45 smithi027 ceph-mon[32179]: Updating smithi039:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T03:13:45.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:45 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:45.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:45 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:45.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:45 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:45.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:45 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:45.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:45 smithi027 ceph-mon[32179]: Added host smithi165 2024-05-21T03:13:45.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:45 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:13:45.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:45 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:46.840 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:46 smithi027 ceph-mon[32179]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:13:46.840 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:46 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:47.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.092+0000 7f1ee9547700 1 -- 172.21.15.27:0/2961892084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ee41031b0 msgr2=0x7f1ee4103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:47.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.092+0000 7f1ee9547700 1 --2- 172.21.15.27:0/2961892084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ee41031b0 0x7f1ee4103590 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4009910 tx=0x7f1ed4009c20 comp rx=0 tx=0).stop 2024-05-21T03:13:47.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.092+0000 7f1ee9547700 1 -- 172.21.15.27:0/2961892084 shutdown_connections 2024-05-21T03:13:47.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.092+0000 7f1ee9547700 1 --2- 172.21.15.27:0/2961892084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ee41031b0 0x7f1ee4103590 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:47.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.092+0000 7f1ee9547700 1 -- 172.21.15.27:0/2961892084 >> 172.21.15.27:0/2961892084 conn(0x7f1ee40fea80 msgr2=0x7f1ee4100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:47.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.092+0000 7f1ee9547700 1 -- 172.21.15.27:0/2961892084 shutdown_connections 2024-05-21T03:13:47.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.092+0000 7f1ee9547700 1 -- 172.21.15.27:0/2961892084 wait complete. 2024-05-21T03:13:47.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.093+0000 7f1ee9547700 1 Processor -- start 2024-05-21T03:13:47.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.093+0000 7f1ee9547700 1 -- start start 2024-05-21T03:13:47.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.093+0000 7f1ee9547700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ee41031b0 0x7f1ee4075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:47.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.093+0000 7f1ee9547700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee410d430 con 0x7f1ee41031b0 2024-05-21T03:13:47.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.094+0000 7f1ee2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ee41031b0 0x7f1ee4075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:47.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.094+0000 7f1ee2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ee41031b0 0x7f1ee4075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56428/0 (socket says 172.21.15.27:56428) 2024-05-21T03:13:47.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.094+0000 7f1ee2ffd700 1 -- 172.21.15.27:0/808059446 learned_addr learned my addr 172.21.15.27:0/808059446 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:13:47.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.094+0000 7f1ee2ffd700 1 -- 172.21.15.27:0/808059446 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee4077730 con 0x7f1ee41031b0 2024-05-21T03:13:47.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.094+0000 7f1ee2ffd700 1 --2- 172.21.15.27:0/808059446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ee41031b0 0x7f1ee4075840 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1ed40056d0 tx=0x7f1ed40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:47.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.094+0000 7f1ecbfff700 1 -- 172.21.15.27:0/808059446 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ed4026070 con 0x7f1ee41031b0 2024-05-21T03:13:47.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.094+0000 7f1ee9547700 1 -- 172.21.15.27:0/808059446 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ee4075de0 con 0x7f1ee41031b0 2024-05-21T03:13:47.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.094+0000 7f1ecbfff700 1 -- 172.21.15.27:0/808059446 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ed400bc00 con 0x7f1ee41031b0 2024-05-21T03:13:47.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.094+0000 7f1ecbfff700 1 -- 172.21.15.27:0/808059446 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ed4003920 con 0x7f1ee41031b0 2024-05-21T03:13:47.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.094+0000 7f1ee9547700 1 -- 172.21.15.27:0/808059446 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ee40762a0 con 0x7f1ee41031b0 2024-05-21T03:13:47.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.095+0000 7f1ee9547700 1 -- 172.21.15.27:0/808059446 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ee4104690 con 0x7f1ee41031b0 2024-05-21T03:13:47.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.095+0000 7f1ecbfff700 1 -- 172.21.15.27:0/808059446 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f1ed4029070 con 0x7f1ee41031b0 2024-05-21T03:13:47.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.096+0000 7f1ecbfff700 1 --2- 172.21.15.27:0/808059446 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f1ecc03e570 0x7f1ecc040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:47.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.096+0000 7f1ecbfff700 1 -- 172.21.15.27:0/808059446 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1ed401e070 con 0x7f1ee41031b0 2024-05-21T03:13:47.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.096+0000 7f1ee27fc700 1 --2- 172.21.15.27:0/808059446 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f1ecc03e570 0x7f1ecc040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:47.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.096+0000 7f1ee27fc700 1 --2- 172.21.15.27:0/808059446 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f1ecc03e570 0x7f1ecc040a30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8006fd0 tx=0x7f1ed8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:47.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.101+0000 7f1ecbfff700 1 -- 172.21.15.27:0/808059446 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1ed4013070 con 0x7f1ee41031b0 2024-05-21T03:13:47.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.322+0000 7f1ee9547700 1 -- 172.21.15.27:0/808059446 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1ee4076d70 con 0x7f1ecc03e570 2024-05-21T03:13:47.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.323+0000 7f1ecbfff700 1 -- 172.21.15.27:0/808059446 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f1ee4076d70 con 0x7f1ecc03e570 2024-05-21T03:13:47.323 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:13:47.323 INFO:teuthology.orchestra.run.smithi027.stdout:[{"addr": "172.21.15.27", "hostname": "smithi027", "labels": [], "status": ""}, {"addr": "172.21.15.39", "hostname": "smithi039", "labels": [], "status": ""}, {"addr": "172.21.15.165", "hostname": "smithi165", "labels": [], "status": ""}] 2024-05-21T03:13:47.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.326+0000 7f1ee9547700 1 -- 172.21.15.27:0/808059446 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f1ecc03e570 msgr2=0x7f1ecc040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:47.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.326+0000 7f1ee9547700 1 --2- 172.21.15.27:0/808059446 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f1ecc03e570 0x7f1ecc040a30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8006fd0 tx=0x7f1ed8006e40 comp rx=0 tx=0).stop 2024-05-21T03:13:47.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.326+0000 7f1ee9547700 1 -- 172.21.15.27:0/808059446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ee41031b0 msgr2=0x7f1ee4075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:47.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.326+0000 7f1ee9547700 1 --2- 172.21.15.27:0/808059446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ee41031b0 0x7f1ee4075840 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1ed40056d0 tx=0x7f1ed40057b0 comp rx=0 tx=0).stop 2024-05-21T03:13:47.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.326+0000 7f1ee9547700 1 -- 172.21.15.27:0/808059446 shutdown_connections 2024-05-21T03:13:47.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.326+0000 7f1ee9547700 1 --2- 172.21.15.27:0/808059446 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f1ecc03e570 0x7f1ecc040a30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:47.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.326+0000 7f1ee9547700 1 --2- 172.21.15.27:0/808059446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ee41031b0 0x7f1ee4075840 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:47.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.326+0000 7f1ee9547700 1 -- 172.21.15.27:0/808059446 >> 172.21.15.27:0/808059446 conn(0x7f1ee40fea80 msgr2=0x7f1ee4104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:47.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.326+0000 7f1ee9547700 1 -- 172.21.15.27:0/808059446 shutdown_connections 2024-05-21T03:13:47.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:47.326+0000 7f1ee9547700 1 -- 172.21.15.27:0/808059446 wait complete. 2024-05-21T03:13:47.932 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-21T03:13:47.933 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd crush tunables default 2024-05-21T03:13:48.175 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:13:48.607 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:48 smithi027 ceph-mon[32179]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:13:48.607 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:48 smithi027 ceph-mon[32179]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:13:48.607 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:48 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:49.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:49.665+0000 7f50df1ba700 1 -- 172.21.15.27:0/1575867243 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50d81040f0 msgr2=0x7f50d81044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:49.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:49.665+0000 7f50df1ba700 1 --2- 172.21.15.27:0/1575867243 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50d81040f0 0x7f50d81044d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009910 tx=0x7f50c8009c20 comp rx=0 tx=0).stop 2024-05-21T03:13:49.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:49.665+0000 7f50df1ba700 1 -- 172.21.15.27:0/1575867243 shutdown_connections 2024-05-21T03:13:49.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:49.665+0000 7f50df1ba700 1 --2- 172.21.15.27:0/1575867243 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50d81040f0 0x7f50d81044d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:49.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:49.665+0000 7f50df1ba700 1 -- 172.21.15.27:0/1575867243 >> 172.21.15.27:0/1575867243 conn(0x7f50d80fd940 msgr2=0x7f50d80ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:49.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:49.665+0000 7f50df1ba700 1 -- 172.21.15.27:0/1575867243 shutdown_connections 2024-05-21T03:13:49.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:49.665+0000 7f50df1ba700 1 -- 172.21.15.27:0/1575867243 wait complete. 2024-05-21T03:13:49.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:49.666+0000 7f50df1ba700 1 Processor -- start 2024-05-21T03:13:49.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:49.666+0000 7f50df1ba700 1 -- start start 2024-05-21T03:13:49.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:49.666+0000 7f50df1ba700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50d81040f0 0x7f50d819e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:49.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:49.666+0000 7f50df1ba700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f50d810d490 con 0x7f50d81040f0 2024-05-21T03:13:49.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:49.666+0000 7f50dcf56700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50d81040f0 0x7f50d819e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:49.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:49.666+0000 7f50dcf56700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50d81040f0 0x7f50d819e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56450/0 (socket says 172.21.15.27:56450) 2024-05-21T03:13:49.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:49.666+0000 7f50dcf56700 1 -- 172.21.15.27:0/3327598686 learned_addr learned my addr 172.21.15.27:0/3327598686 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:13:49.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:49.666+0000 7f50dcf56700 1 -- 172.21.15.27:0/3327598686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d819ec00 con 0x7f50d81040f0 2024-05-21T03:13:49.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:49.666+0000 7f50dcf56700 1 --2- 172.21.15.27:0/3327598686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50d81040f0 0x7f50d819e6c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f50c80056d0 tx=0x7f50c80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:49.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:49.666+0000 7f50d5ffb700 1 -- 172.21.15.27:0/3327598686 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50c8026070 con 0x7f50d81040f0 2024-05-21T03:13:49.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:49.667+0000 7f50d5ffb700 1 -- 172.21.15.27:0/3327598686 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50c8004370 con 0x7f50d81040f0 2024-05-21T03:13:49.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:49.667+0000 7f50df1ba700 1 -- 172.21.15.27:0/3327598686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50d819ee90 con 0x7f50d81040f0 2024-05-21T03:13:49.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:49.667+0000 7f50d5ffb700 1 -- 172.21.15.27:0/3327598686 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50c80039b0 con 0x7f50d81040f0 2024-05-21T03:13:49.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:49.667+0000 7f50df1ba700 1 -- 172.21.15.27:0/3327598686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50d8198840 con 0x7f50d81040f0 2024-05-21T03:13:49.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:49.668+0000 7f50df1ba700 1 -- 172.21.15.27:0/3327598686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50d806bad0 con 0x7f50d81040f0 2024-05-21T03:13:49.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:49.668+0000 7f50d5ffb700 1 -- 172.21.15.27:0/3327598686 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f50c8029070 con 0x7f50d81040f0 2024-05-21T03:13:49.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:49.668+0000 7f50d5ffb700 1 --2- 172.21.15.27:0/3327598686 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f50b803e570 0x7f50b8040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:49.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:49.668+0000 7f50d5ffb700 1 -- 172.21.15.27:0/3327598686 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f50c801e070 con 0x7f50d81040f0 2024-05-21T03:13:49.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:49.668+0000 7f50d7fff700 1 --2- 172.21.15.27:0/3327598686 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f50b803e570 0x7f50b8040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:49.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:49.668+0000 7f50d7fff700 1 --2- 172.21.15.27:0/3327598686 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f50b803e570 0x7f50b8040a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f50c000ad30 tx=0x7f50c00093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:49.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:49.673+0000 7f50d5ffb700 1 -- 172.21.15.27:0/3327598686 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f50c8013070 con 0x7f50d81040f0 2024-05-21T03:13:49.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:49.890+0000 7f50df1ba700 1 -- 172.21.15.27:0/3327598686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f50d8064ba0 con 0x7f50d81040f0 2024-05-21T03:13:50.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:50.351+0000 7f50d5ffb700 1 -- 172.21.15.27:0/3327598686 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f50c8030090 con 0x7f50d81040f0 2024-05-21T03:13:50.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:50.355+0000 7f50df1ba700 1 -- 172.21.15.27:0/3327598686 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f50b803e570 msgr2=0x7f50b8040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:50.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:50.355+0000 7f50df1ba700 1 --2- 172.21.15.27:0/3327598686 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f50b803e570 0x7f50b8040a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f50c000ad30 tx=0x7f50c00093f0 comp rx=0 tx=0).stop 2024-05-21T03:13:50.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:50.355+0000 7f50df1ba700 1 -- 172.21.15.27:0/3327598686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50d81040f0 msgr2=0x7f50d819e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:50.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:50.355+0000 7f50df1ba700 1 --2- 172.21.15.27:0/3327598686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50d81040f0 0x7f50d819e6c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f50c80056d0 tx=0x7f50c80057b0 comp rx=0 tx=0).stop 2024-05-21T03:13:50.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:50.355+0000 7f50df1ba700 1 -- 172.21.15.27:0/3327598686 shutdown_connections 2024-05-21T03:13:50.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:50.355+0000 7f50df1ba700 1 --2- 172.21.15.27:0/3327598686 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f50b803e570 0x7f50b8040a30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:50.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:50.355+0000 7f50df1ba700 1 --2- 172.21.15.27:0/3327598686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50d81040f0 0x7f50d819e6c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:50.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:50.355+0000 7f50df1ba700 1 -- 172.21.15.27:0/3327598686 >> 172.21.15.27:0/3327598686 conn(0x7f50d80fd940 msgr2=0x7f50d8106eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:50.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:50.355+0000 7f50df1ba700 1 -- 172.21.15.27:0/3327598686 shutdown_connections 2024-05-21T03:13:50.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:13:50.356+0000 7f50df1ba700 1 -- 172.21.15.27:0/3327598686 wait complete. 2024-05-21T03:13:50.356 INFO:teuthology.orchestra.run.smithi027.stderr:adjusted tunables profile to default 2024-05-21T03:13:50.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:50 smithi027 ceph-mon[32179]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:13:50.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:50 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3327598686' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-21T03:13:50.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:50 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:50.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:50 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:50.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:50 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:50.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:50 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:50.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:50 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi165", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:13:50.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:50 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:13:50.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:50 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:13:51.054 INFO:tasks.cephadm:Adding mon.a on smithi027 2024-05-21T03:13:51.054 INFO:tasks.cephadm:Adding mon.b on smithi039 2024-05-21T03:13:51.055 INFO:tasks.cephadm:Adding mon.c on smithi165 2024-05-21T03:13:51.055 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph orch apply mon '3;smithi027:172.21.15.27=a;smithi039:172.21.15.39=b;smithi165:172.21.15.165=c' 2024-05-21T03:13:51.283 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T03:13:51.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:51 smithi027 ceph-mon[32179]: Updating smithi165:/etc/ceph/ceph.conf 2024-05-21T03:13:51.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:51 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3327598686' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-21T03:13:51.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:51 smithi027 ceph-mon[32179]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T03:13:51.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:51 smithi027 ceph-mon[32179]: Updating smithi165:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T03:13:51.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:51 smithi027 ceph-mon[32179]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-05-21T03:13:51.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:51 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:51.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:51 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:51.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:51 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:52.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:52 smithi027 ceph-mon[32179]: Updating smithi165:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T03:13:52.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:52 smithi027 ceph-mon[32179]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:13:52.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.709+0000 7f94e1cb3700 1 -- 172.21.15.165:0/2739694192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94dc1020f0 msgr2=0x7f94dc1024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:52.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.709+0000 7f94e1cb3700 1 --2- 172.21.15.165:0/2739694192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94dc1020f0 0x7f94dc1024d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f94cc00b0a0 tx=0x7f94cc00b3b0 comp rx=0 tx=0).stop 2024-05-21T03:13:52.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.709+0000 7f94e1cb3700 1 -- 172.21.15.165:0/2739694192 shutdown_connections 2024-05-21T03:13:52.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.709+0000 7f94e1cb3700 1 --2- 172.21.15.165:0/2739694192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94dc1020f0 0x7f94dc1024d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:52.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.709+0000 7f94e1cb3700 1 -- 172.21.15.165:0/2739694192 >> 172.21.15.165:0/2739694192 conn(0x7f94dc0779d0 msgr2=0x7f94dc077de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:52.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.709+0000 7f94e1cb3700 1 -- 172.21.15.165:0/2739694192 shutdown_connections 2024-05-21T03:13:52.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.710+0000 7f94e1cb3700 1 -- 172.21.15.165:0/2739694192 wait complete. 2024-05-21T03:13:52.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.710+0000 7f94e1cb3700 1 Processor -- start 2024-05-21T03:13:52.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.710+0000 7f94e1cb3700 1 -- start start 2024-05-21T03:13:52.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.711+0000 7f94e1cb3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94dc1020f0 0x7f94dc19d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:52.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.711+0000 7f94e1cb3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc10c350 con 0x7f94dc1020f0 2024-05-21T03:13:52.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.711+0000 7f94db7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94dc1020f0 0x7f94dc19d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:52.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.711+0000 7f94db7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94dc1020f0 0x7f94dc19d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.165:47994/0 (socket says 172.21.15.165:47994) 2024-05-21T03:13:52.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.711+0000 7f94db7fe700 1 -- 172.21.15.165:0/848696838 learned_addr learned my addr 172.21.15.165:0/848696838 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-05-21T03:13:52.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.711+0000 7f94db7fe700 1 -- 172.21.15.165:0/848696838 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94dc19db10 con 0x7f94dc1020f0 2024-05-21T03:13:52.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.711+0000 7f94db7fe700 1 --2- 172.21.15.165:0/848696838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94dc1020f0 0x7f94dc19d5d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f94cc006130 tx=0x7f94cc0097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:52.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.711+0000 7f94d8ff9700 1 -- 172.21.15.165:0/848696838 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94cc021020 con 0x7f94dc1020f0 2024-05-21T03:13:52.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.712+0000 7f94d8ff9700 1 -- 172.21.15.165:0/848696838 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94cc0079b0 con 0x7f94dc1020f0 2024-05-21T03:13:52.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.712+0000 7f94e1cb3700 1 -- 172.21.15.165:0/848696838 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94dc19dda0 con 0x7f94dc1020f0 2024-05-21T03:13:52.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.712+0000 7f94d8ff9700 1 -- 172.21.15.165:0/848696838 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94cc017040 con 0x7f94dc1020f0 2024-05-21T03:13:52.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.712+0000 7f94e1cb3700 1 -- 172.21.15.165:0/848696838 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94dc1976a0 con 0x7f94dc1020f0 2024-05-21T03:13:52.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.713+0000 7f94e1cb3700 1 -- 172.21.15.165:0/848696838 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94dc1035b0 con 0x7f94dc1020f0 2024-05-21T03:13:52.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.713+0000 7f94d8ff9700 1 -- 172.21.15.165:0/848696838 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f94cc024020 con 0x7f94dc1020f0 2024-05-21T03:13:52.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.713+0000 7f94d8ff9700 1 --2- 172.21.15.165:0/848696838 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f94c403e510 0x7f94c40409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:52.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.713+0000 7f94d8ff9700 1 -- 172.21.15.165:0/848696838 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f94cc00f050 con 0x7f94dc1020f0 2024-05-21T03:13:52.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.713+0000 7f94daffd700 1 --2- 172.21.15.165:0/848696838 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f94c403e510 0x7f94c40409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:52.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.714+0000 7f94daffd700 1 --2- 172.21.15.165:0/848696838 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f94c403e510 0x7f94c40409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f94d0006fd0 tx=0x7f94d0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:52.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.719+0000 7f94d8ff9700 1 -- 172.21.15.165:0/848696838 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f94cc057030 con 0x7f94dc1020f0 2024-05-21T03:13:52.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.938+0000 7f94e1cb3700 1 -- 172.21.15.165:0/848696838 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi027:172.21.15.27=a;smithi039:172.21.15.39=b;smithi165:172.21.15.165=c", "target": ["mon-mgr", ""]}) -- 0x7f94dc198bd0 con 0x7f94c403e510 2024-05-21T03:13:52.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.944+0000 7f94d8ff9700 1 -- 172.21.15.165:0/848696838 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f94dc198bd0 con 0x7f94c403e510 2024-05-21T03:13:52.944 INFO:teuthology.orchestra.run.smithi165.stdout:Scheduled mon update... 2024-05-21T03:13:52.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.947+0000 7f94e1cb3700 1 -- 172.21.15.165:0/848696838 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f94c403e510 msgr2=0x7f94c40409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:52.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.947+0000 7f94e1cb3700 1 --2- 172.21.15.165:0/848696838 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f94c403e510 0x7f94c40409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f94d0006fd0 tx=0x7f94d0006e40 comp rx=0 tx=0).stop 2024-05-21T03:13:52.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.947+0000 7f94e1cb3700 1 -- 172.21.15.165:0/848696838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94dc1020f0 msgr2=0x7f94dc19d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:52.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.947+0000 7f94e1cb3700 1 --2- 172.21.15.165:0/848696838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94dc1020f0 0x7f94dc19d5d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f94cc006130 tx=0x7f94cc0097d0 comp rx=0 tx=0).stop 2024-05-21T03:13:52.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.947+0000 7f94e1cb3700 1 -- 172.21.15.165:0/848696838 shutdown_connections 2024-05-21T03:13:52.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.947+0000 7f94e1cb3700 1 --2- 172.21.15.165:0/848696838 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f94c403e510 0x7f94c40409d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:52.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.947+0000 7f94e1cb3700 1 --2- 172.21.15.165:0/848696838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94dc1020f0 0x7f94dc19d5d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:52.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.947+0000 7f94e1cb3700 1 -- 172.21.15.165:0/848696838 >> 172.21.15.165:0/848696838 conn(0x7f94dc0779d0 msgr2=0x7f94dc103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:52.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.947+0000 7f94e1cb3700 1 -- 172.21.15.165:0/848696838 shutdown_connections 2024-05-21T03:13:52.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:52.947+0000 7f94e1cb3700 1 -- 172.21.15.165:0/848696838 wait complete. 2024-05-21T03:13:53.569 DEBUG:teuthology.orchestra.run.smithi039:mon.b> sudo journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mon.b.service 2024-05-21T03:13:53.572 DEBUG:teuthology.orchestra.run.smithi165:mon.c> sudo journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mon.c.service 2024-05-21T03:13:53.574 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-21T03:13:53.574 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-21T03:13:54.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:53 smithi027 ceph-mon[32179]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi027:172.21.15.27=a;smithi039:172.21.15.39=b;smithi165:172.21.15.165=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:13:54.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:53 smithi027 ceph-mon[32179]: Saving service mon spec with placement smithi027:172.21.15.27=a;smithi039:172.21.15.39=b;smithi165:172.21.15.165=c;count:3 2024-05-21T03:13:54.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:53 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:54.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:53 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:13:54.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:53 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:13:54.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:53 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:13:54.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:53 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:13:54.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:53 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T03:13:54.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:53 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:13:54.428 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:53 smithi027 ceph-mon[32179]: Deploying daemon mon.c on smithi165 2024-05-21T03:13:54.428 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:53 smithi027 ceph-mon[32179]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:13:55.609 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T03:13:56.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:13:56 smithi027 ceph-mon[32179]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:13:57.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.454+0000 7fcc9a3a6700 1 -- 172.21.15.165:0/1695848011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc941021b0 msgr2=0x7fcc94102590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:57.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.454+0000 7fcc9a3a6700 1 --2- 172.21.15.165:0/1695848011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc941021b0 0x7fcc94102590 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcc84009910 tx=0x7fcc84009c20 comp rx=0 tx=0).stop 2024-05-21T03:13:57.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.454+0000 7fcc9a3a6700 1 -- 172.21.15.165:0/1695848011 shutdown_connections 2024-05-21T03:13:57.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.454+0000 7fcc9a3a6700 1 --2- 172.21.15.165:0/1695848011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc941021b0 0x7fcc94102590 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:57.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.454+0000 7fcc9a3a6700 1 -- 172.21.15.165:0/1695848011 >> 172.21.15.165:0/1695848011 conn(0x7fcc940fda80 msgr2=0x7fcc940ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:57.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.454+0000 7fcc9a3a6700 1 -- 172.21.15.165:0/1695848011 shutdown_connections 2024-05-21T03:13:57.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.454+0000 7fcc9a3a6700 1 -- 172.21.15.165:0/1695848011 wait complete. 2024-05-21T03:13:57.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.454+0000 7fcc9a3a6700 1 Processor -- start 2024-05-21T03:13:57.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.454+0000 7fcc9a3a6700 1 -- start start 2024-05-21T03:13:57.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.455+0000 7fcc9a3a6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc941021b0 0x7fcc9419e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:57.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.455+0000 7fcc9a3a6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9410c430 con 0x7fcc941021b0 2024-05-21T03:13:57.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.455+0000 7fcc93fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc941021b0 0x7fcc9419e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:57.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.455+0000 7fcc93fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc941021b0 0x7fcc9419e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.165:60090/0 (socket says 172.21.15.165:60090) 2024-05-21T03:13:57.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.455+0000 7fcc93fff700 1 -- 172.21.15.165:0/4273183074 learned_addr learned my addr 172.21.15.165:0/4273183074 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-05-21T03:13:57.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.455+0000 7fcc93fff700 1 -- 172.21.15.165:0/4273183074 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc9419ec50 con 0x7fcc941021b0 2024-05-21T03:13:57.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.456+0000 7fcc93fff700 1 --2- 172.21.15.165:0/4273183074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc941021b0 0x7fcc9419e710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcc840056d0 tx=0x7fcc840057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:57.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.456+0000 7fcc917fa700 1 -- 172.21.15.165:0/4273183074 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc84026070 con 0x7fcc941021b0 2024-05-21T03:13:57.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.456+0000 7fcc917fa700 1 -- 172.21.15.165:0/4273183074 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc8400bba0 con 0x7fcc941021b0 2024-05-21T03:13:57.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.456+0000 7fcc9a3a6700 1 -- 172.21.15.165:0/4273183074 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc9419ef40 con 0x7fcc941021b0 2024-05-21T03:13:57.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.456+0000 7fcc9a3a6700 1 -- 172.21.15.165:0/4273183074 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc941988d0 con 0x7fcc941021b0 2024-05-21T03:13:57.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.456+0000 7fcc917fa700 1 -- 172.21.15.165:0/4273183074 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc84003920 con 0x7fcc941021b0 2024-05-21T03:13:57.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.457+0000 7fcc9a3a6700 1 -- 172.21.15.165:0/4273183074 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc94103690 con 0x7fcc941021b0 2024-05-21T03:13:57.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.457+0000 7fcc917fa700 1 -- 172.21.15.165:0/4273183074 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fcc84029070 con 0x7fcc941021b0 2024-05-21T03:13:57.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.457+0000 7fcc917fa700 1 --2- 172.21.15.165:0/4273183074 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fcc7c03e510 0x7fcc7c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:13:57.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.457+0000 7fcc917fa700 1 -- 172.21.15.165:0/4273183074 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcc8401e070 con 0x7fcc941021b0 2024-05-21T03:13:57.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.457+0000 7fcc937fe700 1 --2- 172.21.15.165:0/4273183074 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fcc7c03e510 0x7fcc7c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:13:57.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.458+0000 7fcc937fe700 1 --2- 172.21.15.165:0/4273183074 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fcc7c03e510 0x7fcc7c0409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcc88006fd0 tx=0x7fcc88006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:13:57.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.463+0000 7fcc917fa700 1 -- 172.21.15.165:0/4273183074 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcc84013070 con 0x7fcc941021b0 2024-05-21T03:13:57.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.760+0000 7fcc9a3a6700 1 -- 172.21.15.165:0/4273183074 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fcc94199e00 con 0x7fcc941021b0 2024-05-21T03:13:57.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.761+0000 7fcc917fa700 1 -- 172.21.15.165:0/4273183074 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fcc84030090 con 0x7fcc941021b0 2024-05-21T03:13:57.761 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-21T03:13:57.761 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":1,"fsid":"c9ca94be-171f-11ef-bc9a-c7b262605968","modified":"2024-05-21T03:12:14.443931Z","created":"2024-05-21T03:12:14.443931Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-21T03:13:57.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.763+0000 7fcc9a3a6700 1 -- 172.21.15.165:0/4273183074 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fcc7c03e510 msgr2=0x7fcc7c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:57.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.763+0000 7fcc9a3a6700 1 --2- 172.21.15.165:0/4273183074 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fcc7c03e510 0x7fcc7c0409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcc88006fd0 tx=0x7fcc88006e40 comp rx=0 tx=0).stop 2024-05-21T03:13:57.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.763+0000 7fcc9a3a6700 1 -- 172.21.15.165:0/4273183074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc941021b0 msgr2=0x7fcc9419e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:13:57.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.763+0000 7fcc9a3a6700 1 --2- 172.21.15.165:0/4273183074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc941021b0 0x7fcc9419e710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcc840056d0 tx=0x7fcc840057b0 comp rx=0 tx=0).stop 2024-05-21T03:13:57.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.763+0000 7fcc9a3a6700 1 -- 172.21.15.165:0/4273183074 shutdown_connections 2024-05-21T03:13:57.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.764+0000 7fcc9a3a6700 1 --2- 172.21.15.165:0/4273183074 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fcc7c03e510 0x7fcc7c0409d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:57.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.764+0000 7fcc9a3a6700 1 --2- 172.21.15.165:0/4273183074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc941021b0 0x7fcc9419e710 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:13:57.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.764+0000 7fcc9a3a6700 1 -- 172.21.15.165:0/4273183074 >> 172.21.15.165:0/4273183074 conn(0x7fcc940fda80 msgr2=0x7fcc9406b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:13:57.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.764+0000 7fcc9a3a6700 1 -- 172.21.15.165:0/4273183074 shutdown_connections 2024-05-21T03:13:57.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:13:57.764+0000 7fcc9a3a6700 1 -- 172.21.15.165:0/4273183074 wait complete. 2024-05-21T03:13:57.764 INFO:teuthology.orchestra.run.smithi165.stderr:dumped monmap epoch 1 2024-05-21T03:13:58.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:13:57 smithi165 systemd[1]: Started Ceph mon.c for c9ca94be-171f-11ef-bc9a-c7b262605968. 2024-05-21T03:13:59.401 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-21T03:13:59.401 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-21T03:13:59.636 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T03:14:03.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:02 smithi027 ceph-mon[32179]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:03.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:02 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T03:14:03.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:02 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T03:14:03.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:02 smithi027 ceph-mon[32179]: mon.a calling monitor election 2024-05-21T03:14:03.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:02 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T03:14:03.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:02 smithi027 ceph-mon[32179]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:03.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:02 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T03:14:03.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:02 smithi027 ceph-mon[32179]: mon.c calling monitor election 2024-05-21T03:14:03.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:02 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T03:14:03.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:02 smithi027 ceph-mon[32179]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:03.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:02 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T03:14:03.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:02 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T03:14:03.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:02 smithi027 ceph-mon[32179]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-21T03:14:03.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:02 smithi027 ceph-mon[32179]: monmap epoch 2 2024-05-21T03:14:03.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:02 smithi027 ceph-mon[32179]: fsid c9ca94be-171f-11ef-bc9a-c7b262605968 2024-05-21T03:14:03.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:02 smithi027 ceph-mon[32179]: last_changed 2024-05-21T03:13:57.824930+0000 2024-05-21T03:14:03.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:02 smithi027 ceph-mon[32179]: created 2024-05-21T03:12:14.443931+0000 2024-05-21T03:14:03.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:02 smithi027 ceph-mon[32179]: min_mon_release 19 (squid) 2024-05-21T03:14:03.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:02 smithi027 ceph-mon[32179]: election_strategy: 1 2024-05-21T03:14:03.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:02 smithi027 ceph-mon[32179]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-05-21T03:14:03.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:02 smithi027 ceph-mon[32179]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-05-21T03:14:03.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:02 smithi027 ceph-mon[32179]: fsmap 2024-05-21T03:14:03.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:02 smithi027 ceph-mon[32179]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T03:14:03.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:02 smithi027 ceph-mon[32179]: mgrmap e13: x(active, since 49s) 2024-05-21T03:14:03.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:02 smithi027 ceph-mon[32179]: overall HEALTH_OK 2024-05-21T03:14:03.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:02 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:03.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:02 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:03.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:02 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:03.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:02 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T03:14:03.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:02 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:03.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:02 smithi165 ceph-mon[36269]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:03.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:02 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T03:14:03.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:02 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T03:14:03.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:02 smithi165 ceph-mon[36269]: mon.a calling monitor election 2024-05-21T03:14:03.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:02 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T03:14:03.273 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:02 smithi165 ceph-mon[36269]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:03.273 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:02 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T03:14:03.273 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:02 smithi165 ceph-mon[36269]: mon.c calling monitor election 2024-05-21T03:14:03.273 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:02 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T03:14:03.273 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:02 smithi165 ceph-mon[36269]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:03.273 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:02 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T03:14:03.274 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:02 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T03:14:03.274 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:02 smithi165 ceph-mon[36269]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-21T03:14:03.274 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:02 smithi165 ceph-mon[36269]: monmap epoch 2 2024-05-21T03:14:03.274 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:02 smithi165 ceph-mon[36269]: fsid c9ca94be-171f-11ef-bc9a-c7b262605968 2024-05-21T03:14:03.274 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:02 smithi165 ceph-mon[36269]: last_changed 2024-05-21T03:13:57.824930+0000 2024-05-21T03:14:03.275 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:02 smithi165 ceph-mon[36269]: created 2024-05-21T03:12:14.443931+0000 2024-05-21T03:14:03.275 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:02 smithi165 ceph-mon[36269]: min_mon_release 19 (squid) 2024-05-21T03:14:03.275 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:02 smithi165 ceph-mon[36269]: election_strategy: 1 2024-05-21T03:14:03.275 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:02 smithi165 ceph-mon[36269]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-05-21T03:14:03.275 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:02 smithi165 ceph-mon[36269]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-05-21T03:14:03.276 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:02 smithi165 ceph-mon[36269]: fsmap 2024-05-21T03:14:03.276 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:02 smithi165 ceph-mon[36269]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T03:14:03.276 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:02 smithi165 ceph-mon[36269]: mgrmap e13: x(active, since 49s) 2024-05-21T03:14:03.276 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:02 smithi165 ceph-mon[36269]: overall HEALTH_OK 2024-05-21T03:14:03.276 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:02 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:03.276 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:02 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:03.276 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:02 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:03.276 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:02 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T03:14:03.276 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:02 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:04.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:03 smithi165 ceph-mon[36269]: Deploying daemon mon.b on smithi039 2024-05-21T03:14:04.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:03 smithi165 ceph-mon[36269]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:04.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:03 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T03:14:04.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.093+0000 7f52f0f9b700 1 -- 172.21.15.165:0/3354651591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52d40035c0 msgr2=0x7f52d4005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:14:04.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.093+0000 7f52f0f9b700 1 --2- 172.21.15.165:0/3354651591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52d40035c0 0x7f52d4005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f52ec079670 tx=0x7f52e000feb0 comp rx=0 tx=0).stop 2024-05-21T03:14:04.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.093+0000 7f52f0f9b700 1 -- 172.21.15.165:0/3354651591 shutdown_connections 2024-05-21T03:14:04.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.093+0000 7f52f0f9b700 1 --2- 172.21.15.165:0/3354651591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52d40035c0 0x7f52d4005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:04.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.093+0000 7f52f0f9b700 1 -- 172.21.15.165:0/3354651591 >> 172.21.15.165:0/3354651591 conn(0x7f52ec078360 msgr2=0x7f52ec078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:14:04.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.093+0000 7f52f0f9b700 1 -- 172.21.15.165:0/3354651591 shutdown_connections 2024-05-21T03:14:04.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.093+0000 7f52f0f9b700 1 -- 172.21.15.165:0/3354651591 wait complete. 2024-05-21T03:14:04.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.095+0000 7f52f0f9b700 1 Processor -- start 2024-05-21T03:14:04.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.095+0000 7f52f0f9b700 1 -- start start 2024-05-21T03:14:04.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.095+0000 7f52f0f9b700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52d40035c0 0x7f52ec1a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:14:04.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.095+0000 7f52f0f9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52ec1a0fa0 0x7f52ec19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:14:04.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.095+0000 7f52f0f9b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f52ec10afd0 con 0x7f52ec1a0fa0 2024-05-21T03:14:04.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.095+0000 7f52f0f9b700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f52ec10ae50 con 0x7f52d40035c0 2024-05-21T03:14:04.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.095+0000 7f52ea59c700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52d40035c0 0x7f52ec1a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:14:04.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.095+0000 7f52ea59c700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52d40035c0 0x7f52ec1a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44446/0 (socket says 172.21.15.165:44446) 2024-05-21T03:14:04.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.095+0000 7f52e9d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52ec1a0fa0 0x7f52ec19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:14:04.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.095+0000 7f52ea59c700 1 -- 172.21.15.165:0/530889185 learned_addr learned my addr 172.21.15.165:0/530889185 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-05-21T03:14:04.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.095+0000 7f52ea59c700 1 -- 172.21.15.165:0/530889185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52d40035c0 msgr2=0x7f52ec1a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-21T03:14:04.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.095+0000 7f52ea59c700 1 -- 172.21.15.165:0/530889185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52d40035c0 msgr2=0x7f52ec1a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-21T03:14:04.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.095+0000 7f52ea59c700 1 --2- 172.21.15.165:0/530889185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52d40035c0 0x7f52ec1a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-21T03:14:04.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.096+0000 7f52ea59c700 1 --2- 172.21.15.165:0/530889185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52d40035c0 0x7f52ec1a0a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T03:14:04.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.096+0000 7f52e9d9b700 1 -- 172.21.15.165:0/530889185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52d40035c0 msgr2=0x7f52ec1a0a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T03:14:04.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.096+0000 7f52e9d9b700 1 --2- 172.21.15.165:0/530889185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52d40035c0 0x7f52ec1a0a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:04.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.096+0000 7f52e9d9b700 1 -- 172.21.15.165:0/530889185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52ec1a1520 con 0x7f52ec1a0fa0 2024-05-21T03:14:04.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.096+0000 7f52e9d9b700 1 --2- 172.21.15.165:0/530889185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52ec1a0fa0 0x7f52ec19ab30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00b4d0 tx=0x7f52dc00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:14:04.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.096+0000 7f52db7fe700 1 -- 172.21.15.165:0/530889185 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f52dc011630 con 0x7f52ec1a0fa0 2024-05-21T03:14:04.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.096+0000 7f52db7fe700 1 -- 172.21.15.165:0/530889185 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52dc011c80 con 0x7f52ec1a0fa0 2024-05-21T03:14:04.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.096+0000 7f52f0f9b700 1 -- 172.21.15.165:0/530889185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52ec19b0d0 con 0x7f52ec1a0fa0 2024-05-21T03:14:04.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.096+0000 7f52db7fe700 1 -- 172.21.15.165:0/530889185 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f52dc010da0 con 0x7f52ec1a0fa0 2024-05-21T03:14:04.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.096+0000 7f52f0f9b700 1 -- 172.21.15.165:0/530889185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52ec19b660 con 0x7f52ec1a0fa0 2024-05-21T03:14:04.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.097+0000 7f52db7fe700 1 -- 172.21.15.165:0/530889185 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f52dc018020 con 0x7f52ec1a0fa0 2024-05-21T03:14:04.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.098+0000 7f52f0f9b700 1 -- 172.21.15.165:0/530889185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52ec06b4a0 con 0x7f52ec1a0fa0 2024-05-21T03:14:04.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.098+0000 7f52db7fe700 1 --2- 172.21.15.165:0/530889185 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f52cc03e2b0 0x7f52cc040770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:14:04.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.098+0000 7f52db7fe700 1 -- 172.21.15.165:0/530889185 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f52dc0521d0 con 0x7f52ec1a0fa0 2024-05-21T03:14:04.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.098+0000 7f52ea59c700 1 --2- 172.21.15.165:0/530889185 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f52cc03e2b0 0x7f52cc040770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:14:04.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.098+0000 7f52ea59c700 1 --2- 172.21.15.165:0/530889185 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f52cc03e2b0 0x7f52cc040770 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f52e000fa60 tx=0x7f52e0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:14:04.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.103+0000 7f52db7fe700 1 -- 172.21.15.165:0/530889185 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f52dc01f020 con 0x7f52ec1a0fa0 2024-05-21T03:14:04.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:03 smithi027 ceph-mon[32179]: Deploying daemon mon.b on smithi039 2024-05-21T03:14:04.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:03 smithi027 ceph-mon[32179]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:04.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:03 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T03:14:04.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.398+0000 7f52f0f9b700 1 -- 172.21.15.165:0/530889185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f52ec19b9d0 con 0x7f52ec1a0fa0 2024-05-21T03:14:04.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.399+0000 7f52db7fe700 1 -- 172.21.15.165:0/530889185 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f52dc0187b0 con 0x7f52ec1a0fa0 2024-05-21T03:14:04.399 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-21T03:14:04.399 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":2,"fsid":"c9ca94be-171f-11ef-bc9a-c7b262605968","modified":"2024-05-21T03:13:57.824930Z","created":"2024-05-21T03:12:14.443931Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:3300","nonce":0},{"type":"v1","addr":"172.21.15.165:6789","nonce":0}]},"addr":"172.21.15.165:6789/0","public_addr":"172.21.15.165:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-21T03:14:04.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.401+0000 7f52f0f9b700 1 -- 172.21.15.165:0/530889185 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f52cc03e2b0 msgr2=0x7f52cc040770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:14:04.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.401+0000 7f52f0f9b700 1 --2- 172.21.15.165:0/530889185 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f52cc03e2b0 0x7f52cc040770 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f52e000fa60 tx=0x7f52e0005960 comp rx=0 tx=0).stop 2024-05-21T03:14:04.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.401+0000 7f52f0f9b700 1 -- 172.21.15.165:0/530889185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52ec1a0fa0 msgr2=0x7f52ec19ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:14:04.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.401+0000 7f52f0f9b700 1 --2- 172.21.15.165:0/530889185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52ec1a0fa0 0x7f52ec19ab30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00b4d0 tx=0x7f52dc00b890 comp rx=0 tx=0).stop 2024-05-21T03:14:04.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.402+0000 7f52f0f9b700 1 -- 172.21.15.165:0/530889185 shutdown_connections 2024-05-21T03:14:04.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.402+0000 7f52f0f9b700 1 --2- 172.21.15.165:0/530889185 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f52cc03e2b0 0x7f52cc040770 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:04.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.402+0000 7f52f0f9b700 1 --2- 172.21.15.165:0/530889185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52d40035c0 0x7f52ec1a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:04.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.402+0000 7f52f0f9b700 1 --2- 172.21.15.165:0/530889185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52ec1a0fa0 0x7f52ec19ab30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:04.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.402+0000 7f52f0f9b700 1 -- 172.21.15.165:0/530889185 >> 172.21.15.165:0/530889185 conn(0x7f52ec078360 msgr2=0x7f52ec102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:14:04.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.402+0000 7f52f0f9b700 1 -- 172.21.15.165:0/530889185 shutdown_connections 2024-05-21T03:14:04.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:04.402+0000 7f52f0f9b700 1 -- 172.21.15.165:0/530889185 wait complete. 2024-05-21T03:14:04.402 INFO:teuthology.orchestra.run.smithi165.stderr:dumped monmap epoch 2 2024-05-21T03:14:04.968 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:04 smithi165 ceph-mon[36269]: from='client.? 172.21.15.165:0/530889185' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-21T03:14:05.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:04 smithi027 ceph-mon[32179]: from='client.? 172.21.15.165:0/530889185' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-21T03:14:05.970 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-21T03:14:05.971 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-21T03:14:06.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:05 smithi027 ceph-mon[32179]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:06.217 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T03:14:06.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:05 smithi165 ceph-mon[36269]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:06.947 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:06 smithi039 ceph-mon[36429]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-21T03:14:06.948 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:06 smithi039 bash[36410]: e7b68b78f66c430aabdf5943ce1597171fce4f5e4b63fe0889a7f791bb9d68c8 2024-05-21T03:14:07.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:06 smithi039 systemd[1]: Started Ceph mon.b for c9ca94be-171f-11ef-bc9a-c7b262605968. 2024-05-21T03:14:12.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T03:14:12.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: mon.a calling monitor election 2024-05-21T03:14:12.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: mon.c calling monitor election 2024-05-21T03:14:12.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:14:12.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T03:14:12.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:12.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:14:12.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:14:12.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: mon.b calling monitor election 2024-05-21T03:14:12.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:12.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:14:12.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:14:12.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:12.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:14:12.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-21T03:14:12.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: monmap epoch 3 2024-05-21T03:14:12.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: fsid c9ca94be-171f-11ef-bc9a-c7b262605968 2024-05-21T03:14:12.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: last_changed 2024-05-21T03:14:06.814495+0000 2024-05-21T03:14:12.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: created 2024-05-21T03:12:14.443931+0000 2024-05-21T03:14:12.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: min_mon_release 19 (squid) 2024-05-21T03:14:12.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: election_strategy: 1 2024-05-21T03:14:12.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-05-21T03:14:12.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-05-21T03:14:12.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: 2: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.b 2024-05-21T03:14:12.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: fsmap 2024-05-21T03:14:12.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T03:14:12.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: mgrmap e13: x(active, since 58s) 2024-05-21T03:14:12.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: overall HEALTH_OK 2024-05-21T03:14:12.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:12.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:12.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:12.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:11 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:12.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.188+0000 7f3ef7b1f700 1 -- 172.21.15.165:0/2272780806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3edc0057d0 msgr2=0x7f3edc007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:14:12.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.188+0000 7f3ef7b1f700 1 --2- 172.21.15.165:0/2272780806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3edc0057d0 0x7f3edc007c60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0101e80 tx=0x7f3eec00fe60 comp rx=0 tx=0).stop 2024-05-21T03:14:12.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.188+0000 7f3ef7b1f700 1 -- 172.21.15.165:0/2272780806 shutdown_connections 2024-05-21T03:14:12.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.188+0000 7f3ef7b1f700 1 --2- 172.21.15.165:0/2272780806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3edc0057d0 0x7f3edc007c60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:12.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.188+0000 7f3ef7b1f700 1 -- 172.21.15.165:0/2272780806 >> 172.21.15.165:0/2272780806 conn(0x7f3ef00fda80 msgr2=0x7f3ef00ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:14:12.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.188+0000 7f3ef7b1f700 1 -- 172.21.15.165:0/2272780806 shutdown_connections 2024-05-21T03:14:12.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.188+0000 7f3ef7b1f700 1 -- 172.21.15.165:0/2272780806 wait complete. 2024-05-21T03:14:12.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.189+0000 7f3ef7b1f700 1 Processor -- start 2024-05-21T03:14:12.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.189+0000 7f3ef7b1f700 1 -- start start 2024-05-21T03:14:12.189 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T03:14:12.189 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: mon.a calling monitor election 2024-05-21T03:14:12.189 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: mon.c calling monitor election 2024-05-21T03:14:12.189 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:14:12.190 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T03:14:12.190 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:12.190 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:14:12.190 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:14:12.190 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: mon.b calling monitor election 2024-05-21T03:14:12.190 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:12.190 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:14:12.190 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:14:12.190 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:12.190 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:14:12.190 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-21T03:14:12.190 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: monmap epoch 3 2024-05-21T03:14:12.190 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: fsid c9ca94be-171f-11ef-bc9a-c7b262605968 2024-05-21T03:14:12.190 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: last_changed 2024-05-21T03:14:06.814495+0000 2024-05-21T03:14:12.191 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: created 2024-05-21T03:12:14.443931+0000 2024-05-21T03:14:12.191 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: min_mon_release 19 (squid) 2024-05-21T03:14:12.191 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: election_strategy: 1 2024-05-21T03:14:12.191 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-05-21T03:14:12.191 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-05-21T03:14:12.191 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: 2: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.b 2024-05-21T03:14:12.191 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: fsmap 2024-05-21T03:14:12.191 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T03:14:12.191 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: mgrmap e13: x(active, since 58s) 2024-05-21T03:14:12.191 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: overall HEALTH_OK 2024-05-21T03:14:12.191 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:12.191 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:12.191 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:12.191 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:11 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:12.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.189+0000 7f3ef7b1f700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3edc0057d0 0x7f3ef019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:14:12.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.189+0000 7f3ef7b1f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ef019b3b0 0x7f3ef019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:14:12.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.189+0000 7f3ef7b1f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ef019fc80 0x7f3ef01a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:14:12.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.189+0000 7f3ef7b1f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef0108e30 con 0x7f3ef019fc80 2024-05-21T03:14:12.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.189+0000 7f3ef7b1f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef006c5e0 con 0x7f3ef019b3b0 2024-05-21T03:14:12.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.189+0000 7f3ef7b1f700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef0108fb0 con 0x7f3edc0057d0 2024-05-21T03:14:12.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.189+0000 7f3ef58bb700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3edc0057d0 0x7f3ef019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:14:12.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.189+0000 7f3ef58bb700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3edc0057d0 0x7f3ef019ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56860/0 (socket says 172.21.15.165:56860) 2024-05-21T03:14:12.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.189+0000 7f3ef58bb700 1 -- 172.21.15.165:0/3618739017 learned_addr learned my addr 172.21.15.165:0/3618739017 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-05-21T03:14:12.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.190+0000 7f3ef58bb700 1 -- 172.21.15.165:0/3618739017 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3edc0057d0 msgr2=0x7f3ef019ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-21T03:14:12.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.190+0000 7f3ef58bb700 1 -- 172.21.15.165:0/3618739017 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3edc0057d0 msgr2=0x7f3ef019ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-21T03:14:12.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.190+0000 7f3ef58bb700 1 --2- 172.21.15.165:0/3618739017 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3edc0057d0 0x7f3ef019ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-21T03:14:12.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.190+0000 7f3ef58bb700 1 --2- 172.21.15.165:0/3618739017 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3edc0057d0 0x7f3ef019ae70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T03:14:12.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.190+0000 7f3ef50ba700 1 --2- 172.21.15.165:0/3618739017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ef019b3b0 0x7f3ef019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:14:12.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.190+0000 7f3ef60bc700 1 --2- 172.21.15.165:0/3618739017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ef019fc80 0x7f3ef01a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:14:12.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.190+0000 7f3ef50ba700 1 -- 172.21.15.165:0/3618739017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ef019b3b0 msgr2=0x7f3ef019f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-21T03:14:12.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.190+0000 7f3ef50ba700 1 -- 172.21.15.165:0/3618739017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ef019b3b0 msgr2=0x7f3ef019f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-21T03:14:12.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.190+0000 7f3ef50ba700 1 --2- 172.21.15.165:0/3618739017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ef019b3b0 0x7f3ef019f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-21T03:14:12.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.190+0000 7f3ef50ba700 1 --2- 172.21.15.165:0/3618739017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ef019b3b0 0x7f3ef019f740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T03:14:12.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.190+0000 7f3ef60bc700 1 -- 172.21.15.165:0/3618739017 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3edc0057d0 msgr2=0x7f3ef019ae70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T03:14:12.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.190+0000 7f3ef60bc700 1 --2- 172.21.15.165:0/3618739017 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3edc0057d0 0x7f3ef019ae70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:12.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.190+0000 7f3ef60bc700 1 -- 172.21.15.165:0/3618739017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ef019b3b0 msgr2=0x7f3ef019f740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T03:14:12.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.190+0000 7f3ef60bc700 1 --2- 172.21.15.165:0/3618739017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ef019b3b0 0x7f3ef019f740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:12.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.190+0000 7f3ef60bc700 1 -- 172.21.15.165:0/3618739017 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef01a79f0 con 0x7f3ef019fc80 2024-05-21T03:14:12.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.190+0000 7f3ef60bc700 1 --2- 172.21.15.165:0/3618739017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ef019fc80 0x7f3ef01a0130 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3ee800bd50 tx=0x7f3ee800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:14:12.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.191+0000 7f3ee6ffd700 1 -- 172.21.15.165:0/3618739017 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8019070 con 0x7f3ef019fc80 2024-05-21T03:14:12.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.191+0000 7f3ee6ffd700 1 -- 172.21.15.165:0/3618739017 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ee800cd10 con 0x7f3ef019fc80 2024-05-21T03:14:12.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.191+0000 7f3ef7b1f700 1 -- 172.21.15.165:0/3618739017 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef01a7c20 con 0x7f3ef019fc80 2024-05-21T03:14:12.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.191+0000 7f3ee6ffd700 1 -- 172.21.15.165:0/3618739017 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8012700 con 0x7f3ef019fc80 2024-05-21T03:14:12.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.191+0000 7f3ef7b1f700 1 -- 172.21.15.165:0/3618739017 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef01a8160 con 0x7f3ef019fc80 2024-05-21T03:14:12.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.192+0000 7f3ee6ffd700 1 -- 172.21.15.165:0/3618739017 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f3ee801c070 con 0x7f3ef019fc80 2024-05-21T03:14:12.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.192+0000 7f3ef7b1f700 1 -- 172.21.15.165:0/3618739017 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ef0064ba0 con 0x7f3ef019fc80 2024-05-21T03:14:12.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.192+0000 7f3ee6ffd700 1 --2- 172.21.15.165:0/3618739017 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f3ed403e9a0 0x7f3ed4040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:14:12.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.192+0000 7f3ee6ffd700 1 -- 172.21.15.165:0/3618739017 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3ee80524a0 con 0x7f3ef019fc80 2024-05-21T03:14:12.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.192+0000 7f3ef58bb700 1 --2- 172.21.15.165:0/3618739017 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f3ed403e9a0 0x7f3ed4040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:14:12.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.193+0000 7f3ef58bb700 1 --2- 172.21.15.165:0/3618739017 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f3ed403e9a0 0x7f3ed4040e60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3eec000c00 tx=0x7f3eec014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:14:12.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.198+0000 7f3ee6ffd700 1 -- 172.21.15.165:0/3618739017 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3ee80133a0 con 0x7f3ef019fc80 2024-05-21T03:14:12.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T03:14:12.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: mon.a calling monitor election 2024-05-21T03:14:12.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: mon.c calling monitor election 2024-05-21T03:14:12.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:14:12.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T03:14:12.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:12.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:14:12.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:14:12.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: mon.b calling monitor election 2024-05-21T03:14:12.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:12.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:14:12.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:14:12.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:12.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:14:12.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-21T03:14:12.271 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: monmap epoch 3 2024-05-21T03:14:12.271 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: fsid c9ca94be-171f-11ef-bc9a-c7b262605968 2024-05-21T03:14:12.271 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: last_changed 2024-05-21T03:14:06.814495+0000 2024-05-21T03:14:12.271 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: created 2024-05-21T03:12:14.443931+0000 2024-05-21T03:14:12.271 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: min_mon_release 19 (squid) 2024-05-21T03:14:12.271 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: election_strategy: 1 2024-05-21T03:14:12.271 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-05-21T03:14:12.271 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-05-21T03:14:12.271 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: 2: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.b 2024-05-21T03:14:12.271 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: fsmap 2024-05-21T03:14:12.271 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T03:14:12.271 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: mgrmap e13: x(active, since 58s) 2024-05-21T03:14:12.271 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: overall HEALTH_OK 2024-05-21T03:14:12.271 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:12.272 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:12.272 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:12.272 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:11 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:12.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.497+0000 7f3ef7b1f700 1 -- 172.21.15.165:0/3618739017 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f3ef002e560 con 0x7f3ef019fc80 2024-05-21T03:14:12.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.498+0000 7f3ee6ffd700 1 -- 172.21.15.165:0/3618739017 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7f3ee8014a10 con 0x7f3ef019fc80 2024-05-21T03:14:12.498 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-05-21T03:14:12.498 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":3,"fsid":"c9ca94be-171f-11ef-bc9a-c7b262605968","modified":"2024-05-21T03:14:06.814495Z","created":"2024-05-21T03:12:14.443931Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:3300","nonce":0},{"type":"v1","addr":"172.21.15.165:6789","nonce":0}]},"addr":"172.21.15.165:6789/0","public_addr":"172.21.15.165:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:3300","nonce":0},{"type":"v1","addr":"172.21.15.39:6789","nonce":0}]},"addr":"172.21.15.39:6789/0","public_addr":"172.21.15.39:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-21T03:14:12.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.500+0000 7f3ef7b1f700 1 -- 172.21.15.165:0/3618739017 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f3ed403e9a0 msgr2=0x7f3ed4040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:14:12.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.500+0000 7f3ef7b1f700 1 --2- 172.21.15.165:0/3618739017 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f3ed403e9a0 0x7f3ed4040e60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3eec000c00 tx=0x7f3eec014040 comp rx=0 tx=0).stop 2024-05-21T03:14:12.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.500+0000 7f3ef7b1f700 1 -- 172.21.15.165:0/3618739017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ef019fc80 msgr2=0x7f3ef01a0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:14:12.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.500+0000 7f3ef7b1f700 1 --2- 172.21.15.165:0/3618739017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ef019fc80 0x7f3ef01a0130 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3ee800bd50 tx=0x7f3ee800be30 comp rx=0 tx=0).stop 2024-05-21T03:14:12.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.500+0000 7f3ef7b1f700 1 -- 172.21.15.165:0/3618739017 shutdown_connections 2024-05-21T03:14:12.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.500+0000 7f3ef7b1f700 1 --2- 172.21.15.165:0/3618739017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ef019fc80 0x7f3ef01a0130 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:12.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.500+0000 7f3ef7b1f700 1 --2- 172.21.15.165:0/3618739017 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f3ed403e9a0 0x7f3ed4040e60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:12.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.500+0000 7f3ef7b1f700 1 --2- 172.21.15.165:0/3618739017 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3edc0057d0 0x7f3ef019ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:12.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.500+0000 7f3ef7b1f700 1 --2- 172.21.15.165:0/3618739017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ef019b3b0 0x7f3ef019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:12.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.500+0000 7f3ef7b1f700 1 -- 172.21.15.165:0/3618739017 >> 172.21.15.165:0/3618739017 conn(0x7f3ef00fda80 msgr2=0x7f3ef00fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:14:12.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.500+0000 7f3ef7b1f700 1 -- 172.21.15.165:0/3618739017 shutdown_connections 2024-05-21T03:14:12.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:12.500+0000 7f3ef7b1f700 1 -- 172.21.15.165:0/3618739017 wait complete. 2024-05-21T03:14:12.501 INFO:teuthology.orchestra.run.smithi165.stderr:dumped monmap epoch 3 2024-05-21T03:14:13.079 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:12 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:14:13.079 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:12 smithi165 ceph-mon[36269]: from='client.? 172.21.15.165:0/3618739017' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-21T03:14:13.079 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:12 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:14:13.108 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-21T03:14:13.108 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph config generate-minimal-conf 2024-05-21T03:14:13.140 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:12 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:14:13.140 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:12 smithi027 ceph-mon[32179]: from='client.? 172.21.15.165:0/3618739017' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-21T03:14:13.141 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:12 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:14:13.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:12 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:14:13.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:12 smithi039 ceph-mon[36429]: from='client.? 172.21.15.165:0/3618739017' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-21T03:14:13.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:12 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:14:13.367 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:14:14.638 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:14 smithi027 ceph-mon[32179]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:14.639 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:14 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:14.639 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:14 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:14.639 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:14 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:14.639 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:14 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:14.639 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:14 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:14.639 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:14 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:14.639 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:14 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:14:14.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:14 smithi039 ceph-mon[36429]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:14.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:14 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:14.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:14 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:14.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:14 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:14.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:14 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:14.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:14 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:14.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:14 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:14.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:14 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:14:14.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:14 smithi165 ceph-mon[36269]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:14.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:14 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:14.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:14 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:14.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:14 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:14.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:14 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:14.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:14 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:14.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:14 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:14.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:14 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:14:15.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.007+0000 7fbeb12fd700 1 -- 172.21.15.27:0/3315210497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeac1031b0 msgr2=0x7fbeac103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:14:15.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.007+0000 7fbeb12fd700 1 --2- 172.21.15.27:0/3315210497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeac1031b0 0x7fbeac103590 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c009910 tx=0x7fbe9c009c20 comp rx=0 tx=0).stop 2024-05-21T03:14:15.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.007+0000 7fbeb12fd700 1 -- 172.21.15.27:0/3315210497 shutdown_connections 2024-05-21T03:14:15.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.007+0000 7fbeb12fd700 1 --2- 172.21.15.27:0/3315210497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeac1031b0 0x7fbeac103590 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:15.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.007+0000 7fbeb12fd700 1 -- 172.21.15.27:0/3315210497 >> 172.21.15.27:0/3315210497 conn(0x7fbeac0fea80 msgr2=0x7fbeac100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:14:15.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.007+0000 7fbeb12fd700 1 -- 172.21.15.27:0/3315210497 shutdown_connections 2024-05-21T03:14:15.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.007+0000 7fbeb12fd700 1 -- 172.21.15.27:0/3315210497 wait complete. 2024-05-21T03:14:15.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.008+0000 7fbeb12fd700 1 Processor -- start 2024-05-21T03:14:15.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.008+0000 7fbeb12fd700 1 -- start start 2024-05-21T03:14:15.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.008+0000 7fbeb12fd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeac1031b0 0x7fbeac19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:14:15.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.008+0000 7fbeb12fd700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbeac19ee10 0x7fbeac198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:14:15.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.008+0000 7fbeb12fd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeac199020 0x7fbeac1994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:14:15.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.008+0000 7fbeb12fd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac10d430 con 0x7fbeac1031b0 2024-05-21T03:14:15.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.008+0000 7fbeb12fd700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac10d2b0 con 0x7fbeac199020 2024-05-21T03:14:15.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.008+0000 7fbeb12fd700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac10d5b0 con 0x7fbeac19ee10 2024-05-21T03:14:15.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.008+0000 7fbeaaffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeac1031b0 0x7fbeac19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:14:15.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.008+0000 7fbeaaffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeac1031b0 0x7fbeac19e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44892/0 (socket says 172.21.15.27:44892) 2024-05-21T03:14:15.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.008+0000 7fbeaaffd700 1 -- 172.21.15.27:0/3996424209 learned_addr learned my addr 172.21.15.27:0/3996424209 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:14:15.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.009+0000 7fbeaa7fc700 1 --2- 172.21.15.27:0/3996424209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbeac19ee10 0x7fbeac198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:14:15.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.009+0000 7fbeab7fe700 1 --2- 172.21.15.27:0/3996424209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeac199020 0x7fbeac1994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:14:15.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.009+0000 7fbeaaffd700 1 -- 172.21.15.27:0/3996424209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbeac19ee10 msgr2=0x7fbeac198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:14:15.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.009+0000 7fbeaaffd700 1 --2- 172.21.15.27:0/3996424209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbeac19ee10 0x7fbeac198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:15.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.009+0000 7fbeaaffd700 1 -- 172.21.15.27:0/3996424209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeac199020 msgr2=0x7fbeac1994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:14:15.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.009+0000 7fbeaaffd700 1 --2- 172.21.15.27:0/3996424209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeac199020 0x7fbeac1994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:15.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.009+0000 7fbeaaffd700 1 -- 172.21.15.27:0/3996424209 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeac1a57a0 con 0x7fbeac1031b0 2024-05-21T03:14:15.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.009+0000 7fbeab7fe700 1 --2- 172.21.15.27:0/3996424209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeac199020 0x7fbeac1994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:14:15.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.009+0000 7fbeaaffd700 1 --2- 172.21.15.27:0/3996424209 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeac1031b0 0x7fbeac19e8d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c004f80 tx=0x7fbe9c004fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:14:15.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.009+0000 7fbe93fff700 1 -- 172.21.15.27:0/3996424209 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c026070 con 0x7fbeac1031b0 2024-05-21T03:14:15.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.009+0000 7fbeb12fd700 1 -- 172.21.15.27:0/3996424209 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeac1a59d0 con 0x7fbeac1031b0 2024-05-21T03:14:15.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.009+0000 7fbe93fff700 1 -- 172.21.15.27:0/3996424209 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe9c00b6f0 con 0x7fbeac1031b0 2024-05-21T03:14:15.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.009+0000 7fbe93fff700 1 -- 172.21.15.27:0/3996424209 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c0038e0 con 0x7fbeac1031b0 2024-05-21T03:14:15.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.009+0000 7fbeb12fd700 1 -- 172.21.15.27:0/3996424209 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeac1a5f10 con 0x7fbeac1031b0 2024-05-21T03:14:15.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.010+0000 7fbe93fff700 1 -- 172.21.15.27:0/3996424209 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fbe9c029070 con 0x7fbeac1031b0 2024-05-21T03:14:15.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.010+0000 7fbe93fff700 1 --2- 172.21.15.27:0/3996424209 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fbe9403e9a0 0x7fbe94040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:14:15.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.011+0000 7fbe93fff700 1 -- 172.21.15.27:0/3996424209 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbe9c053330 con 0x7fbeac1031b0 2024-05-21T03:14:15.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.011+0000 7fbeaa7fc700 1 --2- 172.21.15.27:0/3996424209 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fbe9403e9a0 0x7fbe94040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:14:15.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.011+0000 7fbeb12fd700 1 -- 172.21.15.27:0/3996424209 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeac064ba0 con 0x7fbeac1031b0 2024-05-21T03:14:15.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.011+0000 7fbeaa7fc700 1 --2- 172.21.15.27:0/3996424209 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fbe9403e9a0 0x7fbe94040e60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbea0009ad0 tx=0x7fbea0009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:14:15.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.016+0000 7fbe93fff700 1 -- 172.21.15.27:0/3996424209 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbe9c013070 con 0x7fbeac1031b0 2024-05-21T03:14:15.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.234+0000 7fbeb12fd700 1 -- 172.21.15.27:0/3996424209 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fbeac1a6140 con 0x7fbeac1031b0 2024-05-21T03:14:15.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.234+0000 7fbe93fff700 1 -- 172.21.15.27:0/3996424209 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7fbe9c00edf0 con 0x7fbeac1031b0 2024-05-21T03:14:15.235 INFO:teuthology.orchestra.run.smithi027.stdout:# minimal ceph.conf for c9ca94be-171f-11ef-bc9a-c7b262605968 2024-05-21T03:14:15.235 INFO:teuthology.orchestra.run.smithi027.stdout:[global] 2024-05-21T03:14:15.235 INFO:teuthology.orchestra.run.smithi027.stdout: fsid = c9ca94be-171f-11ef-bc9a-c7b262605968 2024-05-21T03:14:15.235 INFO:teuthology.orchestra.run.smithi027.stdout: mon_host = [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] 2024-05-21T03:14:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.236+0000 7fbeb12fd700 1 -- 172.21.15.27:0/3996424209 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fbe9403e9a0 msgr2=0x7fbe94040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:14:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.236+0000 7fbeb12fd700 1 --2- 172.21.15.27:0/3996424209 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fbe9403e9a0 0x7fbe94040e60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbea0009ad0 tx=0x7fbea0009210 comp rx=0 tx=0).stop 2024-05-21T03:14:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.236+0000 7fbeb12fd700 1 -- 172.21.15.27:0/3996424209 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeac1031b0 msgr2=0x7fbeac19e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:14:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.236+0000 7fbeb12fd700 1 --2- 172.21.15.27:0/3996424209 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeac1031b0 0x7fbeac19e8d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c004f80 tx=0x7fbe9c004fb0 comp rx=0 tx=0).stop 2024-05-21T03:14:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.236+0000 7fbeb12fd700 1 -- 172.21.15.27:0/3996424209 shutdown_connections 2024-05-21T03:14:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.236+0000 7fbeb12fd700 1 --2- 172.21.15.27:0/3996424209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeac199020 0x7fbeac1994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.236+0000 7fbeb12fd700 1 --2- 172.21.15.27:0/3996424209 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeac1031b0 0x7fbeac19e8d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.236+0000 7fbeb12fd700 1 --2- 172.21.15.27:0/3996424209 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fbe9403e9a0 0x7fbe94040e60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.236+0000 7fbeb12fd700 1 --2- 172.21.15.27:0/3996424209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbeac19ee10 0x7fbeac198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.236+0000 7fbeb12fd700 1 -- 172.21.15.27:0/3996424209 >> 172.21.15.27:0/3996424209 conn(0x7fbeac0fea80 msgr2=0x7fbeac104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:14:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.237+0000 7fbeb12fd700 1 -- 172.21.15.27:0/3996424209 shutdown_connections 2024-05-21T03:14:15.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:15.237+0000 7fbeb12fd700 1 -- 172.21.15.27:0/3996424209 wait complete. 2024-05-21T03:14:15.537 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:15 smithi027 ceph-mon[32179]: Updating smithi027:/etc/ceph/ceph.conf 2024-05-21T03:14:15.537 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:15 smithi027 ceph-mon[32179]: Updating smithi039:/etc/ceph/ceph.conf 2024-05-21T03:14:15.537 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:15 smithi027 ceph-mon[32179]: Updating smithi165:/etc/ceph/ceph.conf 2024-05-21T03:14:15.537 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:15 smithi027 ceph-mon[32179]: Updating smithi039:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T03:14:15.537 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:15 smithi027 ceph-mon[32179]: Updating smithi027:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T03:14:15.537 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:15 smithi027 ceph-mon[32179]: Updating smithi165:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T03:14:15.538 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:15 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.538 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:15 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.538 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:15 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.538 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:15 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.538 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:15 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.538 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:15 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.538 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:15 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.538 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:15 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.538 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:15 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.538 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:15 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.539 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:15 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.539 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:15 smithi027 ceph-mon[32179]: Reconfiguring mon.a (unknown last config time)... 2024-05-21T03:14:15.539 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:15 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T03:14:15.539 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:15 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T03:14:15.539 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:15 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:15.539 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:15 smithi027 ceph-mon[32179]: Reconfiguring daemon mon.a on smithi027 2024-05-21T03:14:15.539 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:15 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3996424209' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:15.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:15 smithi039 ceph-mon[36429]: Updating smithi027:/etc/ceph/ceph.conf 2024-05-21T03:14:15.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:15 smithi039 ceph-mon[36429]: Updating smithi039:/etc/ceph/ceph.conf 2024-05-21T03:14:15.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:15 smithi039 ceph-mon[36429]: Updating smithi165:/etc/ceph/ceph.conf 2024-05-21T03:14:15.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:15 smithi039 ceph-mon[36429]: Updating smithi039:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T03:14:15.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:15 smithi039 ceph-mon[36429]: Updating smithi027:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T03:14:15.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:15 smithi039 ceph-mon[36429]: Updating smithi165:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T03:14:15.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:15 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:15 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:15 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:15 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:15 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:15 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:15 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:15 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:15 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:15 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:15 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:15 smithi039 ceph-mon[36429]: Reconfiguring mon.a (unknown last config time)... 2024-05-21T03:14:15.771 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:15 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T03:14:15.771 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:15 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T03:14:15.771 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:15 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:15.771 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:15 smithi039 ceph-mon[36429]: Reconfiguring daemon mon.a on smithi027 2024-05-21T03:14:15.771 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:15 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/3996424209' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:15.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:15 smithi165 ceph-mon[36269]: Updating smithi027:/etc/ceph/ceph.conf 2024-05-21T03:14:15.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:15 smithi165 ceph-mon[36269]: Updating smithi039:/etc/ceph/ceph.conf 2024-05-21T03:14:15.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:15 smithi165 ceph-mon[36269]: Updating smithi165:/etc/ceph/ceph.conf 2024-05-21T03:14:15.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:15 smithi165 ceph-mon[36269]: Updating smithi039:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T03:14:15.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:15 smithi165 ceph-mon[36269]: Updating smithi027:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T03:14:15.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:15 smithi165 ceph-mon[36269]: Updating smithi165:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T03:14:15.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:15 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:15 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:15 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:15 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:15 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:15 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:15 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:15 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:15 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:15 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:15 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:15.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:15 smithi165 ceph-mon[36269]: Reconfiguring mon.a (unknown last config time)... 2024-05-21T03:14:15.773 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:15 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T03:14:15.773 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:15 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T03:14:15.773 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:15 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:15.773 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:15 smithi165 ceph-mon[36269]: Reconfiguring daemon mon.a on smithi027 2024-05-21T03:14:15.773 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:15 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/3996424209' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:15.932 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-21T03:14:15.932 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-05-21T03:14:15.932 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.conf 2024-05-21T03:14:16.000 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-05-21T03:14:16.000 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-21T03:14:16.067 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-21T03:14:16.068 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/ceph/ceph.conf 2024-05-21T03:14:16.099 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-21T03:14:16.100 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-21T03:14:16.166 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-05-21T03:14:16.166 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/ceph/ceph.conf 2024-05-21T03:14:16.198 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-05-21T03:14:16.198 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-21T03:14:16.266 INFO:tasks.cephadm:Adding mgr.x on smithi027 2024-05-21T03:14:16.266 INFO:tasks.cephadm:Adding mgr.y on smithi039 2024-05-21T03:14:16.266 INFO:tasks.cephadm:Adding mgr.z on smithi165 2024-05-21T03:14:16.267 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph orch apply mgr '3;smithi027=x;smithi039=y;smithi165=z' 2024-05-21T03:14:16.544 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T03:14:16.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:16 smithi027 ceph-mon[32179]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:16.960 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:16 smithi165 ceph-mon[36269]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:17.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:16 smithi039 ceph-mon[36429]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:17.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.946+0000 7f220845e700 1 -- 172.21.15.165:0/3256588411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22001030e0 msgr2=0x7f2200103500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:14:17.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.946+0000 7f220845e700 1 --2- 172.21.15.165:0/3256588411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22001030e0 0x7f2200103500 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f21f0009910 tx=0x7f21f0009c20 comp rx=0 tx=0).stop 2024-05-21T03:14:17.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.946+0000 7f220845e700 1 -- 172.21.15.165:0/3256588411 shutdown_connections 2024-05-21T03:14:17.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.946+0000 7f220845e700 1 --2- 172.21.15.165:0/3256588411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22001030e0 0x7f2200103500 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:17.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.946+0000 7f220845e700 1 -- 172.21.15.165:0/3256588411 >> 172.21.15.165:0/3256588411 conn(0x7f22000fe680 msgr2=0x7f2200100ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:14:17.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.946+0000 7f220845e700 1 -- 172.21.15.165:0/3256588411 shutdown_connections 2024-05-21T03:14:17.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.946+0000 7f220845e700 1 -- 172.21.15.165:0/3256588411 wait complete. 2024-05-21T03:14:17.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.946+0000 7f220845e700 1 Processor -- start 2024-05-21T03:14:17.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.947+0000 7f220845e700 1 -- start start 2024-05-21T03:14:17.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.947+0000 7f220845e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22001030e0 0x7f22001987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:14:17.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.947+0000 7f220845e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2200198d20 0x7f220019dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:14:17.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.947+0000 7f220845e700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22001991a0 0x7f22001a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:14:17.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.947+0000 7f220845e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2200109020 con 0x7f22001030e0 2024-05-21T03:14:17.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.947+0000 7f220845e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2200108ea0 con 0x7f2200198d20 2024-05-21T03:14:17.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.947+0000 7f220845e700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f22001091a0 con 0x7f22001991a0 2024-05-21T03:14:17.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.947+0000 7f22069fb700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22001991a0 0x7f22001a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:14:17.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.947+0000 7f22061fa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22001030e0 0x7f22001987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:14:17.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.947+0000 7f22069fb700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22001991a0 0x7f22001a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59304/0 (socket says 172.21.15.165:59304) 2024-05-21T03:14:17.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.947+0000 7f22061fa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22001030e0 0x7f22001987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.165:49670/0 (socket says 172.21.15.165:49670) 2024-05-21T03:14:17.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.947+0000 7f22069fb700 1 -- 172.21.15.165:0/560555596 learned_addr learned my addr 172.21.15.165:0/560555596 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-05-21T03:14:17.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.947+0000 7f22059f9700 1 --2- 172.21.15.165:0/560555596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2200198d20 0x7f220019dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:14:17.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.947+0000 7f22069fb700 1 -- 172.21.15.165:0/560555596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2200198d20 msgr2=0x7f220019dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:14:17.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.947+0000 7f22069fb700 1 --2- 172.21.15.165:0/560555596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2200198d20 0x7f220019dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:17.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.947+0000 7f22069fb700 1 -- 172.21.15.165:0/560555596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22001030e0 msgr2=0x7f22001987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:14:17.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.947+0000 7f22069fb700 1 --2- 172.21.15.165:0/560555596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22001030e0 0x7f22001987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:17.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.947+0000 7f22069fb700 1 -- 172.21.15.165:0/560555596 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22001a0bf0 con 0x7f22001991a0 2024-05-21T03:14:17.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.947+0000 7f22061fa700 1 --2- 172.21.15.165:0/560555596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22001030e0 0x7f22001987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:14:17.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.947+0000 7f22069fb700 1 --2- 172.21.15.165:0/560555596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22001991a0 0x7f22001a0320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f21f800ba50 tx=0x7f21f800bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:14:17.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.948+0000 7f21f77fe700 1 -- 172.21.15.165:0/560555596 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f8019070 con 0x7f22001991a0 2024-05-21T03:14:17.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.948+0000 7f220845e700 1 -- 172.21.15.165:0/560555596 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22001a0ee0 con 0x7f22001991a0 2024-05-21T03:14:17.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.948+0000 7f21f77fe700 1 -- 172.21.15.165:0/560555596 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21f8014440 con 0x7f22001991a0 2024-05-21T03:14:17.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.948+0000 7f21f77fe700 1 -- 172.21.15.165:0/560555596 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f800ce60 con 0x7f22001991a0 2024-05-21T03:14:17.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.948+0000 7f220845e700 1 -- 172.21.15.165:0/560555596 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22001a1410 con 0x7f22001991a0 2024-05-21T03:14:17.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.949+0000 7f220845e700 1 -- 172.21.15.165:0/560555596 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2200064ba0 con 0x7f22001991a0 2024-05-21T03:14:17.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.949+0000 7f21f77fe700 1 -- 172.21.15.165:0/560555596 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f21f801c070 con 0x7f22001991a0 2024-05-21T03:14:17.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.949+0000 7f21f77fe700 1 --2- 172.21.15.165:0/560555596 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f21ec03e9a0 0x7f21ec040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:14:17.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.949+0000 7f21f77fe700 1 -- 172.21.15.165:0/560555596 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f21f8051d90 con 0x7f22001991a0 2024-05-21T03:14:17.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.949+0000 7f22061fa700 1 --2- 172.21.15.165:0/560555596 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f21ec03e9a0 0x7f21ec040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:14:17.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.950+0000 7f22061fa700 1 --2- 172.21.15.165:0/560555596 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f21ec03e9a0 0x7f21ec040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f21f0005190 tx=0x7f21f001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:14:17.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:17.954+0000 7f21f77fe700 1 -- 172.21.15.165:0/560555596 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f21f8017120 con 0x7f22001991a0 2024-05-21T03:14:18.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:17 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:18.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:17 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:18.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:17 smithi039 ceph-mon[36429]: Reconfiguring mon.b (monmap changed)... 2024-05-21T03:14:18.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:17 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T03:14:18.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:17 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T03:14:18.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:17 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:18.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:17 smithi039 ceph-mon[36429]: Reconfiguring daemon mon.b on smithi039 2024-05-21T03:14:18.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:17 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:18.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:17 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:18.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:17 smithi165 ceph-mon[36269]: Reconfiguring mon.b (monmap changed)... 2024-05-21T03:14:18.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:17 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T03:14:18.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:17 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T03:14:18.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:17 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:18.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:17 smithi165 ceph-mon[36269]: Reconfiguring daemon mon.b on smithi039 2024-05-21T03:14:18.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:18.168+0000 7f220845e700 1 -- 172.21.15.165:0/560555596 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi027=x;smithi039=y;smithi165=z", "target": ["mon-mgr", ""]}) -- 0x7f2200063a10 con 0x7f21ec03e9a0 2024-05-21T03:14:18.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:18.175+0000 7f21f77fe700 1 -- 172.21.15.165:0/560555596 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2200063a10 con 0x7f21ec03e9a0 2024-05-21T03:14:18.175 INFO:teuthology.orchestra.run.smithi165.stdout:Scheduled mgr update... 2024-05-21T03:14:18.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:17 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:18.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:17 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:18.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:17 smithi027 ceph-mon[32179]: Reconfiguring mon.b (monmap changed)... 2024-05-21T03:14:18.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:17 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T03:14:18.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:17 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T03:14:18.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:17 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:18.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:17 smithi027 ceph-mon[32179]: Reconfiguring daemon mon.b on smithi039 2024-05-21T03:14:18.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:18.177+0000 7f220845e700 1 -- 172.21.15.165:0/560555596 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f21ec03e9a0 msgr2=0x7f21ec040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:14:18.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:18.177+0000 7f220845e700 1 --2- 172.21.15.165:0/560555596 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f21ec03e9a0 0x7f21ec040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f21f0005190 tx=0x7f21f001a040 comp rx=0 tx=0).stop 2024-05-21T03:14:18.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:18.177+0000 7f220845e700 1 -- 172.21.15.165:0/560555596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22001991a0 msgr2=0x7f22001a0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:14:18.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:18.177+0000 7f220845e700 1 --2- 172.21.15.165:0/560555596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22001991a0 0x7f22001a0320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f21f800ba50 tx=0x7f21f800bd60 comp rx=0 tx=0).stop 2024-05-21T03:14:18.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:18.177+0000 7f220845e700 1 -- 172.21.15.165:0/560555596 shutdown_connections 2024-05-21T03:14:18.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:18.177+0000 7f220845e700 1 --2- 172.21.15.165:0/560555596 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f21ec03e9a0 0x7f21ec040e60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:18.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:18.177+0000 7f220845e700 1 --2- 172.21.15.165:0/560555596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22001991a0 0x7f22001a0320 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:18.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:18.177+0000 7f220845e700 1 --2- 172.21.15.165:0/560555596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22001030e0 0x7f22001987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:18.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:18.177+0000 7f220845e700 1 --2- 172.21.15.165:0/560555596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2200198d20 0x7f220019dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:18.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:18.177+0000 7f220845e700 1 -- 172.21.15.165:0/560555596 >> 172.21.15.165:0/560555596 conn(0x7f22000fe680 msgr2=0x7f22000ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:14:18.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:18.178+0000 7f220845e700 1 -- 172.21.15.165:0/560555596 shutdown_connections 2024-05-21T03:14:18.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:14:18.178+0000 7f220845e700 1 -- 172.21.15.165:0/560555596 wait complete. 2024-05-21T03:14:18.757 DEBUG:teuthology.orchestra.run.smithi039:mgr.y> sudo journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mgr.y.service 2024-05-21T03:14:18.759 DEBUG:teuthology.orchestra.run.smithi165:mgr.z> sudo journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mgr.z.service 2024-05-21T03:14:18.761 INFO:tasks.cephadm:Deploying OSDs... 2024-05-21T03:14:18.762 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-05-21T03:14:18.762 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/scratch_devs of=/dev/stdout 2024-05-21T03:14:18.783 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-21T03:14:18.783 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_1 2024-05-21T03:14:18.843 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-21T03:14:18.843 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T03:14:18.843 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-21T03:14:18.843 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T03:14:18.843 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T03:14:18.843 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-05-21 03:13:23.506829550 +0000 2024-05-21T03:14:18.843 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-05-21 03:07:38.284083955 +0000 2024-05-21T03:14:18.843 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-05-21 03:07:38.284083955 +0000 2024-05-21T03:14:18.844 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-05-21 03:07:38.284083955 +0000 2024-05-21T03:14:18.844 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-21T03:14:18.911 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-05-21T03:14:18.912 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-05-21T03:14:18.912 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000201886 s, 2.5 MB/s 2024-05-21T03:14:18.913 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-21T03:14:18.971 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_2 2024-05-21T03:14:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:18 smithi165 ceph-mon[36269]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:18 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:18 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:18 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:18 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T03:14:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:18 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T03:14:19.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:18 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:19.029 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-21T03:14:19.029 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T03:14:19.029 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-21T03:14:19.029 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T03:14:19.029 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T03:14:19.029 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-05-21 03:13:23.549828437 +0000 2024-05-21T03:14:19.029 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-05-21 03:07:38.632074884 +0000 2024-05-21T03:14:19.029 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-05-21 03:07:38.632074884 +0000 2024-05-21T03:14:19.029 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-05-21 03:07:38.632074884 +0000 2024-05-21T03:14:19.030 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-21T03:14:19.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:18 smithi027 ceph-mon[32179]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:19.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:18 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:19.092 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:18 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:19.093 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:18 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:19.093 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:18 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T03:14:19.093 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:18 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T03:14:19.093 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:18 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:19.094 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-05-21T03:14:19.094 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-05-21T03:14:19.095 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000199286 s, 2.6 MB/s 2024-05-21T03:14:19.096 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-21T03:14:19.153 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_3 2024-05-21T03:14:19.209 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-21T03:14:19.210 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T03:14:19.210 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-05-21T03:14:19.210 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T03:14:19.210 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T03:14:19.210 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-05-21 03:13:23.590827376 +0000 2024-05-21T03:14:19.210 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-05-21 03:07:38.969066101 +0000 2024-05-21T03:14:19.211 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-05-21 03:07:38.969066101 +0000 2024-05-21T03:14:19.211 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-05-21 03:07:38.969066101 +0000 2024-05-21T03:14:19.211 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-21T03:14:19.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:18 smithi039 ceph-mon[36429]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:19.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:18 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:19.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:18 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:19.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:18 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:19.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:18 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T03:14:19.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:18 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T03:14:19.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:18 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:19.274 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-05-21T03:14:19.274 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-05-21T03:14:19.275 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000182793 s, 2.8 MB/s 2024-05-21T03:14:19.276 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-21T03:14:19.333 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_4 2024-05-21T03:14:19.389 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-21T03:14:19.390 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T03:14:19.390 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-21T03:14:19.390 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T03:14:19.390 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T03:14:19.390 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-05-21 03:13:23.632826288 +0000 2024-05-21T03:14:19.390 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-05-21 03:07:39.306057317 +0000 2024-05-21T03:14:19.390 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-05-21 03:07:39.306057317 +0000 2024-05-21T03:14:19.390 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-05-21 03:07:39.306057317 +0000 2024-05-21T03:14:19.391 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-21T03:14:19.454 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-05-21T03:14:19.454 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-05-21T03:14:19.455 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.00020187 s, 2.5 MB/s 2024-05-21T03:14:19.456 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-21T03:14:19.513 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-21T03:14:19.513 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/scratch_devs of=/dev/stdout 2024-05-21T03:14:19.532 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-21T03:14:19.532 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_1 2024-05-21T03:14:19.591 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-21T03:14:19.591 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T03:14:19.591 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-05-21T03:14:19.591 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T03:14:19.591 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T03:14:19.591 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-05-21 03:14:06.283688064 +0000 2024-05-21T03:14:19.591 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-05-21 03:08:15.401236481 +0000 2024-05-21T03:14:19.591 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-05-21 03:08:15.401236481 +0000 2024-05-21T03:14:19.591 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-05-21 03:08:15.401236481 +0000 2024-05-21T03:14:19.592 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-21T03:14:19.659 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-05-21T03:14:19.659 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-05-21T03:14:19.659 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.00030475 s, 1.7 MB/s 2024-05-21T03:14:19.660 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-21T03:14:19.719 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_2 2024-05-21T03:14:19.777 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-21T03:14:19.777 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T03:14:19.777 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-21T03:14:19.777 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T03:14:19.777 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T03:14:19.777 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-05-21 03:14:06.283688064 +0000 2024-05-21T03:14:19.777 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-05-21 03:08:15.766229638 +0000 2024-05-21T03:14:19.777 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-05-21 03:08:15.766229638 +0000 2024-05-21T03:14:19.777 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-05-21 03:08:15.766229638 +0000 2024-05-21T03:14:19.778 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-21T03:14:19.844 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-05-21T03:14:19.844 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-05-21T03:14:19.844 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000333403 s, 1.5 MB/s 2024-05-21T03:14:19.845 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-21T03:14:19.903 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_3 2024-05-21T03:14:19.960 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-21T03:14:19.960 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T03:14:19.960 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-21T03:14:19.960 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T03:14:19.960 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T03:14:19.960 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-05-21 03:14:06.284688046 +0000 2024-05-21T03:14:19.960 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-05-21 03:08:16.113223133 +0000 2024-05-21T03:14:19.961 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-05-21 03:08:16.113223133 +0000 2024-05-21T03:14:19.961 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-05-21 03:08:16.113223133 +0000 2024-05-21T03:14:19.961 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-21T03:14:20.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:19 smithi039 ceph-mon[36429]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi027=x;smithi039=y;smithi165=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:14:20.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:19 smithi039 ceph-mon[36429]: Saving service mgr spec with placement smithi027=x;smithi039=y;smithi165=z;count:3 2024-05-21T03:14:20.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:19 smithi039 ceph-mon[36429]: Reconfiguring mon.c (monmap changed)... 2024-05-21T03:14:20.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:19 smithi039 ceph-mon[36429]: Reconfiguring daemon mon.c on smithi165 2024-05-21T03:14:20.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:19 smithi165 ceph-mon[36269]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi027=x;smithi039=y;smithi165=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:14:20.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:19 smithi165 ceph-mon[36269]: Saving service mgr spec with placement smithi027=x;smithi039=y;smithi165=z;count:3 2024-05-21T03:14:20.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:19 smithi165 ceph-mon[36269]: Reconfiguring mon.c (monmap changed)... 2024-05-21T03:14:20.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:19 smithi165 ceph-mon[36269]: Reconfiguring daemon mon.c on smithi165 2024-05-21T03:14:20.025 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-05-21T03:14:20.025 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-05-21T03:14:20.026 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000284288 s, 1.8 MB/s 2024-05-21T03:14:20.027 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-21T03:14:20.084 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_4 2024-05-21T03:14:20.140 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-21T03:14:20.140 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T03:14:20.140 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-21T03:14:20.140 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T03:14:20.141 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T03:14:20.141 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-05-21 03:14:06.284688046 +0000 2024-05-21T03:14:20.141 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-05-21 03:08:16.468216478 +0000 2024-05-21T03:14:20.141 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-05-21 03:08:16.468216478 +0000 2024-05-21T03:14:20.141 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-05-21 03:08:16.468216478 +0000 2024-05-21T03:14:20.141 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-21T03:14:20.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:19 smithi027 ceph-mon[32179]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi027=x;smithi039=y;smithi165=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:14:20.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:19 smithi027 ceph-mon[32179]: Saving service mgr spec with placement smithi027=x;smithi039=y;smithi165=z;count:3 2024-05-21T03:14:20.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:19 smithi027 ceph-mon[32179]: Reconfiguring mon.c (monmap changed)... 2024-05-21T03:14:20.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:19 smithi027 ceph-mon[32179]: Reconfiguring daemon mon.c on smithi165 2024-05-21T03:14:20.205 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-05-21T03:14:20.205 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-05-21T03:14:20.205 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000357244 s, 1.4 MB/s 2024-05-21T03:14:20.207 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-21T03:14:20.264 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-05-21T03:14:20.264 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/scratch_devs of=/dev/stdout 2024-05-21T03:14:20.282 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-21T03:14:20.283 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_1 2024-05-21T03:14:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-21T03:14:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T03:14:20.340 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-21T03:14:20.340 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T03:14:20.340 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T03:14:20.340 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-05-21 03:13:56.725542981 +0000 2024-05-21T03:14:20.340 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-05-21 03:08:09.547825558 +0000 2024-05-21T03:14:20.340 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-05-21 03:08:09.547825558 +0000 2024-05-21T03:14:20.340 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-05-21 03:08:09.547825558 +0000 2024-05-21T03:14:20.340 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-21T03:14:20.406 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-05-21T03:14:20.406 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-05-21T03:14:20.406 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000283618 s, 1.8 MB/s 2024-05-21T03:14:20.407 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-21T03:14:20.465 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_2 2024-05-21T03:14:20.522 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-21T03:14:20.522 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T03:14:20.522 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-21T03:14:20.522 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T03:14:20.522 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T03:14:20.523 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-05-21 03:13:56.725542981 +0000 2024-05-21T03:14:20.523 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-05-21 03:08:09.871820517 +0000 2024-05-21T03:14:20.523 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-05-21 03:08:09.871820517 +0000 2024-05-21T03:14:20.523 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-05-21 03:08:09.871820517 +0000 2024-05-21T03:14:20.523 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-21T03:14:20.588 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-05-21T03:14:20.588 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-05-21T03:14:20.588 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000225521 s, 2.3 MB/s 2024-05-21T03:14:20.589 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-21T03:14:20.645 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_3 2024-05-21T03:14:20.701 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-21T03:14:20.701 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T03:14:20.701 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-21T03:14:20.701 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T03:14:20.701 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T03:14:20.701 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-05-21 03:13:56.726542966 +0000 2024-05-21T03:14:20.701 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-05-21 03:08:10.187815602 +0000 2024-05-21T03:14:20.702 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-05-21 03:08:10.187815602 +0000 2024-05-21T03:14:20.702 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-05-21 03:08:10.187815602 +0000 2024-05-21T03:14:20.702 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-21T03:14:20.765 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-05-21T03:14:20.765 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-05-21T03:14:20.765 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000237089 s, 2.2 MB/s 2024-05-21T03:14:20.766 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-21T03:14:20.822 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_4 2024-05-21T03:14:20.877 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-21T03:14:20.878 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T03:14:20.878 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-21T03:14:20.878 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T03:14:20.878 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T03:14:20.878 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-05-21 03:13:56.726542966 +0000 2024-05-21T03:14:20.878 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-05-21 03:08:10.515810499 +0000 2024-05-21T03:14:20.878 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-05-21 03:08:10.515810499 +0000 2024-05-21T03:14:20.878 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-05-21 03:08:10.515810499 +0000 2024-05-21T03:14:20.878 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-21T03:14:20.940 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-05-21T03:14:20.940 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-05-21T03:14:20.940 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000193164 s, 2.7 MB/s 2024-05-21T03:14:20.941 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-21T03:14:20.998 INFO:tasks.cephadm:Deploying osd.0 on smithi027 with /dev/vg_nvme/lv_4... 2024-05-21T03:14:20.999 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-21T03:14:21.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:20 smithi165 ceph-mon[36269]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:21.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:20 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:21.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:20 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:21.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:20 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:14:21.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:20 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:21.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:20 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:14:21.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:20 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:21.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:20 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T03:14:21.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:20 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-21T03:14:21.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:20 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T03:14:21.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:20 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:21.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:20 smithi027 ceph-mon[32179]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:21.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:20 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:21.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:20 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:21.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:20 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:14:21.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:20 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:21.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:20 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:14:21.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:20 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:21.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:20 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T03:14:21.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:20 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-21T03:14:21.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:20 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T03:14:21.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:20 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:21.246 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:14:21.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:20 smithi039 ceph-mon[36429]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:21.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:20 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:21.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:20 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:21.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:20 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:14:21.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:20 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:21.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:20 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:14:21.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:20 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:21.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:20 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T03:14:21.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:20 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-21T03:14:21.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:20 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T03:14:21.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:20 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:21.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:21 smithi027 ceph-mon[32179]: Deploying daemon mgr.z on smithi165 2024-05-21T03:14:22.190 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:21 smithi165 ceph-mon[36269]: Deploying daemon mgr.z on smithi165 2024-05-21T03:14:22.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:21 smithi039 ceph-mon[36429]: Deploying daemon mgr.z on smithi165 2024-05-21T03:14:22.521 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:22 smithi165 systemd[1]: Starting Ceph mgr.z for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:14:22.877 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:22 smithi165 ceph-mon[36269]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:22.877 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:22 smithi165 podman[38098]: 2024-05-21 03:14:22.519152815 +0000 UTC m=+0.017645958 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T03:14:22.878 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:22 smithi165 podman[38098]: 2024-05-21 03:14:22.642477178 +0000 UTC m=+0.140970321 container create cd5326e5d3958a25c9ff5681bbcee2664423324810903e02e29bc51e8bd74949 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-21T03:14:22.878 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:22 smithi165 podman[38098]: 2024-05-21 03:14:22.820001402 +0000 UTC m=+0.318494558 container init cd5326e5d3958a25c9ff5681bbcee2664423324810903e02e29bc51e8bd74949 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T03:14:22.878 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:22 smithi165 podman[38098]: 2024-05-21 03:14:22.82277548 +0000 UTC m=+0.321268626 container start cd5326e5d3958a25c9ff5681bbcee2664423324810903e02e29bc51e8bd74949 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea) 2024-05-21T03:14:22.878 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:22 smithi165 ceph-mgr[38117]: -- 172.21.15.165:0/1124842039 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b02e7c85a0 con 0x55b02e79c800 2024-05-21T03:14:22.878 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:22 smithi165 bash[38098]: cd5326e5d3958a25c9ff5681bbcee2664423324810903e02e29bc51e8bd74949 2024-05-21T03:14:23.049 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-21T03:14:23.050 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-21T03:14:23.050 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2024-05-21T03:14:23.050 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2024-05-21T03:14:23.050 INFO:teuthology.orchestra.run.smithi027.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110767 s, 947 MB/s 2024-05-21T03:14:23.050 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 2024-05-21T03:14:23.050 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2024-05-21T03:14:23.161 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:22 smithi165 systemd[1]: Started Ceph mgr.z for c9ca94be-171f-11ef-bc9a-c7b262605968. 2024-05-21T03:14:23.161 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:22 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:14:22.976+0000 7f68b0249200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T03:14:23.161 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:23 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:14:23.161+0000 7f68b0249200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T03:14:23.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:22 smithi027 ceph-mon[32179]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:23.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:22 smithi039 ceph-mon[36429]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:23.521 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:23 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:14:23.264+0000 7f68b0249200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T03:14:23.830 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi027:vg_nvme/lv_4 2024-05-21T03:14:23.945 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:23 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:14:23.573+0000 7f68b0249200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T03:14:23.945 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:23 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:14:23.658+0000 7f68b0249200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T03:14:24.081 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:14:24.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:23 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:24.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:23 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:24.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:23 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:24.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:23 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T03:14:24.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:23 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-21T03:14:24.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:23 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T03:14:24.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:23 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:24.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:23 smithi027 ceph-mon[32179]: Deploying daemon mgr.y on smithi039 2024-05-21T03:14:24.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:23 smithi027 ceph-mon[32179]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:24.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:23 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:24.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:23 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:24.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:23 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:24.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:23 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T03:14:24.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:23 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-21T03:14:24.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:23 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T03:14:24.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:23 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:24.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:23 smithi039 ceph-mon[36429]: Deploying daemon mgr.y on smithi039 2024-05-21T03:14:24.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:23 smithi039 ceph-mon[36429]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:24.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:23 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:24.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:23 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:24.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:23 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:24.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:23 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T03:14:24.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:23 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-21T03:14:24.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:23 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T03:14:24.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:23 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:24.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:23 smithi165 ceph-mon[36269]: Deploying daemon mgr.y on smithi039 2024-05-21T03:14:24.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:23 smithi165 ceph-mon[36269]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:25.203 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:24 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:14:24.853+0000 7f68b0249200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T03:14:25.203 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:25 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:14:25.026+0000 7f68b0249200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T03:14:25.203 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:25 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:14:25.202+0000 7f68b0249200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T03:14:25.273 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:25 smithi039 systemd[1]: Starting Ceph mgr.y for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:14:25.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.496+0000 7fd2734ae700 1 -- 172.21.15.27:0/2301925108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd26c1035a0 msgr2=0x7fd26c1039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:14:25.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.496+0000 7fd2734ae700 1 --2- 172.21.15.27:0/2301925108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd26c1035a0 0x7fd26c1039c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd2680099c0 tx=0x7fd268009cd0 comp rx=0 tx=0).stop 2024-05-21T03:14:25.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.497+0000 7fd2734ae700 1 -- 172.21.15.27:0/2301925108 shutdown_connections 2024-05-21T03:14:25.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.497+0000 7fd2734ae700 1 --2- 172.21.15.27:0/2301925108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd26c105140 0x7fd26c10b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:25.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.497+0000 7fd2734ae700 1 --2- 172.21.15.27:0/2301925108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd26c1035a0 0x7fd26c1039c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:25.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.497+0000 7fd2734ae700 1 --2- 172.21.15.27:0/2301925108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd26c1047a0 0x7fd26c104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:25.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.497+0000 7fd2734ae700 1 -- 172.21.15.27:0/2301925108 >> 172.21.15.27:0/2301925108 conn(0x7fd26c0feb40 msgr2=0x7fd26c100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:14:25.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.497+0000 7fd2734ae700 1 -- 172.21.15.27:0/2301925108 shutdown_connections 2024-05-21T03:14:25.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.497+0000 7fd2734ae700 1 -- 172.21.15.27:0/2301925108 wait complete. 2024-05-21T03:14:25.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.497+0000 7fd2734ae700 1 Processor -- start 2024-05-21T03:14:25.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.497+0000 7fd2734ae700 1 -- start start 2024-05-21T03:14:25.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.498+0000 7fd2734ae700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd26c1035a0 0x7fd26c0772e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:14:25.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.498+0000 7fd2734ae700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd26c1047a0 0x7fd26c075930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:14:25.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.498+0000 7fd2734ae700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd26c105140 0x7fd26c075e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:14:25.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.498+0000 7fd2734ae700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c10f020 con 0x7fd26c105140 2024-05-21T03:14:25.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.498+0000 7fd2734ae700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c10eea0 con 0x7fd26c1035a0 2024-05-21T03:14:25.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.498+0000 7fd2734ae700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c10f1a0 con 0x7fd26c1047a0 2024-05-21T03:14:25.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.498+0000 7fd271a4b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd26c105140 0x7fd26c075e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:14:25.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.498+0000 7fd271a4b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd26c105140 0x7fd26c075e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47986/0 (socket says 172.21.15.27:47986) 2024-05-21T03:14:25.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.498+0000 7fd27124a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd26c1035a0 0x7fd26c0772e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:14:25.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.498+0000 7fd271a4b700 1 -- 172.21.15.27:0/2932484978 learned_addr learned my addr 172.21.15.27:0/2932484978 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:14:25.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.498+0000 7fd270a49700 1 --2- 172.21.15.27:0/2932484978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd26c1047a0 0x7fd26c075930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:14:25.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.498+0000 7fd271a4b700 1 -- 172.21.15.27:0/2932484978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd26c1047a0 msgr2=0x7fd26c075930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:14:25.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.498+0000 7fd271a4b700 1 --2- 172.21.15.27:0/2932484978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd26c1047a0 0x7fd26c075930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:25.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.498+0000 7fd271a4b700 1 -- 172.21.15.27:0/2932484978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd26c1035a0 msgr2=0x7fd26c0772e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:14:25.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.498+0000 7fd271a4b700 1 --2- 172.21.15.27:0/2932484978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd26c1035a0 0x7fd26c0772e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:25.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.498+0000 7fd271a4b700 1 -- 172.21.15.27:0/2932484978 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd26c076720 con 0x7fd26c105140 2024-05-21T03:14:25.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.498+0000 7fd271a4b700 1 --2- 172.21.15.27:0/2932484978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd26c105140 0x7fd26c075e70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd264007ee0 tx=0x7fd26400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:14:25.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.498+0000 7fd25e7fc700 1 -- 172.21.15.27:0/2932484978 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26400dd20 con 0x7fd26c105140 2024-05-21T03:14:25.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.498+0000 7fd25e7fc700 1 -- 172.21.15.27:0/2932484978 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd264004020 con 0x7fd26c105140 2024-05-21T03:14:25.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.498+0000 7fd2734ae700 1 -- 172.21.15.27:0/2932484978 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd26c1a38b0 con 0x7fd26c105140 2024-05-21T03:14:25.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.498+0000 7fd25e7fc700 1 -- 172.21.15.27:0/2932484978 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd264015660 con 0x7fd26c105140 2024-05-21T03:14:25.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.499+0000 7fd2734ae700 1 -- 172.21.15.27:0/2932484978 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd26c1a3d70 con 0x7fd26c105140 2024-05-21T03:14:25.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.500+0000 7fd2734ae700 1 -- 172.21.15.27:0/2932484978 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd26c06a230 con 0x7fd26c105140 2024-05-21T03:14:25.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.500+0000 7fd25e7fc700 1 -- 172.21.15.27:0/2932484978 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fd264003a00 con 0x7fd26c105140 2024-05-21T03:14:25.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.500+0000 7fd25e7fc700 1 --2- 172.21.15.27:0/2932484978 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd25803e750 0x7fd258040c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:14:25.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.500+0000 7fd25e7fc700 1 -- 172.21.15.27:0/2932484978 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd264052260 con 0x7fd26c105140 2024-05-21T03:14:25.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.500+0000 7fd27124a700 1 --2- 172.21.15.27:0/2932484978 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd25803e750 0x7fd258040c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:14:25.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.500+0000 7fd27124a700 1 --2- 172.21.15.27:0/2932484978 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd25803e750 0x7fd258040c10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd268006010 tx=0x7fd268000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:14:25.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.505+0000 7fd25e7fc700 1 -- 172.21.15.27:0/2932484978 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd264010070 con 0x7fd26c105140 2024-05-21T03:14:25.604 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:25 smithi039 podman[37785]: 2024-05-21 03:14:25.255052351 +0000 UTC m=+0.021392965 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T03:14:25.604 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:25 smithi039 podman[37785]: 2024-05-21 03:14:25.357374601 +0000 UTC m=+0.123715192 container create fac9a0fe8ebc20b23a541902ffd6821aed728df818d32da298155cc3dc1ba468 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-21T03:14:25.604 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:25 smithi039 podman[37785]: 2024-05-21 03:14:25.601337343 +0000 UTC m=+0.367677943 container init fac9a0fe8ebc20b23a541902ffd6821aed728df818d32da298155cc3dc1ba468 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T03:14:25.604 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:25 smithi039 podman[37785]: 2024-05-21 03:14:25.604728334 +0000 UTC m=+0.371068934 container start fac9a0fe8ebc20b23a541902ffd6821aed728df818d32da298155cc3dc1ba468 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea) 2024-05-21T03:14:25.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:25.732+0000 7fd2734ae700 1 -- 172.21.15.27:0/2932484978 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd26c063980 con 0x7fd25803e750 2024-05-21T03:14:25.771 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:25 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:14:25.506+0000 7f68b0249200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T03:14:25.771 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:25 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:14:25.599+0000 7f68b0249200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T03:14:25.979 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:25 smithi039 ceph-mgr[37811]: -- 172.21.15.39:0/4202993689 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56405fa925a0 con 0x56405fa66c00 2024-05-21T03:14:25.979 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:25 smithi039 bash[37785]: fac9a0fe8ebc20b23a541902ffd6821aed728df818d32da298155cc3dc1ba468 2024-05-21T03:14:25.979 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:25 smithi039 systemd[1]: Started Ceph mgr.y for c9ca94be-171f-11ef-bc9a-c7b262605968. 2024-05-21T03:14:25.979 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:25 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:14:25.781+0000 7f2586958200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T03:14:25.979 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:25 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:14:25.979+0000 7f2586958200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T03:14:26.186 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:25 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:14:25.808+0000 7f68b0249200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T03:14:26.186 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:26 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:14:26.006+0000 7f68b0249200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T03:14:26.269 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:26 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:14:26.089+0000 7f2586958200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T03:14:26.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:26 smithi165 ceph-mon[36269]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:26.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:26 smithi165 ceph-mon[36269]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:14:26.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:26 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:14:26.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:26 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:14:26.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:26 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:26.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:26 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:26.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:26 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:26.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:26 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:26.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:26 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:26.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:26 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:14:26.590 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:26 smithi027 ceph-mon[32179]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:26.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:26 smithi027 ceph-mon[32179]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:14:26.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:26 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:14:26.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:26 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:14:26.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:26 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:26.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:26 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:26.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:26 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:26.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:26 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:26.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:26 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:26.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:26 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:14:26.769 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:26 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:14:26.425+0000 7f2586958200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T03:14:26.769 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:26 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:14:26.528+0000 7f2586958200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T03:14:26.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:26 smithi039 ceph-mon[36429]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:26.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:26 smithi039 ceph-mon[36429]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:14:26.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:26 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:14:26.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:26 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:14:26.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:26 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:26.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:26 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:26.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:26 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:26.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:26 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:26.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:26 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:26.771 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:26 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:14:26.895 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:26 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:14:26.644+0000 7f68b0249200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T03:14:26.896 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:26 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:14:26.865+0000 7f68b0249200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-21T03:14:27.771 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:27 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:14:27.305+0000 7f68b0249200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T03:14:27.772 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:27 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:14:27.403+0000 7f68b0249200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T03:14:28.132 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:27 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:14:27.776+0000 7f68b0249200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T03:14:28.132 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:28 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:14:28.132+0000 7f68b0249200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T03:14:28.162 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:27 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:14:27.813+0000 7f2586958200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T03:14:28.163 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:27 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:14:27.987+0000 7f2586958200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T03:14:28.163 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:28 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:14:28.163+0000 7f2586958200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T03:14:28.464 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:28 smithi039 ceph-mon[36429]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:28.464 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:28 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:28.464 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:28 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:28.464 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:28 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:28.465 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:28 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:28.465 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:28 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:28.465 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:28 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:14:28.465 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:28 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:28.465 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:28 smithi039 ceph-mon[36429]: Reconfiguring mgr.x (unknown last config time)... 2024-05-21T03:14:28.465 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:28 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T03:14:28.465 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:28 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T03:14:28.465 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:28 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:28.465 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:28 smithi039 ceph-mon[36429]: Reconfiguring daemon mgr.x on smithi027 2024-05-21T03:14:28.472 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:28 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:14:28.226+0000 7f68b0249200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T03:14:28.473 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:28 smithi165 ceph-mon[36269]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:28.473 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:28 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:28.473 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:28 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:28.473 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:28 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:28.473 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:28 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:28.473 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:28 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:28.473 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:28 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:14:28.473 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:28 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:28.473 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:28 smithi165 ceph-mon[36269]: Reconfiguring mgr.x (unknown last config time)... 2024-05-21T03:14:28.473 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:28 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T03:14:28.473 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:28 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T03:14:28.473 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:28 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:28.473 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:28 smithi165 ceph-mon[36269]: Reconfiguring daemon mgr.x on smithi027 2024-05-21T03:14:28.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:28 smithi027 ceph-mon[32179]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:28.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:28 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:28.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:28 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:28.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:28 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:28.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:28 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:28.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:28 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:28.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:28 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:14:28.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:28 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:28.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:28 smithi027 ceph-mon[32179]: Reconfiguring mgr.x (unknown last config time)... 2024-05-21T03:14:28.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:28 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T03:14:28.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:28 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T03:14:28.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:28 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:28.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:28 smithi027 ceph-mon[32179]: Reconfiguring daemon mgr.x on smithi027 2024-05-21T03:14:28.769 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:28 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:14:28.464+0000 7f2586958200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T03:14:28.769 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:28 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:14:28.560+0000 7f2586958200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T03:14:28.771 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:28 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:14:28.472+0000 7f68b0249200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T03:14:28.771 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:28 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:14:28.606+0000 7f68b0249200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T03:14:28.771 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:28 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:14:28.701+0000 7f68b0249200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T03:14:29.269 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:28 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:14:28.770+0000 7f2586958200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T03:14:29.269 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:28 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:14:28.969+0000 7f2586958200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T03:14:29.350 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:29 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:14:29.031+0000 7f68b0249200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T03:14:29.351 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:29 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:14:29.124+0000 7f68b0249200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T03:14:29.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:29 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:29.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:29 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/1830230659' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c745c09f-9a76-4d05-af75-c7554f6d5e6e"}]: dispatch 2024-05-21T03:14:29.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:29 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/1830230659' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c745c09f-9a76-4d05-af75-c7554f6d5e6e"}]': finished 2024-05-21T03:14:29.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:29 smithi027 ceph-mon[32179]: osdmap e5: 1 total, 0 up, 1 in 2024-05-21T03:14:29.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:29 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T03:14:29.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:29 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3351729992' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:14:29.728 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:29 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:29.728 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:29 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/1830230659' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c745c09f-9a76-4d05-af75-c7554f6d5e6e"}]: dispatch 2024-05-21T03:14:29.728 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:29 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/1830230659' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c745c09f-9a76-4d05-af75-c7554f6d5e6e"}]': finished 2024-05-21T03:14:29.728 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:29 smithi165 ceph-mon[36269]: osdmap e5: 1 total, 0 up, 1 in 2024-05-21T03:14:29.728 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:29 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T03:14:29.728 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:29 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/3351729992' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:14:29.769 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:29 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:14:29.594+0000 7f2586958200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T03:14:29.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:29 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:29.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:29 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/1830230659' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c745c09f-9a76-4d05-af75-c7554f6d5e6e"}]: dispatch 2024-05-21T03:14:29.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:29 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/1830230659' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c745c09f-9a76-4d05-af75-c7554f6d5e6e"}]': finished 2024-05-21T03:14:29.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:29 smithi039 ceph-mon[36429]: osdmap e5: 1 total, 0 up, 1 in 2024-05-21T03:14:29.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:29 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T03:14:29.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:29 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/3351729992' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:14:30.021 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:29 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:14:29.728+0000 7f68b0249200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T03:14:30.021 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:14:29 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:14:29.946+0000 7f68b0249200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T03:14:30.260 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:29 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:14:29.824+0000 7f2586958200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-21T03:14:30.519 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:30 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:14:30.261+0000 7f2586958200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T03:14:30.519 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:30 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:14:30.360+0000 7f2586958200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T03:14:30.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:30.607+0000 7fd25e7fc700 1 -- 172.21.15.27:0/2932484978 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 14) ==== 99937+0+0 (secure 0 0 0) 0x7fd264003cf0 con 0x7fd26c105140 2024-05-21T03:14:30.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:30 smithi027 ceph-mon[32179]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:30.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:30 smithi027 ceph-mon[32179]: Standby manager daemon z started 2024-05-21T03:14:30.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:30 smithi027 ceph-mon[32179]: from='mgr.? 172.21.15.165:0/2824862567' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-21T03:14:30.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:30 smithi027 ceph-mon[32179]: from='mgr.? 172.21.15.165:0/2824862567' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T03:14:30.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:30 smithi027 ceph-mon[32179]: from='mgr.? 172.21.15.165:0/2824862567' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-21T03:14:30.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:30 smithi027 ceph-mon[32179]: from='mgr.? 172.21.15.165:0/2824862567' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T03:14:31.019 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:30 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:14:30.754+0000 7f2586958200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T03:14:31.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:30 smithi039 ceph-mon[36429]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:31.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:30 smithi039 ceph-mon[36429]: Standby manager daemon z started 2024-05-21T03:14:31.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:30 smithi039 ceph-mon[36429]: from='mgr.? 172.21.15.165:0/2824862567' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-21T03:14:31.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:30 smithi039 ceph-mon[36429]: from='mgr.? 172.21.15.165:0/2824862567' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T03:14:31.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:30 smithi039 ceph-mon[36429]: from='mgr.? 172.21.15.165:0/2824862567' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-21T03:14:31.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:30 smithi039 ceph-mon[36429]: from='mgr.? 172.21.15.165:0/2824862567' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T03:14:31.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:30 smithi165 ceph-mon[36269]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:31.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:30 smithi165 ceph-mon[36269]: Standby manager daemon z started 2024-05-21T03:14:31.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:30 smithi165 ceph-mon[36269]: from='mgr.? 172.21.15.165:0/2824862567' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-21T03:14:31.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:30 smithi165 ceph-mon[36269]: from='mgr.? 172.21.15.165:0/2824862567' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T03:14:31.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:30 smithi165 ceph-mon[36269]: from='mgr.? 172.21.15.165:0/2824862567' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-21T03:14:31.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:30 smithi165 ceph-mon[36269]: from='mgr.? 172.21.15.165:0/2824862567' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T03:14:31.474 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:31 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:14:31.125+0000 7f2586958200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T03:14:31.474 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:31 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:14:31.220+0000 7f2586958200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T03:14:31.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:31 smithi039 ceph-mon[36429]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-21T03:14:31.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:31 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-21T03:14:31.769 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:31 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:14:31.474+0000 7f2586958200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T03:14:31.769 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:31 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:14:31.613+0000 7f2586958200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T03:14:31.769 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:31 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:14:31.712+0000 7f2586958200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T03:14:31.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:31 smithi027 ceph-mon[32179]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-21T03:14:31.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:31 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-21T03:14:32.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:31 smithi165 ceph-mon[36269]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-21T03:14:32.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:31 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-21T03:14:32.519 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:32 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:14:32.062+0000 7f2586958200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T03:14:32.519 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:32 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:14:32.158+0000 7f2586958200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T03:14:32.790 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:32 smithi027 ceph-mon[32179]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:33.010 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:32 smithi039 ceph-mon[36429]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:33.010 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:32 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:14:32.788+0000 7f2586958200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T03:14:33.010 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:14:33 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:14:33.010+0000 7f2586958200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T03:14:33.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:32 smithi165 ceph-mon[36269]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:33.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:33.629+0000 7fd25e7fc700 1 -- 172.21.15.27:0/2932484978 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fd264019cb0 con 0x7fd26c105140 2024-05-21T03:14:33.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:33 smithi027 ceph-mon[32179]: Standby manager daemon y started 2024-05-21T03:14:33.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:33 smithi027 ceph-mon[32179]: from='mgr.? 172.21.15.39:0/2778243226' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-21T03:14:33.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:33 smithi027 ceph-mon[32179]: from='mgr.? 172.21.15.39:0/2778243226' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T03:14:33.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:33 smithi027 ceph-mon[32179]: from='mgr.? 172.21.15.39:0/2778243226' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-21T03:14:33.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:33 smithi027 ceph-mon[32179]: from='mgr.? 172.21.15.39:0/2778243226' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T03:14:34.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:33 smithi039 ceph-mon[36429]: Standby manager daemon y started 2024-05-21T03:14:34.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:33 smithi039 ceph-mon[36429]: from='mgr.? 172.21.15.39:0/2778243226' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-21T03:14:34.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:33 smithi039 ceph-mon[36429]: from='mgr.? 172.21.15.39:0/2778243226' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T03:14:34.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:33 smithi039 ceph-mon[36429]: from='mgr.? 172.21.15.39:0/2778243226' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-21T03:14:34.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:33 smithi039 ceph-mon[36429]: from='mgr.? 172.21.15.39:0/2778243226' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T03:14:34.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:33 smithi165 ceph-mon[36269]: Standby manager daemon y started 2024-05-21T03:14:34.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:33 smithi165 ceph-mon[36269]: from='mgr.? 172.21.15.39:0/2778243226' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-21T03:14:34.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:33 smithi165 ceph-mon[36269]: from='mgr.? 172.21.15.39:0/2778243226' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T03:14:34.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:33 smithi165 ceph-mon[36269]: from='mgr.? 172.21.15.39:0/2778243226' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-21T03:14:34.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:33 smithi165 ceph-mon[36269]: from='mgr.? 172.21.15.39:0/2778243226' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T03:14:34.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:34 smithi027 ceph-mon[32179]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:34.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:34 smithi027 ceph-mon[32179]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-21T03:14:34.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:34 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-21T03:14:34.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:34 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:34.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:34 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:34.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:34 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:14:34.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:34 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:34.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:34 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:14:34.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:34 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:35.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:34 smithi039 ceph-mon[36429]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:35.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:34 smithi039 ceph-mon[36429]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-21T03:14:35.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:34 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-21T03:14:35.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:34 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:35.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:34 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:35.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:34 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:14:35.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:34 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:35.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:34 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:14:35.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:34 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:35.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:34 smithi165 ceph-mon[36269]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:35.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:34 smithi165 ceph-mon[36269]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-21T03:14:35.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:34 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-21T03:14:35.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:34 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:35.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:34 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:35.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:34 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:14:35.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:34 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:35.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:34 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:14:35.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:34 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:36.737 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:36 smithi027 ceph-mon[32179]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:36.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:36 smithi039 ceph-mon[36429]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:37.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:36 smithi165 ceph-mon[36269]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:37.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:37 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-21T03:14:37.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:37 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:37.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:37 smithi027 ceph-mon[32179]: Deploying daemon osd.0 on smithi027 2024-05-21T03:14:38.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:37 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-21T03:14:38.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:37 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:38.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:37 smithi039 ceph-mon[36429]: Deploying daemon osd.0 on smithi027 2024-05-21T03:14:38.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:37 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-21T03:14:38.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:37 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:38.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:37 smithi165 ceph-mon[36269]: Deploying daemon osd.0 on smithi027 2024-05-21T03:14:38.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:38 smithi027 ceph-mon[32179]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:39.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:38 smithi039 ceph-mon[36429]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:39.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:38 smithi165 ceph-mon[36269]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:40.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:40 smithi027 ceph-mon[32179]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:41.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:40 smithi039 ceph-mon[36429]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:41.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:40 smithi165 ceph-mon[36269]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:42.894 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:42 smithi027 ceph-mon[32179]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:42.894 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:42 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:42.895 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:42 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:42.895 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:42 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:14:43.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:42 smithi039 ceph-mon[36429]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:43.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:42 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:43.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:42 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:43.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:42 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:14:43.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:42 smithi165 ceph-mon[36269]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:43.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:42 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:43.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:42 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:43.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:42 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:14:44.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:44 smithi027 ceph-mon[32179]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:45.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:44 smithi039 ceph-mon[36429]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:45.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:44 smithi165 ceph-mon[36269]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:46.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:46.191+0000 7fd25e7fc700 1 -- 172.21.15.27:0/2932484978 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd26c063980 con 0x7fd25803e750 2024-05-21T03:14:46.192 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 0 on host 'smithi027' 2024-05-21T03:14:46.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:46.193+0000 7fd2734ae700 1 -- 172.21.15.27:0/2932484978 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd25803e750 msgr2=0x7fd258040c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:14:46.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:46.193+0000 7fd2734ae700 1 --2- 172.21.15.27:0/2932484978 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd25803e750 0x7fd258040c10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd268006010 tx=0x7fd268000bc0 comp rx=0 tx=0).stop 2024-05-21T03:14:46.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:46.193+0000 7fd2734ae700 1 -- 172.21.15.27:0/2932484978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd26c105140 msgr2=0x7fd26c075e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:14:46.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:46.193+0000 7fd2734ae700 1 --2- 172.21.15.27:0/2932484978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd26c105140 0x7fd26c075e70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd264007ee0 tx=0x7fd26400d3b0 comp rx=0 tx=0).stop 2024-05-21T03:14:46.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:46.194+0000 7fd2734ae700 1 -- 172.21.15.27:0/2932484978 shutdown_connections 2024-05-21T03:14:46.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:46.194+0000 7fd2734ae700 1 --2- 172.21.15.27:0/2932484978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd26c105140 0x7fd26c075e70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:46.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:46.194+0000 7fd2734ae700 1 --2- 172.21.15.27:0/2932484978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd26c1035a0 0x7fd26c0772e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:46.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:46.194+0000 7fd2734ae700 1 --2- 172.21.15.27:0/2932484978 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd25803e750 0x7fd258040c10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:46.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:46.194+0000 7fd2734ae700 1 --2- 172.21.15.27:0/2932484978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd26c1047a0 0x7fd26c075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:46.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:46.194+0000 7fd2734ae700 1 -- 172.21.15.27:0/2932484978 >> 172.21.15.27:0/2932484978 conn(0x7fd26c0feb40 msgr2=0x7fd26c107b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:14:46.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:46.194+0000 7fd2734ae700 1 -- 172.21.15.27:0/2932484978 shutdown_connections 2024-05-21T03:14:46.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:14:46.194+0000 7fd2734ae700 1 -- 172.21.15.27:0/2932484978 wait complete. 2024-05-21T03:14:46.426 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:46 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:46.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:46 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:46.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:46 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:46.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:46 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:14:46.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:46 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:46.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:46 smithi027 ceph-mon[32179]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:46.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:46 smithi027 ceph-mon[32179]: from='osd.0 [v2:172.21.15.27:6802/1399996449,v1:172.21.15.27:6803/1399996449]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-21T03:14:46.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:46 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:46.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:46 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:46.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:46 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:14:46.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:46 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:46.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:46 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:46.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:46 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:46.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:46 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:14:46.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:46 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:46.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:46 smithi039 ceph-mon[36429]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:46.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:46 smithi039 ceph-mon[36429]: from='osd.0 [v2:172.21.15.27:6802/1399996449,v1:172.21.15.27:6803/1399996449]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-21T03:14:46.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:46 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:46.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:46 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:46.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:46 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:14:46.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:46 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:46.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:46 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:46.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:46 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:46.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:46 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:14:46.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:46 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:46.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:46 smithi165 ceph-mon[36269]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:46.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:46 smithi165 ceph-mon[36269]: from='osd.0 [v2:172.21.15.27:6802/1399996449,v1:172.21.15.27:6803/1399996449]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-21T03:14:46.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:46 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:46.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:46 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:46.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:46 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:14:46.829 DEBUG:teuthology.orchestra.run.smithi027:osd.0> sudo journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.0.service 2024-05-21T03:14:46.832 INFO:tasks.cephadm:Deploying osd.1 on smithi039 with /dev/vg_nvme/lv_4... 2024-05-21T03:14:46.832 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-21T03:14:47.085 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T03:14:47.426 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:47 smithi027 ceph-mon[32179]: from='osd.0 [v2:172.21.15.27:6802/1399996449,v1:172.21.15.27:6803/1399996449]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-21T03:14:47.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:47 smithi027 ceph-mon[32179]: osdmap e6: 1 total, 0 up, 1 in 2024-05-21T03:14:47.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:47 smithi027 ceph-mon[32179]: from='osd.0 [v2:172.21.15.27:6802/1399996449,v1:172.21.15.27:6803/1399996449]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-21T03:14:47.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:47 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T03:14:47.427 INFO:journalctl@ceph.osd.0.smithi027.stdout:May 21 03:14:47 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-0[44703]: 2024-05-21T03:14:47.274+0000 7f6c7afa3700 -1 osd.0 0 waiting for initial osdmap 2024-05-21T03:14:47.427 INFO:journalctl@ceph.osd.0.smithi027.stdout:May 21 03:14:47 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-0[44703]: 2024-05-21T03:14:47.277+0000 7f6c76d96700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T03:14:47.505 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:47 smithi039 ceph-mon[36429]: from='osd.0 [v2:172.21.15.27:6802/1399996449,v1:172.21.15.27:6803/1399996449]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-21T03:14:47.506 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:47 smithi039 ceph-mon[36429]: osdmap e6: 1 total, 0 up, 1 in 2024-05-21T03:14:47.506 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:47 smithi039 ceph-mon[36429]: from='osd.0 [v2:172.21.15.27:6802/1399996449,v1:172.21.15.27:6803/1399996449]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-21T03:14:47.506 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:47 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T03:14:47.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:47 smithi165 ceph-mon[36269]: from='osd.0 [v2:172.21.15.27:6802/1399996449,v1:172.21.15.27:6803/1399996449]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-21T03:14:47.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:47 smithi165 ceph-mon[36269]: osdmap e6: 1 total, 0 up, 1 in 2024-05-21T03:14:47.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:47 smithi165 ceph-mon[36269]: from='osd.0 [v2:172.21.15.27:6802/1399996449,v1:172.21.15.27:6803/1399996449]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-21T03:14:47.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:47 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T03:14:48.382 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:48 smithi039 ceph-mon[36429]: from='osd.0 [v2:172.21.15.27:6802/1399996449,v1:172.21.15.27:6803/1399996449]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-05-21T03:14:48.383 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:48 smithi039 ceph-mon[36429]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:48.383 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:48 smithi039 ceph-mon[36429]: osdmap e7: 1 total, 0 up, 1 in 2024-05-21T03:14:48.383 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:48 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T03:14:48.383 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:48 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T03:14:48.383 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:48 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T03:14:48.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:48 smithi027 ceph-mon[32179]: from='osd.0 [v2:172.21.15.27:6802/1399996449,v1:172.21.15.27:6803/1399996449]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-05-21T03:14:48.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:48 smithi027 ceph-mon[32179]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:48.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:48 smithi027 ceph-mon[32179]: osdmap e7: 1 total, 0 up, 1 in 2024-05-21T03:14:48.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:48 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T03:14:48.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:48 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T03:14:48.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:48 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T03:14:48.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:48 smithi165 ceph-mon[36269]: from='osd.0 [v2:172.21.15.27:6802/1399996449,v1:172.21.15.27:6803/1399996449]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-05-21T03:14:48.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:48 smithi165 ceph-mon[36269]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:48.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:48 smithi165 ceph-mon[36269]: osdmap e7: 1 total, 0 up, 1 in 2024-05-21T03:14:48.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:48 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T03:14:48.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:48 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T03:14:48.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:48 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T03:14:48.961 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-21T03:14:48.961 INFO:teuthology.orchestra.run.smithi039.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-21T03:14:48.961 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10+0 records in 2024-05-21T03:14:48.962 INFO:teuthology.orchestra.run.smithi039.stderr:10+0 records out 2024-05-21T03:14:48.962 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115547 s, 907 MB/s 2024-05-21T03:14:48.962 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping successful for: 2024-05-21T03:14:49.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:49 smithi027 ceph-mon[32179]: purged_snaps scrub starts 2024-05-21T03:14:49.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:49 smithi027 ceph-mon[32179]: purged_snaps scrub ok 2024-05-21T03:14:49.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:49 smithi027 ceph-mon[32179]: osd.0 [v2:172.21.15.27:6802/1399996449,v1:172.21.15.27:6803/1399996449] boot 2024-05-21T03:14:49.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:49 smithi027 ceph-mon[32179]: osdmap e8: 1 total, 1 up, 1 in 2024-05-21T03:14:49.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:49 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T03:14:49.524 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:49 smithi039 ceph-mon[36429]: purged_snaps scrub starts 2024-05-21T03:14:49.524 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:49 smithi039 ceph-mon[36429]: purged_snaps scrub ok 2024-05-21T03:14:49.525 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:49 smithi039 ceph-mon[36429]: osd.0 [v2:172.21.15.27:6802/1399996449,v1:172.21.15.27:6803/1399996449] boot 2024-05-21T03:14:49.525 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:49 smithi039 ceph-mon[36429]: osdmap e8: 1 total, 1 up, 1 in 2024-05-21T03:14:49.525 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:49 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T03:14:49.553 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi039:vg_nvme/lv_4 2024-05-21T03:14:49.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:49 smithi165 ceph-mon[36269]: purged_snaps scrub starts 2024-05-21T03:14:49.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:49 smithi165 ceph-mon[36269]: purged_snaps scrub ok 2024-05-21T03:14:49.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:49 smithi165 ceph-mon[36269]: osd.0 [v2:172.21.15.27:6802/1399996449,v1:172.21.15.27:6803/1399996449] boot 2024-05-21T03:14:49.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:49 smithi165 ceph-mon[36269]: osdmap e8: 1 total, 1 up, 1 in 2024-05-21T03:14:49.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:49 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T03:14:49.820 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T03:14:50.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:50 smithi039 ceph-mon[36429]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:50.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:50 smithi039 ceph-mon[36429]: osdmap e9: 1 total, 1 up, 1 in 2024-05-21T03:14:50.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:50 smithi039 ceph-mon[36429]: Detected new or changed devices on smithi027 2024-05-21T03:14:50.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:50 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:50.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:50 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:50.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:50 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:14:50.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:50 smithi039 ceph-mon[36429]: Adjusting osd_memory_target on smithi027 to 17088M 2024-05-21T03:14:50.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:50 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:50.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:50 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:50.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:50 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:14:50.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:50 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:50.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:50 smithi027 ceph-mon[32179]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:50.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:50 smithi027 ceph-mon[32179]: osdmap e9: 1 total, 1 up, 1 in 2024-05-21T03:14:50.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:50 smithi027 ceph-mon[32179]: Detected new or changed devices on smithi027 2024-05-21T03:14:50.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:50 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:50.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:50 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:50.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:50 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:14:50.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:50 smithi027 ceph-mon[32179]: Adjusting osd_memory_target on smithi027 to 17088M 2024-05-21T03:14:50.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:50 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:50.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:50 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:50.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:50 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:14:50.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:50 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:50.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:50 smithi165 ceph-mon[36269]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T03:14:50.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:50 smithi165 ceph-mon[36269]: osdmap e9: 1 total, 1 up, 1 in 2024-05-21T03:14:50.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:50 smithi165 ceph-mon[36269]: Detected new or changed devices on smithi027 2024-05-21T03:14:50.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:50 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:50.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:50 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:50.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:50 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:14:50.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:50 smithi165 ceph-mon[36269]: Adjusting osd_memory_target on smithi027 to 17088M 2024-05-21T03:14:50.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:50 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:50.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:50 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:50.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:50 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:14:50.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:50 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:14:51.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.354+0000 7f87747d3700 1 -- 172.21.15.39:0/3647997394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f876c104040 msgr2=0x7f876c10a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:14:51.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.354+0000 7f87747d3700 1 --2- 172.21.15.39:0/3647997394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f876c104040 0x7f876c10a8f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f876400b0d0 tx=0x7f876400b3e0 comp rx=0 tx=0).stop 2024-05-21T03:14:51.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.355+0000 7f87747d3700 1 -- 172.21.15.39:0/3647997394 shutdown_connections 2024-05-21T03:14:51.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.355+0000 7f87747d3700 1 --2- 172.21.15.39:0/3647997394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f876c104040 0x7f876c10a8f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:51.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.355+0000 7f87747d3700 1 --2- 172.21.15.39:0/3647997394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f876c1024a0 0x7f876c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:51.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.355+0000 7f87747d3700 1 --2- 172.21.15.39:0/3647997394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f876c1036a0 0x7f876c103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:51.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.355+0000 7f87747d3700 1 -- 172.21.15.39:0/3647997394 >> 172.21.15.39:0/3647997394 conn(0x7f876c0fda40 msgr2=0x7f876c0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:14:51.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.355+0000 7f87747d3700 1 -- 172.21.15.39:0/3647997394 shutdown_connections 2024-05-21T03:14:51.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.355+0000 7f87747d3700 1 -- 172.21.15.39:0/3647997394 wait complete. 2024-05-21T03:14:51.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.355+0000 7f87747d3700 1 Processor -- start 2024-05-21T03:14:51.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.355+0000 7f87747d3700 1 -- start start 2024-05-21T03:14:51.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.356+0000 7f87747d3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f876c1024a0 0x7f876c106430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:14:51.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.356+0000 7f87747d3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f876c1036a0 0x7f876c104a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:14:51.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.356+0000 7f87747d3700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f876c104040 0x7f876c104fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:14:51.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.356+0000 7f87747d3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f876c10df20 con 0x7f876c1024a0 2024-05-21T03:14:51.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.356+0000 7f87747d3700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f876c10dda0 con 0x7f876c1036a0 2024-05-21T03:14:51.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.356+0000 7f87747d3700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f876c10e0a0 con 0x7f876c104040 2024-05-21T03:14:51.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.356+0000 7f8771d6e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f876c1036a0 0x7f876c104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:14:51.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.356+0000 7f8771d6e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f876c1036a0 0x7f876c104a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36476/0 (socket says 172.21.15.39:36476) 2024-05-21T03:14:51.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.356+0000 7f8771d6e700 1 -- 172.21.15.39:0/4184056279 learned_addr learned my addr 172.21.15.39:0/4184056279 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-21T03:14:51.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.356+0000 7f8772d70700 1 --2- 172.21.15.39:0/4184056279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f876c104040 0x7f876c104fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:14:51.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.356+0000 7f877256f700 1 --2- 172.21.15.39:0/4184056279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f876c1024a0 0x7f876c106430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:14:51.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.356+0000 7f8771d6e700 1 -- 172.21.15.39:0/4184056279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f876c104040 msgr2=0x7f876c104fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:14:51.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.356+0000 7f8771d6e700 1 --2- 172.21.15.39:0/4184056279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f876c104040 0x7f876c104fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:51.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.356+0000 7f8771d6e700 1 -- 172.21.15.39:0/4184056279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f876c1024a0 msgr2=0x7f876c106430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:14:51.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.356+0000 7f8771d6e700 1 --2- 172.21.15.39:0/4184056279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f876c1024a0 0x7f876c106430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:14:51.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.356+0000 7f8771d6e700 1 -- 172.21.15.39:0/4184056279 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876c105870 con 0x7f876c1036a0 2024-05-21T03:14:51.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.356+0000 7f8771d6e700 1 --2- 172.21.15.39:0/4184056279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f876c1036a0 0x7f876c104a80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f875c00c920 tx=0x7f875c00cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:14:51.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.356+0000 7f87637fe700 1 -- 172.21.15.39:0/4184056279 <== mon.2 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c004a30 con 0x7f876c1036a0 2024-05-21T03:14:51.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.357+0000 7f87637fe700 1 -- 172.21.15.39:0/4184056279 <== mon.2 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f875c0054d0 con 0x7f876c1036a0 2024-05-21T03:14:51.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.357+0000 7f87747d3700 1 -- 172.21.15.39:0/4184056279 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f876c1a3750 con 0x7f876c1036a0 2024-05-21T03:14:51.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.357+0000 7f87637fe700 1 -- 172.21.15.39:0/4184056279 <== mon.2 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c007c70 con 0x7f876c1036a0 2024-05-21T03:14:51.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.357+0000 7f87747d3700 1 -- 172.21.15.39:0/4184056279 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f876c1a3c90 con 0x7f876c1036a0 2024-05-21T03:14:51.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.358+0000 7f87747d3700 1 -- 172.21.15.39:0/4184056279 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f876c06a230 con 0x7f876c1036a0 2024-05-21T03:14:51.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.359+0000 7f87637fe700 1 -- 172.21.15.39:0/4184056279 <== mon.2 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f875c004c60 con 0x7f876c1036a0 2024-05-21T03:14:51.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.359+0000 7f87637fe700 1 --2- 172.21.15.39:0/4184056279 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f87580b2870 0x7f87580b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:14:51.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.359+0000 7f87637fe700 1 -- 172.21.15.39:0/4184056279 <== mon.2 v2:172.21.15.39:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f875c0e0590 con 0x7f876c1036a0 2024-05-21T03:14:51.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.360+0000 7f877256f700 1 --2- 172.21.15.39:0/4184056279 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f87580b2870 0x7f87580b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:14:51.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.360+0000 7f877256f700 1 --2- 172.21.15.39:0/4184056279 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f87580b2870 0x7f87580b4d30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f876c103500 tx=0x7f876800a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:14:51.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.364+0000 7f87637fe700 1 -- 172.21.15.39:0/4184056279 <== mon.2 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f875c014030 con 0x7f876c1036a0 2024-05-21T03:14:51.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:14:51.587+0000 7f87747d3700 1 -- 172.21.15.39:0/4184056279 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f876c1a3f50 con 0x7f87580b2870 2024-05-21T03:14:52.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:52 smithi039 ceph-mon[36429]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:14:52.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:52 smithi039 ceph-mon[36429]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:14:52.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:52 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:14:52.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:52 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:14:52.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:52 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:52.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:52 smithi027 ceph-mon[32179]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:14:52.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:52 smithi027 ceph-mon[32179]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:14:52.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:52 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:14:52.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:52 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:14:52.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:52 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:52.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:52 smithi165 ceph-mon[36269]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:14:52.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:52 smithi165 ceph-mon[36269]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:14:52.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:52 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:14:52.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:52 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:14:52.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:52 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:14:54.573 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:54 smithi165 ceph-mon[36269]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:14:54.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:54 smithi027 ceph-mon[32179]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:14:54.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:54 smithi039 ceph-mon[36429]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:14:55.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:55 smithi027 ceph-mon[32179]: from='client.? 172.21.15.39:0/2759179518' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b73aa00-4364-438f-9722-53caa4e9bec5"}]: dispatch 2024-05-21T03:14:55.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:55 smithi027 ceph-mon[32179]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b73aa00-4364-438f-9722-53caa4e9bec5"}]: dispatch 2024-05-21T03:14:55.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:55 smithi027 ceph-mon[32179]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8b73aa00-4364-438f-9722-53caa4e9bec5"}]': finished 2024-05-21T03:14:55.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:55 smithi027 ceph-mon[32179]: osdmap e10: 2 total, 1 up, 2 in 2024-05-21T03:14:55.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:55 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T03:14:55.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:55 smithi027 ceph-mon[32179]: from='client.? 172.21.15.39:0/2948983549' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:14:55.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:55 smithi039 ceph-mon[36429]: from='client.? 172.21.15.39:0/2759179518' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b73aa00-4364-438f-9722-53caa4e9bec5"}]: dispatch 2024-05-21T03:14:55.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:55 smithi039 ceph-mon[36429]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b73aa00-4364-438f-9722-53caa4e9bec5"}]: dispatch 2024-05-21T03:14:55.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:55 smithi039 ceph-mon[36429]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8b73aa00-4364-438f-9722-53caa4e9bec5"}]': finished 2024-05-21T03:14:55.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:55 smithi039 ceph-mon[36429]: osdmap e10: 2 total, 1 up, 2 in 2024-05-21T03:14:55.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:55 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T03:14:55.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:55 smithi039 ceph-mon[36429]: from='client.? 172.21.15.39:0/2948983549' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:14:55.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:55 smithi165 ceph-mon[36269]: from='client.? 172.21.15.39:0/2759179518' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b73aa00-4364-438f-9722-53caa4e9bec5"}]: dispatch 2024-05-21T03:14:55.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:55 smithi165 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b73aa00-4364-438f-9722-53caa4e9bec5"}]: dispatch 2024-05-21T03:14:55.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:55 smithi165 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8b73aa00-4364-438f-9722-53caa4e9bec5"}]': finished 2024-05-21T03:14:55.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:55 smithi165 ceph-mon[36269]: osdmap e10: 2 total, 1 up, 2 in 2024-05-21T03:14:55.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:55 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T03:14:55.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:55 smithi165 ceph-mon[36269]: from='client.? 172.21.15.39:0/2948983549' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:14:57.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:56 smithi039 ceph-mon[36429]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:14:57.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:56 smithi165 ceph-mon[36269]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:14:57.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:56 smithi027 ceph-mon[32179]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:14:59.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:14:58 smithi039 ceph-mon[36429]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:14:59.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:14:58 smithi165 ceph-mon[36269]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:14:59.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:14:58 smithi027 ceph-mon[32179]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:15:00.988 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:00 smithi039 ceph-mon[36429]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:15:01.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:00 smithi165 ceph-mon[36269]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:15:01.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:00 smithi027 ceph-mon[32179]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:15:02.979 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:02 smithi027 ceph-mon[32179]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:15:03.018 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:02 smithi039 ceph-mon[36429]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:15:03.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:02 smithi165 ceph-mon[36269]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:15:03.750 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:03 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-21T03:15:03.750 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:03 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:15:03.751 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:03 smithi039 ceph-mon[36429]: Deploying daemon osd.1 on smithi039 2024-05-21T03:15:04.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:03 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-21T03:15:04.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:03 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:15:04.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:03 smithi165 ceph-mon[36269]: Deploying daemon osd.1 on smithi039 2024-05-21T03:15:04.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:03 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-21T03:15:04.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:03 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:15:04.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:03 smithi027 ceph-mon[32179]: Deploying daemon osd.1 on smithi039 2024-05-21T03:15:04.715 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:04 smithi039 ceph-mon[36429]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:15:05.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:04 smithi165 ceph-mon[36269]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:15:05.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:04 smithi027 ceph-mon[32179]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:15:07.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:06 smithi039 ceph-mon[36429]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:15:07.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:06 smithi165 ceph-mon[36269]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:15:07.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:06 smithi027 ceph-mon[32179]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:15:09.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:08 smithi039 ceph-mon[36429]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:15:09.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:08 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:09.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:08 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:09.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:08 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:15:09.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:08 smithi165 ceph-mon[36269]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:15:09.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:08 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:09.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:08 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:09.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:08 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:15:09.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:08 smithi027 ceph-mon[32179]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:15:09.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:08 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:09.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:08 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:09.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:08 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:15:11.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:10 smithi039 ceph-mon[36429]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:15:11.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:10 smithi165 ceph-mon[36269]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:15:11.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:10 smithi027 ceph-mon[32179]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:15:12.126 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:12 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:12.126 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:12 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:12.126 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:12 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:15:12.126 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:12 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:15:12.126 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:12 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:12.127 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:12 smithi039 ceph-mon[36429]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:15:12.213 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 1 on host 'smithi039' 2024-05-21T03:15:12.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:15:12.211+0000 7f87637fe700 1 -- 172.21.15.39:0/4184056279 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f876c1a3f50 con 0x7f87580b2870 2024-05-21T03:15:12.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:15:12.214+0000 7f87747d3700 1 -- 172.21.15.39:0/4184056279 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f87580b2870 msgr2=0x7f87580b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:15:12.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:15:12.214+0000 7f87747d3700 1 --2- 172.21.15.39:0/4184056279 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f87580b2870 0x7f87580b4d30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f876c103500 tx=0x7f876800a3b0 comp rx=0 tx=0).stop 2024-05-21T03:15:12.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:15:12.214+0000 7f87747d3700 1 -- 172.21.15.39:0/4184056279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f876c1036a0 msgr2=0x7f876c104a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:15:12.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:15:12.214+0000 7f87747d3700 1 --2- 172.21.15.39:0/4184056279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f876c1036a0 0x7f876c104a80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f875c00c920 tx=0x7f875c00cc30 comp rx=0 tx=0).stop 2024-05-21T03:15:12.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:15:12.214+0000 7f87747d3700 1 -- 172.21.15.39:0/4184056279 shutdown_connections 2024-05-21T03:15:12.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:15:12.214+0000 7f87747d3700 1 --2- 172.21.15.39:0/4184056279 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f87580b2870 0x7f87580b4d30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:15:12.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:15:12.214+0000 7f87747d3700 1 --2- 172.21.15.39:0/4184056279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f876c104040 0x7f876c104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:15:12.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:15:12.214+0000 7f87747d3700 1 --2- 172.21.15.39:0/4184056279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f876c1024a0 0x7f876c106430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:15:12.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:15:12.214+0000 7f87747d3700 1 --2- 172.21.15.39:0/4184056279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f876c1036a0 0x7f876c104a80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:15:12.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:15:12.214+0000 7f87747d3700 1 -- 172.21.15.39:0/4184056279 >> 172.21.15.39:0/4184056279 conn(0x7f876c0fda40 msgr2=0x7f876c0ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:15:12.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:15:12.214+0000 7f87747d3700 1 -- 172.21.15.39:0/4184056279 shutdown_connections 2024-05-21T03:15:12.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:15:12.214+0000 7f87747d3700 1 -- 172.21.15.39:0/4184056279 wait complete. 2024-05-21T03:15:12.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:12 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:12.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:12 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:12.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:12 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:15:12.428 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:12 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:15:12.428 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:12 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:12.428 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:12 smithi027 ceph-mon[32179]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:15:12.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:12 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:12.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:12 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:12.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:12 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:15:12.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:12 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:15:12.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:12 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:12.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:12 smithi165 ceph-mon[36269]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:15:12.872 DEBUG:teuthology.orchestra.run.smithi039:osd.1> sudo journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.1.service 2024-05-21T03:15:12.874 INFO:tasks.cephadm:Deploying osd.2 on smithi165 with /dev/vg_nvme/lv_4... 2024-05-21T03:15:12.874 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-21T03:15:13.118 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T03:15:13.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:13 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:13.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:13 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:13.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:13 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:15:13.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:13 smithi027 ceph-mon[32179]: from='osd.1 [v2:172.21.15.39:6800/1776318847,v1:172.21.15.39:6801/1776318847]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-21T03:15:13.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:13 smithi027 ceph-mon[32179]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-21T03:15:13.436 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:13 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:13.436 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:13 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:13.436 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:13 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:15:13.436 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:13 smithi165 ceph-mon[36269]: from='osd.1 [v2:172.21.15.39:6800/1776318847,v1:172.21.15.39:6801/1776318847]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-21T03:15:13.436 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:13 smithi165 ceph-mon[36269]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-21T03:15:13.518 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:13 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:13.518 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:13 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:13.518 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:13 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:15:13.518 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:13 smithi039 ceph-mon[36429]: from='osd.1 [v2:172.21.15.39:6800/1776318847,v1:172.21.15.39:6801/1776318847]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-21T03:15:13.518 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:13 smithi039 ceph-mon[36429]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-21T03:15:14.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:14 smithi027 ceph-mon[32179]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-21T03:15:14.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:14 smithi027 ceph-mon[32179]: osdmap e11: 2 total, 1 up, 2 in 2024-05-21T03:15:14.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:14 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T03:15:14.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:14 smithi027 ceph-mon[32179]: from='osd.1 [v2:172.21.15.39:6800/1776318847,v1:172.21.15.39:6801/1776318847]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-21T03:15:14.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:14 smithi027 ceph-mon[32179]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-21T03:15:14.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:14 smithi027 ceph-mon[32179]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:15:14.457 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:14 smithi165 ceph-mon[36269]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-21T03:15:14.457 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:14 smithi165 ceph-mon[36269]: osdmap e11: 2 total, 1 up, 2 in 2024-05-21T03:15:14.457 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:14 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T03:15:14.457 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:14 smithi165 ceph-mon[36269]: from='osd.1 [v2:172.21.15.39:6800/1776318847,v1:172.21.15.39:6801/1776318847]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-21T03:15:14.457 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:14 smithi165 ceph-mon[36269]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-21T03:15:14.457 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:14 smithi165 ceph-mon[36269]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:15:14.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:14 smithi039 ceph-mon[36429]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-21T03:15:14.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:14 smithi039 ceph-mon[36429]: osdmap e11: 2 total, 1 up, 2 in 2024-05-21T03:15:14.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:14 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T03:15:14.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:14 smithi039 ceph-mon[36429]: from='osd.1 [v2:172.21.15.39:6800/1776318847,v1:172.21.15.39:6801/1776318847]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-21T03:15:14.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:14 smithi039 ceph-mon[36429]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-21T03:15:14.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:14 smithi039 ceph-mon[36429]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:15:15.103 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-21T03:15:15.103 INFO:teuthology.orchestra.run.smithi165.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-21T03:15:15.103 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10+0 records in 2024-05-21T03:15:15.104 INFO:teuthology.orchestra.run.smithi165.stderr:10+0 records out 2024-05-21T03:15:15.104 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114495 s, 916 MB/s 2024-05-21T03:15:15.104 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping successful for: 2024-05-21T03:15:15.400 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:15 smithi039 ceph-mon[36429]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-21T03:15:15.400 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:15 smithi039 ceph-mon[36429]: osdmap e12: 2 total, 1 up, 2 in 2024-05-21T03:15:15.400 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:15 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T03:15:15.400 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:15 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T03:15:15.400 INFO:journalctl@ceph.osd.1.smithi039.stdout:May 21 03:15:15 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-1[41013]: 2024-05-21T03:15:15.165+0000 7fae693ba700 -1 osd.1 0 waiting for initial osdmap 2024-05-21T03:15:15.400 INFO:journalctl@ceph.osd.1.smithi039.stdout:May 21 03:15:15 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-1[41013]: 2024-05-21T03:15:15.205+0000 7fae611a5700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T03:15:15.406 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:15 smithi165 ceph-mon[36269]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-21T03:15:15.406 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:15 smithi165 ceph-mon[36269]: osdmap e12: 2 total, 1 up, 2 in 2024-05-21T03:15:15.406 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:15 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T03:15:15.406 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:15 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T03:15:15.426 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:15 smithi027 ceph-mon[32179]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-21T03:15:15.426 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:15 smithi027 ceph-mon[32179]: osdmap e12: 2 total, 1 up, 2 in 2024-05-21T03:15:15.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:15 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T03:15:15.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:15 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T03:15:15.715 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi165:vg_nvme/lv_4 2024-05-21T03:15:15.966 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T03:15:16.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:16 smithi165 ceph-mon[36269]: purged_snaps scrub starts 2024-05-21T03:15:16.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:16 smithi165 ceph-mon[36269]: purged_snaps scrub ok 2024-05-21T03:15:16.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:16 smithi165 ceph-mon[36269]: from='osd.1 ' entity='osd.1' 2024-05-21T03:15:16.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:16 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T03:15:16.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:16 smithi165 ceph-mon[36269]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:15:16.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:16 smithi165 ceph-mon[36269]: Detected new or changed devices on smithi039 2024-05-21T03:15:16.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:16 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:16.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:16 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:16.378 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:16 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:15:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:16 smithi165 ceph-mon[36269]: Adjusting osd_memory_target on smithi039 to 17088M 2024-05-21T03:15:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:16 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:16 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:15:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:16 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:15:16.379 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:16 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:16.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:16 smithi039 ceph-mon[36429]: purged_snaps scrub starts 2024-05-21T03:15:16.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:16 smithi039 ceph-mon[36429]: purged_snaps scrub ok 2024-05-21T03:15:16.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:16 smithi039 ceph-mon[36429]: from='osd.1 ' entity='osd.1' 2024-05-21T03:15:16.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:16 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T03:15:16.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:16 smithi039 ceph-mon[36429]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:15:16.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:16 smithi039 ceph-mon[36429]: Detected new or changed devices on smithi039 2024-05-21T03:15:16.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:16 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:16.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:16 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:16.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:16 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:15:16.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:16 smithi039 ceph-mon[36429]: Adjusting osd_memory_target on smithi039 to 17088M 2024-05-21T03:15:16.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:16 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:16.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:16 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:15:16.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:16 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:15:16.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:16 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:16.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:16 smithi027 ceph-mon[32179]: purged_snaps scrub starts 2024-05-21T03:15:16.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:16 smithi027 ceph-mon[32179]: purged_snaps scrub ok 2024-05-21T03:15:16.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:16 smithi027 ceph-mon[32179]: from='osd.1 ' entity='osd.1' 2024-05-21T03:15:16.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:16 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T03:15:16.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:16 smithi027 ceph-mon[32179]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T03:15:16.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:16 smithi027 ceph-mon[32179]: Detected new or changed devices on smithi039 2024-05-21T03:15:16.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:16 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:16.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:16 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:16.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:16 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:15:16.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:16 smithi027 ceph-mon[32179]: Adjusting osd_memory_target on smithi039 to 17088M 2024-05-21T03:15:16.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:16 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:16.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:16 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:15:16.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:16 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:15:16.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:16 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:17.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:17 smithi165 ceph-mon[36269]: osd.1 [v2:172.21.15.39:6800/1776318847,v1:172.21.15.39:6801/1776318847] boot 2024-05-21T03:15:17.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:17 smithi165 ceph-mon[36269]: osdmap e13: 2 total, 2 up, 2 in 2024-05-21T03:15:17.317 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:17 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T03:15:17.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:17 smithi039 ceph-mon[36429]: osd.1 [v2:172.21.15.39:6800/1776318847,v1:172.21.15.39:6801/1776318847] boot 2024-05-21T03:15:17.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:17 smithi039 ceph-mon[36429]: osdmap e13: 2 total, 2 up, 2 in 2024-05-21T03:15:17.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:17 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T03:15:17.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.551+0000 7f7e519c9700 1 -- 172.21.15.165:0/243472973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e4c1036a0 msgr2=0x7f7e4c103b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:15:17.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.551+0000 7f7e519c9700 1 --2- 172.21.15.165:0/243472973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e4c1036a0 0x7f7e4c103b00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7e40009990 tx=0x7f7e40009ca0 comp rx=0 tx=0).stop 2024-05-21T03:15:17.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.551+0000 7f7e519c9700 1 -- 172.21.15.165:0/243472973 shutdown_connections 2024-05-21T03:15:17.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.551+0000 7f7e519c9700 1 --2- 172.21.15.165:0/243472973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e4c104040 0x7f7e4c10a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:15:17.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.551+0000 7f7e519c9700 1 --2- 172.21.15.165:0/243472973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e4c1024a0 0x7f7e4c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:15:17.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.551+0000 7f7e519c9700 1 --2- 172.21.15.165:0/243472973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e4c1036a0 0x7f7e4c103b00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:15:17.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.551+0000 7f7e519c9700 1 -- 172.21.15.165:0/243472973 >> 172.21.15.165:0/243472973 conn(0x7f7e4c0fda60 msgr2=0x7f7e4c0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:15:17.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.551+0000 7f7e519c9700 1 -- 172.21.15.165:0/243472973 shutdown_connections 2024-05-21T03:15:17.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.551+0000 7f7e519c9700 1 -- 172.21.15.165:0/243472973 wait complete. 2024-05-21T03:15:17.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.551+0000 7f7e519c9700 1 Processor -- start 2024-05-21T03:15:17.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.551+0000 7f7e519c9700 1 -- start start 2024-05-21T03:15:17.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.552+0000 7f7e519c9700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e4c1024a0 0x7f7e4c199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:15:17.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.552+0000 7f7e519c9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e4c1036a0 0x7f7e4c199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:15:17.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.552+0000 7f7e519c9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e4c104040 0x7f7e4c1a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:15:17.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.552+0000 7f7e519c9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4c10df90 con 0x7f7e4c104040 2024-05-21T03:15:17.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.552+0000 7f7e519c9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4c10de10 con 0x7f7e4c1036a0 2024-05-21T03:15:17.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.552+0000 7f7e519c9700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4c10e110 con 0x7f7e4c1024a0 2024-05-21T03:15:17.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.552+0000 7f7e4affd700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e4c1024a0 0x7f7e4c199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:15:17.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.552+0000 7f7e4affd700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e4c1024a0 0x7f7e4c199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44576/0 (socket says 172.21.15.165:44576) 2024-05-21T03:15:17.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.552+0000 7f7e4affd700 1 -- 172.21.15.165:0/1355004456 learned_addr learned my addr 172.21.15.165:0/1355004456 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-05-21T03:15:17.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.552+0000 7f7e4a7fc700 1 --2- 172.21.15.165:0/1355004456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e4c1036a0 0x7f7e4c199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:15:17.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.552+0000 7f7e4b7fe700 1 --2- 172.21.15.165:0/1355004456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e4c104040 0x7f7e4c1a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:15:17.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.552+0000 7f7e4affd700 1 -- 172.21.15.165:0/1355004456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e4c1036a0 msgr2=0x7f7e4c199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:15:17.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.552+0000 7f7e4affd700 1 --2- 172.21.15.165:0/1355004456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e4c1036a0 0x7f7e4c199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:15:17.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.552+0000 7f7e4affd700 1 -- 172.21.15.165:0/1355004456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e4c104040 msgr2=0x7f7e4c1a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:15:17.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.552+0000 7f7e4affd700 1 --2- 172.21.15.165:0/1355004456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e4c104040 0x7f7e4c1a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:15:17.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.552+0000 7f7e4affd700 1 -- 172.21.15.165:0/1355004456 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e4c1a1380 con 0x7f7e4c1024a0 2024-05-21T03:15:17.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.552+0000 7f7e4affd700 1 --2- 172.21.15.165:0/1355004456 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e4c1024a0 0x7f7e4c199720 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c00d9b0 tx=0x7f7e3c00dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:15:17.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.552+0000 7f7e509c7700 1 -- 172.21.15.165:0/1355004456 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c017030 con 0x7f7e4c1024a0 2024-05-21T03:15:17.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.552+0000 7f7e519c9700 1 -- 172.21.15.165:0/1355004456 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e4c1a1670 con 0x7f7e4c1024a0 2024-05-21T03:15:17.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.552+0000 7f7e509c7700 1 -- 172.21.15.165:0/1355004456 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e3c0044d0 con 0x7f7e4c1024a0 2024-05-21T03:15:17.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.552+0000 7f7e509c7700 1 -- 172.21.15.165:0/1355004456 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c0039e0 con 0x7f7e4c1024a0 2024-05-21T03:15:17.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.552+0000 7f7e519c9700 1 -- 172.21.15.165:0/1355004456 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e4c1a1bb0 con 0x7f7e4c1024a0 2024-05-21T03:15:17.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.553+0000 7f7e519c9700 1 -- 172.21.15.165:0/1355004456 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e4c06a230 con 0x7f7e4c1024a0 2024-05-21T03:15:17.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.554+0000 7f7e509c7700 1 -- 172.21.15.165:0/1355004456 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f7e3c01a070 con 0x7f7e4c1024a0 2024-05-21T03:15:17.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.555+0000 7f7e509c7700 1 --2- 172.21.15.165:0/1355004456 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f7e340b27f0 0x7f7e340b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:15:17.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.555+0000 7f7e509c7700 1 -- 172.21.15.165:0/1355004456 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f7e3c0e0690 con 0x7f7e4c1024a0 2024-05-21T03:15:17.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.555+0000 7f7e4a7fc700 1 --2- 172.21.15.165:0/1355004456 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f7e340b27f0 0x7f7e340b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:15:17.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.556+0000 7f7e4a7fc700 1 --2- 172.21.15.165:0/1355004456 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f7e340b27f0 0x7f7e340b4cb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000b5c0 tx=0x7f7e40005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:15:17.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.559+0000 7f7e509c7700 1 -- 172.21.15.165:0/1355004456 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7e3c0a9c00 con 0x7f7e4c1024a0 2024-05-21T03:15:17.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:17 smithi027 ceph-mon[32179]: osd.1 [v2:172.21.15.39:6800/1776318847,v1:172.21.15.39:6801/1776318847] boot 2024-05-21T03:15:17.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:17 smithi027 ceph-mon[32179]: osdmap e13: 2 total, 2 up, 2 in 2024-05-21T03:15:17.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:17 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T03:15:17.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:17.768+0000 7f7e519c9700 1 -- 172.21.15.165:0/1355004456 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7e4c06c320 con 0x7f7e340b27f0 2024-05-21T03:15:18.495 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:18 smithi165 ceph-mon[36269]: osdmap e14: 2 total, 2 up, 2 in 2024-05-21T03:15:18.495 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:18 smithi165 ceph-mon[36269]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:18.495 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:18 smithi165 ceph-mon[36269]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:15:18.495 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:18 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:15:18.495 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:18 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:15:18.495 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:18 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:15:18.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:18 smithi039 ceph-mon[36429]: osdmap e14: 2 total, 2 up, 2 in 2024-05-21T03:15:18.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:18 smithi039 ceph-mon[36429]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:18.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:18 smithi039 ceph-mon[36429]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:15:18.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:18 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:15:18.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:18 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:15:18.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:18 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:15:18.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:18 smithi027 ceph-mon[32179]: osdmap e14: 2 total, 2 up, 2 in 2024-05-21T03:15:18.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:18 smithi027 ceph-mon[32179]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:18.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:18 smithi027 ceph-mon[32179]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:15:18.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:18 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:15:18.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:18 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:15:18.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:18 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:15:20.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:20 smithi027 ceph-mon[32179]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:20.736 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:20 smithi165 ceph-mon[36269]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:20.768 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:20 smithi039 ceph-mon[36429]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:21.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:21 smithi027 ceph-mon[32179]: from='client.? 172.21.15.165:0/1604391177' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0822043e-7d3c-4e98-9dc0-94b98b4d562a"}]: dispatch 2024-05-21T03:15:21.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:21 smithi027 ceph-mon[32179]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0822043e-7d3c-4e98-9dc0-94b98b4d562a"}]: dispatch 2024-05-21T03:15:21.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:21 smithi027 ceph-mon[32179]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0822043e-7d3c-4e98-9dc0-94b98b4d562a"}]': finished 2024-05-21T03:15:21.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:21 smithi027 ceph-mon[32179]: osdmap e15: 3 total, 2 up, 3 in 2024-05-21T03:15:21.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:21 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T03:15:21.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:21 smithi027 ceph-mon[32179]: from='client.? 172.21.15.165:0/2206465153' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:15:21.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:21 smithi039 ceph-mon[36429]: from='client.? 172.21.15.165:0/1604391177' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0822043e-7d3c-4e98-9dc0-94b98b4d562a"}]: dispatch 2024-05-21T03:15:21.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:21 smithi039 ceph-mon[36429]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0822043e-7d3c-4e98-9dc0-94b98b4d562a"}]: dispatch 2024-05-21T03:15:21.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:21 smithi039 ceph-mon[36429]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0822043e-7d3c-4e98-9dc0-94b98b4d562a"}]': finished 2024-05-21T03:15:21.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:21 smithi039 ceph-mon[36429]: osdmap e15: 3 total, 2 up, 3 in 2024-05-21T03:15:21.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:21 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T03:15:21.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:21 smithi039 ceph-mon[36429]: from='client.? 172.21.15.165:0/2206465153' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:15:21.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:21 smithi165 ceph-mon[36269]: from='client.? 172.21.15.165:0/1604391177' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0822043e-7d3c-4e98-9dc0-94b98b4d562a"}]: dispatch 2024-05-21T03:15:21.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:21 smithi165 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0822043e-7d3c-4e98-9dc0-94b98b4d562a"}]: dispatch 2024-05-21T03:15:21.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:21 smithi165 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0822043e-7d3c-4e98-9dc0-94b98b4d562a"}]': finished 2024-05-21T03:15:21.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:21 smithi165 ceph-mon[36269]: osdmap e15: 3 total, 2 up, 3 in 2024-05-21T03:15:21.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:21 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T03:15:21.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:21 smithi165 ceph-mon[36269]: from='client.? 172.21.15.165:0/2206465153' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:15:22.679 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:22 smithi027 ceph-mon[32179]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:22.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:22 smithi039 ceph-mon[36429]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:22.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:22 smithi165 ceph-mon[36269]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:24.573 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:24 smithi165 ceph-mon[36269]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:24.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:24 smithi039 ceph-mon[36429]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:24.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:24 smithi027 ceph-mon[32179]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:26.719 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:26 smithi165 ceph-mon[36269]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:26.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:26 smithi039 ceph-mon[36429]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:26.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:26 smithi027 ceph-mon[32179]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:28.701 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:28 smithi165 ceph-mon[36269]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:28.701 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:28 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-21T03:15:28.701 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:28 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:15:28.701 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:28 smithi165 ceph-mon[36269]: Deploying daemon osd.2 on smithi165 2024-05-21T03:15:28.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:28 smithi039 ceph-mon[36429]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:28.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:28 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-21T03:15:28.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:28 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:15:28.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:28 smithi039 ceph-mon[36429]: Deploying daemon osd.2 on smithi165 2024-05-21T03:15:28.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:28 smithi027 ceph-mon[32179]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:28.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:28 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-21T03:15:28.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:28 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:15:28.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:28 smithi027 ceph-mon[32179]: Deploying daemon osd.2 on smithi165 2024-05-21T03:15:30.526 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:30 smithi165 ceph-mon[36269]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:30.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:30 smithi039 ceph-mon[36429]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:30.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:30 smithi027 ceph-mon[32179]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:32.550 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:32 smithi165 ceph-mon[36269]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:32.729 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:32 smithi027 ceph-mon[32179]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:32.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:32 smithi039 ceph-mon[36429]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:34.409 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:34 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:34.409 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:34 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:34.409 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:34 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:15:34.409 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:34 smithi165 ceph-mon[36269]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:34.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:34 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:34.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:34 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:34.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:34 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:15:34.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:34 smithi027 ceph-mon[32179]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:34.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:34 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:34.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:34 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:34.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:34 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:15:34.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:34 smithi039 ceph-mon[36429]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:36.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:35 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:36.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:35 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:36.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:35 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:15:36.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:35 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:15:36.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:35 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:36.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:35 smithi027 ceph-mon[32179]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:36.244 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:35 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:36.244 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:35 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:36.244 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:35 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:15:36.244 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:35 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:15:36.244 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:35 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:36.244 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:35 smithi165 ceph-mon[36269]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:36.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:35 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:36.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:35 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:36.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:35 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:15:36.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:35 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:15:36.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:35 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:36.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:35 smithi039 ceph-mon[36429]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:36.593 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 2 on host 'smithi165' 2024-05-21T03:15:36.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:36.593+0000 7f7e509c7700 1 -- 172.21.15.165:0/1355004456 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7e4c06c320 con 0x7f7e340b27f0 2024-05-21T03:15:36.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:36.595+0000 7f7e519c9700 1 -- 172.21.15.165:0/1355004456 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f7e340b27f0 msgr2=0x7f7e340b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:15:36.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:36.595+0000 7f7e519c9700 1 --2- 172.21.15.165:0/1355004456 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f7e340b27f0 0x7f7e340b4cb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000b5c0 tx=0x7f7e40005fb0 comp rx=0 tx=0).stop 2024-05-21T03:15:36.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:36.595+0000 7f7e519c9700 1 -- 172.21.15.165:0/1355004456 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e4c1024a0 msgr2=0x7f7e4c199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:15:36.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:36.595+0000 7f7e519c9700 1 --2- 172.21.15.165:0/1355004456 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e4c1024a0 0x7f7e4c199720 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c00d9b0 tx=0x7f7e3c00dcc0 comp rx=0 tx=0).stop 2024-05-21T03:15:36.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:36.595+0000 7f7e519c9700 1 -- 172.21.15.165:0/1355004456 shutdown_connections 2024-05-21T03:15:36.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:36.595+0000 7f7e519c9700 1 --2- 172.21.15.165:0/1355004456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e4c104040 0x7f7e4c1a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:15:36.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:36.595+0000 7f7e519c9700 1 --2- 172.21.15.165:0/1355004456 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f7e340b27f0 0x7f7e340b4cb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:15:36.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:36.595+0000 7f7e519c9700 1 --2- 172.21.15.165:0/1355004456 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e4c1024a0 0x7f7e4c199720 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:15:36.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:36.595+0000 7f7e519c9700 1 --2- 172.21.15.165:0/1355004456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e4c1036a0 0x7f7e4c199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:15:36.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:36.595+0000 7f7e519c9700 1 -- 172.21.15.165:0/1355004456 >> 172.21.15.165:0/1355004456 conn(0x7f7e4c0fda60 msgr2=0x7f7e4c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:15:36.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:36.595+0000 7f7e519c9700 1 -- 172.21.15.165:0/1355004456 shutdown_connections 2024-05-21T03:15:36.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:15:36.595+0000 7f7e519c9700 1 -- 172.21.15.165:0/1355004456 wait complete. 2024-05-21T03:15:37.273 DEBUG:teuthology.orchestra.run.smithi165:osd.2> sudo journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.2.service 2024-05-21T03:15:37.276 INFO:tasks.cephadm:Deploying osd.3 on smithi027 with /dev/vg_nvme/lv_3... 2024-05-21T03:15:37.276 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-21T03:15:37.535 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:15:37.857 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:37 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:37.858 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:37 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:37.858 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:37 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:15:37.858 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:37 smithi027 ceph-mon[32179]: from='osd.2 [v2:172.21.15.165:6800/703322625,v1:172.21.15.165:6801/703322625]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-21T03:15:37.858 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:37 smithi027 ceph-mon[32179]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-21T03:15:38.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:37 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:38.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:37 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:38.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:37 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:15:38.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:37 smithi039 ceph-mon[36429]: from='osd.2 [v2:172.21.15.165:6800/703322625,v1:172.21.15.165:6801/703322625]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-21T03:15:38.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:37 smithi039 ceph-mon[36429]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-21T03:15:38.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:37 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:38.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:37 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:38.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:37 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:15:38.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:37 smithi165 ceph-mon[36269]: from='osd.2 [v2:172.21.15.165:6800/703322625,v1:172.21.15.165:6801/703322625]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-21T03:15:38.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:37 smithi165 ceph-mon[36269]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-21T03:15:38.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:38 smithi027 ceph-mon[32179]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:38.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:38 smithi027 ceph-mon[32179]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-21T03:15:38.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:38 smithi027 ceph-mon[32179]: osdmap e16: 3 total, 2 up, 3 in 2024-05-21T03:15:38.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:38 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T03:15:38.816 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:38 smithi027 ceph-mon[32179]: from='osd.2 [v2:172.21.15.165:6800/703322625,v1:172.21.15.165:6801/703322625]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-21T03:15:38.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:38 smithi027 ceph-mon[32179]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-21T03:15:39.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:38 smithi039 ceph-mon[36429]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:39.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:38 smithi039 ceph-mon[36429]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-21T03:15:39.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:38 smithi039 ceph-mon[36429]: osdmap e16: 3 total, 2 up, 3 in 2024-05-21T03:15:39.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:38 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T03:15:39.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:38 smithi039 ceph-mon[36429]: from='osd.2 [v2:172.21.15.165:6800/703322625,v1:172.21.15.165:6801/703322625]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-21T03:15:39.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:38 smithi039 ceph-mon[36429]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-21T03:15:39.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:38 smithi165 ceph-mon[36269]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:39.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:38 smithi165 ceph-mon[36269]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-21T03:15:39.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:38 smithi165 ceph-mon[36269]: osdmap e16: 3 total, 2 up, 3 in 2024-05-21T03:15:39.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:38 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T03:15:39.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:38 smithi165 ceph-mon[36269]: from='osd.2 [v2:172.21.15.165:6800/703322625,v1:172.21.15.165:6801/703322625]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-21T03:15:39.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:38 smithi165 ceph-mon[36269]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-21T03:15:39.022 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 21 03:15:38 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-2[41409]: 2024-05-21T03:15:38.598+0000 7fc04c24a700 -1 osd.2 0 waiting for initial osdmap 2024-05-21T03:15:39.022 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 21 03:15:38 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-2[41409]: 2024-05-21T03:15:38.602+0000 7fc04803d700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T03:15:39.534 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-21T03:15:39.534 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-21T03:15:39.534 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2024-05-21T03:15:39.534 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2024-05-21T03:15:39.534 INFO:teuthology.orchestra.run.smithi027.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113003 s, 928 MB/s 2024-05-21T03:15:39.534 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2024-05-21T03:15:39.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:39 smithi027 ceph-mon[32179]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-05-21T03:15:39.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:39 smithi027 ceph-mon[32179]: osdmap e17: 3 total, 2 up, 3 in 2024-05-21T03:15:39.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:39 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T03:15:39.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:39 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T03:15:39.959 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:39 smithi165 ceph-mon[36269]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-05-21T03:15:39.959 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:39 smithi165 ceph-mon[36269]: osdmap e17: 3 total, 2 up, 3 in 2024-05-21T03:15:39.959 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:39 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T03:15:39.960 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:39 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T03:15:40.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:39 smithi039 ceph-mon[36429]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-05-21T03:15:40.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:39 smithi039 ceph-mon[36429]: osdmap e17: 3 total, 2 up, 3 in 2024-05-21T03:15:40.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:39 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T03:15:40.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:39 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T03:15:40.333 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi027:vg_nvme/lv_3 2024-05-21T03:15:40.579 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:15:40.909 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:40 smithi027 ceph-mon[32179]: purged_snaps scrub starts 2024-05-21T03:15:40.909 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:40 smithi027 ceph-mon[32179]: purged_snaps scrub ok 2024-05-21T03:15:40.909 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:40 smithi027 ceph-mon[32179]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:40.909 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:40 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T03:15:40.910 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:40 smithi027 ceph-mon[32179]: osd.2 [v2:172.21.15.165:6800/703322625,v1:172.21.15.165:6801/703322625] boot 2024-05-21T03:15:40.910 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:40 smithi027 ceph-mon[32179]: osdmap e18: 3 total, 3 up, 3 in 2024-05-21T03:15:40.910 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:40 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T03:15:40.910 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:40 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:40.910 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:40 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:40.910 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:40 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:15:40.910 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:40 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:40.910 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:40 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:15:40.910 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:40 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:15:40.910 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:40 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:41.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:40 smithi039 ceph-mon[36429]: purged_snaps scrub starts 2024-05-21T03:15:41.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:40 smithi039 ceph-mon[36429]: purged_snaps scrub ok 2024-05-21T03:15:41.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:40 smithi039 ceph-mon[36429]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:41.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:40 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T03:15:41.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:40 smithi039 ceph-mon[36429]: osd.2 [v2:172.21.15.165:6800/703322625,v1:172.21.15.165:6801/703322625] boot 2024-05-21T03:15:41.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:40 smithi039 ceph-mon[36429]: osdmap e18: 3 total, 3 up, 3 in 2024-05-21T03:15:41.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:40 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T03:15:41.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:40 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:41.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:40 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:41.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:40 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:15:41.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:40 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:41.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:40 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:15:41.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:40 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:15:41.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:40 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:41.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:40 smithi165 ceph-mon[36269]: purged_snaps scrub starts 2024-05-21T03:15:41.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:40 smithi165 ceph-mon[36269]: purged_snaps scrub ok 2024-05-21T03:15:41.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:40 smithi165 ceph-mon[36269]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T03:15:41.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:40 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T03:15:41.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:40 smithi165 ceph-mon[36269]: osd.2 [v2:172.21.15.165:6800/703322625,v1:172.21.15.165:6801/703322625] boot 2024-05-21T03:15:41.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:40 smithi165 ceph-mon[36269]: osdmap e18: 3 total, 3 up, 3 in 2024-05-21T03:15:41.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:40 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T03:15:41.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:40 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:41.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:40 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:41.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:40 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:15:41.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:40 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:41.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:40 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:15:41.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:40 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:15:41.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:40 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:41.786 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:41 smithi027 ceph-mon[32179]: Detected new or changed devices on smithi165 2024-05-21T03:15:41.787 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:41 smithi027 ceph-mon[32179]: Adjusting osd_memory_target on smithi165 to 17088M 2024-05-21T03:15:41.787 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:41 smithi027 ceph-mon[32179]: osdmap e19: 3 total, 3 up, 3 in 2024-05-21T03:15:41.787 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:41 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-21T03:15:42.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:41 smithi039 ceph-mon[36429]: Detected new or changed devices on smithi165 2024-05-21T03:15:42.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:41 smithi039 ceph-mon[36429]: Adjusting osd_memory_target on smithi165 to 17088M 2024-05-21T03:15:42.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:41 smithi039 ceph-mon[36429]: osdmap e19: 3 total, 3 up, 3 in 2024-05-21T03:15:42.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:41 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-21T03:15:42.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:41 smithi165 ceph-mon[36269]: Detected new or changed devices on smithi165 2024-05-21T03:15:42.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:41 smithi165 ceph-mon[36269]: Adjusting osd_memory_target on smithi165 to 17088M 2024-05-21T03:15:42.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:41 smithi165 ceph-mon[36269]: osdmap e19: 3 total, 3 up, 3 in 2024-05-21T03:15:42.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:41 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-21T03:15:42.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.310+0000 7ffa22749700 1 -- 172.21.15.27:0/2387300854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa1c1036a0 msgr2=0x7ffa1c103b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:15:42.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.310+0000 7ffa22749700 1 --2- 172.21.15.27:0/2387300854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa1c1036a0 0x7ffa1c103b00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ffa100099c0 tx=0x7ffa10009cd0 comp rx=0 tx=0).stop 2024-05-21T03:15:42.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.310+0000 7ffa22749700 1 -- 172.21.15.27:0/2387300854 shutdown_connections 2024-05-21T03:15:42.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.310+0000 7ffa22749700 1 --2- 172.21.15.27:0/2387300854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa1c104040 0x7ffa1c10a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:15:42.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.310+0000 7ffa22749700 1 --2- 172.21.15.27:0/2387300854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa1c1024a0 0x7ffa1c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:15:42.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.310+0000 7ffa22749700 1 --2- 172.21.15.27:0/2387300854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa1c1036a0 0x7ffa1c103b00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:15:42.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.310+0000 7ffa22749700 1 -- 172.21.15.27:0/2387300854 >> 172.21.15.27:0/2387300854 conn(0x7ffa1c0fda40 msgr2=0x7ffa1c0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:15:42.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.310+0000 7ffa22749700 1 -- 172.21.15.27:0/2387300854 shutdown_connections 2024-05-21T03:15:42.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.310+0000 7ffa22749700 1 -- 172.21.15.27:0/2387300854 wait complete. 2024-05-21T03:15:42.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.310+0000 7ffa22749700 1 Processor -- start 2024-05-21T03:15:42.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.310+0000 7ffa22749700 1 -- start start 2024-05-21T03:15:42.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.311+0000 7ffa22749700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa1c1024a0 0x7ffa1c106430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:15:42.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.311+0000 7ffa22749700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa1c1036a0 0x7ffa1c104a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:15:42.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.311+0000 7ffa22749700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa1c104040 0x7ffa1c104fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:15:42.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.311+0000 7ffa22749700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1c10df20 con 0x7ffa1c1024a0 2024-05-21T03:15:42.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.311+0000 7ffa22749700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1c10dda0 con 0x7ffa1c104040 2024-05-21T03:15:42.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.311+0000 7ffa22749700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1c10e0a0 con 0x7ffa1c1036a0 2024-05-21T03:15:42.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.311+0000 7ffa1bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa1c1024a0 0x7ffa1c106430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:15:42.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.311+0000 7ffa1bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa1c1024a0 0x7ffa1c106430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56156/0 (socket says 172.21.15.27:56156) 2024-05-21T03:15:42.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.311+0000 7ffa1bfff700 1 -- 172.21.15.27:0/922378689 learned_addr learned my addr 172.21.15.27:0/922378689 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:15:42.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.311+0000 7ffa1b7fe700 1 --2- 172.21.15.27:0/922378689 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa1c1036a0 0x7ffa1c104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:15:42.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.311+0000 7ffa20ce6700 1 --2- 172.21.15.27:0/922378689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa1c104040 0x7ffa1c104fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:15:42.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.311+0000 7ffa1bfff700 1 -- 172.21.15.27:0/922378689 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa1c1036a0 msgr2=0x7ffa1c104a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:15:42.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.311+0000 7ffa1bfff700 1 --2- 172.21.15.27:0/922378689 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa1c1036a0 0x7ffa1c104a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:15:42.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.311+0000 7ffa1bfff700 1 -- 172.21.15.27:0/922378689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa1c104040 msgr2=0x7ffa1c104fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:15:42.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.311+0000 7ffa1bfff700 1 --2- 172.21.15.27:0/922378689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa1c104040 0x7ffa1c104fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:15:42.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.311+0000 7ffa1bfff700 1 -- 172.21.15.27:0/922378689 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1c105870 con 0x7ffa1c1024a0 2024-05-21T03:15:42.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.312+0000 7ffa1bfff700 1 --2- 172.21.15.27:0/922378689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa1c1024a0 0x7ffa1c106430 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00c970 tx=0x7ffa0c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:15:42.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.312+0000 7ffa197fa700 1 -- 172.21.15.27:0/922378689 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c004220 con 0x7ffa1c1024a0 2024-05-21T03:15:42.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.312+0000 7ffa22749700 1 -- 172.21.15.27:0/922378689 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa1c1a3870 con 0x7ffa1c1024a0 2024-05-21T03:15:42.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.312+0000 7ffa197fa700 1 -- 172.21.15.27:0/922378689 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa0c007d50 con 0x7ffa1c1024a0 2024-05-21T03:15:42.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.312+0000 7ffa197fa700 1 -- 172.21.15.27:0/922378689 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c003d00 con 0x7ffa1c1024a0 2024-05-21T03:15:42.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.312+0000 7ffa22749700 1 -- 172.21.15.27:0/922378689 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa1c1a3d80 con 0x7ffa1c1024a0 2024-05-21T03:15:42.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.313+0000 7ffa22749700 1 -- 172.21.15.27:0/922378689 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa1c06a230 con 0x7ffa1c1024a0 2024-05-21T03:15:42.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.314+0000 7ffa197fa700 1 -- 172.21.15.27:0/922378689 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7ffa0c0078a0 con 0x7ffa1c1024a0 2024-05-21T03:15:42.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.315+0000 7ffa197fa700 1 --2- 172.21.15.27:0/922378689 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7ffa040b27f0 0x7ffa040b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:15:42.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.315+0000 7ffa1b7fe700 1 --2- 172.21.15.27:0/922378689 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7ffa040b27f0 0x7ffa040b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:15:42.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.315+0000 7ffa197fa700 1 -- 172.21.15.27:0/922378689 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3226+0+0 (secure 0 0 0) 0x7ffa0c0e0ca0 con 0x7ffa1c1024a0 2024-05-21T03:15:42.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.316+0000 7ffa1b7fe700 1 --2- 172.21.15.27:0/922378689 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7ffa040b27f0 0x7ffa040b4cb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ffa10009990 tx=0x7ffa1000b870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:15:42.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.319+0000 7ffa197fa700 1 -- 172.21.15.27:0/922378689 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffa0c014030 con 0x7ffa1c1024a0 2024-05-21T03:15:42.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:42.543+0000 7ffa22749700 1 -- 172.21.15.27:0/922378689 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ffa1c1a4040 con 0x7ffa040b27f0 2024-05-21T03:15:42.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:42 smithi027 ceph-mon[32179]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:15:42.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:42 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-21T03:15:42.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:42 smithi027 ceph-mon[32179]: osdmap e20: 3 total, 3 up, 3 in 2024-05-21T03:15:42.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:42 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-21T03:15:42.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:42 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:15:42.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:42 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:15:42.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:42 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:15:43.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:42 smithi039 ceph-mon[36429]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:15:43.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:42 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-21T03:15:43.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:42 smithi039 ceph-mon[36429]: osdmap e20: 3 total, 3 up, 3 in 2024-05-21T03:15:43.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:42 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-21T03:15:43.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:42 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:15:43.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:42 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:15:43.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:42 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:15:43.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:42 smithi165 ceph-mon[36269]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:15:43.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:42 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-21T03:15:43.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:42 smithi165 ceph-mon[36269]: osdmap e20: 3 total, 3 up, 3 in 2024-05-21T03:15:43.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:42 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-21T03:15:43.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:42 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:15:43.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:42 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:15:43.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:42 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:15:43.360 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:43 smithi027 sudo[49498]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-21T03:15:43.361 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:43 smithi027 sudo[49498]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T03:15:43.361 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:43 smithi027 sudo[49498]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T03:15:43.361 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:43 smithi027 sudo[49498]: pam_unix(sudo:session): session closed for user root 2024-05-21T03:15:43.361 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:43 smithi027 sudo[49501]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-21T03:15:43.361 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:43 smithi027 sudo[49501]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T03:15:43.361 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:43 smithi027 sudo[49501]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T03:15:43.361 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:43 smithi027 sudo[49501]: pam_unix(sudo:session): session closed for user root 2024-05-21T03:15:43.361 INFO:journalctl@ceph.osd.0.smithi027.stdout:May 21 03:15:43 smithi027 sudo[49492]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-21T03:15:43.361 INFO:journalctl@ceph.osd.0.smithi027.stdout:May 21 03:15:43 smithi027 sudo[49492]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T03:15:43.361 INFO:journalctl@ceph.osd.0.smithi027.stdout:May 21 03:15:43 smithi027 sudo[49492]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T03:15:43.361 INFO:journalctl@ceph.osd.0.smithi027.stdout:May 21 03:15:43 smithi027 sudo[49492]: pam_unix(sudo:session): session closed for user root 2024-05-21T03:15:43.362 INFO:journalctl@ceph.osd.0.smithi027.stdout:May 21 03:15:43 smithi027 sudo[49495]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-21T03:15:43.362 INFO:journalctl@ceph.osd.0.smithi027.stdout:May 21 03:15:43 smithi027 sudo[49495]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T03:15:43.362 INFO:journalctl@ceph.osd.0.smithi027.stdout:May 21 03:15:43 smithi027 sudo[49495]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T03:15:43.362 INFO:journalctl@ceph.osd.0.smithi027.stdout:May 21 03:15:43 smithi027 sudo[49495]: pam_unix(sudo:session): session closed for user root 2024-05-21T03:15:43.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:43 smithi039 sudo[45129]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-21T03:15:43.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:43 smithi039 sudo[45129]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T03:15:43.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:43 smithi039 sudo[45129]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T03:15:43.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:43 smithi039 sudo[45129]: pam_unix(sudo:session): session closed for user root 2024-05-21T03:15:43.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:43 smithi039 sudo[45132]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-21T03:15:43.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:43 smithi039 sudo[45132]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T03:15:43.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:43 smithi039 sudo[45132]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T03:15:43.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:43 smithi039 sudo[45132]: pam_unix(sudo:session): session closed for user root 2024-05-21T03:15:43.520 INFO:journalctl@ceph.osd.1.smithi039.stdout:May 21 03:15:43 smithi039 sudo[45123]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-21T03:15:43.520 INFO:journalctl@ceph.osd.1.smithi039.stdout:May 21 03:15:43 smithi039 sudo[45123]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T03:15:43.520 INFO:journalctl@ceph.osd.1.smithi039.stdout:May 21 03:15:43 smithi039 sudo[45123]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T03:15:43.520 INFO:journalctl@ceph.osd.1.smithi039.stdout:May 21 03:15:43 smithi039 sudo[45123]: pam_unix(sudo:session): session closed for user root 2024-05-21T03:15:43.520 INFO:journalctl@ceph.osd.1.smithi039.stdout:May 21 03:15:43 smithi039 sudo[45126]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-21T03:15:43.520 INFO:journalctl@ceph.osd.1.smithi039.stdout:May 21 03:15:43 smithi039 sudo[45126]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T03:15:43.520 INFO:journalctl@ceph.osd.1.smithi039.stdout:May 21 03:15:43 smithi039 sudo[45126]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T03:15:43.520 INFO:journalctl@ceph.osd.1.smithi039.stdout:May 21 03:15:43 smithi039 sudo[45126]: pam_unix(sudo:session): session closed for user root 2024-05-21T03:15:43.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:43 smithi165 sudo[45017]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-21T03:15:43.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:43 smithi165 sudo[45017]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T03:15:43.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:43 smithi165 sudo[45017]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T03:15:43.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:43 smithi165 sudo[45017]: pam_unix(sudo:session): session closed for user root 2024-05-21T03:15:43.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:43 smithi165 sudo[45020]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-21T03:15:43.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:43 smithi165 sudo[45020]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T03:15:43.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:43 smithi165 sudo[45020]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T03:15:43.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:43 smithi165 sudo[45020]: pam_unix(sudo:session): session closed for user root 2024-05-21T03:15:43.522 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 21 03:15:43 smithi165 sudo[45011]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-21T03:15:43.522 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 21 03:15:43 smithi165 sudo[45011]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T03:15:43.522 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 21 03:15:43 smithi165 sudo[45011]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T03:15:43.522 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 21 03:15:43 smithi165 sudo[45011]: pam_unix(sudo:session): session closed for user root 2024-05-21T03:15:43.522 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 21 03:15:43 smithi165 sudo[45014]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-21T03:15:43.522 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 21 03:15:43 smithi165 sudo[45014]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T03:15:43.522 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 21 03:15:43 smithi165 sudo[45014]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T03:15:43.522 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 21 03:15:43 smithi165 sudo[45014]: pam_unix(sudo:session): session closed for user root 2024-05-21T03:15:43.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:43 smithi027 ceph-mon[32179]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:15:43.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:43 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-21T03:15:43.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:43 smithi027 ceph-mon[32179]: osdmap e21: 3 total, 3 up, 3 in 2024-05-21T03:15:43.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:43 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:43.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:43 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:43.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:43 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:43.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:43 smithi027 ceph-mon[32179]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T03:15:43.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:43 smithi027 ceph-mon[32179]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T03:15:43.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:43 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T03:15:43.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:43 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:15:43.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:43 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T03:15:43.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:43 smithi027 ceph-mon[32179]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T03:15:43.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:43 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T03:15:43.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:43 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:15:43.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:43 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T03:15:43.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:43 smithi027 ceph-mon[32179]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T03:15:43.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:43 smithi027 ceph-mon[32179]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T03:15:43.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:43 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T03:15:43.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:43 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:15:43.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:43 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T03:15:43.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:43 smithi027 ceph-mon[32179]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T03:15:44.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:43 smithi039 ceph-mon[36429]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:15:44.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:43 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-21T03:15:44.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:43 smithi039 ceph-mon[36429]: osdmap e21: 3 total, 3 up, 3 in 2024-05-21T03:15:44.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:43 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:44.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:43 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:44.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:43 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:44.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:43 smithi039 ceph-mon[36429]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T03:15:44.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:43 smithi039 ceph-mon[36429]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T03:15:44.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:43 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T03:15:44.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:43 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:15:44.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:43 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T03:15:44.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:43 smithi039 ceph-mon[36429]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T03:15:44.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:43 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T03:15:44.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:43 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:15:44.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:43 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T03:15:44.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:43 smithi039 ceph-mon[36429]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T03:15:44.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:43 smithi039 ceph-mon[36429]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T03:15:44.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:43 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T03:15:44.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:43 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:15:44.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:43 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T03:15:44.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:43 smithi039 ceph-mon[36429]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T03:15:44.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:43 smithi165 ceph-mon[36269]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:15:44.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:43 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-21T03:15:44.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:43 smithi165 ceph-mon[36269]: osdmap e21: 3 total, 3 up, 3 in 2024-05-21T03:15:44.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:43 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:44.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:43 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:44.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:43 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:44.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:43 smithi165 ceph-mon[36269]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T03:15:44.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:43 smithi165 ceph-mon[36269]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T03:15:44.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:43 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T03:15:44.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:43 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:15:44.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:43 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T03:15:44.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:43 smithi165 ceph-mon[36269]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T03:15:44.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:43 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T03:15:44.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:43 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:15:44.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:43 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T03:15:44.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:43 smithi165 ceph-mon[36269]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T03:15:44.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:43 smithi165 ceph-mon[36269]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T03:15:44.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:43 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T03:15:44.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:43 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:15:44.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:43 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T03:15:44.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:43 smithi165 ceph-mon[36269]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T03:15:44.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:15:44.289+0000 7ffa197fa700 1 -- 172.21.15.27:0/922378689 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ffa0c0a5a10 con 0x7ffa1c1024a0 2024-05-21T03:15:44.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:44 smithi027 ceph-mon[32179]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:15:44.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:44 smithi027 ceph-mon[32179]: osdmap e22: 3 total, 3 up, 3 in 2024-05-21T03:15:44.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:44 smithi027 ceph-mon[32179]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-21T03:15:45.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:44 smithi039 ceph-mon[36429]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:15:45.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:44 smithi039 ceph-mon[36429]: osdmap e22: 3 total, 3 up, 3 in 2024-05-21T03:15:45.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:44 smithi039 ceph-mon[36429]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-21T03:15:45.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:44 smithi165 ceph-mon[36269]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:15:45.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:44 smithi165 ceph-mon[36269]: osdmap e22: 3 total, 3 up, 3 in 2024-05-21T03:15:45.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:44 smithi165 ceph-mon[36269]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-21T03:15:45.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:45 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3750787837' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c563b1b2-efcb-4f8c-8121-c1938f83131e"}]: dispatch 2024-05-21T03:15:45.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:45 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3750787837' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c563b1b2-efcb-4f8c-8121-c1938f83131e"}]': finished 2024-05-21T03:15:45.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:45 smithi027 ceph-mon[32179]: osdmap e23: 4 total, 3 up, 4 in 2024-05-21T03:15:45.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:45 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T03:15:46.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:45 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/3750787837' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c563b1b2-efcb-4f8c-8121-c1938f83131e"}]: dispatch 2024-05-21T03:15:46.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:45 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/3750787837' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c563b1b2-efcb-4f8c-8121-c1938f83131e"}]': finished 2024-05-21T03:15:46.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:45 smithi039 ceph-mon[36429]: osdmap e23: 4 total, 3 up, 4 in 2024-05-21T03:15:46.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:45 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T03:15:46.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:45 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/3750787837' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c563b1b2-efcb-4f8c-8121-c1938f83131e"}]: dispatch 2024-05-21T03:15:46.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:45 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/3750787837' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c563b1b2-efcb-4f8c-8121-c1938f83131e"}]': finished 2024-05-21T03:15:46.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:45 smithi165 ceph-mon[36269]: osdmap e23: 4 total, 3 up, 4 in 2024-05-21T03:15:46.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:45 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T03:15:46.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:46 smithi027 ceph-mon[32179]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:15:46.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:46 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3523737804' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:15:47.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:46 smithi039 ceph-mon[36429]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:15:47.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:46 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/3523737804' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:15:47.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:46 smithi165 ceph-mon[36269]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:15:47.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:46 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/3523737804' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:15:48.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:48 smithi027 ceph-mon[32179]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:15:49.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:48 smithi039 ceph-mon[36429]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:15:49.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:48 smithi165 ceph-mon[36269]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:15:50.834 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:50 smithi027 ceph-mon[32179]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:15:51.018 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:50 smithi039 ceph-mon[36429]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:15:51.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:50 smithi165 ceph-mon[36269]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:15:52.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:52 smithi027 ceph-mon[32179]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:15:53.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:52 smithi039 ceph-mon[36429]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:15:53.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:52 smithi165 ceph-mon[36269]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:15:53.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:53 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-21T03:15:53.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:53 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:15:54.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:53 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-21T03:15:54.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:53 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:15:54.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:53 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-21T03:15:54.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:53 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:15:54.848 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:54 smithi027 ceph-mon[32179]: Deploying daemon osd.3 on smithi027 2024-05-21T03:15:54.848 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:54 smithi027 ceph-mon[32179]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:15:54.920 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:54 smithi039 ceph-mon[36429]: Deploying daemon osd.3 on smithi027 2024-05-21T03:15:54.920 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:54 smithi039 ceph-mon[36429]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:15:54.922 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:54 smithi165 ceph-mon[36269]: Deploying daemon osd.3 on smithi027 2024-05-21T03:15:54.922 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:54 smithi165 ceph-mon[36269]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:15:56.776 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:56 smithi027 ceph-mon[32179]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:15:57.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:56 smithi039 ceph-mon[36429]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:15:57.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:56 smithi165 ceph-mon[36269]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:15:58.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:58 smithi027 ceph-mon[32179]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:15:58.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:58 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:58.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:58 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:15:58.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:15:58 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:59.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:58 smithi039 ceph-mon[36429]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:15:59.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:58 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:59.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:58 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:15:59.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:15:58 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:59.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:58 smithi165 ceph-mon[36269]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:15:59.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:58 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:15:59.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:58 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:15:59.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:15:58 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:00.892 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:00 smithi027 ceph-mon[32179]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:16:01.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:00 smithi039 ceph-mon[36429]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:16:01.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:00 smithi165 ceph-mon[36269]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:16:02.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:02.113+0000 7ffa197fa700 1 -- 172.21.15.27:0/922378689 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ffa1c1a4040 con 0x7ffa040b27f0 2024-05-21T03:16:02.114 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 3 on host 'smithi027' 2024-05-21T03:16:02.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:02.116+0000 7ffa22749700 1 -- 172.21.15.27:0/922378689 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7ffa040b27f0 msgr2=0x7ffa040b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:16:02.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:02.116+0000 7ffa22749700 1 --2- 172.21.15.27:0/922378689 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7ffa040b27f0 0x7ffa040b4cb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ffa10009990 tx=0x7ffa1000b870 comp rx=0 tx=0).stop 2024-05-21T03:16:02.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:02.116+0000 7ffa22749700 1 -- 172.21.15.27:0/922378689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa1c1024a0 msgr2=0x7ffa1c106430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:16:02.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:02.116+0000 7ffa22749700 1 --2- 172.21.15.27:0/922378689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa1c1024a0 0x7ffa1c106430 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00c970 tx=0x7ffa0c00cc80 comp rx=0 tx=0).stop 2024-05-21T03:16:02.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:02.116+0000 7ffa22749700 1 -- 172.21.15.27:0/922378689 shutdown_connections 2024-05-21T03:16:02.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:02.116+0000 7ffa22749700 1 --2- 172.21.15.27:0/922378689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa1c104040 0x7ffa1c104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:16:02.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:02.116+0000 7ffa22749700 1 --2- 172.21.15.27:0/922378689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa1c1024a0 0x7ffa1c106430 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:16:02.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:02.116+0000 7ffa22749700 1 --2- 172.21.15.27:0/922378689 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7ffa040b27f0 0x7ffa040b4cb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:16:02.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:02.116+0000 7ffa22749700 1 --2- 172.21.15.27:0/922378689 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa1c1036a0 0x7ffa1c104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:16:02.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:02.116+0000 7ffa22749700 1 -- 172.21.15.27:0/922378689 >> 172.21.15.27:0/922378689 conn(0x7ffa1c0fda40 msgr2=0x7ffa1c0ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:16:02.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:02.116+0000 7ffa22749700 1 -- 172.21.15.27:0/922378689 shutdown_connections 2024-05-21T03:16:02.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:02.116+0000 7ffa22749700 1 -- 172.21.15.27:0/922378689 wait complete. 2024-05-21T03:16:02.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:02 smithi027 ceph-mon[32179]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:16:02.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:02 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:02.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:02 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:02.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:02 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:02.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:02 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:16:02.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:02 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:02.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:02 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:02.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:02 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:02.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:02 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:16:02.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:02 smithi027 ceph-mon[32179]: from='osd.3 [v2:172.21.15.27:6810/1933403135,v1:172.21.15.27:6811/1933403135]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-21T03:16:02.731 DEBUG:teuthology.orchestra.run.smithi027:osd.3> sudo journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.3.service 2024-05-21T03:16:02.734 INFO:tasks.cephadm:Deploying osd.4 on smithi039 with /dev/vg_nvme/lv_3... 2024-05-21T03:16:02.734 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-21T03:16:02.761 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:02 smithi039 ceph-mon[36429]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:16:02.761 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:02 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:02.762 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:02 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:02.762 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:02 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:02.762 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:02 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:16:02.762 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:02 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:02.762 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:02 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:02.762 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:02 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:02.762 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:02 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:16:02.762 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:02 smithi039 ceph-mon[36429]: from='osd.3 [v2:172.21.15.27:6810/1933403135,v1:172.21.15.27:6811/1933403135]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-21T03:16:02.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:02 smithi165 ceph-mon[36269]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:16:02.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:02 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:02.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:02 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:02.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:02 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:02.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:02 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:16:02.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:02 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:02.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:02 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:02.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:02 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:02.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:02 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:16:02.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:02 smithi165 ceph-mon[36269]: from='osd.3 [v2:172.21.15.27:6810/1933403135,v1:172.21.15.27:6811/1933403135]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-21T03:16:02.988 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T03:16:04.316 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:04 smithi039 ceph-mon[36429]: from='osd.3 [v2:172.21.15.27:6810/1933403135,v1:172.21.15.27:6811/1933403135]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-21T03:16:04.316 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:04 smithi039 ceph-mon[36429]: osdmap e24: 4 total, 3 up, 4 in 2024-05-21T03:16:04.316 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:04 smithi039 ceph-mon[36429]: from='osd.3 [v2:172.21.15.27:6810/1933403135,v1:172.21.15.27:6811/1933403135]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-21T03:16:04.316 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:04 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T03:16:04.316 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:04 smithi039 ceph-mon[36429]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:16:04.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:04 smithi027 ceph-mon[32179]: from='osd.3 [v2:172.21.15.27:6810/1933403135,v1:172.21.15.27:6811/1933403135]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-21T03:16:04.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:04 smithi027 ceph-mon[32179]: osdmap e24: 4 total, 3 up, 4 in 2024-05-21T03:16:04.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:04 smithi027 ceph-mon[32179]: from='osd.3 [v2:172.21.15.27:6810/1933403135,v1:172.21.15.27:6811/1933403135]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-21T03:16:04.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:04 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T03:16:04.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:04 smithi027 ceph-mon[32179]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:16:04.367 INFO:journalctl@ceph.osd.3.smithi027.stdout:May 21 03:16:04 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-3[51625]: 2024-05-21T03:16:04.124+0000 7f37de264700 -1 osd.3 0 waiting for initial osdmap 2024-05-21T03:16:04.367 INFO:journalctl@ceph.osd.3.smithi027.stdout:May 21 03:16:04 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-3[51625]: 2024-05-21T03:16:04.133+0000 7f37da057700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T03:16:04.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:04 smithi165 ceph-mon[36269]: from='osd.3 [v2:172.21.15.27:6810/1933403135,v1:172.21.15.27:6811/1933403135]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-21T03:16:04.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:04 smithi165 ceph-mon[36269]: osdmap e24: 4 total, 3 up, 4 in 2024-05-21T03:16:04.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:04 smithi165 ceph-mon[36269]: from='osd.3 [v2:172.21.15.27:6810/1933403135,v1:172.21.15.27:6811/1933403135]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-21T03:16:04.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:04 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T03:16:04.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:04 smithi165 ceph-mon[36269]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-21T03:16:04.872 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-21T03:16:04.872 INFO:teuthology.orchestra.run.smithi039.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-21T03:16:04.872 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10+0 records in 2024-05-21T03:16:04.872 INFO:teuthology.orchestra.run.smithi039.stderr:10+0 records out 2024-05-21T03:16:04.872 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0134342 s, 781 MB/s 2024-05-21T03:16:04.872 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping successful for: 2024-05-21T03:16:05.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:05 smithi027 ceph-mon[32179]: from='osd.3 [v2:172.21.15.27:6810/1933403135,v1:172.21.15.27:6811/1933403135]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-05-21T03:16:05.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:05 smithi027 ceph-mon[32179]: osdmap e25: 4 total, 3 up, 4 in 2024-05-21T03:16:05.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:05 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T03:16:05.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:05 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T03:16:05.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:05 smithi027 ceph-mon[32179]: osd.3 [v2:172.21.15.27:6810/1933403135,v1:172.21.15.27:6811/1933403135] boot 2024-05-21T03:16:05.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:05 smithi027 ceph-mon[32179]: osdmap e26: 4 total, 4 up, 4 in 2024-05-21T03:16:05.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:05 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T03:16:05.487 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:05 smithi039 ceph-mon[36429]: from='osd.3 [v2:172.21.15.27:6810/1933403135,v1:172.21.15.27:6811/1933403135]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-05-21T03:16:05.487 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:05 smithi039 ceph-mon[36429]: osdmap e25: 4 total, 3 up, 4 in 2024-05-21T03:16:05.487 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:05 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T03:16:05.487 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:05 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T03:16:05.487 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:05 smithi039 ceph-mon[36429]: osd.3 [v2:172.21.15.27:6810/1933403135,v1:172.21.15.27:6811/1933403135] boot 2024-05-21T03:16:05.488 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:05 smithi039 ceph-mon[36429]: osdmap e26: 4 total, 4 up, 4 in 2024-05-21T03:16:05.488 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:05 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T03:16:05.515 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi039:vg_nvme/lv_3 2024-05-21T03:16:05.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:05 smithi165 ceph-mon[36269]: from='osd.3 [v2:172.21.15.27:6810/1933403135,v1:172.21.15.27:6811/1933403135]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-05-21T03:16:05.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:05 smithi165 ceph-mon[36269]: osdmap e25: 4 total, 3 up, 4 in 2024-05-21T03:16:05.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:05 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T03:16:05.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:05 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T03:16:05.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:05 smithi165 ceph-mon[36269]: osd.3 [v2:172.21.15.27:6810/1933403135,v1:172.21.15.27:6811/1933403135] boot 2024-05-21T03:16:05.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:05 smithi165 ceph-mon[36269]: osdmap e26: 4 total, 4 up, 4 in 2024-05-21T03:16:05.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:05 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T03:16:05.807 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T03:16:06.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:06 smithi027 ceph-mon[32179]: purged_snaps scrub starts 2024-05-21T03:16:06.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:06 smithi027 ceph-mon[32179]: purged_snaps scrub ok 2024-05-21T03:16:06.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:06 smithi027 ceph-mon[32179]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T03:16:06.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:06 smithi027 ceph-mon[32179]: Detected new or changed devices on smithi027 2024-05-21T03:16:06.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:06 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:06.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:06 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:06.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:06 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:16:06.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:06 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:16:06.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:06 smithi027 ceph-mon[32179]: Adjusting osd_memory_target on smithi027 to 8544M 2024-05-21T03:16:06.428 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:06 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:06.428 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:06 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:06.428 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:06 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:16:06.428 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:06 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:06.428 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:06 smithi027 ceph-mon[32179]: osdmap e27: 4 total, 4 up, 4 in 2024-05-21T03:16:06.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:06 smithi039 ceph-mon[36429]: purged_snaps scrub starts 2024-05-21T03:16:06.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:06 smithi039 ceph-mon[36429]: purged_snaps scrub ok 2024-05-21T03:16:06.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:06 smithi039 ceph-mon[36429]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T03:16:06.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:06 smithi039 ceph-mon[36429]: Detected new or changed devices on smithi027 2024-05-21T03:16:06.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:06 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:06.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:06 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:06.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:06 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:16:06.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:06 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:16:06.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:06 smithi039 ceph-mon[36429]: Adjusting osd_memory_target on smithi027 to 8544M 2024-05-21T03:16:06.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:06 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:06.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:06 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:06.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:06 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:16:06.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:06 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:06.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:06 smithi039 ceph-mon[36429]: osdmap e27: 4 total, 4 up, 4 in 2024-05-21T03:16:06.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:06 smithi165 ceph-mon[36269]: purged_snaps scrub starts 2024-05-21T03:16:06.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:06 smithi165 ceph-mon[36269]: purged_snaps scrub ok 2024-05-21T03:16:06.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:06 smithi165 ceph-mon[36269]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T03:16:06.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:06 smithi165 ceph-mon[36269]: Detected new or changed devices on smithi027 2024-05-21T03:16:06.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:06 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:06.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:06 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:06.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:06 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:16:06.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:06 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:16:06.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:06 smithi165 ceph-mon[36269]: Adjusting osd_memory_target on smithi027 to 8544M 2024-05-21T03:16:06.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:06 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:06.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:06 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:06.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:06 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:16:06.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:06 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:06.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:06 smithi165 ceph-mon[36269]: osdmap e27: 4 total, 4 up, 4 in 2024-05-21T03:16:07.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.223+0000 7fb758d75700 1 -- 172.21.15.39:0/3471414029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7541024c0 msgr2=0x7fb7541028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:16:07.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.223+0000 7fb758d75700 1 --2- 172.21.15.39:0/3471414029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7541024c0 0x7fb7541028e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb7440099c0 tx=0x7fb744009cd0 comp rx=0 tx=0).stop 2024-05-21T03:16:07.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.223+0000 7fb758d75700 1 -- 172.21.15.39:0/3471414029 shutdown_connections 2024-05-21T03:16:07.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.223+0000 7fb758d75700 1 --2- 172.21.15.39:0/3471414029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb754104060 0x7fb75410a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:16:07.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.223+0000 7fb758d75700 1 --2- 172.21.15.39:0/3471414029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7541024c0 0x7fb7541028e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:16:07.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.223+0000 7fb758d75700 1 --2- 172.21.15.39:0/3471414029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7541036c0 0x7fb754103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:16:07.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.223+0000 7fb758d75700 1 -- 172.21.15.39:0/3471414029 >> 172.21.15.39:0/3471414029 conn(0x7fb7540fda60 msgr2=0x7fb7540ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:16:07.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.223+0000 7fb758d75700 1 -- 172.21.15.39:0/3471414029 shutdown_connections 2024-05-21T03:16:07.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.223+0000 7fb758d75700 1 -- 172.21.15.39:0/3471414029 wait complete. 2024-05-21T03:16:07.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.224+0000 7fb758d75700 1 Processor -- start 2024-05-21T03:16:07.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.224+0000 7fb758d75700 1 -- start start 2024-05-21T03:16:07.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.224+0000 7fb758d75700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7541024c0 0x7fb754199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:16:07.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.224+0000 7fb758d75700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7541036c0 0x7fb754199d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:16:07.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.224+0000 7fb758d75700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb754104060 0x7fb7541a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:16:07.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.224+0000 7fb758d75700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb75410e030 con 0x7fb7541036c0 2024-05-21T03:16:07.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.224+0000 7fb758d75700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb75410deb0 con 0x7fb7541024c0 2024-05-21T03:16:07.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.224+0000 7fb758d75700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb75410e1b0 con 0x7fb754104060 2024-05-21T03:16:07.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.224+0000 7fb75259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7541024c0 0x7fb754199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:16:07.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.225+0000 7fb75259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7541024c0 0x7fb754199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34650/0 (socket says 172.21.15.39:34650) 2024-05-21T03:16:07.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.225+0000 7fb75259c700 1 -- 172.21.15.39:0/3935225423 learned_addr learned my addr 172.21.15.39:0/3935225423 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-21T03:16:07.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.225+0000 7fb751d9b700 1 --2- 172.21.15.39:0/3935225423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7541036c0 0x7fb754199d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:16:07.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.225+0000 7fb752d9d700 1 --2- 172.21.15.39:0/3935225423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb754104060 0x7fb7541a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:16:07.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.225+0000 7fb75259c700 1 -- 172.21.15.39:0/3935225423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb754104060 msgr2=0x7fb7541a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:16:07.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.225+0000 7fb75259c700 1 --2- 172.21.15.39:0/3935225423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb754104060 0x7fb7541a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:16:07.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.225+0000 7fb75259c700 1 -- 172.21.15.39:0/3935225423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7541036c0 msgr2=0x7fb754199d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:16:07.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.225+0000 7fb75259c700 1 --2- 172.21.15.39:0/3935225423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7541036c0 0x7fb754199d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:16:07.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.225+0000 7fb75259c700 1 -- 172.21.15.39:0/3935225423 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7541a1430 con 0x7fb7541024c0 2024-05-21T03:16:07.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.225+0000 7fb752d9d700 1 --2- 172.21.15.39:0/3935225423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb754104060 0x7fb7541a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:16:07.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.225+0000 7fb75259c700 1 --2- 172.21.15.39:0/3935225423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7541024c0 0x7fb754199840 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb744009f50 tx=0x7fb7440047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:16:07.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.225+0000 7fb74b7fe700 1 -- 172.21.15.39:0/3935225423 <== mon.2 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb744027070 con 0x7fb7541024c0 2024-05-21T03:16:07.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.225+0000 7fb758d75700 1 -- 172.21.15.39:0/3935225423 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7541a16c0 con 0x7fb7541024c0 2024-05-21T03:16:07.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.225+0000 7fb74b7fe700 1 -- 172.21.15.39:0/3935225423 <== mon.2 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb744022470 con 0x7fb7541024c0 2024-05-21T03:16:07.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.225+0000 7fb74b7fe700 1 -- 172.21.15.39:0/3935225423 <== mon.2 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74400f6c0 con 0x7fb7541024c0 2024-05-21T03:16:07.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.225+0000 7fb758d75700 1 -- 172.21.15.39:0/3935225423 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7541a1c20 con 0x7fb7541024c0 2024-05-21T03:16:07.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.226+0000 7fb758d75700 1 -- 172.21.15.39:0/3935225423 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb75406a230 con 0x7fb7541024c0 2024-05-21T03:16:07.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.228+0000 7fb74b7fe700 1 -- 172.21.15.39:0/3935225423 <== mon.2 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb74402a070 con 0x7fb7541024c0 2024-05-21T03:16:07.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.228+0000 7fb74b7fe700 1 --2- 172.21.15.39:0/3935225423 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb7400b2880 0x7fb7400b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:16:07.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.228+0000 7fb74b7fe700 1 -- 172.21.15.39:0/3935225423 <== mon.2 v2:172.21.15.39:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7fb7440e1f50 con 0x7fb7541024c0 2024-05-21T03:16:07.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.229+0000 7fb751d9b700 1 --2- 172.21.15.39:0/3935225423 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb7400b2880 0x7fb7400b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:16:07.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.229+0000 7fb751d9b700 1 --2- 172.21.15.39:0/3935225423 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb7400b2880 0x7fb7400b4d40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb73c009a10 tx=0x7fb73c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:16:07.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.233+0000 7fb74b7fe700 1 -- 172.21.15.39:0/3935225423 <== mon.2 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb744013070 con 0x7fb7541024c0 2024-05-21T03:16:07.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:07.457+0000 7fb758d75700 1 -- 172.21.15.39:0/3935225423 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb7540ffc40 con 0x7fb7400b2880 2024-05-21T03:16:08.392 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:08 smithi039 ceph-mon[36429]: osdmap e28: 4 total, 4 up, 4 in 2024-05-21T03:16:08.393 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:08 smithi039 ceph-mon[36429]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T03:16:08.393 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:08 smithi039 ceph-mon[36429]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:16:08.393 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:08 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:16:08.393 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:08 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:16:08.393 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:08 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:08.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:08 smithi027 ceph-mon[32179]: osdmap e28: 4 total, 4 up, 4 in 2024-05-21T03:16:08.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:08 smithi027 ceph-mon[32179]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T03:16:08.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:08 smithi027 ceph-mon[32179]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:16:08.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:08 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:16:08.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:08 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:16:08.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:08 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:08.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:08 smithi165 ceph-mon[36269]: osdmap e28: 4 total, 4 up, 4 in 2024-05-21T03:16:08.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:08 smithi165 ceph-mon[36269]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T03:16:08.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:08 smithi165 ceph-mon[36269]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:16:08.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:08 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:16:08.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:08 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:16:08.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:08 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:10.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:10 smithi027 ceph-mon[32179]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T03:16:10.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:10 smithi027 ceph-mon[32179]: from='client.? 172.21.15.39:0/597600578' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c793b87-b045-4b4a-89e7-87ec36deeb0e"}]: dispatch 2024-05-21T03:16:10.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:10 smithi027 ceph-mon[32179]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c793b87-b045-4b4a-89e7-87ec36deeb0e"}]: dispatch 2024-05-21T03:16:10.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:10 smithi027 ceph-mon[32179]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c793b87-b045-4b4a-89e7-87ec36deeb0e"}]': finished 2024-05-21T03:16:10.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:10 smithi027 ceph-mon[32179]: osdmap e29: 5 total, 4 up, 5 in 2024-05-21T03:16:10.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:10 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T03:16:10.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:10 smithi039 ceph-mon[36429]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T03:16:10.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:10 smithi039 ceph-mon[36429]: from='client.? 172.21.15.39:0/597600578' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c793b87-b045-4b4a-89e7-87ec36deeb0e"}]: dispatch 2024-05-21T03:16:10.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:10 smithi039 ceph-mon[36429]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c793b87-b045-4b4a-89e7-87ec36deeb0e"}]: dispatch 2024-05-21T03:16:10.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:10 smithi039 ceph-mon[36429]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c793b87-b045-4b4a-89e7-87ec36deeb0e"}]': finished 2024-05-21T03:16:10.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:10 smithi039 ceph-mon[36429]: osdmap e29: 5 total, 4 up, 5 in 2024-05-21T03:16:10.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:10 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T03:16:10.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:10 smithi165 ceph-mon[36269]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T03:16:10.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:10 smithi165 ceph-mon[36269]: from='client.? 172.21.15.39:0/597600578' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c793b87-b045-4b4a-89e7-87ec36deeb0e"}]: dispatch 2024-05-21T03:16:10.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:10 smithi165 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c793b87-b045-4b4a-89e7-87ec36deeb0e"}]: dispatch 2024-05-21T03:16:10.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:10 smithi165 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c793b87-b045-4b4a-89e7-87ec36deeb0e"}]': finished 2024-05-21T03:16:10.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:10 smithi165 ceph-mon[36269]: osdmap e29: 5 total, 4 up, 5 in 2024-05-21T03:16:10.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:10 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T03:16:11.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:11 smithi027 ceph-mon[32179]: from='client.? 172.21.15.39:0/4134679085' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:16:12.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:11 smithi039 ceph-mon[36429]: from='client.? 172.21.15.39:0/4134679085' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:16:12.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:11 smithi165 ceph-mon[36269]: from='client.? 172.21.15.39:0/4134679085' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:16:12.895 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:12 smithi039 ceph-mon[36429]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T03:16:12.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:12 smithi027 ceph-mon[32179]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T03:16:13.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:12 smithi165 ceph-mon[36269]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T03:16:14.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:14 smithi027 ceph-mon[32179]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-21T03:16:15.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:14 smithi039 ceph-mon[36429]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-21T03:16:15.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:14 smithi165 ceph-mon[36269]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-21T03:16:16.869 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:16 smithi039 ceph-mon[36429]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-21T03:16:16.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:16 smithi027 ceph-mon[32179]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-21T03:16:17.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:16 smithi165 ceph-mon[36269]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-21T03:16:18.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:18 smithi039 ceph-mon[36429]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T03:16:18.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:18 smithi027 ceph-mon[32179]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T03:16:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:18 smithi165 ceph-mon[36269]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T03:16:19.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:19 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-21T03:16:19.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:19 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:19.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:19 smithi039 ceph-mon[36429]: Deploying daemon osd.4 on smithi039 2024-05-21T03:16:19.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:19 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-21T03:16:19.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:19 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:19.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:19 smithi027 ceph-mon[32179]: Deploying daemon osd.4 on smithi039 2024-05-21T03:16:20.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:19 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-21T03:16:20.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:19 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:20.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:19 smithi165 ceph-mon[36269]: Deploying daemon osd.4 on smithi039 2024-05-21T03:16:20.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:20 smithi027 ceph-mon[32179]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T03:16:20.953 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:20 smithi039 ceph-mon[36429]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T03:16:21.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:20 smithi165 ceph-mon[36269]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T03:16:22.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:22 smithi039 ceph-mon[36429]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-21T03:16:22.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:22 smithi027 ceph-mon[32179]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-21T03:16:23.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:22 smithi165 ceph-mon[36269]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-21T03:16:24.920 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:24 smithi039 ceph-mon[36429]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T03:16:24.921 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:24 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:24.921 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:24 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:24.921 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:24 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:16:24.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:24 smithi027 ceph-mon[32179]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T03:16:24.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:24 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:24.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:24 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:24.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:24 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:16:25.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:24 smithi165 ceph-mon[36269]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T03:16:25.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:24 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:25.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:24 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:25.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:24 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:16:26.920 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:26 smithi039 ceph-mon[36429]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T03:16:26.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:26 smithi027 ceph-mon[32179]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T03:16:27.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:26 smithi165 ceph-mon[36269]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T03:16:27.632 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 4 on host 'smithi039' 2024-05-21T03:16:27.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:27.632+0000 7fb74b7fe700 1 -- 172.21.15.39:0/3935225423 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb7540ffc40 con 0x7fb7400b2880 2024-05-21T03:16:27.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:27.635+0000 7fb758d75700 1 -- 172.21.15.39:0/3935225423 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb7400b2880 msgr2=0x7fb7400b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:16:27.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:27.635+0000 7fb758d75700 1 --2- 172.21.15.39:0/3935225423 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb7400b2880 0x7fb7400b4d40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb73c009a10 tx=0x7fb73c009190 comp rx=0 tx=0).stop 2024-05-21T03:16:27.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:27.635+0000 7fb758d75700 1 -- 172.21.15.39:0/3935225423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7541024c0 msgr2=0x7fb754199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:16:27.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:27.635+0000 7fb758d75700 1 --2- 172.21.15.39:0/3935225423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7541024c0 0x7fb754199840 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb744009f50 tx=0x7fb7440047b0 comp rx=0 tx=0).stop 2024-05-21T03:16:27.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:27.635+0000 7fb758d75700 1 -- 172.21.15.39:0/3935225423 shutdown_connections 2024-05-21T03:16:27.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:27.635+0000 7fb758d75700 1 --2- 172.21.15.39:0/3935225423 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb7400b2880 0x7fb7400b4d40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:16:27.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:27.635+0000 7fb758d75700 1 --2- 172.21.15.39:0/3935225423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb754104060 0x7fb7541a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:16:27.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:27.635+0000 7fb758d75700 1 --2- 172.21.15.39:0/3935225423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7541024c0 0x7fb754199840 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:16:27.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:27.635+0000 7fb758d75700 1 --2- 172.21.15.39:0/3935225423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7541036c0 0x7fb754199d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:16:27.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:27.635+0000 7fb758d75700 1 -- 172.21.15.39:0/3935225423 >> 172.21.15.39:0/3935225423 conn(0x7fb7540fda60 msgr2=0x7fb7540ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:16:27.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:27.635+0000 7fb758d75700 1 -- 172.21.15.39:0/3935225423 shutdown_connections 2024-05-21T03:16:27.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:16:27.635+0000 7fb758d75700 1 -- 172.21.15.39:0/3935225423 wait complete. 2024-05-21T03:16:28.202 DEBUG:teuthology.orchestra.run.smithi039:osd.4> sudo journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.4.service 2024-05-21T03:16:28.204 INFO:tasks.cephadm:Deploying osd.5 on smithi165 with /dev/vg_nvme/lv_3... 2024-05-21T03:16:28.205 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-21T03:16:28.453 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T03:16:28.891 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:28 smithi165 ceph-mon[36269]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T03:16:28.891 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:28 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:28.891 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:28 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:28.891 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:28 smithi165 ceph-mon[36269]: from='osd.4 [v2:172.21.15.39:6808/155579331,v1:172.21.15.39:6809/155579331]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-21T03:16:28.891 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:28 smithi165 ceph-mon[36269]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-21T03:16:28.891 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:28 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:28.892 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:28 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:28.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:28 smithi027 ceph-mon[32179]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T03:16:28.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:28 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:28.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:28 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:28.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:28 smithi027 ceph-mon[32179]: from='osd.4 [v2:172.21.15.39:6808/155579331,v1:172.21.15.39:6809/155579331]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-21T03:16:28.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:28 smithi027 ceph-mon[32179]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-21T03:16:28.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:28 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:28.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:28 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:28.951 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:28 smithi039 ceph-mon[36429]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T03:16:28.951 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:28 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:28.951 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:28 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:28.952 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:28 smithi039 ceph-mon[36429]: from='osd.4 [v2:172.21.15.39:6808/155579331,v1:172.21.15.39:6809/155579331]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-21T03:16:28.952 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:28 smithi039 ceph-mon[36429]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-21T03:16:28.952 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:28 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:28.952 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:28 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:29.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:29 smithi165 ceph-mon[36269]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-21T03:16:29.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:29 smithi165 ceph-mon[36269]: osdmap e30: 5 total, 4 up, 5 in 2024-05-21T03:16:29.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:29 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T03:16:29.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:29 smithi165 ceph-mon[36269]: from='osd.4 [v2:172.21.15.39:6808/155579331,v1:172.21.15.39:6809/155579331]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-21T03:16:29.806 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:29 smithi165 ceph-mon[36269]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-21T03:16:29.978 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:29 smithi039 ceph-mon[36429]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-21T03:16:29.978 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:29 smithi039 ceph-mon[36429]: osdmap e30: 5 total, 4 up, 5 in 2024-05-21T03:16:29.978 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:29 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T03:16:29.978 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:29 smithi039 ceph-mon[36429]: from='osd.4 [v2:172.21.15.39:6808/155579331,v1:172.21.15.39:6809/155579331]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-21T03:16:29.979 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:29 smithi039 ceph-mon[36429]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-21T03:16:30.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:29 smithi027 ceph-mon[32179]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-21T03:16:30.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:29 smithi027 ceph-mon[32179]: osdmap e30: 5 total, 4 up, 5 in 2024-05-21T03:16:30.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:29 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T03:16:30.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:29 smithi027 ceph-mon[32179]: from='osd.4 [v2:172.21.15.39:6808/155579331,v1:172.21.15.39:6809/155579331]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-21T03:16:30.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:29 smithi027 ceph-mon[32179]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-21T03:16:30.340 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-21T03:16:30.340 INFO:teuthology.orchestra.run.smithi165.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-21T03:16:30.340 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10+0 records in 2024-05-21T03:16:30.341 INFO:teuthology.orchestra.run.smithi165.stderr:10+0 records out 2024-05-21T03:16:30.341 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0119988 s, 874 MB/s 2024-05-21T03:16:30.341 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping successful for: 2024-05-21T03:16:30.959 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:30 smithi039 ceph-mon[36429]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T03:16:30.959 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:30 smithi039 ceph-mon[36429]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-21T03:16:30.959 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:30 smithi039 ceph-mon[36429]: osdmap e31: 5 total, 4 up, 5 in 2024-05-21T03:16:30.959 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:30 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T03:16:30.959 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:30 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T03:16:30.959 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:30 smithi039 ceph-mon[36429]: from='osd.4 ' entity='osd.4' 2024-05-21T03:16:30.959 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:16:30 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:16:30.629+0000 7fa08f2c2700 -1 osd.4 0 waiting for initial osdmap 2024-05-21T03:16:30.960 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:16:30 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:16:30.651+0000 7fa0870ad700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T03:16:30.975 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:30 smithi165 ceph-mon[36269]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T03:16:30.975 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:30 smithi165 ceph-mon[36269]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-21T03:16:30.975 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:30 smithi165 ceph-mon[36269]: osdmap e31: 5 total, 4 up, 5 in 2024-05-21T03:16:30.976 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:30 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T03:16:30.976 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:30 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T03:16:30.976 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:30 smithi165 ceph-mon[36269]: from='osd.4 ' entity='osd.4' 2024-05-21T03:16:31.004 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi165:vg_nvme/lv_3 2024-05-21T03:16:31.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:30 smithi027 ceph-mon[32179]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T03:16:31.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:30 smithi027 ceph-mon[32179]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-21T03:16:31.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:30 smithi027 ceph-mon[32179]: osdmap e31: 5 total, 4 up, 5 in 2024-05-21T03:16:31.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:30 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T03:16:31.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:30 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T03:16:31.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:30 smithi027 ceph-mon[32179]: from='osd.4 ' entity='osd.4' 2024-05-21T03:16:31.240 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T03:16:32.006 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:31 smithi165 ceph-mon[36269]: purged_snaps scrub starts 2024-05-21T03:16:32.006 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:31 smithi165 ceph-mon[36269]: purged_snaps scrub ok 2024-05-21T03:16:32.007 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:31 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T03:16:32.007 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:31 smithi165 ceph-mon[36269]: osd.4 [v2:172.21.15.39:6808/155579331,v1:172.21.15.39:6809/155579331] boot 2024-05-21T03:16:32.007 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:31 smithi165 ceph-mon[36269]: osdmap e32: 5 total, 5 up, 5 in 2024-05-21T03:16:32.007 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:31 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T03:16:32.007 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:31 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:32.007 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:31 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:32.007 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:31 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:16:32.007 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:31 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:16:32.007 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:31 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:32.007 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:31 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:32.007 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:31 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:16:32.007 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:31 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:32.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:31 smithi039 ceph-mon[36429]: purged_snaps scrub starts 2024-05-21T03:16:32.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:31 smithi039 ceph-mon[36429]: purged_snaps scrub ok 2024-05-21T03:16:32.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:31 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T03:16:32.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:31 smithi039 ceph-mon[36429]: osd.4 [v2:172.21.15.39:6808/155579331,v1:172.21.15.39:6809/155579331] boot 2024-05-21T03:16:32.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:31 smithi039 ceph-mon[36429]: osdmap e32: 5 total, 5 up, 5 in 2024-05-21T03:16:32.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:31 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T03:16:32.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:31 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:32.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:31 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:32.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:31 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:16:32.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:31 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:16:32.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:31 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:32.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:31 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:32.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:31 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:16:32.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:31 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:32.078 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:31 smithi027 ceph-mon[32179]: purged_snaps scrub starts 2024-05-21T03:16:32.078 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:31 smithi027 ceph-mon[32179]: purged_snaps scrub ok 2024-05-21T03:16:32.078 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:31 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T03:16:32.078 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:31 smithi027 ceph-mon[32179]: osd.4 [v2:172.21.15.39:6808/155579331,v1:172.21.15.39:6809/155579331] boot 2024-05-21T03:16:32.079 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:31 smithi027 ceph-mon[32179]: osdmap e32: 5 total, 5 up, 5 in 2024-05-21T03:16:32.079 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:31 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T03:16:32.079 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:31 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:32.079 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:31 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:32.079 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:31 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:16:32.079 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:31 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:16:32.079 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:31 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:32.079 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:31 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:32.079 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:31 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:16:32.079 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:31 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:32.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.754+0000 7f8bf6e12700 1 -- 172.21.15.165:0/3522438812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bf0105140 msgr2=0x7f8bf010b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:16:32.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.754+0000 7f8bf6e12700 1 --2- 172.21.15.165:0/3522438812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bf0105140 0x7f8bf010b9f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8be800b0a0 tx=0x7f8be800b3b0 comp rx=0 tx=0).stop 2024-05-21T03:16:32.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.754+0000 7f8bf6e12700 1 -- 172.21.15.165:0/3522438812 shutdown_connections 2024-05-21T03:16:32.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.754+0000 7f8bf6e12700 1 --2- 172.21.15.165:0/3522438812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bf0105140 0x7f8bf010b9f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:16:32.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.754+0000 7f8bf6e12700 1 --2- 172.21.15.165:0/3522438812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf01035a0 0x7f8bf01039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:16:32.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.754+0000 7f8bf6e12700 1 --2- 172.21.15.165:0/3522438812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bf01047a0 0x7f8bf0104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:16:32.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.754+0000 7f8bf6e12700 1 -- 172.21.15.165:0/3522438812 >> 172.21.15.165:0/3522438812 conn(0x7f8bf00feb40 msgr2=0x7f8bf0100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:16:32.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.755+0000 7f8bf6e12700 1 -- 172.21.15.165:0/3522438812 shutdown_connections 2024-05-21T03:16:32.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.755+0000 7f8bf6e12700 1 -- 172.21.15.165:0/3522438812 wait complete. 2024-05-21T03:16:32.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.755+0000 7f8bf6e12700 1 Processor -- start 2024-05-21T03:16:32.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.755+0000 7f8bf6e12700 1 -- start start 2024-05-21T03:16:32.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.755+0000 7f8bf6e12700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bf01035a0 0x7f8bf019b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:16:32.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.755+0000 7f8bf6e12700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf01047a0 0x7f8bf019bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:16:32.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.755+0000 7f8bf6e12700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bf0105140 0x7f8bf01a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:16:32.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.755+0000 7f8bf6e12700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf010f020 con 0x7f8bf01047a0 2024-05-21T03:16:32.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.755+0000 7f8bf6e12700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf010eea0 con 0x7f8bf01035a0 2024-05-21T03:16:32.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.755+0000 7f8bf6e12700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf010f1a0 con 0x7f8bf0105140 2024-05-21T03:16:32.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.755+0000 7f8bf53af700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bf0105140 0x7f8bf01a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:16:32.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.755+0000 7f8bf53af700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bf0105140 0x7f8bf01a2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44630/0 (socket says 172.21.15.165:44630) 2024-05-21T03:16:32.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.755+0000 7f8bf53af700 1 -- 172.21.15.165:0/2500342751 learned_addr learned my addr 172.21.15.165:0/2500342751 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-05-21T03:16:32.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.755+0000 7f8beffff700 1 --2- 172.21.15.165:0/2500342751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf01047a0 0x7f8bf019bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:16:32.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.756+0000 7f8bf4bae700 1 --2- 172.21.15.165:0/2500342751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bf01035a0 0x7f8bf019b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:16:32.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.756+0000 7f8bf53af700 1 -- 172.21.15.165:0/2500342751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bf01035a0 msgr2=0x7f8bf019b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:16:32.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.756+0000 7f8bf53af700 1 --2- 172.21.15.165:0/2500342751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bf01035a0 0x7f8bf019b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:16:32.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.756+0000 7f8bf53af700 1 -- 172.21.15.165:0/2500342751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf01047a0 msgr2=0x7f8bf019bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:16:32.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.756+0000 7f8bf53af700 1 --2- 172.21.15.165:0/2500342751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf01047a0 0x7f8bf019bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:16:32.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.756+0000 7f8bf53af700 1 -- 172.21.15.165:0/2500342751 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf01a3440 con 0x7f8bf0105140 2024-05-21T03:16:32.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.756+0000 7f8bf4bae700 1 --2- 172.21.15.165:0/2500342751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bf01035a0 0x7f8bf019b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:16:32.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.756+0000 7f8bf53af700 1 --2- 172.21.15.165:0/2500342751 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bf0105140 0x7f8bf01a2da0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8be800b070 tx=0x7f8be8004d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:16:32.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.756+0000 7f8bedffb700 1 -- 172.21.15.165:0/2500342751 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be8021070 con 0x7f8bf0105140 2024-05-21T03:16:32.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.756+0000 7f8bedffb700 1 -- 172.21.15.165:0/2500342751 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8be8007c00 con 0x7f8bf0105140 2024-05-21T03:16:32.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.756+0000 7f8bf6e12700 1 -- 172.21.15.165:0/2500342751 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf01a36d0 con 0x7f8bf0105140 2024-05-21T03:16:32.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.756+0000 7f8bedffb700 1 -- 172.21.15.165:0/2500342751 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be80048a0 con 0x7f8bf0105140 2024-05-21T03:16:32.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.756+0000 7f8bf6e12700 1 -- 172.21.15.165:0/2500342751 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf01a3b20 con 0x7f8bf0105140 2024-05-21T03:16:32.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.757+0000 7f8bf6e12700 1 -- 172.21.15.165:0/2500342751 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bf006a230 con 0x7f8bf0105140 2024-05-21T03:16:32.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.759+0000 7f8bedffb700 1 -- 172.21.15.165:0/2500342751 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8be8003f40 con 0x7f8bf0105140 2024-05-21T03:16:32.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.759+0000 7f8bedffb700 1 --2- 172.21.15.165:0/2500342751 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f8be00b2a20 0x7f8be00b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:16:32.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.759+0000 7f8bedffb700 1 -- 172.21.15.165:0/2500342751 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7f8be800f020 con 0x7f8bf0105140 2024-05-21T03:16:32.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.759+0000 7f8bf4bae700 1 --2- 172.21.15.165:0/2500342751 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f8be00b2a20 0x7f8be00b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:16:32.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.760+0000 7f8bf4bae700 1 --2- 172.21.15.165:0/2500342751 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f8be00b2a20 0x7f8be00b4ee0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0104600 tx=0x7f8be4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:16:32.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.763+0000 7f8bedffb700 1 -- 172.21.15.165:0/2500342751 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8be802a020 con 0x7f8bf0105140 2024-05-21T03:16:32.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:32.975+0000 7f8bf6e12700 1 -- 172.21.15.165:0/2500342751 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f8bf0076e90 con 0x7f8be00b2a20 2024-05-21T03:16:32.975 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:32 smithi165 ceph-mon[36269]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T03:16:32.975 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:32 smithi165 ceph-mon[36269]: Detected new or changed devices on smithi039 2024-05-21T03:16:32.975 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:32 smithi165 ceph-mon[36269]: Adjusting osd_memory_target on smithi039 to 8544M 2024-05-21T03:16:32.975 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:32 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:16:32.975 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:32 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:32.976 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:32 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:16:32.976 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:32 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:33.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:32 smithi039 ceph-mon[36429]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T03:16:33.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:32 smithi039 ceph-mon[36429]: Detected new or changed devices on smithi039 2024-05-21T03:16:33.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:32 smithi039 ceph-mon[36429]: Adjusting osd_memory_target on smithi039 to 8544M 2024-05-21T03:16:33.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:32 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:16:33.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:32 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:33.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:32 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:16:33.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:32 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:33.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:32 smithi027 ceph-mon[32179]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T03:16:33.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:32 smithi027 ceph-mon[32179]: Detected new or changed devices on smithi039 2024-05-21T03:16:33.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:32 smithi027 ceph-mon[32179]: Adjusting osd_memory_target on smithi039 to 8544M 2024-05-21T03:16:33.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:32 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:16:33.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:32 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:33.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:32 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:16:33.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:32 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:34.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:33 smithi027 ceph-mon[32179]: osdmap e33: 5 total, 5 up, 5 in 2024-05-21T03:16:34.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:33 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:16:34.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:33 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:16:34.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:33 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:34.223 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:33 smithi165 ceph-mon[36269]: osdmap e33: 5 total, 5 up, 5 in 2024-05-21T03:16:34.224 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:33 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:16:34.224 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:33 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:16:34.224 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:33 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:34.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:33 smithi039 ceph-mon[36429]: osdmap e33: 5 total, 5 up, 5 in 2024-05-21T03:16:34.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:33 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:16:34.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:33 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:16:34.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:33 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:35.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:34 smithi027 ceph-mon[32179]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:16:35.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:34 smithi027 ceph-mon[32179]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:35.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:34 smithi039 ceph-mon[36429]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:16:35.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:34 smithi039 ceph-mon[36429]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:35.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:34 smithi165 ceph-mon[36269]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:16:35.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:34 smithi165 ceph-mon[36269]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:36.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:35 smithi027 ceph-mon[32179]: from='client.? 172.21.15.165:0/2095520940' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "367eeb5a-9d75-4644-bf46-317c03f33d8a"}]: dispatch 2024-05-21T03:16:36.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:35 smithi027 ceph-mon[32179]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "367eeb5a-9d75-4644-bf46-317c03f33d8a"}]: dispatch 2024-05-21T03:16:36.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:35 smithi027 ceph-mon[32179]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "367eeb5a-9d75-4644-bf46-317c03f33d8a"}]': finished 2024-05-21T03:16:36.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:35 smithi027 ceph-mon[32179]: osdmap e34: 6 total, 5 up, 6 in 2024-05-21T03:16:36.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:35 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T03:16:36.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:35 smithi039 ceph-mon[36429]: from='client.? 172.21.15.165:0/2095520940' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "367eeb5a-9d75-4644-bf46-317c03f33d8a"}]: dispatch 2024-05-21T03:16:36.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:35 smithi039 ceph-mon[36429]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "367eeb5a-9d75-4644-bf46-317c03f33d8a"}]: dispatch 2024-05-21T03:16:36.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:35 smithi039 ceph-mon[36429]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "367eeb5a-9d75-4644-bf46-317c03f33d8a"}]': finished 2024-05-21T03:16:36.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:35 smithi039 ceph-mon[36429]: osdmap e34: 6 total, 5 up, 6 in 2024-05-21T03:16:36.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:35 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T03:16:36.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:35 smithi165 ceph-mon[36269]: from='client.? 172.21.15.165:0/2095520940' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "367eeb5a-9d75-4644-bf46-317c03f33d8a"}]: dispatch 2024-05-21T03:16:36.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:35 smithi165 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "367eeb5a-9d75-4644-bf46-317c03f33d8a"}]: dispatch 2024-05-21T03:16:36.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:35 smithi165 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "367eeb5a-9d75-4644-bf46-317c03f33d8a"}]': finished 2024-05-21T03:16:36.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:35 smithi165 ceph-mon[36269]: osdmap e34: 6 total, 5 up, 6 in 2024-05-21T03:16:36.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:35 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T03:16:37.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:36 smithi027 ceph-mon[32179]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:37.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:36 smithi027 ceph-mon[32179]: from='client.? 172.21.15.165:0/1160299808' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:16:37.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:36 smithi039 ceph-mon[36429]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:37.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:36 smithi039 ceph-mon[36429]: from='client.? 172.21.15.165:0/1160299808' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:16:37.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:36 smithi165 ceph-mon[36269]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:37.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:36 smithi165 ceph-mon[36269]: from='client.? 172.21.15.165:0/1160299808' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:16:38.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:37 smithi039 ceph-mon[36429]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:38.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:37 smithi165 ceph-mon[36269]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:38.426 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:37 smithi027 ceph-mon[32179]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:40.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:40 smithi165 ceph-mon[36269]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:40.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:40 smithi027 ceph-mon[32179]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:40.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:40 smithi039 ceph-mon[36429]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:42.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:42 smithi027 ceph-mon[32179]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:42.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:42 smithi039 ceph-mon[36429]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:42.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:42 smithi165 ceph-mon[36269]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:43.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:43 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-21T03:16:43.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:43 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:43.734 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:43 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-21T03:16:43.734 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:43 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:43.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:43 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-21T03:16:43.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:43 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:44.626 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:44 smithi165 ceph-mon[36269]: Deploying daemon osd.5 on smithi165 2024-05-21T03:16:44.627 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:44 smithi165 ceph-mon[36269]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:44.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:44 smithi027 ceph-mon[32179]: Deploying daemon osd.5 on smithi165 2024-05-21T03:16:44.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:44 smithi027 ceph-mon[32179]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:44.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:44 smithi039 ceph-mon[36429]: Deploying daemon osd.5 on smithi165 2024-05-21T03:16:44.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:44 smithi039 ceph-mon[36429]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:46.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:46 smithi165 ceph-mon[36269]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:46.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:46 smithi027 ceph-mon[32179]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:46.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:46 smithi039 ceph-mon[36429]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:48.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:48 smithi027 ceph-mon[32179]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:48.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:48 smithi039 ceph-mon[36429]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:48.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:48 smithi165 ceph-mon[36269]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:49.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:49 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:49.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:49 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:49.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:49 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:16:49.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:49 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:49.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:49 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:49.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:49 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:16:49.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:49 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:49.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:49 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:49.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:49 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:16:50.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:50 smithi039 ceph-mon[36429]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:50.770 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:50 smithi165 ceph-mon[36269]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:50.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:50 smithi027 ceph-mon[32179]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:52.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:52.091+0000 7f8bedffb700 1 -- 172.21.15.165:0/2500342751 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8bf0076e90 con 0x7f8be00b2a20 2024-05-21T03:16:52.092 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 5 on host 'smithi165' 2024-05-21T03:16:52.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:52.093+0000 7f8bf6e12700 1 -- 172.21.15.165:0/2500342751 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f8be00b2a20 msgr2=0x7f8be00b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:16:52.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:52.093+0000 7f8bf6e12700 1 --2- 172.21.15.165:0/2500342751 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f8be00b2a20 0x7f8be00b4ee0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0104600 tx=0x7f8be4009260 comp rx=0 tx=0).stop 2024-05-21T03:16:52.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:52.093+0000 7f8bf6e12700 1 -- 172.21.15.165:0/2500342751 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bf0105140 msgr2=0x7f8bf01a2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:16:52.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:52.093+0000 7f8bf6e12700 1 --2- 172.21.15.165:0/2500342751 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bf0105140 0x7f8bf01a2da0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8be800b070 tx=0x7f8be8004d70 comp rx=0 tx=0).stop 2024-05-21T03:16:52.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:52.093+0000 7f8bf6e12700 1 -- 172.21.15.165:0/2500342751 shutdown_connections 2024-05-21T03:16:52.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:52.093+0000 7f8bf6e12700 1 --2- 172.21.15.165:0/2500342751 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f8be00b2a20 0x7f8be00b4ee0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:16:52.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:52.093+0000 7f8bf6e12700 1 --2- 172.21.15.165:0/2500342751 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bf0105140 0x7f8bf01a2da0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:16:52.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:52.093+0000 7f8bf6e12700 1 --2- 172.21.15.165:0/2500342751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bf01035a0 0x7f8bf019b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:16:52.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:52.093+0000 7f8bf6e12700 1 --2- 172.21.15.165:0/2500342751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf01047a0 0x7f8bf019bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:16:52.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:52.093+0000 7f8bf6e12700 1 -- 172.21.15.165:0/2500342751 >> 172.21.15.165:0/2500342751 conn(0x7f8bf00feb40 msgr2=0x7f8bf0075770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:16:52.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:52.094+0000 7f8bf6e12700 1 -- 172.21.15.165:0/2500342751 shutdown_connections 2024-05-21T03:16:52.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:16:52.094+0000 7f8bf6e12700 1 -- 172.21.15.165:0/2500342751 wait complete. 2024-05-21T03:16:52.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:52 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:52.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:52 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:52.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:52 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:52.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:52 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:16:52.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:52 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:52.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:52 smithi165 ceph-mon[36269]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:52.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:52 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:52.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:52 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:52.370 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:52 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:16:52.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:52 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:52.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:52 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:52.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:52 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:52.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:52 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:16:52.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:52 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:52.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:52 smithi039 ceph-mon[36429]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:52.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:52 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:52.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:52 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:52.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:52 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:16:52.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:52 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:52.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:52 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:52.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:52 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:52.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:52 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:16:52.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:52 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:52.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:52 smithi027 ceph-mon[32179]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:52.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:52 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:52.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:52 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:52.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:52 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:16:52.691 DEBUG:teuthology.orchestra.run.smithi165:osd.5> sudo journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.5.service 2024-05-21T03:16:52.693 INFO:tasks.cephadm:Deploying osd.6 on smithi027 with /dev/vg_nvme/lv_2... 2024-05-21T03:16:52.693 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-21T03:16:52.953 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:16:53.401 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:53 smithi027 ceph-mon[32179]: from='osd.5 [v2:172.21.15.165:6808/386850244,v1:172.21.15.165:6809/386850244]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-21T03:16:53.401 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:53 smithi027 ceph-mon[32179]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-21T03:16:53.513 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:53 smithi165 ceph-mon[36269]: from='osd.5 [v2:172.21.15.165:6808/386850244,v1:172.21.15.165:6809/386850244]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-21T03:16:53.514 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:53 smithi165 ceph-mon[36269]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-21T03:16:53.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:53 smithi039 ceph-mon[36429]: from='osd.5 [v2:172.21.15.165:6808/386850244,v1:172.21.15.165:6809/386850244]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-21T03:16:53.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:53 smithi039 ceph-mon[36429]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-21T03:16:54.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:54 smithi039 ceph-mon[36429]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-21T03:16:54.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:54 smithi039 ceph-mon[36429]: osdmap e35: 6 total, 5 up, 6 in 2024-05-21T03:16:54.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:54 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T03:16:54.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:54 smithi039 ceph-mon[36429]: from='osd.5 [v2:172.21.15.165:6808/386850244,v1:172.21.15.165:6809/386850244]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-21T03:16:54.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:54 smithi039 ceph-mon[36429]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-21T03:16:54.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:54 smithi039 ceph-mon[36429]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:54.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:54 smithi165 ceph-mon[36269]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-21T03:16:54.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:54 smithi165 ceph-mon[36269]: osdmap e35: 6 total, 5 up, 6 in 2024-05-21T03:16:54.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:54 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T03:16:54.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:54 smithi165 ceph-mon[36269]: from='osd.5 [v2:172.21.15.165:6808/386850244,v1:172.21.15.165:6809/386850244]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-21T03:16:54.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:54 smithi165 ceph-mon[36269]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-21T03:16:54.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:54 smithi165 ceph-mon[36269]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:54.525 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:54 smithi027 ceph-mon[32179]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-21T03:16:54.526 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:54 smithi027 ceph-mon[32179]: osdmap e35: 6 total, 5 up, 6 in 2024-05-21T03:16:54.526 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:54 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T03:16:54.526 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:54 smithi027 ceph-mon[32179]: from='osd.5 [v2:172.21.15.165:6808/386850244,v1:172.21.15.165:6809/386850244]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-21T03:16:54.526 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:54 smithi027 ceph-mon[32179]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-21T03:16:54.526 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:54 smithi027 ceph-mon[32179]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:55.128 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-21T03:16:55.128 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-21T03:16:55.128 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2024-05-21T03:16:55.128 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2024-05-21T03:16:55.128 INFO:teuthology.orchestra.run.smithi027.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0121568 s, 863 MB/s 2024-05-21T03:16:55.128 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2024-05-21T03:16:55.254 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:16:55 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:16:55.206+0000 7f49bb79c700 -1 osd.5 0 waiting for initial osdmap 2024-05-21T03:16:55.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:55 smithi027 ceph-mon[32179]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-05-21T03:16:55.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:55 smithi027 ceph-mon[32179]: osdmap e36: 6 total, 5 up, 6 in 2024-05-21T03:16:55.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:55 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T03:16:55.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:55 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T03:16:55.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:55 smithi027 ceph-mon[32179]: from='osd.5 ' entity='osd.5' 2024-05-21T03:16:55.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:55 smithi039 ceph-mon[36429]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-05-21T03:16:55.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:55 smithi039 ceph-mon[36429]: osdmap e36: 6 total, 5 up, 6 in 2024-05-21T03:16:55.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:55 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T03:16:55.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:55 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T03:16:55.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:55 smithi039 ceph-mon[36429]: from='osd.5 ' entity='osd.5' 2024-05-21T03:16:55.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:55 smithi165 ceph-mon[36269]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-05-21T03:16:55.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:55 smithi165 ceph-mon[36269]: osdmap e36: 6 total, 5 up, 6 in 2024-05-21T03:16:55.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:55 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T03:16:55.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:55 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T03:16:55.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:55 smithi165 ceph-mon[36269]: from='osd.5 ' entity='osd.5' 2024-05-21T03:16:55.522 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:16:55 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:16:55.358+0000 7f49b3587700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T03:16:55.737 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi027:vg_nvme/lv_2 2024-05-21T03:16:55.996 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:16:56.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:56 smithi027 ceph-mon[32179]: purged_snaps scrub starts 2024-05-21T03:16:56.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:56 smithi027 ceph-mon[32179]: purged_snaps scrub ok 2024-05-21T03:16:56.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:56 smithi027 ceph-mon[32179]: osdmap e37: 6 total, 5 up, 6 in 2024-05-21T03:16:56.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:56 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T03:16:56.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:56 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T03:16:56.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:56 smithi027 ceph-mon[32179]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:56.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:56 smithi027 ceph-mon[32179]: Detected new or changed devices on smithi165 2024-05-21T03:16:56.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:56 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:56.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:56 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:56.367 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:56 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:16:56.368 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:56 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:16:56.368 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:56 smithi027 ceph-mon[32179]: Adjusting osd_memory_target on smithi165 to 8544M 2024-05-21T03:16:56.368 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:56 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:56.368 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:56 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:56.368 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:56 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:16:56.368 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:56 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:56.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:56 smithi165 ceph-mon[36269]: purged_snaps scrub starts 2024-05-21T03:16:56.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:56 smithi165 ceph-mon[36269]: purged_snaps scrub ok 2024-05-21T03:16:56.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:56 smithi165 ceph-mon[36269]: osdmap e37: 6 total, 5 up, 6 in 2024-05-21T03:16:56.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:56 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T03:16:56.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:56 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T03:16:56.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:56 smithi165 ceph-mon[36269]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:56.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:56 smithi165 ceph-mon[36269]: Detected new or changed devices on smithi165 2024-05-21T03:16:56.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:56 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:56.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:56 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:56.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:56 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:16:56.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:56 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:16:56.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:56 smithi165 ceph-mon[36269]: Adjusting osd_memory_target on smithi165 to 8544M 2024-05-21T03:16:56.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:56 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:56.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:56 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:56.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:56 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:16:56.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:56 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:56.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:56 smithi039 ceph-mon[36429]: purged_snaps scrub starts 2024-05-21T03:16:56.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:56 smithi039 ceph-mon[36429]: purged_snaps scrub ok 2024-05-21T03:16:56.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:56 smithi039 ceph-mon[36429]: osdmap e37: 6 total, 5 up, 6 in 2024-05-21T03:16:56.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:56 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T03:16:56.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:56 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T03:16:56.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:56 smithi039 ceph-mon[36429]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-21T03:16:56.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:56 smithi039 ceph-mon[36429]: Detected new or changed devices on smithi165 2024-05-21T03:16:56.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:56 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:56.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:56 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:56.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:56 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:16:56.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:56 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:16:56.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:56 smithi039 ceph-mon[36429]: Adjusting osd_memory_target on smithi165 to 8544M 2024-05-21T03:16:56.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:56 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:56.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:56 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:56.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:56 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:16:56.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:56 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:16:57.466 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:57 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T03:16:57.466 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:57 smithi027 ceph-mon[32179]: osd.5 [v2:172.21.15.165:6808/386850244,v1:172.21.15.165:6809/386850244] boot 2024-05-21T03:16:57.466 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:57 smithi027 ceph-mon[32179]: osdmap e38: 6 total, 6 up, 6 in 2024-05-21T03:16:57.466 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:57 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T03:16:57.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.686+0000 7fae23e96700 1 -- 172.21.15.27:0/1799896424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae1c1035c0 msgr2=0x7fae1c1039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:16:57.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.686+0000 7fae23e96700 1 --2- 172.21.15.27:0/1799896424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae1c1035c0 0x7fae1c1039e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fae180099c0 tx=0x7fae18009cd0 comp rx=0 tx=0).stop 2024-05-21T03:16:57.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.686+0000 7fae23e96700 1 -- 172.21.15.27:0/1799896424 shutdown_connections 2024-05-21T03:16:57.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.686+0000 7fae23e96700 1 --2- 172.21.15.27:0/1799896424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae1c105160 0x7fae1c10ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:16:57.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.686+0000 7fae23e96700 1 --2- 172.21.15.27:0/1799896424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae1c1035c0 0x7fae1c1039e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:16:57.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.686+0000 7fae23e96700 1 --2- 172.21.15.27:0/1799896424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae1c1047c0 0x7fae1c104c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:16:57.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.686+0000 7fae23e96700 1 -- 172.21.15.27:0/1799896424 >> 172.21.15.27:0/1799896424 conn(0x7fae1c0feb60 msgr2=0x7fae1c100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:16:57.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.686+0000 7fae23e96700 1 -- 172.21.15.27:0/1799896424 shutdown_connections 2024-05-21T03:16:57.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.686+0000 7fae23e96700 1 -- 172.21.15.27:0/1799896424 wait complete. 2024-05-21T03:16:57.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.686+0000 7fae23e96700 1 Processor -- start 2024-05-21T03:16:57.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.686+0000 7fae23e96700 1 -- start start 2024-05-21T03:16:57.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.687+0000 7fae23e96700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae1c1035c0 0x7fae1c0772e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:16:57.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.687+0000 7fae23e96700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae1c1047c0 0x7fae1c075930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:16:57.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.687+0000 7fae23e96700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae1c105160 0x7fae1c075e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:16:57.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.687+0000 7fae23e96700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c10f0a0 con 0x7fae1c105160 2024-05-21T03:16:57.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.687+0000 7fae23e96700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c10ef20 con 0x7fae1c1035c0 2024-05-21T03:16:57.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.687+0000 7fae23e96700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c10f220 con 0x7fae1c1047c0 2024-05-21T03:16:57.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.687+0000 7fae22433700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae1c105160 0x7fae1c075e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:16:57.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.687+0000 7fae22433700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae1c105160 0x7fae1c075e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57734/0 (socket says 172.21.15.27:57734) 2024-05-21T03:16:57.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.687+0000 7fae22433700 1 -- 172.21.15.27:0/4000756663 learned_addr learned my addr 172.21.15.27:0/4000756663 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:16:57.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.687+0000 7fae21c32700 1 --2- 172.21.15.27:0/4000756663 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae1c1035c0 0x7fae1c0772e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:16:57.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.687+0000 7fae21431700 1 --2- 172.21.15.27:0/4000756663 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae1c1047c0 0x7fae1c075930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:16:57.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.687+0000 7fae22433700 1 -- 172.21.15.27:0/4000756663 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae1c1047c0 msgr2=0x7fae1c075930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:16:57.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.687+0000 7fae22433700 1 --2- 172.21.15.27:0/4000756663 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae1c1047c0 0x7fae1c075930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:16:57.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.687+0000 7fae22433700 1 -- 172.21.15.27:0/4000756663 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae1c1035c0 msgr2=0x7fae1c0772e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:16:57.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.687+0000 7fae22433700 1 --2- 172.21.15.27:0/4000756663 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae1c1035c0 0x7fae1c0772e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:16:57.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.687+0000 7fae22433700 1 -- 172.21.15.27:0/4000756663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae1c076720 con 0x7fae1c105160 2024-05-21T03:16:57.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.687+0000 7fae22433700 1 --2- 172.21.15.27:0/4000756663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae1c105160 0x7fae1c075e70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fae14007ee0 tx=0x7fae1400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:16:57.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.687+0000 7fae12ffd700 1 -- 172.21.15.27:0/4000756663 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1400dbd0 con 0x7fae1c105160 2024-05-21T03:16:57.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.688+0000 7fae23e96700 1 -- 172.21.15.27:0/4000756663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae1c1a38a0 con 0x7fae1c105160 2024-05-21T03:16:57.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.688+0000 7fae12ffd700 1 -- 172.21.15.27:0/4000756663 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae1400dd70 con 0x7fae1c105160 2024-05-21T03:16:57.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.688+0000 7fae12ffd700 1 -- 172.21.15.27:0/4000756663 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae14005990 con 0x7fae1c105160 2024-05-21T03:16:57.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.688+0000 7fae23e96700 1 -- 172.21.15.27:0/4000756663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae1c1a3db0 con 0x7fae1c105160 2024-05-21T03:16:57.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.688+0000 7fae23e96700 1 -- 172.21.15.27:0/4000756663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae1c06a230 con 0x7fae1c105160 2024-05-21T03:16:57.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.690+0000 7fae12ffd700 1 -- 172.21.15.27:0/4000756663 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fae1401a020 con 0x7fae1c105160 2024-05-21T03:16:57.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.691+0000 7fae12ffd700 1 --2- 172.21.15.27:0/4000756663 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fae080b2880 0x7fae080b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:16:57.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.691+0000 7fae21c32700 1 --2- 172.21.15.27:0/4000756663 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fae080b2880 0x7fae080b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:16:57.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.691+0000 7fae12ffd700 1 -- 172.21.15.27:0/4000756663 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4533+0+0 (secure 0 0 0) 0x7fae140e1150 con 0x7fae1c105160 2024-05-21T03:16:57.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.691+0000 7fae21c32700 1 --2- 172.21.15.27:0/4000756663 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fae080b2880 0x7fae080b4d40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fae18009f50 tx=0x7fae1800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:16:57.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.694+0000 7fae12ffd700 1 -- 172.21.15.27:0/4000756663 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fae14010070 con 0x7fae1c105160 2024-05-21T03:16:57.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:57 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T03:16:57.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:57 smithi039 ceph-mon[36429]: osd.5 [v2:172.21.15.165:6808/386850244,v1:172.21.15.165:6809/386850244] boot 2024-05-21T03:16:57.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:57 smithi039 ceph-mon[36429]: osdmap e38: 6 total, 6 up, 6 in 2024-05-21T03:16:57.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:57 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T03:16:57.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:57 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T03:16:57.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:57 smithi165 ceph-mon[36269]: osd.5 [v2:172.21.15.165:6808/386850244,v1:172.21.15.165:6809/386850244] boot 2024-05-21T03:16:57.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:57 smithi165 ceph-mon[36269]: osdmap e38: 6 total, 6 up, 6 in 2024-05-21T03:16:57.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:57 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T03:16:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:16:57.918+0000 7fae23e96700 1 -- 172.21.15.27:0/4000756663 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fae1c063a40 con 0x7fae080b2880 2024-05-21T03:16:58.556 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:58 smithi027 ceph-mon[32179]: osdmap e39: 6 total, 6 up, 6 in 2024-05-21T03:16:58.556 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:58 smithi027 ceph-mon[32179]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-21T03:16:58.556 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:58 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:16:58.556 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:58 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:16:58.556 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:58 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:58.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:58 smithi039 ceph-mon[36429]: osdmap e39: 6 total, 6 up, 6 in 2024-05-21T03:16:58.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:58 smithi039 ceph-mon[36429]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-21T03:16:58.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:58 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:16:58.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:58 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:16:58.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:58 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:58.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:58 smithi165 ceph-mon[36269]: osdmap e39: 6 total, 6 up, 6 in 2024-05-21T03:16:58.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:58 smithi165 ceph-mon[36269]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-21T03:16:58.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:58 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:16:58.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:58 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:16:58.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:58 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:16:59.584 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:59 smithi027 ceph-mon[32179]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:16:59.584 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:16:59 smithi027 ceph-mon[32179]: osdmap e40: 6 total, 6 up, 6 in 2024-05-21T03:16:59.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:59 smithi039 ceph-mon[36429]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:16:59.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:16:59 smithi039 ceph-mon[36429]: osdmap e40: 6 total, 6 up, 6 in 2024-05-21T03:16:59.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:59 smithi165 ceph-mon[36269]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:16:59.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:16:59 smithi165 ceph-mon[36269]: osdmap e40: 6 total, 6 up, 6 in 2024-05-21T03:17:00.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:00 smithi027 ceph-mon[32179]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-21T03:17:00.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:00 smithi039 ceph-mon[36429]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-21T03:17:00.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:00 smithi165 ceph-mon[36269]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-21T03:17:01.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:01 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2333544728' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e000167f-24fd-4115-83f0-8afbfa8d8991"}]: dispatch 2024-05-21T03:17:01.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:01 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2333544728' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e000167f-24fd-4115-83f0-8afbfa8d8991"}]': finished 2024-05-21T03:17:01.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:01 smithi027 ceph-mon[32179]: osdmap e41: 7 total, 6 up, 7 in 2024-05-21T03:17:01.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:01 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T03:17:01.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:01 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/1086358734' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:17:01.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:01 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/2333544728' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e000167f-24fd-4115-83f0-8afbfa8d8991"}]: dispatch 2024-05-21T03:17:01.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:01 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/2333544728' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e000167f-24fd-4115-83f0-8afbfa8d8991"}]': finished 2024-05-21T03:17:01.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:01 smithi039 ceph-mon[36429]: osdmap e41: 7 total, 6 up, 7 in 2024-05-21T03:17:01.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:01 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T03:17:01.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:01 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/1086358734' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:17:01.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:01 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/2333544728' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e000167f-24fd-4115-83f0-8afbfa8d8991"}]: dispatch 2024-05-21T03:17:01.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:01 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/2333544728' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e000167f-24fd-4115-83f0-8afbfa8d8991"}]': finished 2024-05-21T03:17:01.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:01 smithi165 ceph-mon[36269]: osdmap e41: 7 total, 6 up, 7 in 2024-05-21T03:17:01.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:01 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T03:17:01.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:01 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/1086358734' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:17:03.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:02 smithi039 ceph-mon[36429]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-21T03:17:03.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:02 smithi165 ceph-mon[36269]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-21T03:17:03.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:02 smithi027 ceph-mon[32179]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-21T03:17:04.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:04 smithi027 ceph-mon[32179]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T03:17:05.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:04 smithi039 ceph-mon[36429]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T03:17:05.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:04 smithi165 ceph-mon[36269]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T03:17:06.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:06 smithi027 ceph-mon[32179]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T03:17:07.018 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:06 smithi039 ceph-mon[36429]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T03:17:07.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:06 smithi165 ceph-mon[36269]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T03:17:09.000 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:08 smithi027 ceph-mon[32179]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-21T03:17:09.000 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:08 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-21T03:17:09.000 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:08 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:17:09.018 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:08 smithi039 ceph-mon[36429]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-21T03:17:09.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:08 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-21T03:17:09.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:08 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:17:09.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:08 smithi165 ceph-mon[36269]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-21T03:17:09.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:08 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-21T03:17:09.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:08 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:17:09.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:09 smithi027 ceph-mon[32179]: Deploying daemon osd.6 on smithi027 2024-05-21T03:17:10.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:09 smithi039 ceph-mon[36429]: Deploying daemon osd.6 on smithi027 2024-05-21T03:17:10.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:09 smithi165 ceph-mon[36269]: Deploying daemon osd.6 on smithi027 2024-05-21T03:17:11.003 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:10 smithi027 ceph-mon[32179]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T03:17:11.018 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:10 smithi039 ceph-mon[36429]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T03:17:11.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:10 smithi165 ceph-mon[36269]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T03:17:12.905 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:12 smithi027 ceph-mon[32179]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-05-21T03:17:13.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:12 smithi039 ceph-mon[36429]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-05-21T03:17:13.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:12 smithi165 ceph-mon[36269]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-05-21T03:17:13.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:13 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:17:13.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:13 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:13.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:13 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:14.018 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:13 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:17:14.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:13 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:14.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:13 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:14.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:13 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:17:14.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:13 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:14.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:13 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:14.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:14 smithi027 ceph-mon[32179]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-21T03:17:15.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:14 smithi039 ceph-mon[36429]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-21T03:17:15.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:14 smithi165 ceph-mon[36269]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-21T03:17:16.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:16 smithi027 ceph-mon[32179]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-21T03:17:16.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:16 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:16.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:16 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:16.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:16 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:17:16.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:16 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:17:17.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:16 smithi039 ceph-mon[36429]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-21T03:17:17.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:16 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:17.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:16 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:17.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:16 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:17:17.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:16 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:17:17.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:16 smithi165 ceph-mon[36269]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-21T03:17:17.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:16 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:17.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:16 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:17.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:16 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:17:17.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:16 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:17:17.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:17:17.687+0000 7fae12ffd700 1 -- 172.21.15.27:0/4000756663 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fae1c063a40 con 0x7fae080b2880 2024-05-21T03:17:17.687 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 6 on host 'smithi027' 2024-05-21T03:17:17.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:17:17.689+0000 7fae23e96700 1 -- 172.21.15.27:0/4000756663 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fae080b2880 msgr2=0x7fae080b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:17:17.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:17:17.689+0000 7fae23e96700 1 --2- 172.21.15.27:0/4000756663 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fae080b2880 0x7fae080b4d40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fae18009f50 tx=0x7fae1800b540 comp rx=0 tx=0).stop 2024-05-21T03:17:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:17:17.689+0000 7fae23e96700 1 -- 172.21.15.27:0/4000756663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae1c105160 msgr2=0x7fae1c075e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:17:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:17:17.689+0000 7fae23e96700 1 --2- 172.21.15.27:0/4000756663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae1c105160 0x7fae1c075e70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fae14007ee0 tx=0x7fae1400d3b0 comp rx=0 tx=0).stop 2024-05-21T03:17:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:17:17.689+0000 7fae23e96700 1 -- 172.21.15.27:0/4000756663 shutdown_connections 2024-05-21T03:17:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:17:17.689+0000 7fae23e96700 1 --2- 172.21.15.27:0/4000756663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae1c105160 0x7fae1c075e70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:17:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:17:17.689+0000 7fae23e96700 1 --2- 172.21.15.27:0/4000756663 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae1c1035c0 0x7fae1c0772e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:17:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:17:17.689+0000 7fae23e96700 1 --2- 172.21.15.27:0/4000756663 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fae080b2880 0x7fae080b4d40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:17:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:17:17.689+0000 7fae23e96700 1 --2- 172.21.15.27:0/4000756663 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae1c1047c0 0x7fae1c075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:17:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:17:17.689+0000 7fae23e96700 1 -- 172.21.15.27:0/4000756663 >> 172.21.15.27:0/4000756663 conn(0x7fae1c0feb60 msgr2=0x7fae1c107b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:17:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:17:17.689+0000 7fae23e96700 1 -- 172.21.15.27:0/4000756663 shutdown_connections 2024-05-21T03:17:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:17:17.689+0000 7fae23e96700 1 -- 172.21.15.27:0/4000756663 wait complete. 2024-05-21T03:17:17.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:17 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:17.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:17 smithi027 ceph-mon[32179]: from='osd.6 [v2:172.21.15.27:6818/3531376679,v1:172.21.15.27:6819/3531376679]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-21T03:17:17.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:17 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:17.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:17 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:17:17.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:17 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:18.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:17 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:18.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:17 smithi039 ceph-mon[36429]: from='osd.6 [v2:172.21.15.27:6818/3531376679,v1:172.21.15.27:6819/3531376679]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-21T03:17:18.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:17 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:18.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:17 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:17:18.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:17 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:18.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:17 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:18.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:17 smithi165 ceph-mon[36269]: from='osd.6 [v2:172.21.15.27:6818/3531376679,v1:172.21.15.27:6819/3531376679]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-21T03:17:18.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:17 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:18.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:17 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:17:18.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:17 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:18.442 DEBUG:teuthology.orchestra.run.smithi027:osd.6> sudo journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.6.service 2024-05-21T03:17:18.444 INFO:tasks.cephadm:Deploying osd.7 on smithi039 with /dev/vg_nvme/lv_2... 2024-05-21T03:17:18.444 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-21T03:17:18.701 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T03:17:19.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:18 smithi039 ceph-mon[36429]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-21T03:17:19.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:18 smithi039 ceph-mon[36429]: from='osd.6 [v2:172.21.15.27:6818/3531376679,v1:172.21.15.27:6819/3531376679]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-21T03:17:19.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:18 smithi039 ceph-mon[36429]: osdmap e42: 7 total, 6 up, 7 in 2024-05-21T03:17:19.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:18 smithi039 ceph-mon[36429]: from='osd.6 [v2:172.21.15.27:6818/3531376679,v1:172.21.15.27:6819/3531376679]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-21T03:17:19.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:18 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T03:17:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:18 smithi165 ceph-mon[36269]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-21T03:17:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:18 smithi165 ceph-mon[36269]: from='osd.6 [v2:172.21.15.27:6818/3531376679,v1:172.21.15.27:6819/3531376679]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-21T03:17:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:18 smithi165 ceph-mon[36269]: osdmap e42: 7 total, 6 up, 7 in 2024-05-21T03:17:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:18 smithi165 ceph-mon[36269]: from='osd.6 [v2:172.21.15.27:6818/3531376679,v1:172.21.15.27:6819/3531376679]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-21T03:17:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:18 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T03:17:19.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:18 smithi027 ceph-mon[32179]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-21T03:17:19.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:18 smithi027 ceph-mon[32179]: from='osd.6 [v2:172.21.15.27:6818/3531376679,v1:172.21.15.27:6819/3531376679]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-21T03:17:19.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:18 smithi027 ceph-mon[32179]: osdmap e42: 7 total, 6 up, 7 in 2024-05-21T03:17:19.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:18 smithi027 ceph-mon[32179]: from='osd.6 [v2:172.21.15.27:6818/3531376679,v1:172.21.15.27:6819/3531376679]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-21T03:17:19.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:18 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T03:17:19.177 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:17:18 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:17:18.741+0000 7f95be2b2700 -1 osd.6 0 waiting for initial osdmap 2024-05-21T03:17:19.177 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:17:18 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:17:18.753+0000 7f95ba0a5700 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T03:17:19.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:19 smithi027 ceph-mon[32179]: from='osd.6 [v2:172.21.15.27:6818/3531376679,v1:172.21.15.27:6819/3531376679]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-05-21T03:17:19.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:19 smithi027 ceph-mon[32179]: osdmap e43: 7 total, 6 up, 7 in 2024-05-21T03:17:19.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:19 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T03:17:19.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:19 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T03:17:20.010 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:19 smithi039 ceph-mon[36429]: from='osd.6 [v2:172.21.15.27:6818/3531376679,v1:172.21.15.27:6819/3531376679]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-05-21T03:17:20.010 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:19 smithi039 ceph-mon[36429]: osdmap e43: 7 total, 6 up, 7 in 2024-05-21T03:17:20.010 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:19 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T03:17:20.010 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:19 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T03:17:20.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:19 smithi165 ceph-mon[36269]: from='osd.6 [v2:172.21.15.27:6818/3531376679,v1:172.21.15.27:6819/3531376679]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-05-21T03:17:20.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:19 smithi165 ceph-mon[36269]: osdmap e43: 7 total, 6 up, 7 in 2024-05-21T03:17:20.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:19 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T03:17:20.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:19 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T03:17:20.580 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-21T03:17:20.580 INFO:teuthology.orchestra.run.smithi039.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-21T03:17:20.581 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10+0 records in 2024-05-21T03:17:20.581 INFO:teuthology.orchestra.run.smithi039.stderr:10+0 records out 2024-05-21T03:17:20.581 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0144442 s, 726 MB/s 2024-05-21T03:17:20.581 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping successful for: 2024-05-21T03:17:20.891 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:20 smithi039 ceph-mon[36429]: purged_snaps scrub starts 2024-05-21T03:17:20.892 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:20 smithi039 ceph-mon[36429]: purged_snaps scrub ok 2024-05-21T03:17:20.892 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:20 smithi039 ceph-mon[36429]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-21T03:17:20.892 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:20 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T03:17:20.892 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:20 smithi039 ceph-mon[36429]: osd.6 [v2:172.21.15.27:6818/3531376679,v1:172.21.15.27:6819/3531376679] boot 2024-05-21T03:17:20.892 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:20 smithi039 ceph-mon[36429]: osdmap e44: 7 total, 7 up, 7 in 2024-05-21T03:17:20.892 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:20 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T03:17:21.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:20 smithi165 ceph-mon[36269]: purged_snaps scrub starts 2024-05-21T03:17:21.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:20 smithi165 ceph-mon[36269]: purged_snaps scrub ok 2024-05-21T03:17:21.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:20 smithi165 ceph-mon[36269]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-21T03:17:21.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:20 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T03:17:21.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:20 smithi165 ceph-mon[36269]: osd.6 [v2:172.21.15.27:6818/3531376679,v1:172.21.15.27:6819/3531376679] boot 2024-05-21T03:17:21.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:20 smithi165 ceph-mon[36269]: osdmap e44: 7 total, 7 up, 7 in 2024-05-21T03:17:21.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:20 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T03:17:21.146 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:20 smithi027 ceph-mon[32179]: purged_snaps scrub starts 2024-05-21T03:17:21.146 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:20 smithi027 ceph-mon[32179]: purged_snaps scrub ok 2024-05-21T03:17:21.146 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:20 smithi027 ceph-mon[32179]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-21T03:17:21.146 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:20 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T03:17:21.146 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:20 smithi027 ceph-mon[32179]: osd.6 [v2:172.21.15.27:6818/3531376679,v1:172.21.15.27:6819/3531376679] boot 2024-05-21T03:17:21.146 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:20 smithi027 ceph-mon[32179]: osdmap e44: 7 total, 7 up, 7 in 2024-05-21T03:17:21.146 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:20 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T03:17:21.196 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi039:vg_nvme/lv_2 2024-05-21T03:17:21.443 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T03:17:21.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:21 smithi027 ceph-mon[32179]: osdmap e45: 7 total, 7 up, 7 in 2024-05-21T03:17:21.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:21 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:21.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:21 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:21.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:21 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:17:21.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:21 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:17:21.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:21 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:17:21.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:21 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:21.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:21 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:17:21.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:21 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:17:21.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:21 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:22.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:21 smithi039 ceph-mon[36429]: osdmap e45: 7 total, 7 up, 7 in 2024-05-21T03:17:22.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:21 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:22.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:21 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:22.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:21 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:17:22.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:21 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:17:22.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:21 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:17:22.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:21 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:22.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:21 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:17:22.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:21 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:17:22.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:21 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:22.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:21 smithi165 ceph-mon[36269]: osdmap e45: 7 total, 7 up, 7 in 2024-05-21T03:17:22.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:21 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:22.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:21 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:22.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:21 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:17:22.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:21 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:17:22.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:21 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:17:22.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:21 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:22.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:21 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:17:22.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:21 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:17:22.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:21 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:22.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.977+0000 7fb92bf7c700 1 -- 172.21.15.39:0/215600218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9241047c0 msgr2=0x7fb924104c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:17:22.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.977+0000 7fb92bf7c700 1 --2- 172.21.15.39:0/215600218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9241047c0 0x7fb924104c20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb9180099c0 tx=0x7fb918009cd0 comp rx=0 tx=0).stop 2024-05-21T03:17:22.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.977+0000 7fb92bf7c700 1 -- 172.21.15.39:0/215600218 shutdown_connections 2024-05-21T03:17:22.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.977+0000 7fb92bf7c700 1 --2- 172.21.15.39:0/215600218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb924105160 0x7fb92410ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:17:22.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.977+0000 7fb92bf7c700 1 --2- 172.21.15.39:0/215600218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9241035c0 0x7fb9241039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:17:22.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.977+0000 7fb92bf7c700 1 --2- 172.21.15.39:0/215600218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9241047c0 0x7fb924104c20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:17:22.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.977+0000 7fb92bf7c700 1 -- 172.21.15.39:0/215600218 >> 172.21.15.39:0/215600218 conn(0x7fb9240feb60 msgr2=0x7fb924100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:17:22.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.977+0000 7fb92bf7c700 1 -- 172.21.15.39:0/215600218 shutdown_connections 2024-05-21T03:17:22.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.977+0000 7fb92bf7c700 1 -- 172.21.15.39:0/215600218 wait complete. 2024-05-21T03:17:22.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.978+0000 7fb92bf7c700 1 Processor -- start 2024-05-21T03:17:22.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.978+0000 7fb92bf7c700 1 -- start start 2024-05-21T03:17:22.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.978+0000 7fb92bf7c700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9241035c0 0x7fb9240772e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:17:22.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.978+0000 7fb92bf7c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9241047c0 0x7fb924075930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:17:22.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.978+0000 7fb92bf7c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb924105160 0x7fb924075e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:17:22.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.978+0000 7fb92bf7c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb92410f050 con 0x7fb9241047c0 2024-05-21T03:17:22.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.978+0000 7fb92bf7c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb92410eed0 con 0x7fb924105160 2024-05-21T03:17:22.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.978+0000 7fb92bf7c700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb92410f1d0 con 0x7fb9241035c0 2024-05-21T03:17:22.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.978+0000 7fb92a519700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb924105160 0x7fb924075e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:17:22.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.978+0000 7fb929517700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9241047c0 0x7fb924075930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:17:22.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.978+0000 7fb92a519700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb924105160 0x7fb924075e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58766/0 (socket says 172.21.15.39:58766) 2024-05-21T03:17:22.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.978+0000 7fb92a519700 1 -- 172.21.15.39:0/2299053294 learned_addr learned my addr 172.21.15.39:0/2299053294 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-21T03:17:22.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.978+0000 7fb929d18700 1 --2- 172.21.15.39:0/2299053294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9241035c0 0x7fb9240772e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:17:22.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.978+0000 7fb92a519700 1 -- 172.21.15.39:0/2299053294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9241035c0 msgr2=0x7fb9240772e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:17:22.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.978+0000 7fb92a519700 1 --2- 172.21.15.39:0/2299053294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9241035c0 0x7fb9240772e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:17:22.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.978+0000 7fb92a519700 1 -- 172.21.15.39:0/2299053294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9241047c0 msgr2=0x7fb924075930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:17:22.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.978+0000 7fb92a519700 1 --2- 172.21.15.39:0/2299053294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9241047c0 0x7fb924075930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:17:22.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.978+0000 7fb92a519700 1 -- 172.21.15.39:0/2299053294 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb924076720 con 0x7fb924105160 2024-05-21T03:17:22.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.979+0000 7fb92a519700 1 --2- 172.21.15.39:0/2299053294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb924105160 0x7fb924075e70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb91c00c900 tx=0x7fb91c00cc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:17:22.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.979+0000 7fb922ffd700 1 -- 172.21.15.39:0/2299053294 <== mon.2 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c0078d0 con 0x7fb924105160 2024-05-21T03:17:22.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.979+0000 7fb922ffd700 1 -- 172.21.15.39:0/2299053294 <== mon.2 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb91c00f450 con 0x7fb924105160 2024-05-21T03:17:22.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.979+0000 7fb92bf7c700 1 -- 172.21.15.39:0/2299053294 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9241a37d0 con 0x7fb924105160 2024-05-21T03:17:22.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.979+0000 7fb922ffd700 1 -- 172.21.15.39:0/2299053294 <== mon.2 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c018620 con 0x7fb924105160 2024-05-21T03:17:22.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.979+0000 7fb92bf7c700 1 -- 172.21.15.39:0/2299053294 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9241a3c90 con 0x7fb924105160 2024-05-21T03:17:22.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.980+0000 7fb92bf7c700 1 -- 172.21.15.39:0/2299053294 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb92406a230 con 0x7fb924105160 2024-05-21T03:17:22.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.982+0000 7fb922ffd700 1 -- 172.21.15.39:0/2299053294 <== mon.2 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb91c007a70 con 0x7fb924105160 2024-05-21T03:17:22.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.983+0000 7fb922ffd700 1 --2- 172.21.15.39:0/2299053294 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb90c0b2880 0x7fb90c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:17:22.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.983+0000 7fb922ffd700 1 -- 172.21.15.39:0/2299053294 <== mon.2 v2:172.21.15.39:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 4965+0+0 (secure 0 0 0) 0x7fb91c0e1410 con 0x7fb924105160 2024-05-21T03:17:22.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.983+0000 7fb929d18700 1 --2- 172.21.15.39:0/2299053294 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb90c0b2880 0x7fb90c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:17:22.983 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:22 smithi039 ceph-mon[36429]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:22.983 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:22 smithi039 ceph-mon[36429]: Detected new or changed devices on smithi027 2024-05-21T03:17:22.983 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:22 smithi039 ceph-mon[36429]: Adjusting osd_memory_target on smithi027 to 5696M 2024-05-21T03:17:22.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.983+0000 7fb929d18700 1 --2- 172.21.15.39:0/2299053294 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb90c0b2880 0x7fb90c0b4d40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb924104620 tx=0x7fb914009180 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:17:22.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:22.987+0000 7fb922ffd700 1 -- 172.21.15.39:0/2299053294 <== mon.2 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb91c014030 con 0x7fb924105160 2024-05-21T03:17:23.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:22 smithi165 ceph-mon[36269]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:23.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:22 smithi165 ceph-mon[36269]: Detected new or changed devices on smithi027 2024-05-21T03:17:23.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:22 smithi165 ceph-mon[36269]: Adjusting osd_memory_target on smithi027 to 5696M 2024-05-21T03:17:23.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:22 smithi027 ceph-mon[32179]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:23.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:22 smithi027 ceph-mon[32179]: Detected new or changed devices on smithi027 2024-05-21T03:17:23.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:22 smithi027 ceph-mon[32179]: Adjusting osd_memory_target on smithi027 to 5696M 2024-05-21T03:17:23.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:23.209+0000 7fb92bf7c700 1 -- 172.21.15.39:0/2299053294 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb924063980 con 0x7fb90c0b2880 2024-05-21T03:17:24.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:23 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:17:24.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:23 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:17:24.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:23 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:17:24.064 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:23 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:17:24.064 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:23 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:17:24.064 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:23 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:17:24.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:23 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:17:24.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:23 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:17:24.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:23 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:17:24.994 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:24 smithi039 ceph-mon[36429]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:17:24.995 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:24 smithi039 ceph-mon[36429]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:25.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:24 smithi165 ceph-mon[36269]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:17:25.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:24 smithi165 ceph-mon[36269]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:25.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:24 smithi027 ceph-mon[32179]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:17:25.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:24 smithi027 ceph-mon[32179]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:27.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:26 smithi027 ceph-mon[32179]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:27.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:26 smithi027 ceph-mon[32179]: from='client.? 172.21.15.39:0/1256869007' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b8d486b-868e-4f33-98a1-b99686adeaaf"}]: dispatch 2024-05-21T03:17:27.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:26 smithi027 ceph-mon[32179]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b8d486b-868e-4f33-98a1-b99686adeaaf"}]: dispatch 2024-05-21T03:17:27.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:26 smithi027 ceph-mon[32179]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b8d486b-868e-4f33-98a1-b99686adeaaf"}]': finished 2024-05-21T03:17:27.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:26 smithi027 ceph-mon[32179]: osdmap e46: 8 total, 7 up, 8 in 2024-05-21T03:17:27.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:26 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T03:17:27.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:26 smithi027 ceph-mon[32179]: from='client.? 172.21.15.39:0/2111137692' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:17:27.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:26 smithi039 ceph-mon[36429]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:27.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:26 smithi039 ceph-mon[36429]: from='client.? 172.21.15.39:0/1256869007' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b8d486b-868e-4f33-98a1-b99686adeaaf"}]: dispatch 2024-05-21T03:17:27.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:26 smithi039 ceph-mon[36429]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b8d486b-868e-4f33-98a1-b99686adeaaf"}]: dispatch 2024-05-21T03:17:27.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:26 smithi039 ceph-mon[36429]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b8d486b-868e-4f33-98a1-b99686adeaaf"}]': finished 2024-05-21T03:17:27.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:26 smithi039 ceph-mon[36429]: osdmap e46: 8 total, 7 up, 8 in 2024-05-21T03:17:27.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:26 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T03:17:27.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:26 smithi039 ceph-mon[36429]: from='client.? 172.21.15.39:0/2111137692' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:17:27.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:26 smithi165 ceph-mon[36269]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:27.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:26 smithi165 ceph-mon[36269]: from='client.? 172.21.15.39:0/1256869007' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b8d486b-868e-4f33-98a1-b99686adeaaf"}]: dispatch 2024-05-21T03:17:27.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:26 smithi165 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b8d486b-868e-4f33-98a1-b99686adeaaf"}]: dispatch 2024-05-21T03:17:27.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:26 smithi165 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b8d486b-868e-4f33-98a1-b99686adeaaf"}]': finished 2024-05-21T03:17:27.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:26 smithi165 ceph-mon[36269]: osdmap e46: 8 total, 7 up, 8 in 2024-05-21T03:17:27.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:26 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T03:17:27.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:26 smithi165 ceph-mon[36269]: from='client.? 172.21.15.39:0/2111137692' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:17:29.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:28 smithi027 ceph-mon[32179]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:29.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:28 smithi039 ceph-mon[36429]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:29.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:28 smithi165 ceph-mon[36269]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:31.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:30 smithi039 ceph-mon[36429]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:31.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:30 smithi027 ceph-mon[32179]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:31.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:30 smithi165 ceph-mon[36269]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:33.131 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:32 smithi039 ceph-mon[36429]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:33.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:32 smithi027 ceph-mon[32179]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:33.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:32 smithi165 ceph-mon[36269]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:35.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:34 smithi027 ceph-mon[32179]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:35.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:34 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-21T03:17:35.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:34 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:17:35.188 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:34 smithi039 ceph-mon[36429]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:35.188 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:34 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-21T03:17:35.188 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:34 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:17:35.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:34 smithi165 ceph-mon[36269]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:35.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:34 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-21T03:17:35.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:34 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:17:36.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:35 smithi039 ceph-mon[36429]: Deploying daemon osd.7 on smithi039 2024-05-21T03:17:36.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:35 smithi165 ceph-mon[36269]: Deploying daemon osd.7 on smithi039 2024-05-21T03:17:36.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:35 smithi027 ceph-mon[32179]: Deploying daemon osd.7 on smithi039 2024-05-21T03:17:36.883 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:36 smithi039 ceph-mon[36429]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:37.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:36 smithi027 ceph-mon[32179]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:37.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:36 smithi165 ceph-mon[36269]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:39.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:38 smithi039 ceph-mon[36429]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:39.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:38 smithi027 ceph-mon[32179]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:39.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:38 smithi165 ceph-mon[36269]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:40.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:40 smithi039 ceph-mon[36429]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:40.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:40 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:40.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:40 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:40.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:40 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:17:40.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:40 smithi165 ceph-mon[36269]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:40.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:40 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:40.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:40 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:40.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:40 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:17:40.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:40 smithi027 ceph-mon[32179]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:40.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:40 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:40.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:40 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:40.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:40 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:17:42.620 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:42 smithi039 ceph-mon[36429]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:42.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:42.674+0000 7fb922ffd700 1 -- 172.21.15.39:0/2299053294 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb924063980 con 0x7fb90c0b2880 2024-05-21T03:17:42.676 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 7 on host 'smithi039' 2024-05-21T03:17:42.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:42.677+0000 7fb92bf7c700 1 -- 172.21.15.39:0/2299053294 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb90c0b2880 msgr2=0x7fb90c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:17:42.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:42.677+0000 7fb92bf7c700 1 --2- 172.21.15.39:0/2299053294 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb90c0b2880 0x7fb90c0b4d40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb924104620 tx=0x7fb914009180 comp rx=0 tx=0).stop 2024-05-21T03:17:42.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:42.677+0000 7fb92bf7c700 1 -- 172.21.15.39:0/2299053294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb924105160 msgr2=0x7fb924075e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:17:42.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:42.677+0000 7fb92bf7c700 1 --2- 172.21.15.39:0/2299053294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb924105160 0x7fb924075e70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb91c00c900 tx=0x7fb91c00cc10 comp rx=0 tx=0).stop 2024-05-21T03:17:42.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:42.677+0000 7fb92bf7c700 1 -- 172.21.15.39:0/2299053294 shutdown_connections 2024-05-21T03:17:42.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:42.677+0000 7fb92bf7c700 1 --2- 172.21.15.39:0/2299053294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb924105160 0x7fb924075e70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:17:42.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:42.677+0000 7fb92bf7c700 1 --2- 172.21.15.39:0/2299053294 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb90c0b2880 0x7fb90c0b4d40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:17:42.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:42.677+0000 7fb92bf7c700 1 --2- 172.21.15.39:0/2299053294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9241035c0 0x7fb9240772e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:17:42.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:42.677+0000 7fb92bf7c700 1 --2- 172.21.15.39:0/2299053294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9241047c0 0x7fb924075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:17:42.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:42.677+0000 7fb92bf7c700 1 -- 172.21.15.39:0/2299053294 >> 172.21.15.39:0/2299053294 conn(0x7fb9240feb60 msgr2=0x7fb924107b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:17:42.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:42.677+0000 7fb92bf7c700 1 -- 172.21.15.39:0/2299053294 shutdown_connections 2024-05-21T03:17:42.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:17:42.677+0000 7fb92bf7c700 1 -- 172.21.15.39:0/2299053294 wait complete. 2024-05-21T03:17:42.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:42 smithi165 ceph-mon[36269]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:42.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:42 smithi027 ceph-mon[32179]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:43.593 DEBUG:teuthology.orchestra.run.smithi039:osd.7> sudo journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.7.service 2024-05-21T03:17:43.596 INFO:tasks.cephadm:Deploying osd.8 on smithi165 with /dev/vg_nvme/lv_2... 2024-05-21T03:17:43.597 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-21T03:17:43.845 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T03:17:43.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:43 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:43.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:43 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:43.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:43 smithi027 ceph-mon[32179]: from='osd.7 [v2:172.21.15.39:6816/1550035852,v1:172.21.15.39:6817/1550035852]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-21T03:17:43.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:43 smithi027 ceph-mon[32179]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-21T03:17:43.967 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:43 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:43.967 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:43 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:43.967 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:43 smithi165 ceph-mon[36269]: from='osd.7 [v2:172.21.15.39:6816/1550035852,v1:172.21.15.39:6817/1550035852]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-21T03:17:43.967 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:43 smithi165 ceph-mon[36269]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-21T03:17:44.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:43 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:44.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:43 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:44.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:43 smithi039 ceph-mon[36429]: from='osd.7 [v2:172.21.15.39:6816/1550035852,v1:172.21.15.39:6817/1550035852]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-21T03:17:44.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:43 smithi039 ceph-mon[36429]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-21T03:17:44.987 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:44 smithi165 ceph-mon[36269]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:44.987 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:44 smithi165 ceph-mon[36269]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-21T03:17:44.987 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:44 smithi165 ceph-mon[36269]: osdmap e47: 8 total, 7 up, 8 in 2024-05-21T03:17:44.987 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:44 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T03:17:44.987 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:44 smithi165 ceph-mon[36269]: from='osd.7 [v2:172.21.15.39:6816/1550035852,v1:172.21.15.39:6817/1550035852]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-21T03:17:44.987 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:44 smithi165 ceph-mon[36269]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-21T03:17:44.987 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:44 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:44.987 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:44 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:45.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:44 smithi039 ceph-mon[36429]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:45.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:44 smithi039 ceph-mon[36429]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-21T03:17:45.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:44 smithi039 ceph-mon[36429]: osdmap e47: 8 total, 7 up, 8 in 2024-05-21T03:17:45.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:44 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T03:17:45.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:44 smithi039 ceph-mon[36429]: from='osd.7 [v2:172.21.15.39:6816/1550035852,v1:172.21.15.39:6817/1550035852]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-21T03:17:45.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:44 smithi039 ceph-mon[36429]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-21T03:17:45.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:44 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:45.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:44 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:45.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:44 smithi027 ceph-mon[32179]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:45.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:44 smithi027 ceph-mon[32179]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-21T03:17:45.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:44 smithi027 ceph-mon[32179]: osdmap e47: 8 total, 7 up, 8 in 2024-05-21T03:17:45.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:44 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T03:17:45.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:44 smithi027 ceph-mon[32179]: from='osd.7 [v2:172.21.15.39:6816/1550035852,v1:172.21.15.39:6817/1550035852]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-21T03:17:45.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:44 smithi027 ceph-mon[32179]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-21T03:17:45.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:44 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:45.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:44 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:45.599 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:17:45 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:17:45.484+0000 7f55704b1700 -1 osd.7 0 waiting for initial osdmap 2024-05-21T03:17:45.599 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:17:45 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:17:45.527+0000 7f556829c700 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T03:17:45.656 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-21T03:17:45.656 INFO:teuthology.orchestra.run.smithi165.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-21T03:17:45.657 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10+0 records in 2024-05-21T03:17:45.657 INFO:teuthology.orchestra.run.smithi165.stderr:10+0 records out 2024-05-21T03:17:45.657 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130404 s, 804 MB/s 2024-05-21T03:17:45.657 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping successful for: 2024-05-21T03:17:45.987 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:45 smithi165 ceph-mon[36269]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-21T03:17:45.987 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:45 smithi165 ceph-mon[36269]: osdmap e48: 8 total, 7 up, 8 in 2024-05-21T03:17:45.987 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:45 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T03:17:45.987 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:45 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T03:17:45.987 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:45 smithi165 ceph-mon[36269]: from='osd.7 ' entity='osd.7' 2024-05-21T03:17:46.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:45 smithi039 ceph-mon[36429]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-21T03:17:46.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:45 smithi039 ceph-mon[36429]: osdmap e48: 8 total, 7 up, 8 in 2024-05-21T03:17:46.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:45 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T03:17:46.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:45 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T03:17:46.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:45 smithi039 ceph-mon[36429]: from='osd.7 ' entity='osd.7' 2024-05-21T03:17:46.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:45 smithi027 ceph-mon[32179]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-21T03:17:46.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:45 smithi027 ceph-mon[32179]: osdmap e48: 8 total, 7 up, 8 in 2024-05-21T03:17:46.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:45 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T03:17:46.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:45 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T03:17:46.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:45 smithi027 ceph-mon[32179]: from='osd.7 ' entity='osd.7' 2024-05-21T03:17:46.255 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi165:vg_nvme/lv_2 2024-05-21T03:17:46.504 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T03:17:46.795 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:46 smithi165 ceph-mon[36269]: purged_snaps scrub starts 2024-05-21T03:17:46.795 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:46 smithi165 ceph-mon[36269]: purged_snaps scrub ok 2024-05-21T03:17:46.795 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:46 smithi165 ceph-mon[36269]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:46.796 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:46 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T03:17:46.796 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:46 smithi165 ceph-mon[36269]: osd.7 [v2:172.21.15.39:6816/1550035852,v1:172.21.15.39:6817/1550035852] boot 2024-05-21T03:17:46.796 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:46 smithi165 ceph-mon[36269]: osdmap e49: 8 total, 8 up, 8 in 2024-05-21T03:17:46.796 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:46 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T03:17:46.989 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:46 smithi039 ceph-mon[36429]: purged_snaps scrub starts 2024-05-21T03:17:46.989 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:46 smithi039 ceph-mon[36429]: purged_snaps scrub ok 2024-05-21T03:17:46.989 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:46 smithi039 ceph-mon[36429]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:46.989 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:46 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T03:17:46.989 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:46 smithi039 ceph-mon[36429]: osd.7 [v2:172.21.15.39:6816/1550035852,v1:172.21.15.39:6817/1550035852] boot 2024-05-21T03:17:46.989 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:46 smithi039 ceph-mon[36429]: osdmap e49: 8 total, 8 up, 8 in 2024-05-21T03:17:46.989 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:46 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T03:17:47.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:46 smithi027 ceph-mon[32179]: purged_snaps scrub starts 2024-05-21T03:17:47.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:46 smithi027 ceph-mon[32179]: purged_snaps scrub ok 2024-05-21T03:17:47.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:46 smithi027 ceph-mon[32179]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-21T03:17:47.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:46 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T03:17:47.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:46 smithi027 ceph-mon[32179]: osd.7 [v2:172.21.15.39:6816/1550035852,v1:172.21.15.39:6817/1550035852] boot 2024-05-21T03:17:47.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:46 smithi027 ceph-mon[32179]: osdmap e49: 8 total, 8 up, 8 in 2024-05-21T03:17:47.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:46 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T03:17:47.748 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:47 smithi165 ceph-mon[36269]: osdmap e50: 8 total, 8 up, 8 in 2024-05-21T03:17:47.748 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:47 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:47.748 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:47 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:47.748 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:47 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:17:47.748 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:47 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:17:47.748 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:47 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:17:47.748 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:47 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:47.748 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:47 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:17:47.749 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:47 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:17:47.749 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:47 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:47.749 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:47 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:17:47.749 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:47 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:17:47.749 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:47 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:17:47.749 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:47 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:47.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.964+0000 7f3db97a0700 1 -- 172.21.15.165:0/707460409 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3db41027a0 msgr2=0x7f3db4102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:17:47.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.964+0000 7f3db97a0700 1 --2- 172.21.15.165:0/707460409 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3db41027a0 0x7f3db4102c20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009990 tx=0x7f3da8009ca0 comp rx=0 tx=0).stop 2024-05-21T03:17:47.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.964+0000 7f3db97a0700 1 -- 172.21.15.165:0/707460409 shutdown_connections 2024-05-21T03:17:47.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.964+0000 7f3db97a0700 1 --2- 172.21.15.165:0/707460409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db4103160 0x7f3db41035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:17:47.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.964+0000 7f3db97a0700 1 --2- 172.21.15.165:0/707460409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3db4101e40 0x7f3db4102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:17:47.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.964+0000 7f3db97a0700 1 --2- 172.21.15.165:0/707460409 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3db41027a0 0x7f3db4102c20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:17:47.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.964+0000 7f3db97a0700 1 -- 172.21.15.165:0/707460409 >> 172.21.15.165:0/707460409 conn(0x7f3db40fdaa0 msgr2=0x7f3db40ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:17:47.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.964+0000 7f3db97a0700 1 -- 172.21.15.165:0/707460409 shutdown_connections 2024-05-21T03:17:47.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.964+0000 7f3db97a0700 1 -- 172.21.15.165:0/707460409 wait complete. 2024-05-21T03:17:47.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.965+0000 7f3db97a0700 1 Processor -- start 2024-05-21T03:17:47.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.965+0000 7f3db97a0700 1 -- start start 2024-05-21T03:17:47.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.965+0000 7f3db97a0700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3db4101e40 0x7f3db419b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:17:47.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.965+0000 7f3db97a0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3db41027a0 0x7f3db419be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:17:47.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.965+0000 7f3db97a0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db4103160 0x7f3db41a2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:17:47.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.965+0000 7f3db97a0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3db4111240 con 0x7f3db41027a0 2024-05-21T03:17:47.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.965+0000 7f3db97a0700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3db41110c0 con 0x7f3db4103160 2024-05-21T03:17:47.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.965+0000 7f3db97a0700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3db41113c0 con 0x7f3db4101e40 2024-05-21T03:17:47.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.965+0000 7f3db2ffd700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3db4101e40 0x7f3db419b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:17:47.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.965+0000 7f3db2ffd700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3db4101e40 0x7f3db419b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49806/0 (socket says 172.21.15.165:49806) 2024-05-21T03:17:47.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.965+0000 7f3db2ffd700 1 -- 172.21.15.165:0/1608557707 learned_addr learned my addr 172.21.15.165:0/1608557707 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-05-21T03:17:47.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.965+0000 7f3db27fc700 1 --2- 172.21.15.165:0/1608557707 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3db41027a0 0x7f3db419be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:17:47.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.966+0000 7f3db37fe700 1 --2- 172.21.15.165:0/1608557707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db4103160 0x7f3db41a2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:17:47.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.966+0000 7f3db2ffd700 1 -- 172.21.15.165:0/1608557707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db4103160 msgr2=0x7f3db41a2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:17:47.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.966+0000 7f3db2ffd700 1 --2- 172.21.15.165:0/1608557707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db4103160 0x7f3db41a2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:17:47.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.966+0000 7f3db2ffd700 1 -- 172.21.15.165:0/1608557707 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3db41027a0 msgr2=0x7f3db419be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:17:47.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.966+0000 7f3db2ffd700 1 --2- 172.21.15.165:0/1608557707 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3db41027a0 0x7f3db419be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:17:47.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.966+0000 7f3db2ffd700 1 -- 172.21.15.165:0/1608557707 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db41a3560 con 0x7f3db4101e40 2024-05-21T03:17:47.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.966+0000 7f3db2ffd700 1 --2- 172.21.15.165:0/1608557707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3db4101e40 0x7f3db419b900 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3da400d980 tx=0x7f3da400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:17:47.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.966+0000 7f3d9bfff700 1 -- 172.21.15.165:0/1608557707 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da4016070 con 0x7f3db4101e40 2024-05-21T03:17:47.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.966+0000 7f3d9bfff700 1 -- 172.21.15.165:0/1608557707 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3da400de70 con 0x7f3db4101e40 2024-05-21T03:17:47.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.966+0000 7f3d9bfff700 1 -- 172.21.15.165:0/1608557707 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da400f3c0 con 0x7f3db4101e40 2024-05-21T03:17:47.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.966+0000 7f3db97a0700 1 -- 172.21.15.165:0/1608557707 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3db41a3850 con 0x7f3db4101e40 2024-05-21T03:17:47.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.966+0000 7f3db97a0700 1 -- 172.21.15.165:0/1608557707 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3db41a3d40 con 0x7f3db4101e40 2024-05-21T03:17:47.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.967+0000 7f3db97a0700 1 -- 172.21.15.165:0/1608557707 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3db406a230 con 0x7f3db4101e40 2024-05-21T03:17:47.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.968+0000 7f3d9bfff700 1 -- 172.21.15.165:0/1608557707 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3da4019070 con 0x7f3db4101e40 2024-05-21T03:17:47.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.969+0000 7f3d9bfff700 1 --2- 172.21.15.165:0/1608557707 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f3d9c0b2900 0x7f3d9c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:17:47.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.969+0000 7f3d9bfff700 1 -- 172.21.15.165:0/1608557707 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5397+0+0 (secure 0 0 0) 0x7f3da40e1130 con 0x7f3db4101e40 2024-05-21T03:17:47.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.969+0000 7f3db27fc700 1 --2- 172.21.15.165:0/1608557707 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f3d9c0b2900 0x7f3d9c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:17:47.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.970+0000 7f3db27fc700 1 --2- 172.21.15.165:0/1608557707 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f3d9c0b2900 0x7f3d9c0b4dc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3da800b5c0 tx=0x7f3da8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:17:47.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:47.973+0000 7f3d9bfff700 1 -- 172.21.15.165:0/1608557707 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3da40a99b0 con 0x7f3db4101e40 2024-05-21T03:17:48.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:47 smithi039 ceph-mon[36429]: osdmap e50: 8 total, 8 up, 8 in 2024-05-21T03:17:48.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:47 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:48.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:47 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:48.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:47 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:17:48.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:47 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:17:48.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:47 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:17:48.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:47 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:48.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:47 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:17:48.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:47 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:17:48.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:47 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:48.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:47 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:17:48.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:47 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:17:48.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:47 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:17:48.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:47 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:48.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:47 smithi027 ceph-mon[32179]: osdmap e50: 8 total, 8 up, 8 in 2024-05-21T03:17:48.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:47 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:48.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:47 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:48.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:47 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:17:48.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:47 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:17:48.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:47 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:17:48.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:47 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:48.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:47 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:17:48.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:47 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:17:48.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:47 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:48.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:47 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:17:48.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:47 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:17:48.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:47 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:17:48.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:47 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:17:48.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:17:48.184+0000 7f3db97a0700 1 -- 172.21.15.165:0/1608557707 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3db4063980 con 0x7f3d9c0b2900 2024-05-21T03:17:49.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:48 smithi039 ceph-mon[36429]: Detected new or changed devices on smithi039 2024-05-21T03:17:49.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:48 smithi039 ceph-mon[36429]: Adjusting osd_memory_target on smithi039 to 5696M 2024-05-21T03:17:49.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:48 smithi039 ceph-mon[36429]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-21T03:17:49.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:48 smithi039 ceph-mon[36429]: osdmap e51: 8 total, 8 up, 8 in 2024-05-21T03:17:49.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:48 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:17:49.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:48 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:17:49.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:48 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:17:49.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:48 smithi165 ceph-mon[36269]: Detected new or changed devices on smithi039 2024-05-21T03:17:49.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:48 smithi165 ceph-mon[36269]: Adjusting osd_memory_target on smithi039 to 5696M 2024-05-21T03:17:49.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:48 smithi165 ceph-mon[36269]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-21T03:17:49.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:48 smithi165 ceph-mon[36269]: osdmap e51: 8 total, 8 up, 8 in 2024-05-21T03:17:49.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:48 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:17:49.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:48 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:17:49.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:48 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:17:49.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:48 smithi027 ceph-mon[32179]: Detected new or changed devices on smithi039 2024-05-21T03:17:49.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:48 smithi027 ceph-mon[32179]: Adjusting osd_memory_target on smithi039 to 5696M 2024-05-21T03:17:49.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:48 smithi027 ceph-mon[32179]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-21T03:17:49.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:48 smithi027 ceph-mon[32179]: osdmap e51: 8 total, 8 up, 8 in 2024-05-21T03:17:49.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:48 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:17:49.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:48 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:17:49.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:48 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:17:49.953 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:49 smithi165 ceph-mon[36269]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:17:50.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:49 smithi039 ceph-mon[36429]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:17:50.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:49 smithi027 ceph-mon[32179]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:17:51.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:50 smithi039 ceph-mon[36429]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-21T03:17:51.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:50 smithi165 ceph-mon[36269]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-21T03:17:51.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:50 smithi027 ceph-mon[32179]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-21T03:17:52.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:51 smithi039 ceph-mon[36429]: from='client.? 172.21.15.165:0/1221579298' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2259f35c-2edd-4331-9a63-8c0c1acbdc71"}]: dispatch 2024-05-21T03:17:52.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:51 smithi039 ceph-mon[36429]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2259f35c-2edd-4331-9a63-8c0c1acbdc71"}]: dispatch 2024-05-21T03:17:52.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:51 smithi039 ceph-mon[36429]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2259f35c-2edd-4331-9a63-8c0c1acbdc71"}]': finished 2024-05-21T03:17:52.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:51 smithi039 ceph-mon[36429]: osdmap e52: 9 total, 8 up, 9 in 2024-05-21T03:17:52.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:51 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T03:17:52.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:51 smithi039 ceph-mon[36429]: from='client.? 172.21.15.165:0/4113478299' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:17:52.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:51 smithi165 ceph-mon[36269]: from='client.? 172.21.15.165:0/1221579298' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2259f35c-2edd-4331-9a63-8c0c1acbdc71"}]: dispatch 2024-05-21T03:17:52.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:51 smithi165 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2259f35c-2edd-4331-9a63-8c0c1acbdc71"}]: dispatch 2024-05-21T03:17:52.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:51 smithi165 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2259f35c-2edd-4331-9a63-8c0c1acbdc71"}]': finished 2024-05-21T03:17:52.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:51 smithi165 ceph-mon[36269]: osdmap e52: 9 total, 8 up, 9 in 2024-05-21T03:17:52.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:51 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T03:17:52.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:51 smithi165 ceph-mon[36269]: from='client.? 172.21.15.165:0/4113478299' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:17:52.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:51 smithi027 ceph-mon[32179]: from='client.? 172.21.15.165:0/1221579298' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2259f35c-2edd-4331-9a63-8c0c1acbdc71"}]: dispatch 2024-05-21T03:17:52.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:51 smithi027 ceph-mon[32179]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2259f35c-2edd-4331-9a63-8c0c1acbdc71"}]: dispatch 2024-05-21T03:17:52.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:51 smithi027 ceph-mon[32179]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2259f35c-2edd-4331-9a63-8c0c1acbdc71"}]': finished 2024-05-21T03:17:52.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:51 smithi027 ceph-mon[32179]: osdmap e52: 9 total, 8 up, 9 in 2024-05-21T03:17:52.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:51 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T03:17:52.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:51 smithi027 ceph-mon[32179]: from='client.? 172.21.15.165:0/4113478299' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:17:53.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:52 smithi027 ceph-mon[32179]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-21T03:17:53.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:52 smithi039 ceph-mon[36429]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-21T03:17:53.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:52 smithi165 ceph-mon[36269]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-21T03:17:55.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:54 smithi165 ceph-mon[36269]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-21T03:17:55.071 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:54 smithi039 ceph-mon[36429]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-21T03:17:55.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:54 smithi027 ceph-mon[32179]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-21T03:17:57.146 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:56 smithi165 ceph-mon[36269]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T03:17:57.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:56 smithi027 ceph-mon[32179]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T03:17:57.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:56 smithi039 ceph-mon[36429]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T03:17:59.078 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:58 smithi165 ceph-mon[36269]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-21T03:17:59.078 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:58 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-21T03:17:59.078 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:58 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:17:59.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:58 smithi027 ceph-mon[32179]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-21T03:17:59.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:58 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-21T03:17:59.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:58 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:17:59.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:58 smithi039 ceph-mon[36429]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-21T03:17:59.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:58 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-21T03:17:59.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:58 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:17:59.926 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:17:59 smithi165 ceph-mon[36269]: Deploying daemon osd.8 on smithi165 2024-05-21T03:18:00.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:17:59 smithi027 ceph-mon[32179]: Deploying daemon osd.8 on smithi165 2024-05-21T03:18:00.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:17:59 smithi039 ceph-mon[36429]: Deploying daemon osd.8 on smithi165 2024-05-21T03:18:00.915 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:00 smithi165 ceph-mon[36269]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T03:18:01.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:00 smithi027 ceph-mon[32179]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T03:18:01.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:00 smithi039 ceph-mon[36429]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T03:18:02.956 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:02 smithi165 ceph-mon[36269]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-21T03:18:03.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:02 smithi027 ceph-mon[32179]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-21T03:18:03.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:02 smithi039 ceph-mon[36429]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-21T03:18:05.090 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:04 smithi165 ceph-mon[36269]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-21T03:18:05.090 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:04 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:05.091 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:04 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:05.091 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:04 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:18:05.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:04 smithi027 ceph-mon[32179]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-21T03:18:05.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:04 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:05.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:04 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:05.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:04 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:18:05.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:04 smithi039 ceph-mon[36429]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-21T03:18:05.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:04 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:05.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:04 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:05.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:04 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:18:07.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:06 smithi165 ceph-mon[36269]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-21T03:18:07.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:06 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:07.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:06 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:07.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:06 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:18:07.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:06 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:18:07.134 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:06 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:07.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:06 smithi027 ceph-mon[32179]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-21T03:18:07.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:06 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:07.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:06 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:07.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:06 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:18:07.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:06 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:18:07.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:06 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:07.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:06 smithi039 ceph-mon[36429]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-21T03:18:07.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:06 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:07.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:06 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:07.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:06 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:18:07.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:06 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:18:07.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:06 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:07.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:18:07.501+0000 7f3d9bfff700 1 -- 172.21.15.165:0/1608557707 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3db4063980 con 0x7f3d9c0b2900 2024-05-21T03:18:07.501 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 8 on host 'smithi165' 2024-05-21T03:18:07.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:18:07.504+0000 7f3db97a0700 1 -- 172.21.15.165:0/1608557707 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f3d9c0b2900 msgr2=0x7f3d9c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:18:07.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:18:07.504+0000 7f3db97a0700 1 --2- 172.21.15.165:0/1608557707 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f3d9c0b2900 0x7f3d9c0b4dc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3da800b5c0 tx=0x7f3da8005fb0 comp rx=0 tx=0).stop 2024-05-21T03:18:07.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:18:07.504+0000 7f3db97a0700 1 -- 172.21.15.165:0/1608557707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3db4101e40 msgr2=0x7f3db419b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:18:07.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:18:07.504+0000 7f3db97a0700 1 --2- 172.21.15.165:0/1608557707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3db4101e40 0x7f3db419b900 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3da400d980 tx=0x7f3da400dc90 comp rx=0 tx=0).stop 2024-05-21T03:18:07.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:18:07.504+0000 7f3db97a0700 1 -- 172.21.15.165:0/1608557707 shutdown_connections 2024-05-21T03:18:07.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:18:07.504+0000 7f3db97a0700 1 --2- 172.21.15.165:0/1608557707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db4103160 0x7f3db41a2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:18:07.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:18:07.504+0000 7f3db97a0700 1 --2- 172.21.15.165:0/1608557707 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f3d9c0b2900 0x7f3d9c0b4dc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:18:07.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:18:07.504+0000 7f3db97a0700 1 --2- 172.21.15.165:0/1608557707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3db4101e40 0x7f3db419b900 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:18:07.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:18:07.504+0000 7f3db97a0700 1 --2- 172.21.15.165:0/1608557707 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3db41027a0 0x7f3db419be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:18:07.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:18:07.504+0000 7f3db97a0700 1 -- 172.21.15.165:0/1608557707 >> 172.21.15.165:0/1608557707 conn(0x7f3db40fdaa0 msgr2=0x7f3db40ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:18:07.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:18:07.504+0000 7f3db97a0700 1 -- 172.21.15.165:0/1608557707 shutdown_connections 2024-05-21T03:18:07.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:18:07.504+0000 7f3db97a0700 1 -- 172.21.15.165:0/1608557707 wait complete. 2024-05-21T03:18:08.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:07 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:18:08.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:07 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:08.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:07 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:08.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:07 smithi165 ceph-mon[36269]: from='osd.8 [v2:172.21.15.165:6816/3184704971,v1:172.21.15.165:6817/3184704971]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-21T03:18:08.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:07 smithi165 ceph-mon[36269]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-21T03:18:08.108 DEBUG:teuthology.orchestra.run.smithi165:osd.8> sudo journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.8.service 2024-05-21T03:18:08.110 INFO:tasks.cephadm:Deploying osd.9 on smithi027 with /dev/vg_nvme/lv_1... 2024-05-21T03:18:08.110 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-21T03:18:08.139 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:07 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:18:08.139 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:07 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:08.139 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:07 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:08.139 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:07 smithi027 ceph-mon[32179]: from='osd.8 [v2:172.21.15.165:6816/3184704971,v1:172.21.15.165:6817/3184704971]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-21T03:18:08.139 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:07 smithi027 ceph-mon[32179]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-21T03:18:08.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:07 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:18:08.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:07 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:08.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:07 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:08.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:07 smithi039 ceph-mon[36429]: from='osd.8 [v2:172.21.15.165:6816/3184704971,v1:172.21.15.165:6817/3184704971]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-21T03:18:08.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:07 smithi039 ceph-mon[36429]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-21T03:18:08.372 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:18:09.168 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:08 smithi165 ceph-mon[36269]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-21T03:18:09.168 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:08 smithi165 ceph-mon[36269]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-21T03:18:09.168 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:08 smithi165 ceph-mon[36269]: osdmap e53: 9 total, 8 up, 9 in 2024-05-21T03:18:09.169 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:08 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T03:18:09.169 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:08 smithi165 ceph-mon[36269]: from='osd.8 [v2:172.21.15.165:6816/3184704971,v1:172.21.15.165:6817/3184704971]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-21T03:18:09.169 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:08 smithi165 ceph-mon[36269]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-21T03:18:09.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:08 smithi027 ceph-mon[32179]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-21T03:18:09.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:08 smithi027 ceph-mon[32179]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-21T03:18:09.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:08 smithi027 ceph-mon[32179]: osdmap e53: 9 total, 8 up, 9 in 2024-05-21T03:18:09.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:08 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T03:18:09.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:08 smithi027 ceph-mon[32179]: from='osd.8 [v2:172.21.15.165:6816/3184704971,v1:172.21.15.165:6817/3184704971]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-21T03:18:09.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:08 smithi027 ceph-mon[32179]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-21T03:18:09.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:08 smithi039 ceph-mon[36429]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-21T03:18:09.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:08 smithi039 ceph-mon[36429]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-21T03:18:09.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:08 smithi039 ceph-mon[36429]: osdmap e53: 9 total, 8 up, 9 in 2024-05-21T03:18:09.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:08 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T03:18:09.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:08 smithi039 ceph-mon[36429]: from='osd.8 [v2:172.21.15.165:6816/3184704971,v1:172.21.15.165:6817/3184704971]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-21T03:18:09.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:08 smithi039 ceph-mon[36429]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-21T03:18:09.771 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:18:09 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:18:09.467+0000 7f9797e1c700 -1 osd.8 0 waiting for initial osdmap 2024-05-21T03:18:09.772 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:18:09 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:18:09.477+0000 7f9793c0f700 -1 osd.8 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T03:18:10.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:10 smithi165 ceph-mon[36269]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-21T03:18:10.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:10 smithi165 ceph-mon[36269]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-05-21T03:18:10.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:10 smithi165 ceph-mon[36269]: osdmap e54: 9 total, 8 up, 9 in 2024-05-21T03:18:10.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:10 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T03:18:10.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:10 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T03:18:10.604 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-21T03:18:10.604 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-21T03:18:10.604 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2024-05-21T03:18:10.604 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2024-05-21T03:18:10.604 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110268 s, 951 MB/s 2024-05-21T03:18:10.604 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2024-05-21T03:18:10.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:10 smithi039 ceph-mon[36429]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-21T03:18:10.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:10 smithi039 ceph-mon[36429]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-05-21T03:18:10.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:10 smithi039 ceph-mon[36429]: osdmap e54: 9 total, 8 up, 9 in 2024-05-21T03:18:10.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:10 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T03:18:10.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:10 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T03:18:10.903 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:10 smithi027 ceph-mon[32179]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-21T03:18:10.903 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:10 smithi027 ceph-mon[32179]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-05-21T03:18:10.903 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:10 smithi027 ceph-mon[32179]: osdmap e54: 9 total, 8 up, 9 in 2024-05-21T03:18:10.903 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:10 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T03:18:10.903 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:10 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T03:18:11.205 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi027:vg_nvme/lv_1 2024-05-21T03:18:11.496 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:18:11.721 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:11 smithi027 ceph-mon[32179]: purged_snaps scrub starts 2024-05-21T03:18:11.721 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:11 smithi027 ceph-mon[32179]: purged_snaps scrub ok 2024-05-21T03:18:11.721 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:11 smithi027 ceph-mon[32179]: osd.8 [v2:172.21.15.165:6816/3184704971,v1:172.21.15.165:6817/3184704971] boot 2024-05-21T03:18:11.721 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:11 smithi027 ceph-mon[32179]: osdmap e55: 9 total, 9 up, 9 in 2024-05-21T03:18:11.721 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:11 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T03:18:11.721 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:11 smithi027 ceph-mon[32179]: Detected new or changed devices on smithi165 2024-05-21T03:18:11.721 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:11 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:11.721 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:11 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:11.721 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:11 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:18:11.721 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:11 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:18:11.721 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:11 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:18:11.722 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:11 smithi027 ceph-mon[32179]: Adjusting osd_memory_target on smithi165 to 5696M 2024-05-21T03:18:11.722 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:11 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:11.722 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:11 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:18:11.722 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:11 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:18:11.722 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:11 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:11.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:11 smithi039 ceph-mon[36429]: purged_snaps scrub starts 2024-05-21T03:18:11.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:11 smithi039 ceph-mon[36429]: purged_snaps scrub ok 2024-05-21T03:18:11.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:11 smithi039 ceph-mon[36429]: osd.8 [v2:172.21.15.165:6816/3184704971,v1:172.21.15.165:6817/3184704971] boot 2024-05-21T03:18:11.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:11 smithi039 ceph-mon[36429]: osdmap e55: 9 total, 9 up, 9 in 2024-05-21T03:18:11.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:11 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T03:18:11.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:11 smithi039 ceph-mon[36429]: Detected new or changed devices on smithi165 2024-05-21T03:18:11.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:11 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:11.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:11 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:11.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:11 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:18:11.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:11 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:18:11.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:11 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:18:11.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:11 smithi039 ceph-mon[36429]: Adjusting osd_memory_target on smithi165 to 5696M 2024-05-21T03:18:11.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:11 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:11.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:11 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:18:11.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:11 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:18:11.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:11 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:11.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:11 smithi165 ceph-mon[36269]: purged_snaps scrub starts 2024-05-21T03:18:11.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:11 smithi165 ceph-mon[36269]: purged_snaps scrub ok 2024-05-21T03:18:11.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:11 smithi165 ceph-mon[36269]: osd.8 [v2:172.21.15.165:6816/3184704971,v1:172.21.15.165:6817/3184704971] boot 2024-05-21T03:18:11.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:11 smithi165 ceph-mon[36269]: osdmap e55: 9 total, 9 up, 9 in 2024-05-21T03:18:11.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:11 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T03:18:11.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:11 smithi165 ceph-mon[36269]: Detected new or changed devices on smithi165 2024-05-21T03:18:11.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:11 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:11.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:11 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:11.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:11 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:18:11.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:11 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:18:11.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:11 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:18:11.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:11 smithi165 ceph-mon[36269]: Adjusting osd_memory_target on smithi165 to 5696M 2024-05-21T03:18:11.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:11 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:11.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:11 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:18:11.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:11 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:18:11.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:11 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:12.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:12 smithi039 ceph-mon[36429]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-21T03:18:12.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:12 smithi039 ceph-mon[36429]: osdmap e56: 9 total, 9 up, 9 in 2024-05-21T03:18:12.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:12 smithi165 ceph-mon[36269]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-21T03:18:12.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:12 smithi165 ceph-mon[36269]: osdmap e56: 9 total, 9 up, 9 in 2024-05-21T03:18:12.832 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:12 smithi027 ceph-mon[32179]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-21T03:18:12.832 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:12 smithi027 ceph-mon[32179]: osdmap e56: 9 total, 9 up, 9 in 2024-05-21T03:18:13.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.084+0000 7fc1af0e8700 1 -- 172.21.15.27:0/2934481581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a81035a0 msgr2=0x7fc1a81039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:18:13.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.084+0000 7fc1af0e8700 1 --2- 172.21.15.27:0/2934481581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a81035a0 0x7fc1a81039c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0099c0 tx=0x7fc19c009cd0 comp rx=0 tx=0).stop 2024-05-21T03:18:13.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.084+0000 7fc1af0e8700 1 -- 172.21.15.27:0/2934481581 shutdown_connections 2024-05-21T03:18:13.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.084+0000 7fc1af0e8700 1 --2- 172.21.15.27:0/2934481581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1a8105140 0x7fc1a810b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:18:13.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.084+0000 7fc1af0e8700 1 --2- 172.21.15.27:0/2934481581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a81035a0 0x7fc1a81039c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:18:13.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.084+0000 7fc1af0e8700 1 --2- 172.21.15.27:0/2934481581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1a81047a0 0x7fc1a8104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:18:13.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.084+0000 7fc1af0e8700 1 -- 172.21.15.27:0/2934481581 >> 172.21.15.27:0/2934481581 conn(0x7fc1a80feb40 msgr2=0x7fc1a8100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:18:13.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.084+0000 7fc1af0e8700 1 -- 172.21.15.27:0/2934481581 shutdown_connections 2024-05-21T03:18:13.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.084+0000 7fc1af0e8700 1 -- 172.21.15.27:0/2934481581 wait complete. 2024-05-21T03:18:13.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.084+0000 7fc1af0e8700 1 Processor -- start 2024-05-21T03:18:13.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.085+0000 7fc1af0e8700 1 -- start start 2024-05-21T03:18:13.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.085+0000 7fc1af0e8700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1a81035a0 0x7fc1a8199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:18:13.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.085+0000 7fc1af0e8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1a81047a0 0x7fc1a8199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:18:13.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.085+0000 7fc1af0e8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a8105140 0x7fc1a81a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:18:13.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.085+0000 7fc1af0e8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a810f040 con 0x7fc1a8105140 2024-05-21T03:18:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.085+0000 7fc1af0e8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a810eec0 con 0x7fc1a81047a0 2024-05-21T03:18:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.085+0000 7fc1af0e8700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a810f1c0 con 0x7fc1a81035a0 2024-05-21T03:18:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.085+0000 7fc1ad685700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a8105140 0x7fc1a81a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:18:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.085+0000 7fc1ad685700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a8105140 0x7fc1a81a0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58072/0 (socket says 172.21.15.27:58072) 2024-05-21T03:18:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.085+0000 7fc1ad685700 1 -- 172.21.15.27:0/332336255 learned_addr learned my addr 172.21.15.27:0/332336255 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:18:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.085+0000 7fc1a7fff700 1 --2- 172.21.15.27:0/332336255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1a81047a0 0x7fc1a8199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:18:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.085+0000 7fc1ace84700 1 --2- 172.21.15.27:0/332336255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1a81035a0 0x7fc1a8199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:18:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.085+0000 7fc1ad685700 1 -- 172.21.15.27:0/332336255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1a81035a0 msgr2=0x7fc1a8199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:18:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.085+0000 7fc1ad685700 1 --2- 172.21.15.27:0/332336255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1a81035a0 0x7fc1a8199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:18:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.085+0000 7fc1ad685700 1 -- 172.21.15.27:0/332336255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1a81047a0 msgr2=0x7fc1a8199c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:18:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.085+0000 7fc1ad685700 1 --2- 172.21.15.27:0/332336255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1a81047a0 0x7fc1a8199c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:18:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.085+0000 7fc1ad685700 1 -- 172.21.15.27:0/332336255 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a81a1370 con 0x7fc1a8105140 2024-05-21T03:18:13.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.085+0000 7fc1a7fff700 1 --2- 172.21.15.27:0/332336255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1a81047a0 0x7fc1a8199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:18:13.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.086+0000 7fc1ad685700 1 --2- 172.21.15.27:0/332336255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a8105140 0x7fc1a81a0cd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0007f10 tx=0x7fc1a000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:18:13.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.086+0000 7fc1a5ffb700 1 -- 172.21.15.27:0/332336255 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0004bb0 con 0x7fc1a8105140 2024-05-21T03:18:13.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.086+0000 7fc1af0e8700 1 -- 172.21.15.27:0/332336255 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a81a1660 con 0x7fc1a8105140 2024-05-21T03:18:13.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.086+0000 7fc1a5ffb700 1 -- 172.21.15.27:0/332336255 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1a0004d50 con 0x7fc1a8105140 2024-05-21T03:18:13.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.086+0000 7fc1a5ffb700 1 -- 172.21.15.27:0/332336255 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a00058e0 con 0x7fc1a8105140 2024-05-21T03:18:13.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.086+0000 7fc1af0e8700 1 -- 172.21.15.27:0/332336255 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a81a1ba0 con 0x7fc1a8105140 2024-05-21T03:18:13.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.087+0000 7fc1af0e8700 1 -- 172.21.15.27:0/332336255 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1a806a230 con 0x7fc1a8105140 2024-05-21T03:18:13.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.088+0000 7fc1a5ffb700 1 -- 172.21.15.27:0/332336255 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc1a001a020 con 0x7fc1a8105140 2024-05-21T03:18:13.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.089+0000 7fc1a5ffb700 1 --2- 172.21.15.27:0/332336255 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fc1980b2900 0x7fc1980b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:18:13.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.089+0000 7fc1ace84700 1 --2- 172.21.15.27:0/332336255 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fc1980b2900 0x7fc1980b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:18:13.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.089+0000 7fc1a5ffb700 1 -- 172.21.15.27:0/332336255 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5829+0+0 (secure 0 0 0) 0x7fc1a00e0f20 con 0x7fc1a8105140 2024-05-21T03:18:13.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.090+0000 7fc1ace84700 1 --2- 172.21.15.27:0/332336255 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fc1980b2900 0x7fc1980b4dc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0038c0 tx=0x7fc19c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:18:13.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.094+0000 7fc1a5ffb700 1 -- 172.21.15.27:0/332336255 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc1a0010070 con 0x7fc1a8105140 2024-05-21T03:18:13.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:13.323+0000 7fc1af0e8700 1 -- 172.21.15.27:0/332336255 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fc1a8063980 con 0x7fc1980b2900 2024-05-21T03:18:13.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:13 smithi039 ceph-mon[36429]: osdmap e57: 9 total, 9 up, 9 in 2024-05-21T03:18:13.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:13 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:18:13.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:13 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:18:13.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:13 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:18:13.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:13 smithi165 ceph-mon[36269]: osdmap e57: 9 total, 9 up, 9 in 2024-05-21T03:18:13.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:13 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:18:13.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:13 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:18:13.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:13 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:18:13.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:13 smithi027 ceph-mon[32179]: osdmap e57: 9 total, 9 up, 9 in 2024-05-21T03:18:13.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:13 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:18:13.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:13 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:18:13.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:13 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:18:14.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:14 smithi027 ceph-mon[32179]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-21T03:18:14.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:14 smithi027 ceph-mon[32179]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:18:14.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:14 smithi039 ceph-mon[36429]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-21T03:18:14.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:14 smithi039 ceph-mon[36429]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:18:14.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:14 smithi165 ceph-mon[36269]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-21T03:18:14.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:14 smithi165 ceph-mon[36269]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:18:16.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:16 smithi039 ceph-mon[36429]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-21T03:18:16.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:16 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/2759210850' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10307ecc-8270-4be2-b62b-9708e09bb207"}]: dispatch 2024-05-21T03:18:16.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:16 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/2759210850' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "10307ecc-8270-4be2-b62b-9708e09bb207"}]': finished 2024-05-21T03:18:16.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:16 smithi039 ceph-mon[36429]: osdmap e58: 10 total, 9 up, 10 in 2024-05-21T03:18:16.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:16 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T03:18:16.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:16 smithi165 ceph-mon[36269]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-21T03:18:16.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:16 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/2759210850' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10307ecc-8270-4be2-b62b-9708e09bb207"}]: dispatch 2024-05-21T03:18:16.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:16 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/2759210850' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "10307ecc-8270-4be2-b62b-9708e09bb207"}]': finished 2024-05-21T03:18:16.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:16 smithi165 ceph-mon[36269]: osdmap e58: 10 total, 9 up, 10 in 2024-05-21T03:18:16.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:16 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T03:18:16.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:16 smithi027 ceph-mon[32179]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-21T03:18:16.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:16 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2759210850' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10307ecc-8270-4be2-b62b-9708e09bb207"}]: dispatch 2024-05-21T03:18:16.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:16 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2759210850' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "10307ecc-8270-4be2-b62b-9708e09bb207"}]': finished 2024-05-21T03:18:16.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:16 smithi027 ceph-mon[32179]: osdmap e58: 10 total, 9 up, 10 in 2024-05-21T03:18:16.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:16 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T03:18:17.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:17 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/561370178' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:18:17.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:17 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/561370178' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:18:17.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:17 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/561370178' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:18:18.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:18 smithi039 ceph-mon[36429]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T03:18:18.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:18 smithi165 ceph-mon[36269]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T03:18:18.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:18 smithi027 ceph-mon[32179]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T03:18:20.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:20 smithi039 ceph-mon[36429]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-21T03:18:20.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:20 smithi165 ceph-mon[36269]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-21T03:18:20.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:20 smithi027 ceph-mon[32179]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-21T03:18:22.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:22 smithi027 ceph-mon[32179]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-21T03:18:22.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:22 smithi039 ceph-mon[36429]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-21T03:18:22.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:22 smithi165 ceph-mon[36269]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-21T03:18:24.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:24 smithi027 ceph-mon[32179]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T03:18:24.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:24 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-21T03:18:24.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:24 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:18:24.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:24 smithi039 ceph-mon[36429]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T03:18:24.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:24 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-21T03:18:24.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:24 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:18:24.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:24 smithi165 ceph-mon[36269]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T03:18:24.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:24 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-21T03:18:24.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:24 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:18:25.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:25 smithi027 ceph-mon[32179]: Deploying daemon osd.9 on smithi027 2024-05-21T03:18:25.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:25 smithi165 ceph-mon[36269]: Deploying daemon osd.9 on smithi027 2024-05-21T03:18:26.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:25 smithi039 ceph-mon[36429]: Deploying daemon osd.9 on smithi027 2024-05-21T03:18:26.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:26 smithi027 ceph-mon[32179]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T03:18:26.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:26 smithi165 ceph-mon[36269]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T03:18:27.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:26 smithi039 ceph-mon[36429]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T03:18:28.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:28 smithi027 ceph-mon[32179]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-21T03:18:29.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:28 smithi039 ceph-mon[36429]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-21T03:18:29.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:28 smithi165 ceph-mon[36269]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-21T03:18:30.792 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:30 smithi027 ceph-mon[32179]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-21T03:18:30.792 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:30 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:30.792 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:30 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:30.792 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:30 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:18:31.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:30 smithi039 ceph-mon[36429]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-21T03:18:31.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:30 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:31.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:30 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:31.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:30 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:18:31.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:30 smithi165 ceph-mon[36269]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-21T03:18:31.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:30 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:31.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:30 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:31.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:30 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:18:32.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:32 smithi027 ceph-mon[32179]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-21T03:18:33.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:32 smithi039 ceph-mon[36429]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-21T03:18:33.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:32 smithi165 ceph-mon[36269]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-21T03:18:34.183 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 9 on host 'smithi027' 2024-05-21T03:18:34.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:34.182+0000 7fc1a5ffb700 1 -- 172.21.15.27:0/332336255 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc1a8063980 con 0x7fc1980b2900 2024-05-21T03:18:34.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:34.185+0000 7fc1af0e8700 1 -- 172.21.15.27:0/332336255 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fc1980b2900 msgr2=0x7fc1980b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:18:34.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:34.185+0000 7fc1af0e8700 1 --2- 172.21.15.27:0/332336255 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fc1980b2900 0x7fc1980b4dc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0038c0 tx=0x7fc19c00b540 comp rx=0 tx=0).stop 2024-05-21T03:18:34.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:34.185+0000 7fc1af0e8700 1 -- 172.21.15.27:0/332336255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a8105140 msgr2=0x7fc1a81a0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:18:34.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:34.185+0000 7fc1af0e8700 1 --2- 172.21.15.27:0/332336255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a8105140 0x7fc1a81a0cd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0007f10 tx=0x7fc1a000d3b0 comp rx=0 tx=0).stop 2024-05-21T03:18:34.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:34.185+0000 7fc1af0e8700 1 -- 172.21.15.27:0/332336255 shutdown_connections 2024-05-21T03:18:34.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:34.185+0000 7fc1af0e8700 1 --2- 172.21.15.27:0/332336255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1a8105140 0x7fc1a81a0cd0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:18:34.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:34.185+0000 7fc1af0e8700 1 --2- 172.21.15.27:0/332336255 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fc1980b2900 0x7fc1980b4dc0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:18:34.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:34.185+0000 7fc1af0e8700 1 --2- 172.21.15.27:0/332336255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1a81035a0 0x7fc1a8199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:18:34.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:34.185+0000 7fc1af0e8700 1 --2- 172.21.15.27:0/332336255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1a81047a0 0x7fc1a8199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:18:34.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:34.185+0000 7fc1af0e8700 1 -- 172.21.15.27:0/332336255 >> 172.21.15.27:0/332336255 conn(0x7fc1a80feb40 msgr2=0x7fc1a8107b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:18:34.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:34.185+0000 7fc1af0e8700 1 -- 172.21.15.27:0/332336255 shutdown_connections 2024-05-21T03:18:34.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:18:34.185+0000 7fc1af0e8700 1 -- 172.21.15.27:0/332336255 wait complete. 2024-05-21T03:18:34.756 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:34 smithi027 ceph-mon[32179]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-21T03:18:34.756 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:34 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:34.756 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:34 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:34.757 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:34 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:18:34.757 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:34 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:18:34.757 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:34 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:34.757 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:34 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:34.757 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:34 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:18:34.757 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:34 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:34.757 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:34 smithi027 ceph-mon[32179]: from='osd.9 [v2:172.21.15.27:6826/2800048793,v1:172.21.15.27:6827/2800048793]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-21T03:18:34.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:34 smithi039 ceph-mon[36429]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-21T03:18:34.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:34 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:34.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:34 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:34.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:34 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:18:34.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:34 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:18:34.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:34 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:34.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:34 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:34.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:34 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:18:34.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:34 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:34.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:34 smithi039 ceph-mon[36429]: from='osd.9 [v2:172.21.15.27:6826/2800048793,v1:172.21.15.27:6827/2800048793]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-21T03:18:34.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:34 smithi165 ceph-mon[36269]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-21T03:18:34.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:34 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:34.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:34 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:34.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:34 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:18:34.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:34 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:18:34.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:34 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:34.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:34 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:34.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:34 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:18:34.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:34 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:34.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:34 smithi165 ceph-mon[36269]: from='osd.9 [v2:172.21.15.27:6826/2800048793,v1:172.21.15.27:6827/2800048793]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-21T03:18:34.782 DEBUG:teuthology.orchestra.run.smithi027:osd.9> sudo journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.9.service 2024-05-21T03:18:34.784 INFO:tasks.cephadm:Deploying osd.10 on smithi039 with /dev/vg_nvme/lv_1... 2024-05-21T03:18:34.784 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-21T03:18:35.041 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T03:18:36.184 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:36 smithi027 ceph-mon[32179]: from='osd.9 [v2:172.21.15.27:6826/2800048793,v1:172.21.15.27:6827/2800048793]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-21T03:18:36.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:36 smithi027 ceph-mon[32179]: osdmap e59: 10 total, 9 up, 10 in 2024-05-21T03:18:36.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:36 smithi027 ceph-mon[32179]: from='osd.9 [v2:172.21.15.27:6826/2800048793,v1:172.21.15.27:6827/2800048793]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-21T03:18:36.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:36 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T03:18:36.185 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:36 smithi027 ceph-mon[32179]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-21T03:18:36.185 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:18:36 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:18:36.172+0000 7f430886e700 -1 osd.9 0 waiting for initial osdmap 2024-05-21T03:18:36.185 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:18:36 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:18:36.184+0000 7f4304661700 -1 osd.9 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T03:18:36.405 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:36 smithi039 ceph-mon[36429]: from='osd.9 [v2:172.21.15.27:6826/2800048793,v1:172.21.15.27:6827/2800048793]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-21T03:18:36.406 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:36 smithi039 ceph-mon[36429]: osdmap e59: 10 total, 9 up, 10 in 2024-05-21T03:18:36.406 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:36 smithi039 ceph-mon[36429]: from='osd.9 [v2:172.21.15.27:6826/2800048793,v1:172.21.15.27:6827/2800048793]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-21T03:18:36.406 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:36 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T03:18:36.406 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:36 smithi039 ceph-mon[36429]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-21T03:18:36.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:36 smithi165 ceph-mon[36269]: from='osd.9 [v2:172.21.15.27:6826/2800048793,v1:172.21.15.27:6827/2800048793]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-21T03:18:36.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:36 smithi165 ceph-mon[36269]: osdmap e59: 10 total, 9 up, 10 in 2024-05-21T03:18:36.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:36 smithi165 ceph-mon[36269]: from='osd.9 [v2:172.21.15.27:6826/2800048793,v1:172.21.15.27:6827/2800048793]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-21T03:18:36.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:36 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T03:18:36.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:36 smithi165 ceph-mon[36269]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-21T03:18:37.011 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-21T03:18:37.011 INFO:teuthology.orchestra.run.smithi039.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-21T03:18:37.011 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10+0 records in 2024-05-21T03:18:37.011 INFO:teuthology.orchestra.run.smithi039.stderr:10+0 records out 2024-05-21T03:18:37.011 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0153074 s, 685 MB/s 2024-05-21T03:18:37.011 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping successful for: 2024-05-21T03:18:37.323 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:37 smithi039 ceph-mon[36429]: from='osd.9 [v2:172.21.15.27:6826/2800048793,v1:172.21.15.27:6827/2800048793]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-05-21T03:18:37.323 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:37 smithi039 ceph-mon[36429]: osdmap e60: 10 total, 9 up, 10 in 2024-05-21T03:18:37.323 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:37 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T03:18:37.323 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:37 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T03:18:37.323 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:37 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T03:18:37.426 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:37 smithi027 ceph-mon[32179]: from='osd.9 [v2:172.21.15.27:6826/2800048793,v1:172.21.15.27:6827/2800048793]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-05-21T03:18:37.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:37 smithi027 ceph-mon[32179]: osdmap e60: 10 total, 9 up, 10 in 2024-05-21T03:18:37.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:37 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T03:18:37.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:37 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T03:18:37.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:37 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T03:18:37.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:37 smithi165 ceph-mon[36269]: from='osd.9 [v2:172.21.15.27:6826/2800048793,v1:172.21.15.27:6827/2800048793]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-05-21T03:18:37.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:37 smithi165 ceph-mon[36269]: osdmap e60: 10 total, 9 up, 10 in 2024-05-21T03:18:37.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:37 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T03:18:37.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:37 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T03:18:37.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:37 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T03:18:37.603 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi039:vg_nvme/lv_1 2024-05-21T03:18:37.854 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T03:18:38.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:38 smithi039 ceph-mon[36429]: purged_snaps scrub starts 2024-05-21T03:18:38.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:38 smithi039 ceph-mon[36429]: purged_snaps scrub ok 2024-05-21T03:18:38.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:38 smithi039 ceph-mon[36429]: osd.9 [v2:172.21.15.27:6826/2800048793,v1:172.21.15.27:6827/2800048793] boot 2024-05-21T03:18:38.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:38 smithi039 ceph-mon[36429]: osdmap e61: 10 total, 10 up, 10 in 2024-05-21T03:18:38.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:38 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T03:18:38.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:38 smithi039 ceph-mon[36429]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-05-21T03:18:38.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:38 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:38.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:38 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:38.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:38 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:18:38.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:38 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:18:38.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:38 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:18:38.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:38 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:18:38.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:38 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:38.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:38 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:18:38.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:38 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:18:38.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:38 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:38.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:38 smithi165 ceph-mon[36269]: purged_snaps scrub starts 2024-05-21T03:18:38.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:38 smithi165 ceph-mon[36269]: purged_snaps scrub ok 2024-05-21T03:18:38.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:38 smithi165 ceph-mon[36269]: osd.9 [v2:172.21.15.27:6826/2800048793,v1:172.21.15.27:6827/2800048793] boot 2024-05-21T03:18:38.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:38 smithi165 ceph-mon[36269]: osdmap e61: 10 total, 10 up, 10 in 2024-05-21T03:18:38.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:38 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T03:18:38.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:38 smithi165 ceph-mon[36269]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-05-21T03:18:38.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:38 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:38.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:38 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:38.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:38 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:18:38.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:38 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:18:38.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:38 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:18:38.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:38 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:18:38.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:38 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:38.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:38 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:18:38.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:38 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:18:38.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:38 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:38.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:38 smithi027 ceph-mon[32179]: purged_snaps scrub starts 2024-05-21T03:18:38.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:38 smithi027 ceph-mon[32179]: purged_snaps scrub ok 2024-05-21T03:18:38.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:38 smithi027 ceph-mon[32179]: osd.9 [v2:172.21.15.27:6826/2800048793,v1:172.21.15.27:6827/2800048793] boot 2024-05-21T03:18:38.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:38 smithi027 ceph-mon[32179]: osdmap e61: 10 total, 10 up, 10 in 2024-05-21T03:18:38.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:38 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T03:18:38.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:38 smithi027 ceph-mon[32179]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-05-21T03:18:38.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:38 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:38.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:38 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:38.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:38 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:18:38.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:38 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:18:38.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:38 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:18:38.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:38 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:18:38.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:38 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:38.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:38 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:18:38.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:38 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:18:38.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:38 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:39.383 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:39 smithi039 ceph-mon[36429]: Detected new or changed devices on smithi027 2024-05-21T03:18:39.383 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:39 smithi039 ceph-mon[36429]: Adjusting osd_memory_target on smithi027 to 4272M 2024-05-21T03:18:39.383 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:39 smithi039 ceph-mon[36429]: osdmap e62: 10 total, 10 up, 10 in 2024-05-21T03:18:39.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:39 smithi165 ceph-mon[36269]: Detected new or changed devices on smithi027 2024-05-21T03:18:39.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:39 smithi165 ceph-mon[36269]: Adjusting osd_memory_target on smithi027 to 4272M 2024-05-21T03:18:39.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:39 smithi165 ceph-mon[36269]: osdmap e62: 10 total, 10 up, 10 in 2024-05-21T03:18:39.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.617+0000 7f39e5fa9700 1 -- 172.21.15.39:0/3187785532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39e01024a0 msgr2=0x7f39e01028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:18:39.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.617+0000 7f39e5fa9700 1 --2- 172.21.15.39:0/3187785532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39e01024a0 0x7f39e01028c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f39d800b0d0 tx=0x7f39d800b3e0 comp rx=0 tx=0).stop 2024-05-21T03:18:39.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.617+0000 7f39e5fa9700 1 -- 172.21.15.39:0/3187785532 shutdown_connections 2024-05-21T03:18:39.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.617+0000 7f39e5fa9700 1 --2- 172.21.15.39:0/3187785532 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39e0104040 0x7f39e010a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:18:39.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.617+0000 7f39e5fa9700 1 --2- 172.21.15.39:0/3187785532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39e01024a0 0x7f39e01028c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:18:39.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.617+0000 7f39e5fa9700 1 --2- 172.21.15.39:0/3187785532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39e01036a0 0x7f39e0103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:18:39.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.617+0000 7f39e5fa9700 1 -- 172.21.15.39:0/3187785532 >> 172.21.15.39:0/3187785532 conn(0x7f39e00fda40 msgr2=0x7f39e00ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:18:39.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.617+0000 7f39e5fa9700 1 -- 172.21.15.39:0/3187785532 shutdown_connections 2024-05-21T03:18:39.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.617+0000 7f39e5fa9700 1 -- 172.21.15.39:0/3187785532 wait complete. 2024-05-21T03:18:39.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.618+0000 7f39e5fa9700 1 Processor -- start 2024-05-21T03:18:39.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.618+0000 7f39e5fa9700 1 -- start start 2024-05-21T03:18:39.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.618+0000 7f39e5fa9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39e01024a0 0x7f39e0198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:18:39.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.618+0000 7f39e5fa9700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39e01036a0 0x7f39e0198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:18:39.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.619+0000 7f39e5fa9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39e0104040 0x7f39e019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:18:39.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.619+0000 7f39e5fa9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f39e010df30 con 0x7f39e01024a0 2024-05-21T03:18:39.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.619+0000 7f39e5fa9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f39e010ddb0 con 0x7f39e0104040 2024-05-21T03:18:39.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.619+0000 7f39e5fa9700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f39e010e0b0 con 0x7f39e01036a0 2024-05-21T03:18:39.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.619+0000 7f39dffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39e0104040 0x7f39e019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:18:39.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.619+0000 7f39dffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39e0104040 0x7f39e019fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51056/0 (socket says 172.21.15.39:51056) 2024-05-21T03:18:39.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.619+0000 7f39dffff700 1 -- 172.21.15.39:0/3664225907 learned_addr learned my addr 172.21.15.39:0/3664225907 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-21T03:18:39.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.619+0000 7f39df7fe700 1 --2- 172.21.15.39:0/3664225907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39e01024a0 0x7f39e0198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:18:39.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.619+0000 7f39deffd700 1 --2- 172.21.15.39:0/3664225907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39e01036a0 0x7f39e0198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:18:39.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.619+0000 7f39dffff700 1 -- 172.21.15.39:0/3664225907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39e01036a0 msgr2=0x7f39e0198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:18:39.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.619+0000 7f39dffff700 1 --2- 172.21.15.39:0/3664225907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39e01036a0 0x7f39e0198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:18:39.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.619+0000 7f39dffff700 1 -- 172.21.15.39:0/3664225907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39e01024a0 msgr2=0x7f39e0198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:18:39.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.619+0000 7f39dffff700 1 --2- 172.21.15.39:0/3664225907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39e01024a0 0x7f39e0198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:18:39.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.619+0000 7f39dffff700 1 -- 172.21.15.39:0/3664225907 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e01a02a0 con 0x7f39e0104040 2024-05-21T03:18:39.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.619+0000 7f39df7fe700 1 --2- 172.21.15.39:0/3664225907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39e01024a0 0x7f39e0198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:18:39.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.619+0000 7f39dffff700 1 --2- 172.21.15.39:0/3664225907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39e0104040 0x7f39e019fc00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f39d00098d0 tx=0x7f39d0009be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:18:39.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.619+0000 7f39dcff9700 1 -- 172.21.15.39:0/3664225907 <== mon.2 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d000d020 con 0x7f39e0104040 2024-05-21T03:18:39.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.619+0000 7f39e5fa9700 1 -- 172.21.15.39:0/3664225907 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e006b280 con 0x7f39e0104040 2024-05-21T03:18:39.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.619+0000 7f39dcff9700 1 -- 172.21.15.39:0/3664225907 <== mon.2 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39d000d1c0 con 0x7f39e0104040 2024-05-21T03:18:39.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.619+0000 7f39dcff9700 1 -- 172.21.15.39:0/3664225907 <== mon.2 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d000c460 con 0x7f39e0104040 2024-05-21T03:18:39.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.619+0000 7f39e5fa9700 1 -- 172.21.15.39:0/3664225907 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e006b7c0 con 0x7f39e0104040 2024-05-21T03:18:39.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.621+0000 7f39e5fa9700 1 -- 172.21.15.39:0/3664225907 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39e006a230 con 0x7f39e0104040 2024-05-21T03:18:39.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.622+0000 7f39dcff9700 1 -- 172.21.15.39:0/3664225907 <== mon.2 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f39d001f020 con 0x7f39e0104040 2024-05-21T03:18:39.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.622+0000 7f39dcff9700 1 --2- 172.21.15.39:0/3664225907 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f39cc0b2880 0x7f39cc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:18:39.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.623+0000 7f39dcff9700 1 -- 172.21.15.39:0/3664225907 <== mon.2 v2:172.21.15.39:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6261+0+0 (secure 0 0 0) 0x7f39d00e3420 con 0x7f39e0104040 2024-05-21T03:18:39.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.623+0000 7f39df7fe700 1 --2- 172.21.15.39:0/3664225907 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f39cc0b2880 0x7f39cc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:18:39.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.623+0000 7f39df7fe700 1 --2- 172.21.15.39:0/3664225907 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f39cc0b2880 0x7f39cc0b4d40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009fd0 tx=0x7f39d800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:18:39.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.626+0000 7f39dcff9700 1 -- 172.21.15.39:0/3664225907 <== mon.2 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f39d0026030 con 0x7f39e0104040 2024-05-21T03:18:39.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:39 smithi027 ceph-mon[32179]: Detected new or changed devices on smithi027 2024-05-21T03:18:39.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:39 smithi027 ceph-mon[32179]: Adjusting osd_memory_target on smithi027 to 4272M 2024-05-21T03:18:39.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:39 smithi027 ceph-mon[32179]: osdmap e62: 10 total, 10 up, 10 in 2024-05-21T03:18:39.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:18:39.851+0000 7f39e5fa9700 1 -- 172.21.15.39:0/3664225907 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f39e006ba20 con 0x7f39cc0b2880 2024-05-21T03:18:40.476 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:40 smithi039 ceph-mon[36429]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:18:40.476 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:40 smithi039 ceph-mon[36429]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:18:40.477 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:40 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:18:40.477 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:40 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:18:40.477 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:40 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:18:40.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:40 smithi165 ceph-mon[36269]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:18:40.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:40 smithi165 ceph-mon[36269]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:18:40.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:40 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:18:40.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:40 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:18:40.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:40 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:18:40.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:40 smithi027 ceph-mon[32179]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:18:40.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:40 smithi027 ceph-mon[32179]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:18:40.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:40 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:18:40.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:40 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:18:40.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:40 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:18:42.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:42 smithi027 ceph-mon[32179]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:18:42.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:42 smithi039 ceph-mon[36429]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:18:42.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:42 smithi165 ceph-mon[36269]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:18:43.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:43 smithi027 ceph-mon[32179]: from='client.? 172.21.15.39:0/1209708647' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfe1b8ca-ff8a-4b7f-ade5-ad4f00cf60cf"}]: dispatch 2024-05-21T03:18:43.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:43 smithi027 ceph-mon[32179]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfe1b8ca-ff8a-4b7f-ade5-ad4f00cf60cf"}]: dispatch 2024-05-21T03:18:43.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:43 smithi027 ceph-mon[32179]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dfe1b8ca-ff8a-4b7f-ade5-ad4f00cf60cf"}]': finished 2024-05-21T03:18:43.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:43 smithi027 ceph-mon[32179]: osdmap e63: 11 total, 10 up, 11 in 2024-05-21T03:18:43.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:43 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T03:18:43.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:43 smithi027 ceph-mon[32179]: from='client.? 172.21.15.39:0/875758292' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:18:43.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:43 smithi039 ceph-mon[36429]: from='client.? 172.21.15.39:0/1209708647' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfe1b8ca-ff8a-4b7f-ade5-ad4f00cf60cf"}]: dispatch 2024-05-21T03:18:43.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:43 smithi039 ceph-mon[36429]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfe1b8ca-ff8a-4b7f-ade5-ad4f00cf60cf"}]: dispatch 2024-05-21T03:18:43.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:43 smithi039 ceph-mon[36429]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dfe1b8ca-ff8a-4b7f-ade5-ad4f00cf60cf"}]': finished 2024-05-21T03:18:43.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:43 smithi039 ceph-mon[36429]: osdmap e63: 11 total, 10 up, 11 in 2024-05-21T03:18:43.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:43 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T03:18:43.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:43 smithi039 ceph-mon[36429]: from='client.? 172.21.15.39:0/875758292' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:18:43.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:43 smithi165 ceph-mon[36269]: from='client.? 172.21.15.39:0/1209708647' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfe1b8ca-ff8a-4b7f-ade5-ad4f00cf60cf"}]: dispatch 2024-05-21T03:18:43.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:43 smithi165 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfe1b8ca-ff8a-4b7f-ade5-ad4f00cf60cf"}]: dispatch 2024-05-21T03:18:43.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:43 smithi165 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dfe1b8ca-ff8a-4b7f-ade5-ad4f00cf60cf"}]': finished 2024-05-21T03:18:43.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:43 smithi165 ceph-mon[36269]: osdmap e63: 11 total, 10 up, 11 in 2024-05-21T03:18:43.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:43 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T03:18:43.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:43 smithi165 ceph-mon[36269]: from='client.? 172.21.15.39:0/875758292' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:18:45.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:44 smithi027 ceph-mon[32179]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:18:45.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:44 smithi039 ceph-mon[36429]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:18:45.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:44 smithi165 ceph-mon[36269]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:18:47.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:46 smithi027 ceph-mon[32179]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:18:47.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:46 smithi039 ceph-mon[36429]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:18:47.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:46 smithi165 ceph-mon[36269]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:18:49.110 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:48 smithi039 ceph-mon[36429]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:18:49.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:48 smithi027 ceph-mon[32179]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:18:49.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:48 smithi165 ceph-mon[36269]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:18:50.954 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:50 smithi039 ceph-mon[36429]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:18:51.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:50 smithi027 ceph-mon[32179]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:18:51.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:50 smithi165 ceph-mon[36269]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:18:52.063 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:51 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-21T03:18:52.064 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:51 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:18:52.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:51 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-21T03:18:52.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:51 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:18:52.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:51 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-21T03:18:52.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:51 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:18:53.074 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:52 smithi039 ceph-mon[36429]: Deploying daemon osd.10 on smithi039 2024-05-21T03:18:53.186 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:52 smithi039 ceph-mon[36429]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:18:53.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:52 smithi027 ceph-mon[32179]: Deploying daemon osd.10 on smithi039 2024-05-21T03:18:53.187 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:52 smithi027 ceph-mon[32179]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:18:53.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:52 smithi165 ceph-mon[36269]: Deploying daemon osd.10 on smithi039 2024-05-21T03:18:53.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:52 smithi165 ceph-mon[36269]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:18:54.985 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:54 smithi039 ceph-mon[36429]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:18:55.073 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:54 smithi165 ceph-mon[36269]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:18:55.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:54 smithi027 ceph-mon[32179]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:18:57.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:56 smithi027 ceph-mon[32179]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:18:57.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:56 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:57.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:56 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:18:57.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:56 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:57.183 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:56 smithi039 ceph-mon[36429]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:18:57.183 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:56 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:57.184 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:56 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:18:57.184 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:56 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:57.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:56 smithi165 ceph-mon[36269]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:18:57.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:56 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:57.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:56 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:18:57.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:56 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:18:59.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:18:58 smithi039 ceph-mon[36429]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:18:59.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:18:58 smithi027 ceph-mon[32179]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:18:59.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:18:58 smithi165 ceph-mon[36269]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:19:00.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:19:00.573+0000 7f39dcff9700 1 -- 172.21.15.39:0/3664225907 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f39e006ba20 con 0x7f39cc0b2880 2024-05-21T03:19:00.573 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 10 on host 'smithi039' 2024-05-21T03:19:00.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:19:00.576+0000 7f39e5fa9700 1 -- 172.21.15.39:0/3664225907 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f39cc0b2880 msgr2=0x7f39cc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:00.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:19:00.576+0000 7f39e5fa9700 1 --2- 172.21.15.39:0/3664225907 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f39cc0b2880 0x7f39cc0b4d40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009fd0 tx=0x7f39d800b040 comp rx=0 tx=0).stop 2024-05-21T03:19:00.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:19:00.576+0000 7f39e5fa9700 1 -- 172.21.15.39:0/3664225907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39e0104040 msgr2=0x7f39e019fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:00.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:19:00.576+0000 7f39e5fa9700 1 --2- 172.21.15.39:0/3664225907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39e0104040 0x7f39e019fc00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f39d00098d0 tx=0x7f39d0009be0 comp rx=0 tx=0).stop 2024-05-21T03:19:00.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:19:00.576+0000 7f39e5fa9700 1 -- 172.21.15.39:0/3664225907 shutdown_connections 2024-05-21T03:19:00.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:19:00.576+0000 7f39e5fa9700 1 --2- 172.21.15.39:0/3664225907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39e0104040 0x7f39e019fc00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:00.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:19:00.576+0000 7f39e5fa9700 1 --2- 172.21.15.39:0/3664225907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39e01024a0 0x7f39e0198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:00.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:19:00.576+0000 7f39e5fa9700 1 --2- 172.21.15.39:0/3664225907 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f39cc0b2880 0x7f39cc0b4d40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:00.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:19:00.576+0000 7f39e5fa9700 1 --2- 172.21.15.39:0/3664225907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39e01036a0 0x7f39e0198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:00.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:19:00.576+0000 7f39e5fa9700 1 -- 172.21.15.39:0/3664225907 >> 172.21.15.39:0/3664225907 conn(0x7f39e00fda40 msgr2=0x7f39e0106ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:19:00.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:19:00.576+0000 7f39e5fa9700 1 -- 172.21.15.39:0/3664225907 shutdown_connections 2024-05-21T03:19:00.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-21T03:19:00.576+0000 7f39e5fa9700 1 -- 172.21.15.39:0/3664225907 wait complete. 2024-05-21T03:19:01.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:00 smithi039 ceph-mon[36429]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:19:01.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:00 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:01.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:00 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:01.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:00 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:01.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:00 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:19:01.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:00 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:01.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:00 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:01.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:00 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:19:01.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:00 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:01.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:00 smithi039 ceph-mon[36429]: from='osd.10 [v2:172.21.15.39:6824/363500811,v1:172.21.15.39:6825/363500811]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-21T03:19:01.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:00 smithi039 ceph-mon[36429]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-21T03:19:01.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:00 smithi165 ceph-mon[36269]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:19:01.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:00 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:01.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:00 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:01.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:00 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:01.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:00 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:19:01.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:00 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:01.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:00 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:01.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:00 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:19:01.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:00 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:01.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:00 smithi165 ceph-mon[36269]: from='osd.10 [v2:172.21.15.39:6824/363500811,v1:172.21.15.39:6825/363500811]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-21T03:19:01.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:00 smithi165 ceph-mon[36269]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-21T03:19:01.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:00 smithi027 ceph-mon[32179]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:19:01.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:00 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:01.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:00 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:01.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:00 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:01.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:00 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:19:01.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:00 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:01.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:00 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:01.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:00 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:19:01.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:00 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:01.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:00 smithi027 ceph-mon[32179]: from='osd.10 [v2:172.21.15.39:6824/363500811,v1:172.21.15.39:6825/363500811]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-21T03:19:01.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:00 smithi027 ceph-mon[32179]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-21T03:19:01.250 DEBUG:teuthology.orchestra.run.smithi039:osd.10> sudo journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.10.service 2024-05-21T03:19:01.252 INFO:tasks.cephadm:Deploying osd.11 on smithi165 with /dev/vg_nvme/lv_1... 2024-05-21T03:19:01.253 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-21T03:19:01.499 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T03:19:02.722 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:02 smithi039 ceph-mon[36429]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:19:02.722 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:02 smithi039 ceph-mon[36429]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-21T03:19:02.722 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:02 smithi039 ceph-mon[36429]: osdmap e64: 11 total, 10 up, 11 in 2024-05-21T03:19:02.723 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:02 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T03:19:02.723 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:02 smithi039 ceph-mon[36429]: from='osd.10 [v2:172.21.15.39:6824/363500811,v1:172.21.15.39:6825/363500811]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-21T03:19:02.723 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:02 smithi039 ceph-mon[36429]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-21T03:19:02.723 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:19:02 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:19:02.554+0000 7fb4b65d7700 -1 osd.10 0 waiting for initial osdmap 2024-05-21T03:19:02.723 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:19:02 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:19:02.568+0000 7fb4b23ca700 -1 osd.10 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T03:19:02.797 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:02 smithi165 ceph-mon[36269]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:19:02.797 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:02 smithi165 ceph-mon[36269]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-21T03:19:02.797 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:02 smithi165 ceph-mon[36269]: osdmap e64: 11 total, 10 up, 11 in 2024-05-21T03:19:02.797 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:02 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T03:19:02.797 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:02 smithi165 ceph-mon[36269]: from='osd.10 [v2:172.21.15.39:6824/363500811,v1:172.21.15.39:6825/363500811]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-21T03:19:02.797 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:02 smithi165 ceph-mon[36269]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-21T03:19:02.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:02 smithi027 ceph-mon[32179]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:19:02.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:02 smithi027 ceph-mon[32179]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-21T03:19:02.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:02 smithi027 ceph-mon[32179]: osdmap e64: 11 total, 10 up, 11 in 2024-05-21T03:19:02.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:02 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T03:19:02.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:02 smithi027 ceph-mon[32179]: from='osd.10 [v2:172.21.15.39:6824/363500811,v1:172.21.15.39:6825/363500811]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-21T03:19:02.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:02 smithi027 ceph-mon[32179]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-21T03:19:03.518 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-21T03:19:03.518 INFO:teuthology.orchestra.run.smithi165.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-21T03:19:03.518 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10+0 records in 2024-05-21T03:19:03.518 INFO:teuthology.orchestra.run.smithi165.stderr:10+0 records out 2024-05-21T03:19:03.519 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116327 s, 901 MB/s 2024-05-21T03:19:03.519 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping successful for: 2024-05-21T03:19:03.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:03 smithi165 ceph-mon[36269]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-21T03:19:03.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:03 smithi165 ceph-mon[36269]: osdmap e65: 11 total, 10 up, 11 in 2024-05-21T03:19:03.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:03 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T03:19:03.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:03 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T03:19:03.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:03 smithi165 ceph-mon[36269]: osd.10 [v2:172.21.15.39:6824/363500811,v1:172.21.15.39:6825/363500811] boot 2024-05-21T03:19:03.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:03 smithi165 ceph-mon[36269]: osdmap e66: 11 total, 11 up, 11 in 2024-05-21T03:19:03.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:03 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T03:19:03.866 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:03 smithi039 ceph-mon[36429]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-21T03:19:03.866 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:03 smithi039 ceph-mon[36429]: osdmap e65: 11 total, 10 up, 11 in 2024-05-21T03:19:03.867 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:03 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T03:19:03.867 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:03 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T03:19:03.867 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:03 smithi039 ceph-mon[36429]: osd.10 [v2:172.21.15.39:6824/363500811,v1:172.21.15.39:6825/363500811] boot 2024-05-21T03:19:03.867 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:03 smithi039 ceph-mon[36429]: osdmap e66: 11 total, 11 up, 11 in 2024-05-21T03:19:03.867 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:03 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T03:19:03.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:03 smithi027 ceph-mon[32179]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-21T03:19:03.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:03 smithi027 ceph-mon[32179]: osdmap e65: 11 total, 10 up, 11 in 2024-05-21T03:19:03.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:03 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T03:19:03.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:03 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T03:19:03.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:03 smithi027 ceph-mon[32179]: osd.10 [v2:172.21.15.39:6824/363500811,v1:172.21.15.39:6825/363500811] boot 2024-05-21T03:19:03.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:03 smithi027 ceph-mon[32179]: osdmap e66: 11 total, 11 up, 11 in 2024-05-21T03:19:03.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:03 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T03:19:04.162 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi165:vg_nvme/lv_1 2024-05-21T03:19:04.442 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T03:19:04.812 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:04 smithi165 ceph-mon[36269]: purged_snaps scrub starts 2024-05-21T03:19:04.812 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:04 smithi165 ceph-mon[36269]: purged_snaps scrub ok 2024-05-21T03:19:04.812 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:04 smithi165 ceph-mon[36269]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:19:04.812 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:04 smithi165 ceph-mon[36269]: osdmap e67: 11 total, 11 up, 11 in 2024-05-21T03:19:04.813 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:04 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:04.813 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:04 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:04.813 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:04 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:19:04.813 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:04 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:19:04.813 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:04 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:19:04.813 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:04 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:19:04.813 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:04 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:04.813 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:04 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:04.813 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:04 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:19:04.813 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:04 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:04.828 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:04 smithi027 ceph-mon[32179]: purged_snaps scrub starts 2024-05-21T03:19:04.828 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:04 smithi027 ceph-mon[32179]: purged_snaps scrub ok 2024-05-21T03:19:04.828 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:04 smithi027 ceph-mon[32179]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:19:04.828 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:04 smithi027 ceph-mon[32179]: osdmap e67: 11 total, 11 up, 11 in 2024-05-21T03:19:04.829 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:04 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:04.829 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:04 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:04.829 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:04 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:19:04.829 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:04 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:19:04.829 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:04 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:19:04.829 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:04 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:19:04.829 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:04 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:04.829 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:04 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:04.829 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:04 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:19:04.829 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:04 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:05.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:04 smithi039 ceph-mon[36429]: purged_snaps scrub starts 2024-05-21T03:19:05.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:04 smithi039 ceph-mon[36429]: purged_snaps scrub ok 2024-05-21T03:19:05.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:04 smithi039 ceph-mon[36429]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T03:19:05.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:04 smithi039 ceph-mon[36429]: osdmap e67: 11 total, 11 up, 11 in 2024-05-21T03:19:05.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:04 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:05.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:04 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:05.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:04 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:19:05.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:04 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:19:05.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:04 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:19:05.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:04 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:19:05.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:04 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:05.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:04 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:05.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:04 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:19:05.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:04 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:05.812 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:05 smithi165 ceph-mon[36269]: Detected new or changed devices on smithi039 2024-05-21T03:19:05.812 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:05 smithi165 ceph-mon[36269]: Adjusting osd_memory_target on smithi039 to 4272M 2024-05-21T03:19:05.812 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:05 smithi165 ceph-mon[36269]: osdmap e68: 11 total, 11 up, 11 in 2024-05-21T03:19:05.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:05 smithi027 ceph-mon[32179]: Detected new or changed devices on smithi039 2024-05-21T03:19:05.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:05 smithi027 ceph-mon[32179]: Adjusting osd_memory_target on smithi039 to 4272M 2024-05-21T03:19:05.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:05 smithi027 ceph-mon[32179]: osdmap e68: 11 total, 11 up, 11 in 2024-05-21T03:19:06.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:05 smithi039 ceph-mon[36429]: Detected new or changed devices on smithi039 2024-05-21T03:19:06.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:05 smithi039 ceph-mon[36429]: Adjusting osd_memory_target on smithi039 to 4272M 2024-05-21T03:19:06.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:05 smithi039 ceph-mon[36429]: osdmap e68: 11 total, 11 up, 11 in 2024-05-21T03:19:06.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.169+0000 7f7d45c4b700 1 -- 172.21.15.165:0/764093827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d401047a0 msgr2=0x7f7d40104c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:06.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.169+0000 7f7d45c4b700 1 --2- 172.21.15.165:0/764093827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d401047a0 0x7f7d40104c00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7d34009a10 tx=0x7f7d34009d20 comp rx=0 tx=0).stop 2024-05-21T03:19:06.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.170+0000 7f7d45c4b700 1 -- 172.21.15.165:0/764093827 shutdown_connections 2024-05-21T03:19:06.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.170+0000 7f7d45c4b700 1 --2- 172.21.15.165:0/764093827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d40105140 0x7f7d4010b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:06.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.170+0000 7f7d45c4b700 1 --2- 172.21.15.165:0/764093827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d401035a0 0x7f7d401039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:06.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.170+0000 7f7d45c4b700 1 --2- 172.21.15.165:0/764093827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d401047a0 0x7f7d40104c00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:06.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.170+0000 7f7d45c4b700 1 -- 172.21.15.165:0/764093827 >> 172.21.15.165:0/764093827 conn(0x7f7d400feb40 msgr2=0x7f7d40100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:19:06.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.170+0000 7f7d45c4b700 1 -- 172.21.15.165:0/764093827 shutdown_connections 2024-05-21T03:19:06.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.170+0000 7f7d45c4b700 1 -- 172.21.15.165:0/764093827 wait complete. 2024-05-21T03:19:06.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.170+0000 7f7d45c4b700 1 Processor -- start 2024-05-21T03:19:06.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.170+0000 7f7d45c4b700 1 -- start start 2024-05-21T03:19:06.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.171+0000 7f7d45c4b700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d401035a0 0x7f7d401974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:06.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.171+0000 7f7d45c4b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d401047a0 0x7f7d40197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:06.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.171+0000 7f7d45c4b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d40105140 0x7f7d4019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:06.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.171+0000 7f7d45c4b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4010ceb0 con 0x7f7d401047a0 2024-05-21T03:19:06.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.171+0000 7f7d45c4b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4010cd30 con 0x7f7d40105140 2024-05-21T03:19:06.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.171+0000 7f7d45c4b700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4010d030 con 0x7f7d401035a0 2024-05-21T03:19:06.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.171+0000 7f7d3f7fe700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d401035a0 0x7f7d401974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:06.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.171+0000 7f7d3f7fe700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d401035a0 0x7f7d401974f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:58774/0 (socket says 172.21.15.165:58774) 2024-05-21T03:19:06.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.171+0000 7f7d3f7fe700 1 -- 172.21.15.165:0/3777630902 learned_addr learned my addr 172.21.15.165:0/3777630902 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-05-21T03:19:06.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.171+0000 7f7d3ffff700 1 --2- 172.21.15.165:0/3777630902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d40105140 0x7f7d4019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:06.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.171+0000 7f7d3effd700 1 --2- 172.21.15.165:0/3777630902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d401047a0 0x7f7d40197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:06.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.171+0000 7f7d3f7fe700 1 -- 172.21.15.165:0/3777630902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d40105140 msgr2=0x7f7d4019eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:06.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.171+0000 7f7d3f7fe700 1 --2- 172.21.15.165:0/3777630902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d40105140 0x7f7d4019eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:06.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.171+0000 7f7d3f7fe700 1 -- 172.21.15.165:0/3777630902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d401047a0 msgr2=0x7f7d40197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:06.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.171+0000 7f7d3f7fe700 1 --2- 172.21.15.165:0/3777630902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d401047a0 0x7f7d40197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:06.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.171+0000 7f7d3f7fe700 1 -- 172.21.15.165:0/3777630902 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d4019f150 con 0x7f7d401035a0 2024-05-21T03:19:06.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.171+0000 7f7d3f7fe700 1 --2- 172.21.15.165:0/3777630902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d401035a0 0x7f7d401974f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000d9b0 tx=0x7f7d3000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:19:06.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.171+0000 7f7d3cff9700 1 -- 172.21.15.165:0/3777630902 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d30016070 con 0x7f7d401035a0 2024-05-21T03:19:06.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.171+0000 7f7d45c4b700 1 -- 172.21.15.165:0/3777630902 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d4019f440 con 0x7f7d401035a0 2024-05-21T03:19:06.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.171+0000 7f7d3cff9700 1 -- 172.21.15.165:0/3777630902 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d3000be00 con 0x7f7d401035a0 2024-05-21T03:19:06.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.171+0000 7f7d3cff9700 1 -- 172.21.15.165:0/3777630902 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d30010e30 con 0x7f7d401035a0 2024-05-21T03:19:06.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.172+0000 7f7d45c4b700 1 -- 172.21.15.165:0/3777630902 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d4019f9d0 con 0x7f7d401035a0 2024-05-21T03:19:06.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.172+0000 7f7d45c4b700 1 -- 172.21.15.165:0/3777630902 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d4006a230 con 0x7f7d401035a0 2024-05-21T03:19:06.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.174+0000 7f7d3cff9700 1 -- 172.21.15.165:0/3777630902 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7d30019070 con 0x7f7d401035a0 2024-05-21T03:19:06.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.174+0000 7f7d3cff9700 1 --2- 172.21.15.165:0/3777630902 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f7d280b2880 0x7f7d280b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:06.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.175+0000 7f7d3effd700 1 --2- 172.21.15.165:0/3777630902 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f7d280b2880 0x7f7d280b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:06.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.175+0000 7f7d3cff9700 1 -- 172.21.15.165:0/3777630902 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6694+0+0 (secure 0 0 0) 0x7f7d300e2020 con 0x7f7d401035a0 2024-05-21T03:19:06.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.175+0000 7f7d3effd700 1 --2- 172.21.15.165:0/3777630902 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f7d280b2880 0x7f7d280b4d40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7d34009960 tx=0x7f7d34005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:19:06.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.178+0000 7f7d3cff9700 1 -- 172.21.15.165:0/3777630902 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7d30020030 con 0x7f7d401035a0 2024-05-21T03:19:06.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:06.390+0000 7f7d45c4b700 1 -- 172.21.15.165:0/3777630902 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f7d40063a40 con 0x7f7d280b2880 2024-05-21T03:19:06.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:06 smithi165 ceph-mon[36269]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-05-21T03:19:06.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:06 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:19:06.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:06 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:19:06.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:06 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:06.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:06 smithi027 ceph-mon[32179]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-05-21T03:19:06.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:06 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:19:06.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:06 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:19:06.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:06 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:07.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:06 smithi039 ceph-mon[36429]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-05-21T03:19:07.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:06 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T03:19:07.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:06 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T03:19:07.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:06 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:07.918 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:07 smithi039 ceph-mon[36429]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:19:07.926 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:07 smithi165 ceph-mon[36269]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:19:07.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:07 smithi027 ceph-mon[32179]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:19:08.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:08 smithi027 ceph-mon[32179]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-05-21T03:19:09.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:08 smithi039 ceph-mon[36429]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-05-21T03:19:09.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:08 smithi165 ceph-mon[36269]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-05-21T03:19:09.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:09 smithi165 ceph-mon[36269]: from='client.? 172.21.15.165:0/3077035445' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9531145-5d09-4dbd-a050-3159ee09db24"}]: dispatch 2024-05-21T03:19:09.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:09 smithi165 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9531145-5d09-4dbd-a050-3159ee09db24"}]: dispatch 2024-05-21T03:19:09.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:09 smithi165 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f9531145-5d09-4dbd-a050-3159ee09db24"}]': finished 2024-05-21T03:19:09.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:09 smithi165 ceph-mon[36269]: osdmap e69: 12 total, 11 up, 12 in 2024-05-21T03:19:09.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:09 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T03:19:09.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:09 smithi027 ceph-mon[32179]: from='client.? 172.21.15.165:0/3077035445' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9531145-5d09-4dbd-a050-3159ee09db24"}]: dispatch 2024-05-21T03:19:09.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:09 smithi027 ceph-mon[32179]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9531145-5d09-4dbd-a050-3159ee09db24"}]: dispatch 2024-05-21T03:19:09.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:09 smithi027 ceph-mon[32179]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f9531145-5d09-4dbd-a050-3159ee09db24"}]': finished 2024-05-21T03:19:09.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:09 smithi027 ceph-mon[32179]: osdmap e69: 12 total, 11 up, 12 in 2024-05-21T03:19:09.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:09 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T03:19:10.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:09 smithi039 ceph-mon[36429]: from='client.? 172.21.15.165:0/3077035445' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9531145-5d09-4dbd-a050-3159ee09db24"}]: dispatch 2024-05-21T03:19:10.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:09 smithi039 ceph-mon[36429]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9531145-5d09-4dbd-a050-3159ee09db24"}]: dispatch 2024-05-21T03:19:10.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:09 smithi039 ceph-mon[36429]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f9531145-5d09-4dbd-a050-3159ee09db24"}]': finished 2024-05-21T03:19:10.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:09 smithi039 ceph-mon[36429]: osdmap e69: 12 total, 11 up, 12 in 2024-05-21T03:19:10.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:09 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T03:19:10.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:10 smithi027 ceph-mon[32179]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T03:19:10.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:10 smithi027 ceph-mon[32179]: from='client.? 172.21.15.165:0/1434034009' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:19:11.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:10 smithi039 ceph-mon[36429]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T03:19:11.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:10 smithi039 ceph-mon[36429]: from='client.? 172.21.15.165:0/1434034009' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:19:11.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:10 smithi165 ceph-mon[36269]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T03:19:11.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:10 smithi165 ceph-mon[36269]: from='client.? 172.21.15.165:0/1434034009' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T03:19:12.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:12 smithi027 ceph-mon[32179]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-05-21T03:19:12.966 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:12 smithi165 ceph-mon[36269]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-05-21T03:19:13.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:12 smithi039 ceph-mon[36429]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-05-21T03:19:14.893 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:14 smithi165 ceph-mon[36269]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-21T03:19:14.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:14 smithi027 ceph-mon[32179]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-21T03:19:15.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:14 smithi039 ceph-mon[36429]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-21T03:19:16.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:16 smithi165 ceph-mon[36269]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T03:19:16.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:16 smithi027 ceph-mon[32179]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T03:19:17.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:16 smithi039 ceph-mon[36429]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T03:19:17.704 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:17 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-21T03:19:17.704 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:17 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:17.704 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:17 smithi165 ceph-mon[36269]: Deploying daemon osd.11 on smithi165 2024-05-21T03:19:17.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:17 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-21T03:19:17.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:17 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:17.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:17 smithi027 ceph-mon[32179]: Deploying daemon osd.11 on smithi165 2024-05-21T03:19:18.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:17 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-21T03:19:18.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:17 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:18.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:17 smithi039 ceph-mon[36429]: Deploying daemon osd.11 on smithi165 2024-05-21T03:19:18.731 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:18 smithi165 ceph-mon[36269]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-21T03:19:18.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:18 smithi027 ceph-mon[32179]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-21T03:19:19.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:18 smithi039 ceph-mon[36429]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-21T03:19:20.865 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:20 smithi165 ceph-mon[36269]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-21T03:19:20.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:20 smithi027 ceph-mon[32179]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-21T03:19:21.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:20 smithi039 ceph-mon[36429]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-21T03:19:22.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:22 smithi165 ceph-mon[36269]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-21T03:19:22.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:22 smithi027 ceph-mon[32179]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-21T03:19:23.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:22 smithi039 ceph-mon[36429]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-21T03:19:23.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:23 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:23.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:23 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:23.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:23 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:19:24.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:23 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:24.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:23 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:24.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:23 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:19:24.022 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:23 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:24.022 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:23 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:24.022 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:23 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:19:24.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:24 smithi165 ceph-mon[36269]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-21T03:19:24.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:24 smithi027 ceph-mon[32179]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-21T03:19:25.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:24 smithi039 ceph-mon[36429]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-21T03:19:26.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:26.415+0000 7f7d3cff9700 1 -- 172.21.15.165:0/3777630902 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f7d40063a40 con 0x7f7d280b2880 2024-05-21T03:19:26.417 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 11 on host 'smithi165' 2024-05-21T03:19:26.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:26.418+0000 7f7d45c4b700 1 -- 172.21.15.165:0/3777630902 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f7d280b2880 msgr2=0x7f7d280b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:26.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:26.418+0000 7f7d45c4b700 1 --2- 172.21.15.165:0/3777630902 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f7d280b2880 0x7f7d280b4d40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7d34009960 tx=0x7f7d34005fb0 comp rx=0 tx=0).stop 2024-05-21T03:19:26.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:26.418+0000 7f7d45c4b700 1 -- 172.21.15.165:0/3777630902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d401035a0 msgr2=0x7f7d401974f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:26.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:26.418+0000 7f7d45c4b700 1 --2- 172.21.15.165:0/3777630902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d401035a0 0x7f7d401974f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000d9b0 tx=0x7f7d3000dcc0 comp rx=0 tx=0).stop 2024-05-21T03:19:26.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:26.418+0000 7f7d45c4b700 1 -- 172.21.15.165:0/3777630902 shutdown_connections 2024-05-21T03:19:26.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:26.418+0000 7f7d45c4b700 1 --2- 172.21.15.165:0/3777630902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d40105140 0x7f7d4019eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:26.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:26.418+0000 7f7d45c4b700 1 --2- 172.21.15.165:0/3777630902 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f7d280b2880 0x7f7d280b4d40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:26.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:26.418+0000 7f7d45c4b700 1 --2- 172.21.15.165:0/3777630902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d401035a0 0x7f7d401974f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:26.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:26.418+0000 7f7d45c4b700 1 --2- 172.21.15.165:0/3777630902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d401047a0 0x7f7d40197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:26.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:26.418+0000 7f7d45c4b700 1 -- 172.21.15.165:0/3777630902 >> 172.21.15.165:0/3777630902 conn(0x7f7d400feb40 msgr2=0x7f7d40107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:19:26.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:26.418+0000 7f7d45c4b700 1 -- 172.21.15.165:0/3777630902 shutdown_connections 2024-05-21T03:19:26.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:26.418+0000 7f7d45c4b700 1 -- 172.21.15.165:0/3777630902 wait complete. 2024-05-21T03:19:26.768 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:26 smithi165 ceph-mon[36269]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-21T03:19:26.768 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:26 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:26.768 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:26 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:26.768 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:26 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:26.768 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:26 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:19:26.769 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:26 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:26.769 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:26 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:26.769 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:26 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:26.769 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:26 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:19:26.769 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:26 smithi165 ceph-mon[36269]: from='osd.11 [v2:172.21.15.165:6824/2274332243,v1:172.21.15.165:6825/2274332243]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-21T03:19:26.769 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:26 smithi165 ceph-mon[36269]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-21T03:19:26.821 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:26 smithi039 ceph-mon[36429]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-21T03:19:26.822 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:26 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:26.822 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:26 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:26.822 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:26 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:26.822 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:26 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:19:26.822 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:26 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:26.822 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:26 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:26.822 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:26 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:26.822 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:26 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:19:26.822 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:26 smithi039 ceph-mon[36429]: from='osd.11 [v2:172.21.15.165:6824/2274332243,v1:172.21.15.165:6825/2274332243]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-21T03:19:26.822 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:26 smithi039 ceph-mon[36429]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-21T03:19:26.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:26 smithi027 ceph-mon[32179]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-21T03:19:26.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:26 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:26.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:26 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:26.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:26 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:26.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:26 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:19:26.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:26 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:26.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:26 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:26.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:26 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:26.929 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:26 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:19:26.929 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:26 smithi027 ceph-mon[32179]: from='osd.11 [v2:172.21.15.165:6824/2274332243,v1:172.21.15.165:6825/2274332243]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-21T03:19:26.929 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:26 smithi027 ceph-mon[32179]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-21T03:19:27.083 DEBUG:teuthology.orchestra.run.smithi165:osd.11> sudo journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.11.service 2024-05-21T03:19:27.085 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-21T03:19:27.085 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd stat -f json 2024-05-21T03:19:27.343 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:19:28.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:28 smithi165 ceph-mon[36269]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-21T03:19:28.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:28 smithi165 ceph-mon[36269]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-21T03:19:28.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:28 smithi165 ceph-mon[36269]: osdmap e70: 12 total, 11 up, 12 in 2024-05-21T03:19:28.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:28 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T03:19:28.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:28 smithi165 ceph-mon[36269]: from='osd.11 [v2:172.21.15.165:6824/2274332243,v1:172.21.15.165:6825/2274332243]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-21T03:19:28.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:28 smithi165 ceph-mon[36269]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-21T03:19:28.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:28 smithi027 ceph-mon[32179]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-21T03:19:28.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:28 smithi027 ceph-mon[32179]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-21T03:19:28.661 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:28 smithi027 ceph-mon[32179]: osdmap e70: 12 total, 11 up, 12 in 2024-05-21T03:19:28.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:28 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T03:19:28.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:28 smithi027 ceph-mon[32179]: from='osd.11 [v2:172.21.15.165:6824/2274332243,v1:172.21.15.165:6825/2274332243]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-21T03:19:28.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:28 smithi027 ceph-mon[32179]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-21T03:19:28.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:28 smithi039 ceph-mon[36429]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-21T03:19:28.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:28 smithi039 ceph-mon[36429]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-21T03:19:28.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:28 smithi039 ceph-mon[36429]: osdmap e70: 12 total, 11 up, 12 in 2024-05-21T03:19:28.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:28 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T03:19:28.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:28 smithi039 ceph-mon[36429]: from='osd.11 [v2:172.21.15.165:6824/2274332243,v1:172.21.15.165:6825/2274332243]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-21T03:19:28.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:28 smithi039 ceph-mon[36429]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-05-21T03:19:29.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.314+0000 7f7c3df77700 1 -- 172.21.15.27:0/92341338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c38109280 msgr2=0x7f7c3810b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:29.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.314+0000 7f7c3df77700 1 --2- 172.21.15.27:0/92341338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c38109280 0x7f7c3810b6f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000b0a0 tx=0x7f7c3000b3b0 comp rx=0 tx=0).stop 2024-05-21T03:19:29.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.314+0000 7f7c3df77700 1 -- 172.21.15.27:0/92341338 shutdown_connections 2024-05-21T03:19:29.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.314+0000 7f7c3df77700 1 --2- 172.21.15.27:0/92341338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c38109280 0x7f7c3810b6f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:29.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.314+0000 7f7c3df77700 1 --2- 172.21.15.27:0/92341338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c38104020 0x7f7c38106410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:29.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.314+0000 7f7c3df77700 1 --2- 172.21.15.27:0/92341338 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c38106950 0x7f7c38108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:29.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.314+0000 7f7c3df77700 1 -- 172.21.15.27:0/92341338 >> 172.21.15.27:0/92341338 conn(0x7f7c380fda30 msgr2=0x7f7c380ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:19:29.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.314+0000 7f7c3df77700 1 -- 172.21.15.27:0/92341338 shutdown_connections 2024-05-21T03:19:29.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.314+0000 7f7c3df77700 1 -- 172.21.15.27:0/92341338 wait complete. 2024-05-21T03:19:29.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.314+0000 7f7c3df77700 1 Processor -- start 2024-05-21T03:19:29.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.315+0000 7f7c3df77700 1 -- start start 2024-05-21T03:19:29.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.315+0000 7f7c3df77700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c38104020 0x7f7c38199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:29.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.315+0000 7f7c3df77700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c38106950 0x7f7c38199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:29.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.315+0000 7f7c3df77700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c38109280 0x7f7c381a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:29.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.315+0000 7f7c3df77700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3810f010 con 0x7f7c38104020 2024-05-21T03:19:29.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.315+0000 7f7c3df77700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3810ee90 con 0x7f7c38109280 2024-05-21T03:19:29.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.315+0000 7f7c3df77700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3810f190 con 0x7f7c38106950 2024-05-21T03:19:29.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.315+0000 7f7c377fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c38104020 0x7f7c38199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:29.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.315+0000 7f7c377fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c38104020 0x7f7c38199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58542/0 (socket says 172.21.15.27:58542) 2024-05-21T03:19:29.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.315+0000 7f7c377fe700 1 -- 172.21.15.27:0/4118398040 learned_addr learned my addr 172.21.15.27:0/4118398040 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:19:29.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.315+0000 7f7c36ffd700 1 --2- 172.21.15.27:0/4118398040 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c38106950 0x7f7c38199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:29.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.315+0000 7f7c37fff700 1 --2- 172.21.15.27:0/4118398040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c38109280 0x7f7c381a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:29.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.315+0000 7f7c377fe700 1 -- 172.21.15.27:0/4118398040 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c38106950 msgr2=0x7f7c38199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:29.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.315+0000 7f7c377fe700 1 --2- 172.21.15.27:0/4118398040 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c38106950 0x7f7c38199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:29.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.315+0000 7f7c377fe700 1 -- 172.21.15.27:0/4118398040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c38109280 msgr2=0x7f7c381a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:29.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.315+0000 7f7c377fe700 1 --2- 172.21.15.27:0/4118398040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c38109280 0x7f7c381a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:29.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.315+0000 7f7c377fe700 1 -- 172.21.15.27:0/4118398040 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c381a12b0 con 0x7f7c38104020 2024-05-21T03:19:29.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.315+0000 7f7c36ffd700 1 --2- 172.21.15.27:0/4118398040 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c38106950 0x7f7c38199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:19:29.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.316+0000 7f7c377fe700 1 --2- 172.21.15.27:0/4118398040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c38104020 0x7f7c38199650 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800d9b0 tx=0x7f7c2800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:19:29.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.316+0000 7f7c34ff9700 1 -- 172.21.15.27:0/4118398040 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c28016070 con 0x7f7c38104020 2024-05-21T03:19:29.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.316+0000 7f7c3df77700 1 -- 172.21.15.27:0/4118398040 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c381a15a0 con 0x7f7c38104020 2024-05-21T03:19:29.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.316+0000 7f7c34ff9700 1 -- 172.21.15.27:0/4118398040 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c28004830 con 0x7f7c38104020 2024-05-21T03:19:29.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.316+0000 7f7c34ff9700 1 -- 172.21.15.27:0/4118398040 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c28005440 con 0x7f7c38104020 2024-05-21T03:19:29.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.316+0000 7f7c3df77700 1 -- 172.21.15.27:0/4118398040 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c381a1b30 con 0x7f7c38104020 2024-05-21T03:19:29.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.317+0000 7f7c3df77700 1 -- 172.21.15.27:0/4118398040 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c3806a230 con 0x7f7c38104020 2024-05-21T03:19:29.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.318+0000 7f7c34ff9700 1 -- 172.21.15.27:0/4118398040 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7c280055e0 con 0x7f7c38104020 2024-05-21T03:19:29.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.319+0000 7f7c34ff9700 1 --2- 172.21.15.27:0/4118398040 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f7c200b2880 0x7f7c200b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:29.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.319+0000 7f7c36ffd700 1 --2- 172.21.15.27:0/4118398040 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f7c200b2880 0x7f7c200b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:29.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.319+0000 7f7c34ff9700 1 -- 172.21.15.27:0/4118398040 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6847+0+0 (secure 0 0 0) 0x7f7c280e19c0 con 0x7f7c38104020 2024-05-21T03:19:29.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.320+0000 7f7c36ffd700 1 --2- 172.21.15.27:0/4118398040 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f7c200b2880 0x7f7c200b4d40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c009fd0 tx=0x7f7c2c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:19:29.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.328+0000 7f7c34ff9700 1 -- 172.21.15.27:0/4118398040 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7c280e3e20 con 0x7f7c38104020 2024-05-21T03:19:29.497 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:29 smithi027 ceph-mon[32179]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-05-21T03:19:29.497 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:29 smithi027 ceph-mon[32179]: osdmap e71: 12 total, 11 up, 12 in 2024-05-21T03:19:29.497 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:29 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T03:19:29.498 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:29 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T03:19:29.498 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:29 smithi027 ceph-mon[32179]: from='osd.11 ' entity='osd.11' 2024-05-21T03:19:29.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.548+0000 7f7c3df77700 1 -- 172.21.15.27:0/4118398040 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f7c38064150 con 0x7f7c38104020 2024-05-21T03:19:29.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.549+0000 7f7c34ff9700 1 -- 172.21.15.27:0/4118398040 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7f7c2801f020 con 0x7f7c38104020 2024-05-21T03:19:29.549 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:19:29.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.553+0000 7f7c3df77700 1 -- 172.21.15.27:0/4118398040 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f7c200b2880 msgr2=0x7f7c200b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:29.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.553+0000 7f7c3df77700 1 --2- 172.21.15.27:0/4118398040 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f7c200b2880 0x7f7c200b4d40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c009fd0 tx=0x7f7c2c009260 comp rx=0 tx=0).stop 2024-05-21T03:19:29.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.553+0000 7f7c3df77700 1 -- 172.21.15.27:0/4118398040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c38104020 msgr2=0x7f7c38199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:29.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.553+0000 7f7c3df77700 1 --2- 172.21.15.27:0/4118398040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c38104020 0x7f7c38199650 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800d9b0 tx=0x7f7c2800dcc0 comp rx=0 tx=0).stop 2024-05-21T03:19:29.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.553+0000 7f7c3df77700 1 -- 172.21.15.27:0/4118398040 shutdown_connections 2024-05-21T03:19:29.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.553+0000 7f7c3df77700 1 --2- 172.21.15.27:0/4118398040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c38109280 0x7f7c381a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:29.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.553+0000 7f7c3df77700 1 --2- 172.21.15.27:0/4118398040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c38104020 0x7f7c38199650 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:29.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.553+0000 7f7c3df77700 1 --2- 172.21.15.27:0/4118398040 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f7c200b2880 0x7f7c200b4d40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:29.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.553+0000 7f7c3df77700 1 --2- 172.21.15.27:0/4118398040 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c38106950 0x7f7c38199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:29.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.553+0000 7f7c3df77700 1 -- 172.21.15.27:0/4118398040 >> 172.21.15.27:0/4118398040 conn(0x7f7c380fda30 msgr2=0x7f7c38104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:19:29.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.553+0000 7f7c3df77700 1 -- 172.21.15.27:0/4118398040 shutdown_connections 2024-05-21T03:19:29.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:29.553+0000 7f7c3df77700 1 -- 172.21.15.27:0/4118398040 wait complete. 2024-05-21T03:19:29.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:29 smithi039 ceph-mon[36429]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-05-21T03:19:29.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:29 smithi039 ceph-mon[36429]: osdmap e71: 12 total, 11 up, 12 in 2024-05-21T03:19:29.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:29 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T03:19:29.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:29 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T03:19:29.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:29 smithi039 ceph-mon[36429]: from='osd.11 ' entity='osd.11' 2024-05-21T03:19:29.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:29 smithi165 ceph-mon[36269]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-05-21T03:19:29.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:29 smithi165 ceph-mon[36269]: osdmap e71: 12 total, 11 up, 12 in 2024-05-21T03:19:29.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:29 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T03:19:29.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:29 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T03:19:29.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:29 smithi165 ceph-mon[36269]: from='osd.11 ' entity='osd.11' 2024-05-21T03:19:29.771 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:19:29 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:19:29.292+0000 7f7108072700 -1 osd.11 0 waiting for initial osdmap 2024-05-21T03:19:29.772 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:19:29 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:19:29.468+0000 7f70ffe5d700 -1 osd.11 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T03:19:30.163 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":71,"num_osds":12,"num_up_osds":11,"osd_up_since":1716261543,"num_in_osds":12,"osd_in_since":1716261549,"num_remapped_pgs":0} 2024-05-21T03:19:30.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:30 smithi027 ceph-mon[32179]: purged_snaps scrub starts 2024-05-21T03:19:30.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:30 smithi027 ceph-mon[32179]: purged_snaps scrub ok 2024-05-21T03:19:30.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:30 smithi027 ceph-mon[32179]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-21T03:19:30.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:30 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T03:19:30.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:30 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/4118398040' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-21T03:19:30.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:30 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:30.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:30 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:30.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:30 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:19:30.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:30 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:19:30.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:30 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:19:30.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:30 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:19:30.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:30 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:30.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:30 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:30.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:30 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:19:30.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:30 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:30.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:30 smithi039 ceph-mon[36429]: purged_snaps scrub starts 2024-05-21T03:19:30.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:30 smithi039 ceph-mon[36429]: purged_snaps scrub ok 2024-05-21T03:19:30.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:30 smithi039 ceph-mon[36429]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-21T03:19:30.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:30 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T03:19:30.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:30 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/4118398040' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-21T03:19:30.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:30 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:30.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:30 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:30.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:30 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:19:30.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:30 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:19:30.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:30 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:19:30.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:30 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:19:30.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:30 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:30.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:30 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:30.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:30 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:19:30.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:30 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:30.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:30 smithi165 ceph-mon[36269]: purged_snaps scrub starts 2024-05-21T03:19:30.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:30 smithi165 ceph-mon[36269]: purged_snaps scrub ok 2024-05-21T03:19:30.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:30 smithi165 ceph-mon[36269]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-21T03:19:30.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:30 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T03:19:30.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:30 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/4118398040' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-21T03:19:30.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:30 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:30.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:30 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:30.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:30 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:19:30.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:30 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:19:30.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:30 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:19:30.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:30 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:19:30.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:30 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:30.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:30 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:30.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:30 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:19:30.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:30 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:31.164 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd stat -f json 2024-05-21T03:19:31.422 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:19:31.545 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:31 smithi027 ceph-mon[32179]: Detected new or changed devices on smithi165 2024-05-21T03:19:31.545 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:31 smithi027 ceph-mon[32179]: Adjusting osd_memory_target on smithi165 to 4272M 2024-05-21T03:19:31.545 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:31 smithi027 ceph-mon[32179]: osd.11 [v2:172.21.15.165:6824/2274332243,v1:172.21.15.165:6825/2274332243] boot 2024-05-21T03:19:31.545 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:31 smithi027 ceph-mon[32179]: osdmap e72: 12 total, 12 up, 12 in 2024-05-21T03:19:31.545 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:31 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T03:19:31.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:31 smithi039 ceph-mon[36429]: Detected new or changed devices on smithi165 2024-05-21T03:19:31.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:31 smithi039 ceph-mon[36429]: Adjusting osd_memory_target on smithi165 to 4272M 2024-05-21T03:19:31.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:31 smithi039 ceph-mon[36429]: osd.11 [v2:172.21.15.165:6824/2274332243,v1:172.21.15.165:6825/2274332243] boot 2024-05-21T03:19:31.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:31 smithi039 ceph-mon[36429]: osdmap e72: 12 total, 12 up, 12 in 2024-05-21T03:19:31.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:31 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T03:19:31.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:31 smithi165 ceph-mon[36269]: Detected new or changed devices on smithi165 2024-05-21T03:19:31.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:31 smithi165 ceph-mon[36269]: Adjusting osd_memory_target on smithi165 to 4272M 2024-05-21T03:19:31.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:31 smithi165 ceph-mon[36269]: osd.11 [v2:172.21.15.165:6824/2274332243,v1:172.21.15.165:6825/2274332243] boot 2024-05-21T03:19:31.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:31 smithi165 ceph-mon[36269]: osdmap e72: 12 total, 12 up, 12 in 2024-05-21T03:19:31.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:31 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T03:19:32.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:32 smithi027 ceph-mon[32179]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-21T03:19:32.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:32 smithi027 ceph-mon[32179]: osdmap e73: 12 total, 12 up, 12 in 2024-05-21T03:19:32.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:32 smithi039 ceph-mon[36429]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-21T03:19:32.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:32 smithi039 ceph-mon[36429]: osdmap e73: 12 total, 12 up, 12 in 2024-05-21T03:19:32.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:32 smithi165 ceph-mon[36269]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-21T03:19:32.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:32 smithi165 ceph-mon[36269]: osdmap e73: 12 total, 12 up, 12 in 2024-05-21T03:19:33.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.265+0000 7f62f8233700 1 -- 172.21.15.27:0/2477256092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62f0104040 msgr2=0x7f62f010a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:33.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.265+0000 7f62f8233700 1 --2- 172.21.15.27:0/2477256092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62f0104040 0x7f62f010a8f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f62e800b0d0 tx=0x7f62e800b3e0 comp rx=0 tx=0).stop 2024-05-21T03:19:33.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.265+0000 7f62f8233700 1 -- 172.21.15.27:0/2477256092 shutdown_connections 2024-05-21T03:19:33.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.265+0000 7f62f8233700 1 --2- 172.21.15.27:0/2477256092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62f0104040 0x7f62f010a8f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:33.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.265+0000 7f62f8233700 1 --2- 172.21.15.27:0/2477256092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62f01024a0 0x7f62f01028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:33.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.265+0000 7f62f8233700 1 --2- 172.21.15.27:0/2477256092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62f01036a0 0x7f62f0103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:33.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.265+0000 7f62f8233700 1 -- 172.21.15.27:0/2477256092 >> 172.21.15.27:0/2477256092 conn(0x7f62f00fda40 msgr2=0x7f62f00ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:19:33.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.265+0000 7f62f8233700 1 -- 172.21.15.27:0/2477256092 shutdown_connections 2024-05-21T03:19:33.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.265+0000 7f62f8233700 1 -- 172.21.15.27:0/2477256092 wait complete. 2024-05-21T03:19:33.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.266+0000 7f62f8233700 1 Processor -- start 2024-05-21T03:19:33.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.266+0000 7f62f8233700 1 -- start start 2024-05-21T03:19:33.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.266+0000 7f62f8233700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62f01024a0 0x7f62f0199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:33.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.266+0000 7f62f8233700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62f01036a0 0x7f62f0199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:33.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.266+0000 7f62f8233700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62f0104040 0x7f62f01a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:33.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.266+0000 7f62f8233700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f62f010df90 con 0x7f62f01024a0 2024-05-21T03:19:33.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.266+0000 7f62f8233700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f62f010de10 con 0x7f62f0104040 2024-05-21T03:19:33.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.266+0000 7f62f8233700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f62f010e110 con 0x7f62f01036a0 2024-05-21T03:19:33.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.266+0000 7f62f5fcf700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62f01024a0 0x7f62f0199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:33.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.266+0000 7f62f5fcf700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62f01024a0 0x7f62f0199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33870/0 (socket says 172.21.15.27:33870) 2024-05-21T03:19:33.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.266+0000 7f62f5fcf700 1 -- 172.21.15.27:0/2766931555 learned_addr learned my addr 172.21.15.27:0/2766931555 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:19:33.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.266+0000 7f62f67d0700 1 --2- 172.21.15.27:0/2766931555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62f0104040 0x7f62f01a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:33.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.266+0000 7f62f57ce700 1 --2- 172.21.15.27:0/2766931555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62f01036a0 0x7f62f0199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:33.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.267+0000 7f62f5fcf700 1 -- 172.21.15.27:0/2766931555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62f01036a0 msgr2=0x7f62f0199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:33.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.267+0000 7f62f5fcf700 1 --2- 172.21.15.27:0/2766931555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62f01036a0 0x7f62f0199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:33.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.267+0000 7f62f5fcf700 1 -- 172.21.15.27:0/2766931555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62f0104040 msgr2=0x7f62f01a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:33.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.267+0000 7f62f5fcf700 1 --2- 172.21.15.27:0/2766931555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62f0104040 0x7f62f01a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:33.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.267+0000 7f62f5fcf700 1 -- 172.21.15.27:0/2766931555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f01a13b0 con 0x7f62f01024a0 2024-05-21T03:19:33.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.267+0000 7f62f57ce700 1 --2- 172.21.15.27:0/2766931555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62f01036a0 0x7f62f0199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:19:33.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.267+0000 7f62f5fcf700 1 --2- 172.21.15.27:0/2766931555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62f01024a0 0x7f62f0199750 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f62ec00d9b0 tx=0x7f62ec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:19:33.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.267+0000 7f62e2ffd700 1 -- 172.21.15.27:0/2766931555 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec017030 con 0x7f62f01024a0 2024-05-21T03:19:33.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.267+0000 7f62e2ffd700 1 -- 172.21.15.27:0/2766931555 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62ec0044d0 con 0x7f62f01024a0 2024-05-21T03:19:33.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.267+0000 7f62f8233700 1 -- 172.21.15.27:0/2766931555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f01a16a0 con 0x7f62f01024a0 2024-05-21T03:19:33.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.267+0000 7f62e2ffd700 1 -- 172.21.15.27:0/2766931555 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec0039e0 con 0x7f62f01024a0 2024-05-21T03:19:33.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.267+0000 7f62f8233700 1 -- 172.21.15.27:0/2766931555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f01a1cb0 con 0x7f62f01024a0 2024-05-21T03:19:33.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.268+0000 7f62f8233700 1 -- 172.21.15.27:0/2766931555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62f006a230 con 0x7f62f01024a0 2024-05-21T03:19:33.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.269+0000 7f62e2ffd700 1 -- 172.21.15.27:0/2766931555 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f62ec01a070 con 0x7f62f01024a0 2024-05-21T03:19:33.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.270+0000 7f62e2ffd700 1 --2- 172.21.15.27:0/2766931555 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f62dc0b2900 0x7f62dc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:33.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.270+0000 7f62e2ffd700 1 -- 172.21.15.27:0/2766931555 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f62ec0e19d0 con 0x7f62f01024a0 2024-05-21T03:19:33.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.270+0000 7f62f57ce700 1 --2- 172.21.15.27:0/2766931555 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f62dc0b2900 0x7f62dc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:33.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.271+0000 7f62f57ce700 1 --2- 172.21.15.27:0/2766931555 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f62dc0b2900 0x7f62dc0b4dc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f62e40099d0 tx=0x7f62e4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:19:33.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.274+0000 7f62e2ffd700 1 -- 172.21.15.27:0/2766931555 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f62ec0a9c10 con 0x7f62f01024a0 2024-05-21T03:19:33.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.488+0000 7f62f8233700 1 -- 172.21.15.27:0/2766931555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f62f01a1fd0 con 0x7f62f01024a0 2024-05-21T03:19:33.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.488+0000 7f62e2ffd700 1 -- 172.21.15.27:0/2766931555 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7f62ec0a9510 con 0x7f62f01024a0 2024-05-21T03:19:33.488 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:19:33.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.491+0000 7f62f8233700 1 -- 172.21.15.27:0/2766931555 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f62dc0b2900 msgr2=0x7f62dc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:33.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.491+0000 7f62f8233700 1 --2- 172.21.15.27:0/2766931555 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f62dc0b2900 0x7f62dc0b4dc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f62e40099d0 tx=0x7f62e4008040 comp rx=0 tx=0).stop 2024-05-21T03:19:33.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.491+0000 7f62f8233700 1 -- 172.21.15.27:0/2766931555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62f01024a0 msgr2=0x7f62f0199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:33.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.491+0000 7f62f8233700 1 --2- 172.21.15.27:0/2766931555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62f01024a0 0x7f62f0199750 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f62ec00d9b0 tx=0x7f62ec00dcc0 comp rx=0 tx=0).stop 2024-05-21T03:19:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.491+0000 7f62f8233700 1 -- 172.21.15.27:0/2766931555 shutdown_connections 2024-05-21T03:19:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.491+0000 7f62f8233700 1 --2- 172.21.15.27:0/2766931555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62f0104040 0x7f62f01a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.491+0000 7f62f8233700 1 --2- 172.21.15.27:0/2766931555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62f01024a0 0x7f62f0199750 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.491+0000 7f62f8233700 1 --2- 172.21.15.27:0/2766931555 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f62dc0b2900 0x7f62dc0b4dc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.491+0000 7f62f8233700 1 --2- 172.21.15.27:0/2766931555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62f01036a0 0x7f62f0199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.491+0000 7f62f8233700 1 -- 172.21.15.27:0/2766931555 >> 172.21.15.27:0/2766931555 conn(0x7f62f00fda40 msgr2=0x7f62f00ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:19:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.491+0000 7f62f8233700 1 -- 172.21.15.27:0/2766931555 shutdown_connections 2024-05-21T03:19:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:33.491+0000 7f62f8233700 1 -- 172.21.15.27:0/2766931555 wait complete. 2024-05-21T03:19:34.205 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":73,"num_osds":12,"num_up_osds":12,"osd_up_since":1716261570,"num_in_osds":12,"osd_in_since":1716261549,"num_remapped_pgs":0} 2024-05-21T03:19:34.206 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-21T03:19:34.426 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:34 smithi027 ceph-mon[32179]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:34.426 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:34 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2766931555' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-21T03:19:34.454 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:19:34.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:34 smithi039 ceph-mon[36429]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:34.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:34 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/2766931555' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-21T03:19:34.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:34 smithi165 ceph-mon[36269]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:34.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:34 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/2766931555' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-21T03:19:36.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.052+0000 7fbbff0c5700 1 -- 172.21.15.27:0/339828542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbf81041a0 msgr2=0x7fbbf81081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:36.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.052+0000 7fbbff0c5700 1 --2- 172.21.15.27:0/339828542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbf81041a0 0x7fbbf81081c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbbe40099c0 tx=0x7fbbe4009cd0 comp rx=0 tx=0).stop 2024-05-21T03:19:36.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.052+0000 7fbbff0c5700 1 -- 172.21.15.27:0/339828542 shutdown_connections 2024-05-21T03:19:36.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.052+0000 7fbbff0c5700 1 --2- 172.21.15.27:0/339828542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbf81088f0 0x7fbbf8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:36.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.052+0000 7fbbff0c5700 1 --2- 172.21.15.27:0/339828542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbf81037f0 0x7fbbf8103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:36.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.052+0000 7fbbff0c5700 1 --2- 172.21.15.27:0/339828542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbf81041a0 0x7fbbf81081c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:36.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.052+0000 7fbbff0c5700 1 -- 172.21.15.27:0/339828542 >> 172.21.15.27:0/339828542 conn(0x7fbbf807a6a0 msgr2=0x7fbbf807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:19:36.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.052+0000 7fbbff0c5700 1 -- 172.21.15.27:0/339828542 shutdown_connections 2024-05-21T03:19:36.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.052+0000 7fbbff0c5700 1 -- 172.21.15.27:0/339828542 wait complete. 2024-05-21T03:19:36.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.052+0000 7fbbff0c5700 1 Processor -- start 2024-05-21T03:19:36.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.052+0000 7fbbff0c5700 1 -- start start 2024-05-21T03:19:36.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.053+0000 7fbbff0c5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbf81037f0 0x7fbbf819be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:36.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.053+0000 7fbbff0c5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbf81041a0 0x7fbbf819c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:36.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.053+0000 7fbbff0c5700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbf81088f0 0x7fbbf81a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:36.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.053+0000 7fbbff0c5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf8113730 con 0x7fbbf81041a0 2024-05-21T03:19:36.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.053+0000 7fbbff0c5700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf81135b0 con 0x7fbbf81037f0 2024-05-21T03:19:36.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.053+0000 7fbbff0c5700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf81138b0 con 0x7fbbf81088f0 2024-05-21T03:19:36.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.053+0000 7fbbf7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbf81041a0 0x7fbbf819c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:36.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.053+0000 7fbbf7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbf81041a0 0x7fbbf819c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33886/0 (socket says 172.21.15.27:33886) 2024-05-21T03:19:36.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.053+0000 7fbbf7fff700 1 -- 172.21.15.27:0/410065805 learned_addr learned my addr 172.21.15.27:0/410065805 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:19:36.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.053+0000 7fbbfd662700 1 --2- 172.21.15.27:0/410065805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbf81088f0 0x7fbbf81a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:36.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.053+0000 7fbbfce61700 1 --2- 172.21.15.27:0/410065805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbf81037f0 0x7fbbf819be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:36.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.053+0000 7fbbfd662700 1 -- 172.21.15.27:0/410065805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbf81037f0 msgr2=0x7fbbf819be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:36.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.053+0000 7fbbfd662700 1 --2- 172.21.15.27:0/410065805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbf81037f0 0x7fbbf819be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:36.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.053+0000 7fbbfd662700 1 -- 172.21.15.27:0/410065805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbf81041a0 msgr2=0x7fbbf819c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:36.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.053+0000 7fbbfd662700 1 --2- 172.21.15.27:0/410065805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbf81041a0 0x7fbbf819c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:36.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.053+0000 7fbbfd662700 1 -- 172.21.15.27:0/410065805 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf81a0e40 con 0x7fbbf81088f0 2024-05-21T03:19:36.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.054+0000 7fbbfd662700 1 --2- 172.21.15.27:0/410065805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbf81088f0 0x7fbbf81a0700 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0007f10 tx=0x7fbbf000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:19:36.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.054+0000 7fbbf5ffb700 1 -- 172.21.15.27:0/410065805 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0004db0 con 0x7fbbf81088f0 2024-05-21T03:19:36.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.054+0000 7fbbff0c5700 1 -- 172.21.15.27:0/410065805 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf81a1130 con 0x7fbbf81088f0 2024-05-21T03:19:36.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.054+0000 7fbbf5ffb700 1 -- 172.21.15.27:0/410065805 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbf0009e20 con 0x7fbbf81088f0 2024-05-21T03:19:36.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.054+0000 7fbbf5ffb700 1 -- 172.21.15.27:0/410065805 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0015400 con 0x7fbbf81088f0 2024-05-21T03:19:36.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.054+0000 7fbbff0c5700 1 -- 172.21.15.27:0/410065805 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf81a89b0 con 0x7fbbf81088f0 2024-05-21T03:19:36.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.055+0000 7fbbff0c5700 1 -- 172.21.15.27:0/410065805 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbf806a230 con 0x7fbbf81088f0 2024-05-21T03:19:36.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.056+0000 7fbbf5ffb700 1 -- 172.21.15.27:0/410065805 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fbbf0005260 con 0x7fbbf81088f0 2024-05-21T03:19:36.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.057+0000 7fbbf5ffb700 1 --2- 172.21.15.27:0/410065805 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fbbe80b2880 0x7fbbe80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:36.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.058+0000 7fbbfce61700 1 --2- 172.21.15.27:0/410065805 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fbbe80b2880 0x7fbbe80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:36.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.058+0000 7fbbf5ffb700 1 -- 172.21.15.27:0/410065805 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fbbf00e1c70 con 0x7fbbf81088f0 2024-05-21T03:19:36.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.058+0000 7fbbfce61700 1 --2- 172.21.15.27:0/410065805 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fbbe80b2880 0x7fbbe80b4d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009a10 tx=0x7fbbec009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:19:36.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.061+0000 7fbbf5ffb700 1 -- 172.21.15.27:0/410065805 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbbf0010070 con 0x7fbbf81088f0 2024-05-21T03:19:36.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.281+0000 7fbbff0c5700 1 -- 172.21.15.27:0/410065805 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbbf81a8b80 con 0x7fbbf81088f0 2024-05-21T03:19:36.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.282+0000 7fbbf5ffb700 1 -- 172.21.15.27:0/410065805 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+18394 (secure 0 0 0) 0x7fbbf00a9d90 con 0x7fbbf81088f0 2024-05-21T03:19:36.282 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:19:36.282 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":73,"fsid":"c9ca94be-171f-11ef-bc9a-c7b262605968","created":"2024-05-21T03:12:18.217236+0000","modified":"2024-05-21T03:19:31.389844+0000","last_up_change":"2024-05-21T03:19:30.382150+0000","last_in_change":"2024-05-21T03:19:09.058249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T03:15:41.340194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c745c09f-9a76-4d05-af75-c7554f6d5e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6803","nonce":1399996449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6805","nonce":1399996449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6809","nonce":1399996449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6807","nonce":1399996449}]},"public_addr":"172.21.15.27:6803/1399996449","cluster_addr":"172.21.15.27:6805/1399996449","heartbeat_back_addr":"172.21.15.27:6809/1399996449","heartbeat_front_addr":"172.21.15.27:6807/1399996449","state":["exists","up"]},{"osd":1,"uuid":"8b73aa00-4364-438f-9722-53caa4e9bec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6801","nonce":1776318847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6803","nonce":1776318847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6807","nonce":1776318847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6805","nonce":1776318847}]},"public_addr":"172.21.15.39:6801/1776318847","cluster_addr":"172.21.15.39:6803/1776318847","heartbeat_back_addr":"172.21.15.39:6807/1776318847","heartbeat_front_addr":"172.21.15.39:6805/1776318847","state":["exists","up"]},{"osd":2,"uuid":"0822043e-7d3c-4e98-9dc0-94b98b4d562a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6801","nonce":703322625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6803","nonce":703322625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6807","nonce":703322625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6805","nonce":703322625}]},"public_addr":"172.21.15.165:6801/703322625","cluster_addr":"172.21.15.165:6803/703322625","heartbeat_back_addr":"172.21.15.165:6807/703322625","heartbeat_front_addr":"172.21.15.165:6805/703322625","state":["exists","up"]},{"osd":3,"uuid":"c563b1b2-efcb-4f8c-8121-c1938f83131e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6811","nonce":1933403135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6813","nonce":1933403135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6817","nonce":1933403135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6815","nonce":1933403135}]},"public_addr":"172.21.15.27:6811/1933403135","cluster_addr":"172.21.15.27:6813/1933403135","heartbeat_back_addr":"172.21.15.27:6817/1933403135","heartbeat_front_addr":"172.21.15.27:6815/1933403135","state":["exists","up"]},{"osd":4,"uuid":"0c793b87-b045-4b4a-89e7-87ec36deeb0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6809","nonce":155579331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6811","nonce":155579331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6815","nonce":155579331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6813","nonce":155579331}]},"public_addr":"172.21.15.39:6809/155579331","cluster_addr":"172.21.15.39:6811/155579331","heartbeat_back_addr":"172.21.15.39:6815/155579331","heartbeat_front_addr":"172.21.15.39:6813/155579331","state":["exists","up"]},{"osd":5,"uuid":"367eeb5a-9d75-4644-bf46-317c03f33d8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6809","nonce":386850244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6811","nonce":386850244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6815","nonce":386850244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6813","nonce":386850244}]},"public_addr":"172.21.15.165:6809/386850244","cluster_addr":"172.21.15.165:6811/386850244","heartbeat_back_addr":"172.21.15.165:6815/386850244","heartbeat_front_addr":"172.21.15.165:6813/386850244","state":["exists","up"]},{"osd":6,"uuid":"e000167f-24fd-4115-83f0-8afbfa8d8991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6819","nonce":3531376679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6821","nonce":3531376679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6825","nonce":3531376679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6823","nonce":3531376679}]},"public_addr":"172.21.15.27:6819/3531376679","cluster_addr":"172.21.15.27:6821/3531376679","heartbeat_back_addr":"172.21.15.27:6825/3531376679","heartbeat_front_addr":"172.21.15.27:6823/3531376679","state":["exists","up"]},{"osd":7,"uuid":"4b8d486b-868e-4f33-98a1-b99686adeaaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6817","nonce":1550035852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6819","nonce":1550035852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6823","nonce":1550035852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6821","nonce":1550035852}]},"public_addr":"172.21.15.39:6817/1550035852","cluster_addr":"172.21.15.39:6819/1550035852","heartbeat_back_addr":"172.21.15.39:6823/1550035852","heartbeat_front_addr":"172.21.15.39:6821/1550035852","state":["exists","up"]},{"osd":8,"uuid":"2259f35c-2edd-4331-9a63-8c0c1acbdc71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6817","nonce":3184704971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6819","nonce":3184704971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6823","nonce":3184704971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6821","nonce":3184704971}]},"public_addr":"172.21.15.165:6817/3184704971","cluster_addr":"172.21.15.165:6819/3184704971","heartbeat_back_addr":"172.21.15.165:6823/3184704971","heartbeat_front_addr":"172.21.15.165:6821/3184704971","state":["exists","up"]},{"osd":9,"uuid":"10307ecc-8270-4be2-b62b-9708e09bb207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6827","nonce":2800048793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6829","nonce":2800048793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6833","nonce":2800048793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6831","nonce":2800048793}]},"public_addr":"172.21.15.27:6827/2800048793","cluster_addr":"172.21.15.27:6829/2800048793","heartbeat_back_addr":"172.21.15.27:6833/2800048793","heartbeat_front_addr":"172.21.15.27:6831/2800048793","state":["exists","up"]},{"osd":10,"uuid":"dfe1b8ca-ff8a-4b7f-ade5-ad4f00cf60cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6825","nonce":363500811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6827","nonce":363500811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6831","nonce":363500811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6829","nonce":363500811}]},"public_addr":"172.21.15.39:6825/363500811","cluster_addr":"172.21.15.39:6827/363500811","heartbeat_back_addr":"172.21.15.39:6831/363500811","heartbeat_front_addr":"172.21.15.39:6829/363500811","state":["exists","up"]},{"osd":11,"uuid":"f9531145-5d09-4dbd-a050-3159ee09db24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6825","nonce":2274332243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6827","nonce":2274332243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6831","nonce":2274332243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6829","nonce":2274332243}]},"public_addr":"172.21.15.165:6825/2274332243","cluster_addr":"172.21.15.165:6827/2274332243","heartbeat_back_addr":"172.21.15.165:6831/2274332243","heartbeat_front_addr":"172.21.15.165:6829/2274332243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:14:47.077860+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:15:13.272879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:15:38.046651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:03.376483+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:29.179280+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:53.391261+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:17:18.472829+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:17:44.349925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:18:08.652193+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:18:35.282669+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:19:01.637151+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:19:27.461566+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6800/1185518061":"2024-05-22T03:13:13.249220+0000","172.21.15.27:6801/1185518061":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/4194431212":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/1890032101":"2024-05-22T03:13:13.249220+0000","172.21.15.27:6800/175229478":"2024-05-22T03:12:45.739195+0000","172.21.15.27:0/999325706":"2024-05-22T03:12:45.739195+0000","172.21.15.27:6801/175229478":"2024-05-22T03:12:45.739195+0000","172.21.15.27:0/4184128736":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/3000197506":"2024-05-22T03:12:45.739195+0000","172.21.15.27:0/972677319":"2024-05-22T03:12:45.739195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T03:19:36.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.285+0000 7fbbff0c5700 1 -- 172.21.15.27:0/410065805 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fbbe80b2880 msgr2=0x7fbbe80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:36.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.285+0000 7fbbff0c5700 1 --2- 172.21.15.27:0/410065805 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fbbe80b2880 0x7fbbe80b4d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009a10 tx=0x7fbbec009190 comp rx=0 tx=0).stop 2024-05-21T03:19:36.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.285+0000 7fbbff0c5700 1 -- 172.21.15.27:0/410065805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbf81088f0 msgr2=0x7fbbf81a0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:36.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.285+0000 7fbbff0c5700 1 --2- 172.21.15.27:0/410065805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbf81088f0 0x7fbbf81a0700 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0007f10 tx=0x7fbbf000d3b0 comp rx=0 tx=0).stop 2024-05-21T03:19:36.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.285+0000 7fbbff0c5700 1 -- 172.21.15.27:0/410065805 shutdown_connections 2024-05-21T03:19:36.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.285+0000 7fbbff0c5700 1 --2- 172.21.15.27:0/410065805 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fbbe80b2880 0x7fbbe80b4d40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:36.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.285+0000 7fbbff0c5700 1 --2- 172.21.15.27:0/410065805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbf81088f0 0x7fbbf81a0700 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:36.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.285+0000 7fbbff0c5700 1 --2- 172.21.15.27:0/410065805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbf81037f0 0x7fbbf819be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:36.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.285+0000 7fbbff0c5700 1 --2- 172.21.15.27:0/410065805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbf81041a0 0x7fbbf819c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:36.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.285+0000 7fbbff0c5700 1 -- 172.21.15.27:0/410065805 >> 172.21.15.27:0/410065805 conn(0x7fbbf807a6a0 msgr2=0x7fbbf8109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:19:36.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.285+0000 7fbbff0c5700 1 -- 172.21.15.27:0/410065805 shutdown_connections 2024-05-21T03:19:36.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:36.285+0000 7fbbff0c5700 1 -- 172.21.15.27:0/410065805 wait complete. 2024-05-21T03:19:36.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:36 smithi027 ceph-mon[32179]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:36.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:36 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/410065805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:19:36.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:36 smithi039 ceph-mon[36429]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:36.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:36 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/410065805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:19:36.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:36 smithi165 ceph-mon[36269]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:36.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:36 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/410065805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:19:37.014 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-21T03:15:41.340194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-21T03:19:37.014 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-21T03:19:37.266 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:19:38.667 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:38 smithi027 ceph-mon[32179]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:38.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:38 smithi039 ceph-mon[36429]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:38.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:38 smithi165 ceph-mon[36269]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:38.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.997+0000 7fd4be577700 1 -- 172.21.15.27:0/1704447511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4b81088f0 msgr2=0x7fd4b8110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:38.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.997+0000 7fd4be577700 1 --2- 172.21.15.27:0/1704447511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4b81088f0 0x7fd4b8110470 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000b0d0 tx=0x7fd4b000b3e0 comp rx=0 tx=0).stop 2024-05-21T03:19:38.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.997+0000 7fd4be577700 1 -- 172.21.15.27:0/1704447511 shutdown_connections 2024-05-21T03:19:38.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.997+0000 7fd4be577700 1 --2- 172.21.15.27:0/1704447511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4b81088f0 0x7fd4b8110470 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:38.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.997+0000 7fd4be577700 1 --2- 172.21.15.27:0/1704447511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4b81037f0 0x7fd4b8103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:38.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.997+0000 7fd4be577700 1 --2- 172.21.15.27:0/1704447511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4b81041a0 0x7fd4b81081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:38.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.997+0000 7fd4be577700 1 -- 172.21.15.27:0/1704447511 >> 172.21.15.27:0/1704447511 conn(0x7fd4b807a6a0 msgr2=0x7fd4b807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:19:38.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.997+0000 7fd4be577700 1 -- 172.21.15.27:0/1704447511 shutdown_connections 2024-05-21T03:19:38.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.998+0000 7fd4be577700 1 -- 172.21.15.27:0/1704447511 wait complete. 2024-05-21T03:19:38.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.998+0000 7fd4be577700 1 Processor -- start 2024-05-21T03:19:38.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.998+0000 7fd4be577700 1 -- start start 2024-05-21T03:19:39.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.998+0000 7fd4be577700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4b81037f0 0x7fd4b8075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:39.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.998+0000 7fd4be577700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4b81041a0 0x7fd4b8075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:39.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.998+0000 7fd4be577700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4b81088f0 0x7fd4b80762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:39.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.998+0000 7fd4be577700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b81136c0 con 0x7fd4b81041a0 2024-05-21T03:19:39.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.998+0000 7fd4be577700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b8113540 con 0x7fd4b81088f0 2024-05-21T03:19:39.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.998+0000 7fd4be577700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b8113840 con 0x7fd4b81037f0 2024-05-21T03:19:39.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.999+0000 7fd4affff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4b81041a0 0x7fd4b8075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:39.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.999+0000 7fd4affff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4b81041a0 0x7fd4b8075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33892/0 (socket says 172.21.15.27:33892) 2024-05-21T03:19:39.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.999+0000 7fd4affff700 1 -- 172.21.15.27:0/3616326894 learned_addr learned my addr 172.21.15.27:0/3616326894 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:19:39.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.999+0000 7fd4b7fff700 1 --2- 172.21.15.27:0/3616326894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4b81037f0 0x7fd4b8075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:39.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.999+0000 7fd4bcb14700 1 --2- 172.21.15.27:0/3616326894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4b81088f0 0x7fd4b80762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:39.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.999+0000 7fd4affff700 1 -- 172.21.15.27:0/3616326894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4b81037f0 msgr2=0x7fd4b8075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:39.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.999+0000 7fd4affff700 1 --2- 172.21.15.27:0/3616326894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4b81037f0 0x7fd4b8075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:39.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.999+0000 7fd4affff700 1 -- 172.21.15.27:0/3616326894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4b81088f0 msgr2=0x7fd4b80762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:39.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.999+0000 7fd4affff700 1 --2- 172.21.15.27:0/3616326894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4b81088f0 0x7fd4b80762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:39.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.999+0000 7fd4affff700 1 -- 172.21.15.27:0/3616326894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b81a8040 con 0x7fd4b81041a0 2024-05-21T03:19:39.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.999+0000 7fd4affff700 1 --2- 172.21.15.27:0/3616326894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4b81041a0 0x7fd4b8075d80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd4a000d950 tx=0x7fd4a000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:19:39.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.999+0000 7fd4b5ffb700 1 -- 172.21.15.27:0/3616326894 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0016070 con 0x7fd4b81041a0 2024-05-21T03:19:39.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.999+0000 7fd4be577700 1 -- 172.21.15.27:0/3616326894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4b81a8330 con 0x7fd4b81041a0 2024-05-21T03:19:39.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.999+0000 7fd4b5ffb700 1 -- 172.21.15.27:0/3616326894 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4a000fd40 con 0x7fd4b81041a0 2024-05-21T03:19:39.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.999+0000 7fd4b5ffb700 1 -- 172.21.15.27:0/3616326894 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a000e3b0 con 0x7fd4b81041a0 2024-05-21T03:19:39.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:38.999+0000 7fd4be577700 1 -- 172.21.15.27:0/3616326894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4b81a88c0 con 0x7fd4b81041a0 2024-05-21T03:19:39.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:39.000+0000 7fd4be577700 1 -- 172.21.15.27:0/3616326894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4b806a230 con 0x7fd4b81041a0 2024-05-21T03:19:39.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:39.002+0000 7fd4b5ffb700 1 -- 172.21.15.27:0/3616326894 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd4a0019070 con 0x7fd4b81041a0 2024-05-21T03:19:39.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:39.002+0000 7fd4b5ffb700 1 --2- 172.21.15.27:0/3616326894 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd4a40b2900 0x7fd4a40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:39.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:39.003+0000 7fd4b7fff700 1 --2- 172.21.15.27:0/3616326894 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd4a40b2900 0x7fd4a40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:39.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:39.003+0000 7fd4b5ffb700 1 -- 172.21.15.27:0/3616326894 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fd4a00e16d0 con 0x7fd4b81041a0 2024-05-21T03:19:39.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:39.003+0000 7fd4b7fff700 1 --2- 172.21.15.27:0/3616326894 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd4a40b2900 0x7fd4a40b4dc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8007be0 tx=0x7fd4a80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:19:39.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:39.006+0000 7fd4b5ffb700 1 -- 172.21.15.27:0/3616326894 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd4a00a9890 con 0x7fd4b81041a0 2024-05-21T03:19:39.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:39.218+0000 7fd4be577700 1 -- 172.21.15.27:0/3616326894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd4b802e560 con 0x7fd4b81041a0 2024-05-21T03:19:39.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:39.218+0000 7fd4b5ffb700 1 -- 172.21.15.27:0/3616326894 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7fd4a00a9190 con 0x7fd4b81041a0 2024-05-21T03:19:39.218 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-05-21T03:19:39.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:39.220+0000 7fd4be577700 1 -- 172.21.15.27:0/3616326894 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd4a40b2900 msgr2=0x7fd4a40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:39.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:39.220+0000 7fd4be577700 1 --2- 172.21.15.27:0/3616326894 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd4a40b2900 0x7fd4a40b4dc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8007be0 tx=0x7fd4a80073d0 comp rx=0 tx=0).stop 2024-05-21T03:19:39.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:39.220+0000 7fd4be577700 1 -- 172.21.15.27:0/3616326894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4b81041a0 msgr2=0x7fd4b8075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:39.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:39.220+0000 7fd4be577700 1 --2- 172.21.15.27:0/3616326894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4b81041a0 0x7fd4b8075d80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd4a000d950 tx=0x7fd4a000dc60 comp rx=0 tx=0).stop 2024-05-21T03:19:39.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:39.221+0000 7fd4be577700 1 -- 172.21.15.27:0/3616326894 shutdown_connections 2024-05-21T03:19:39.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:39.221+0000 7fd4be577700 1 --2- 172.21.15.27:0/3616326894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4b81088f0 0x7fd4b80762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:39.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:39.221+0000 7fd4be577700 1 --2- 172.21.15.27:0/3616326894 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd4a40b2900 0x7fd4a40b4dc0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:39.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:39.221+0000 7fd4be577700 1 --2- 172.21.15.27:0/3616326894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4b81037f0 0x7fd4b8075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:39.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:39.221+0000 7fd4be577700 1 --2- 172.21.15.27:0/3616326894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4b81041a0 0x7fd4b8075d80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:39.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:39.221+0000 7fd4be577700 1 -- 172.21.15.27:0/3616326894 >> 172.21.15.27:0/3616326894 conn(0x7fd4b807a6a0 msgr2=0x7fd4b8101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:19:39.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:39.221+0000 7fd4be577700 1 -- 172.21.15.27:0/3616326894 shutdown_connections 2024-05-21T03:19:39.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:39.221+0000 7fd4be577700 1 -- 172.21.15.27:0/3616326894 wait complete. 2024-05-21T03:19:39.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:39 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3616326894' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T03:19:39.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:39 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/3616326894' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T03:19:39.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:39 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/3616326894' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T03:19:39.777 INFO:tasks.cephadm:Adding mds.a on smithi027 2024-05-21T03:19:39.777 INFO:tasks.cephadm:Adding mds.d on smithi027 2024-05-21T03:19:39.777 INFO:tasks.cephadm:Adding mds.g on smithi027 2024-05-21T03:19:39.777 INFO:tasks.cephadm:Adding mds.j on smithi027 2024-05-21T03:19:39.777 INFO:tasks.cephadm:Adding mds.b on smithi039 2024-05-21T03:19:39.777 INFO:tasks.cephadm:Adding mds.e on smithi039 2024-05-21T03:19:39.777 INFO:tasks.cephadm:Adding mds.h on smithi039 2024-05-21T03:19:39.777 INFO:tasks.cephadm:Adding mds.k on smithi039 2024-05-21T03:19:39.777 INFO:tasks.cephadm:Adding mds.c on smithi165 2024-05-21T03:19:39.778 INFO:tasks.cephadm:Adding mds.f on smithi165 2024-05-21T03:19:39.778 INFO:tasks.cephadm:Adding mds.i on smithi165 2024-05-21T03:19:39.778 INFO:tasks.cephadm:Adding mds.l on smithi165 2024-05-21T03:19:39.778 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph orch apply mds all '12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi039=b;smithi039=e;smithi039=h;smithi039=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l' 2024-05-21T03:19:40.038 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T03:19:40.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:40 smithi027 ceph-mon[32179]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:40.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:40 smithi039 ceph-mon[36429]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:40.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:40 smithi165 ceph-mon[36269]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:41.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.627+0000 7f31090c6700 1 -- 172.21.15.165:0/2450550421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31041036e0 msgr2=0x7f3104103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:41.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.627+0000 7f31090c6700 1 --2- 172.21.15.165:0/2450550421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31041036e0 0x7f3104103b40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009990 tx=0x7f30f8009ca0 comp rx=0 tx=0).stop 2024-05-21T03:19:41.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.628+0000 7f31090c6700 1 -- 172.21.15.165:0/2450550421 shutdown_connections 2024-05-21T03:19:41.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.628+0000 7f31090c6700 1 --2- 172.21.15.165:0/2450550421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3104104080 0x7f310410a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:41.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.628+0000 7f31090c6700 1 --2- 172.21.15.165:0/2450550421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31041024e0 0x7f3104102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:41.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.628+0000 7f31090c6700 1 --2- 172.21.15.165:0/2450550421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31041036e0 0x7f3104103b40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:41.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.628+0000 7f31090c6700 1 -- 172.21.15.165:0/2450550421 >> 172.21.15.165:0/2450550421 conn(0x7f31040fdaa0 msgr2=0x7f31040ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:19:41.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.628+0000 7f31090c6700 1 -- 172.21.15.165:0/2450550421 shutdown_connections 2024-05-21T03:19:41.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.628+0000 7f31090c6700 1 -- 172.21.15.165:0/2450550421 wait complete. 2024-05-21T03:19:41.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.628+0000 7f31090c6700 1 Processor -- start 2024-05-21T03:19:41.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.629+0000 7f31090c6700 1 -- start start 2024-05-21T03:19:41.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.629+0000 7f31090c6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31041024e0 0x7f3104199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:41.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.629+0000 7f31090c6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31041036e0 0x7f3104199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:41.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.629+0000 7f31090c6700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3104104080 0x7f31041a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:41.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.629+0000 7f31090c6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f310410df80 con 0x7f31041036e0 2024-05-21T03:19:41.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.629+0000 7f31090c6700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f310410de00 con 0x7f31041024e0 2024-05-21T03:19:41.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.629+0000 7f31090c6700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f310410e100 con 0x7f3104104080 2024-05-21T03:19:41.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.629+0000 7f310359e700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3104104080 0x7f31041a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:41.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.629+0000 7f310359e700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3104104080 0x7f31041a0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38792/0 (socket says 172.21.15.165:38792) 2024-05-21T03:19:41.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.629+0000 7f310359e700 1 -- 172.21.15.165:0/2942131500 learned_addr learned my addr 172.21.15.165:0/2942131500 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-05-21T03:19:41.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.629+0000 7f310259c700 1 --2- 172.21.15.165:0/2942131500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31041036e0 0x7f3104199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:41.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.629+0000 7f3102d9d700 1 --2- 172.21.15.165:0/2942131500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31041024e0 0x7f3104199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:41.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.629+0000 7f310359e700 1 -- 172.21.15.165:0/2942131500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31041024e0 msgr2=0x7f3104199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:41.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.629+0000 7f310359e700 1 --2- 172.21.15.165:0/2942131500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31041024e0 0x7f3104199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:41.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.629+0000 7f310359e700 1 -- 172.21.15.165:0/2942131500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31041036e0 msgr2=0x7f3104199c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:41.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.629+0000 7f310359e700 1 --2- 172.21.15.165:0/2942131500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31041036e0 0x7f3104199c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:41.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.629+0000 7f310359e700 1 -- 172.21.15.165:0/2942131500 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31041a1370 con 0x7f3104104080 2024-05-21T03:19:41.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.629+0000 7f3102d9d700 1 --2- 172.21.15.165:0/2942131500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31041024e0 0x7f3104199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:19:41.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.629+0000 7f310359e700 1 --2- 172.21.15.165:0/2942131500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3104104080 0x7f31041a0cd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f30fc007f10 tx=0x7f30fc00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:19:41.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.629+0000 7f30ebfff700 1 -- 172.21.15.165:0/2942131500 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30fc004bb0 con 0x7f3104104080 2024-05-21T03:19:41.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.630+0000 7f31090c6700 1 -- 172.21.15.165:0/2942131500 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31041a1660 con 0x7f3104104080 2024-05-21T03:19:41.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.630+0000 7f30ebfff700 1 -- 172.21.15.165:0/2942131500 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30fc004d50 con 0x7f3104104080 2024-05-21T03:19:41.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.630+0000 7f30ebfff700 1 -- 172.21.15.165:0/2942131500 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30fc005a30 con 0x7f3104104080 2024-05-21T03:19:41.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.630+0000 7f31090c6700 1 -- 172.21.15.165:0/2942131500 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31041a1ba0 con 0x7f3104104080 2024-05-21T03:19:41.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.631+0000 7f31090c6700 1 -- 172.21.15.165:0/2942131500 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f310406a230 con 0x7f3104104080 2024-05-21T03:19:41.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.632+0000 7f30ebfff700 1 -- 172.21.15.165:0/2942131500 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f30fc01a020 con 0x7f3104104080 2024-05-21T03:19:41.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.632+0000 7f30ebfff700 1 --2- 172.21.15.165:0/2942131500 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f30ec0b2880 0x7f30ec0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:41.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.633+0000 7f30ebfff700 1 -- 172.21.15.165:0/2942131500 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f30fc0e14d0 con 0x7f3104104080 2024-05-21T03:19:41.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.633+0000 7f3102d9d700 1 --2- 172.21.15.165:0/2942131500 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f30ec0b2880 0x7f30ec0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:41.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.633+0000 7f3102d9d700 1 --2- 172.21.15.165:0/2942131500 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f30ec0b2880 0x7f30ec0b4d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3104103540 tx=0x7f30f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:19:41.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.637+0000 7f30ebfff700 1 -- 172.21.15.165:0/2942131500 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f30fc010070 con 0x7f3104104080 2024-05-21T03:19:41.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.849+0000 7f31090c6700 1 -- 172.21.15.165:0/2942131500 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi039=b;smithi039=e;smithi039=h;smithi039=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l", "target": ["mon-mgr", ""]}) -- 0x7f310406c480 con 0x7f30ec0b2880 2024-05-21T03:19:41.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.857+0000 7f30ebfff700 1 -- 172.21.15.165:0/2942131500 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f310406c480 con 0x7f30ec0b2880 2024-05-21T03:19:41.857 INFO:teuthology.orchestra.run.smithi165.stdout:Scheduled mds.all update... 2024-05-21T03:19:41.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.859+0000 7f31090c6700 1 -- 172.21.15.165:0/2942131500 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f30ec0b2880 msgr2=0x7f30ec0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:41.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.859+0000 7f31090c6700 1 --2- 172.21.15.165:0/2942131500 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f30ec0b2880 0x7f30ec0b4d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3104103540 tx=0x7f30f4009260 comp rx=0 tx=0).stop 2024-05-21T03:19:41.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.859+0000 7f31090c6700 1 -- 172.21.15.165:0/2942131500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3104104080 msgr2=0x7f31041a0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:41.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.859+0000 7f31090c6700 1 --2- 172.21.15.165:0/2942131500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3104104080 0x7f31041a0cd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f30fc007f10 tx=0x7f30fc00d3b0 comp rx=0 tx=0).stop 2024-05-21T03:19:41.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.860+0000 7f31090c6700 1 -- 172.21.15.165:0/2942131500 shutdown_connections 2024-05-21T03:19:41.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.860+0000 7f31090c6700 1 --2- 172.21.15.165:0/2942131500 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f30ec0b2880 0x7f30ec0b4d40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:41.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.860+0000 7f31090c6700 1 --2- 172.21.15.165:0/2942131500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3104104080 0x7f31041a0cd0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:41.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.860+0000 7f31090c6700 1 --2- 172.21.15.165:0/2942131500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31041024e0 0x7f3104199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:41.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.860+0000 7f31090c6700 1 --2- 172.21.15.165:0/2942131500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31041036e0 0x7f3104199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:41.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.860+0000 7f31090c6700 1 -- 172.21.15.165:0/2942131500 >> 172.21.15.165:0/2942131500 conn(0x7f31040fdaa0 msgr2=0x7f310406bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:19:41.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.860+0000 7f31090c6700 1 -- 172.21.15.165:0/2942131500 shutdown_connections 2024-05-21T03:19:41.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-05-21T03:19:41.860+0000 7f31090c6700 1 -- 172.21.15.165:0/2942131500 wait complete. 2024-05-21T03:19:42.444 DEBUG:teuthology.orchestra.run.smithi027:mds.a> sudo journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.a.service 2024-05-21T03:19:42.447 DEBUG:teuthology.orchestra.run.smithi027:mds.d> sudo journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.d.service 2024-05-21T03:19:42.490 DEBUG:teuthology.orchestra.run.smithi027:mds.g> sudo journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.g.service 2024-05-21T03:19:42.533 DEBUG:teuthology.orchestra.run.smithi027:mds.j> sudo journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.j.service 2024-05-21T03:19:42.576 DEBUG:teuthology.orchestra.run.smithi039:mds.b> sudo journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.b.service 2024-05-21T03:19:42.578 DEBUG:teuthology.orchestra.run.smithi039:mds.e> sudo journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.e.service 2024-05-21T03:19:42.621 DEBUG:teuthology.orchestra.run.smithi039:mds.h> sudo journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.h.service 2024-05-21T03:19:42.665 DEBUG:teuthology.orchestra.run.smithi039:mds.k> sudo journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.k.service 2024-05-21T03:19:42.708 DEBUG:teuthology.orchestra.run.smithi165:mds.c> sudo journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.c.service 2024-05-21T03:19:42.710 DEBUG:teuthology.orchestra.run.smithi165:mds.f> sudo journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.f.service 2024-05-21T03:19:42.754 DEBUG:teuthology.orchestra.run.smithi165:mds.i> sudo journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.i.service 2024-05-21T03:19:42.796 DEBUG:teuthology.orchestra.run.smithi165:mds.l> sudo journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.l.service 2024-05-21T03:19:42.839 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-21T03:19:42.839 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:19:43.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:42 smithi165 ceph-mon[36269]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:43.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:42 smithi165 ceph-mon[36269]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi039=b;smithi039=e;smithi039=h;smithi039=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:19:43.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:42 smithi165 ceph-mon[36269]: Saving service mds.all spec with placement smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi039=b;smithi039=e;smithi039=h;smithi039=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l;count:12 2024-05-21T03:19:43.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:42 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:43.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:42 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:19:43.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:42 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:43.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:42 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:19:43.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:42 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:43.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:42 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:19:43.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:42 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:19:43.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:42 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:43.091 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:19:43.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:42 smithi027 ceph-mon[32179]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:43.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:42 smithi027 ceph-mon[32179]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi039=b;smithi039=e;smithi039=h;smithi039=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:19:43.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:42 smithi027 ceph-mon[32179]: Saving service mds.all spec with placement smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi039=b;smithi039=e;smithi039=h;smithi039=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l;count:12 2024-05-21T03:19:43.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:42 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:43.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:42 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:19:43.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:42 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:43.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:42 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:19:43.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:42 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:43.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:42 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:19:43.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:42 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:19:43.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:42 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:43.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:42 smithi039 ceph-mon[36429]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:43.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:42 smithi039 ceph-mon[36429]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi039=b;smithi039=e;smithi039=h;smithi039=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:19:43.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:42 smithi039 ceph-mon[36429]: Saving service mds.all spec with placement smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi039=b;smithi039=e;smithi039=h;smithi039=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l;count:12 2024-05-21T03:19:43.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:42 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:43.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:42 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:19:43.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:42 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:43.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:42 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:19:43.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:42 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:43.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:42 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:19:43.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:42 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:19:43.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:42 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:44.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:43 smithi165 ceph-mon[36269]: Deploying daemon mds.l on smithi165 2024-05-21T03:19:44.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:43 smithi027 ceph-mon[32179]: Deploying daemon mds.l on smithi165 2024-05-21T03:19:44.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:43 smithi039 ceph-mon[36429]: Deploying daemon mds.l on smithi165 2024-05-21T03:19:44.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.679+0000 7f84bf326700 1 -- 172.21.15.27:0/2204997384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84b8101ee0 msgr2=0x7f84b8102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:44.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.679+0000 7f84bf326700 1 --2- 172.21.15.27:0/2204997384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84b8101ee0 0x7f84b8102360 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f84a8009990 tx=0x7f84a8009ca0 comp rx=0 tx=0).stop 2024-05-21T03:19:44.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.679+0000 7f84bf326700 1 -- 172.21.15.27:0/2204997384 shutdown_connections 2024-05-21T03:19:44.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.679+0000 7f84bf326700 1 --2- 172.21.15.27:0/2204997384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84b81028a0 0x7f84b8110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:44.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.679+0000 7f84bf326700 1 --2- 172.21.15.27:0/2204997384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84b8108600 0x7f84b81089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:44.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.679+0000 7f84bf326700 1 --2- 172.21.15.27:0/2204997384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84b8101ee0 0x7f84b8102360 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:44.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.679+0000 7f84bf326700 1 -- 172.21.15.27:0/2204997384 >> 172.21.15.27:0/2204997384 conn(0x7f84b8077760 msgr2=0x7f84b8077b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:19:44.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.679+0000 7f84bf326700 1 -- 172.21.15.27:0/2204997384 shutdown_connections 2024-05-21T03:19:44.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.679+0000 7f84bf326700 1 -- 172.21.15.27:0/2204997384 wait complete. 2024-05-21T03:19:44.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.679+0000 7f84bf326700 1 Processor -- start 2024-05-21T03:19:44.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.679+0000 7f84bf326700 1 -- start start 2024-05-21T03:19:44.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.680+0000 7f84bf326700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84b8101ee0 0x7f84b819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:44.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.680+0000 7f84bf326700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84b81028a0 0x7f84b819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:44.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.680+0000 7f84bf326700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84b8108600 0x7f84b81999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:44.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.680+0000 7f84bf326700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f84b8113750 con 0x7f84b8101ee0 2024-05-21T03:19:44.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.680+0000 7f84bf326700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f84b81135d0 con 0x7f84b81028a0 2024-05-21T03:19:44.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.680+0000 7f84bf326700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f84b81138d0 con 0x7f84b8108600 2024-05-21T03:19:44.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.680+0000 7f84bd0c2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84b8101ee0 0x7f84b819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:44.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.680+0000 7f84bd0c2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84b8101ee0 0x7f84b819f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58728/0 (socket says 172.21.15.27:58728) 2024-05-21T03:19:44.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.680+0000 7f84bd0c2700 1 -- 172.21.15.27:0/2572794708 learned_addr learned my addr 172.21.15.27:0/2572794708 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:19:44.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.680+0000 7f84bc8c1700 1 --2- 172.21.15.27:0/2572794708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84b81028a0 0x7f84b819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:44.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.680+0000 7f84bd8c3700 1 --2- 172.21.15.27:0/2572794708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84b8108600 0x7f84b81999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:44.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.680+0000 7f84bd0c2700 1 -- 172.21.15.27:0/2572794708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84b8108600 msgr2=0x7f84b81999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:44.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.680+0000 7f84bd0c2700 1 --2- 172.21.15.27:0/2572794708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84b8108600 0x7f84b81999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:44.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.680+0000 7f84bd0c2700 1 -- 172.21.15.27:0/2572794708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84b81028a0 msgr2=0x7f84b819fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:44.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.680+0000 7f84bd0c2700 1 --2- 172.21.15.27:0/2572794708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84b81028a0 0x7f84b819fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:44.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.680+0000 7f84bd0c2700 1 -- 172.21.15.27:0/2572794708 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b819a110 con 0x7f84b8101ee0 2024-05-21T03:19:44.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.680+0000 7f84bd0c2700 1 --2- 172.21.15.27:0/2572794708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84b8101ee0 0x7f84b819f7f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f84b400c970 tx=0x7f84b400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:19:44.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.681+0000 7f84ae7fc700 1 -- 172.21.15.27:0/2572794708 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b4004220 con 0x7f84b8101ee0 2024-05-21T03:19:44.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.681+0000 7f84bf326700 1 -- 172.21.15.27:0/2572794708 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84b819a400 con 0x7f84b8101ee0 2024-05-21T03:19:44.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.681+0000 7f84ae7fc700 1 -- 172.21.15.27:0/2572794708 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84b40043c0 con 0x7f84b8101ee0 2024-05-21T03:19:44.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.681+0000 7f84ae7fc700 1 -- 172.21.15.27:0/2572794708 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b4003be0 con 0x7f84b8101ee0 2024-05-21T03:19:44.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.681+0000 7f84bf326700 1 -- 172.21.15.27:0/2572794708 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84b81a66c0 con 0x7f84b8101ee0 2024-05-21T03:19:44.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.682+0000 7f84bf326700 1 -- 172.21.15.27:0/2572794708 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84b806a230 con 0x7f84b8101ee0 2024-05-21T03:19:44.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.683+0000 7f84ae7fc700 1 -- 172.21.15.27:0/2572794708 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f84b40076d0 con 0x7f84b8101ee0 2024-05-21T03:19:44.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.684+0000 7f84ae7fc700 1 --2- 172.21.15.27:0/2572794708 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f84a40b2880 0x7f84a40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:44.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.684+0000 7f84bc8c1700 1 --2- 172.21.15.27:0/2572794708 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f84a40b2880 0x7f84a40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:44.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.684+0000 7f84ae7fc700 1 -- 172.21.15.27:0/2572794708 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f84b40e14e0 con 0x7f84b8101ee0 2024-05-21T03:19:44.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.684+0000 7f84bc8c1700 1 --2- 172.21.15.27:0/2572794708 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f84a40b2880 0x7f84a40b4d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f84a800b2e0 tx=0x7f84a8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:19:44.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.688+0000 7f84ae7fc700 1 -- 172.21.15.27:0/2572794708 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f84b4014030 con 0x7f84b8101ee0 2024-05-21T03:19:44.719 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 21 03:19:44 smithi165 systemd[1]: Starting Ceph mds.l for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:19:44.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.954+0000 7f84bf326700 1 -- 172.21.15.27:0/2572794708 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84b819aee0 con 0x7f84b8101ee0 2024-05-21T03:19:44.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.955+0000 7f84ae7fc700 1 -- 172.21.15.27:0/2572794708 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f84b40a96a0 con 0x7f84b8101ee0 2024-05-21T03:19:44.956 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:19:44.956 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-21T03:19:44.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.957+0000 7f84bf326700 1 -- 172.21.15.27:0/2572794708 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f84a40b2880 msgr2=0x7f84a40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:44.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.957+0000 7f84bf326700 1 --2- 172.21.15.27:0/2572794708 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f84a40b2880 0x7f84a40b4d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f84a800b2e0 tx=0x7f84a8005960 comp rx=0 tx=0).stop 2024-05-21T03:19:44.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.957+0000 7f84bf326700 1 -- 172.21.15.27:0/2572794708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84b8101ee0 msgr2=0x7f84b819f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:44.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.957+0000 7f84bf326700 1 --2- 172.21.15.27:0/2572794708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84b8101ee0 0x7f84b819f7f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f84b400c970 tx=0x7f84b400cc80 comp rx=0 tx=0).stop 2024-05-21T03:19:44.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.957+0000 7f84bf326700 1 -- 172.21.15.27:0/2572794708 shutdown_connections 2024-05-21T03:19:44.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.957+0000 7f84bf326700 1 --2- 172.21.15.27:0/2572794708 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f84a40b2880 0x7f84a40b4d40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:44.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.957+0000 7f84bf326700 1 --2- 172.21.15.27:0/2572794708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84b8108600 0x7f84b81999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:44.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.957+0000 7f84bf326700 1 --2- 172.21.15.27:0/2572794708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84b8101ee0 0x7f84b819f7f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:44.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.957+0000 7f84bf326700 1 --2- 172.21.15.27:0/2572794708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84b81028a0 0x7f84b819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:44.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.957+0000 7f84bf326700 1 -- 172.21.15.27:0/2572794708 >> 172.21.15.27:0/2572794708 conn(0x7f84b8077760 msgr2=0x7f84b81009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:19:44.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.957+0000 7f84bf326700 1 -- 172.21.15.27:0/2572794708 shutdown_connections 2024-05-21T03:19:44.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:44.957+0000 7f84bf326700 1 -- 172.21.15.27:0/2572794708 wait complete. 2024-05-21T03:19:44.960 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 1 2024-05-21T03:19:44.996 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:44 smithi165 ceph-mon[36269]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:44.997 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 21 03:19:44 smithi165 podman[65963]: 2024-05-21 03:19:44.622438478 +0000 UTC m=+0.016963190 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T03:19:44.997 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 21 03:19:44 smithi165 podman[65963]: 2024-05-21 03:19:44.777823681 +0000 UTC m=+0.172348374 container create 3b6d506082ca28485d84139410a31df5360ea437f138d620060ab75907f4ebb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-21T03:19:45.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:44 smithi027 ceph-mon[32179]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:45.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:44 smithi039 ceph-mon[36429]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:45.271 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 21 03:19:44 smithi165 podman[65963]: 2024-05-21 03:19:44.997001818 +0000 UTC m=+0.391526521 container init 3b6d506082ca28485d84139410a31df5360ea437f138d620060ab75907f4ebb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-l, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-05-21T03:19:45.272 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 21 03:19:44 smithi165 podman[65963]: 2024-05-21 03:19:44.999672816 +0000 UTC m=+0.394197521 container start 3b6d506082ca28485d84139410a31df5360ea437f138d620060ab75907f4ebb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-l, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-21T03:19:45.272 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 21 03:19:45 smithi165 ceph-mds[65982]: -- 172.21.15.165:0/520343015 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559be42c7a40 con 0x559be429a800 2024-05-21T03:19:45.272 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 21 03:19:45 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-l[65978]: starting mds.l at 2024-05-21T03:19:45.272 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 21 03:19:45 smithi165 bash[65963]: 3b6d506082ca28485d84139410a31df5360ea437f138d620060ab75907f4ebb5 2024-05-21T03:19:45.272 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 21 03:19:45 smithi165 systemd[1]: Started Ceph mds.l for c9ca94be-171f-11ef-bc9a-c7b262605968. 2024-05-21T03:19:46.120 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:45 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/2572794708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:19:46.120 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:45 smithi039 ceph-mon[36429]: mds.? [v2:172.21.15.165:6832/4176762898,v1:172.21.15.165:6833/4176762898] up:boot 2024-05-21T03:19:46.120 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:45 smithi039 ceph-mon[36429]: fsmap 1 up:standby 2024-05-21T03:19:46.120 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:45 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T03:19:46.120 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:45 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:46.120 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:45 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:46.120 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:45 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:46.120 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:45 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:19:46.120 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:45 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:19:46.121 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:45 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:46.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:45 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2572794708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:19:46.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:45 smithi027 ceph-mon[32179]: mds.? [v2:172.21.15.165:6832/4176762898,v1:172.21.15.165:6833/4176762898] up:boot 2024-05-21T03:19:46.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:45 smithi027 ceph-mon[32179]: fsmap 1 up:standby 2024-05-21T03:19:46.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:45 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T03:19:46.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:45 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:46.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:45 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:46.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:45 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:46.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:45 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:19:46.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:45 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:19:46.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:45 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:46.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:45 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/2572794708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:19:46.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:45 smithi165 ceph-mon[36269]: mds.? [v2:172.21.15.165:6832/4176762898,v1:172.21.15.165:6833/4176762898] up:boot 2024-05-21T03:19:46.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:45 smithi165 ceph-mon[36269]: fsmap 1 up:standby 2024-05-21T03:19:46.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:45 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T03:19:46.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:45 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:46.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:45 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:46.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:45 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:46.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:45 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:19:46.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:45 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:19:46.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:45 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:47.134 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:46 smithi039 ceph-mon[36429]: Deploying daemon mds.k on smithi039 2024-05-21T03:19:47.134 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:46 smithi039 ceph-mon[36429]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:47.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:46 smithi027 ceph-mon[32179]: Deploying daemon mds.k on smithi039 2024-05-21T03:19:47.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:46 smithi027 ceph-mon[32179]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:47.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:46 smithi165 ceph-mon[36269]: Deploying daemon mds.k on smithi039 2024-05-21T03:19:47.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:46 smithi165 ceph-mon[36269]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:47.519 INFO:journalctl@ceph.mds.k.smithi039.stdout:May 21 03:19:47 smithi039 systemd[1]: Starting Ceph mds.k for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:19:47.741 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:19:47.839 INFO:journalctl@ceph.mds.k.smithi039.stdout:May 21 03:19:47 smithi039 podman[65511]: 2024-05-21 03:19:47.535978159 +0000 UTC m=+0.105940413 container create 8ad1a58936f916c735955ffffd45f65a57c02138f9d1c8bf83ca0af704b06f8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-k, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS) 2024-05-21T03:19:47.839 INFO:journalctl@ceph.mds.k.smithi039.stdout:May 21 03:19:47 smithi039 podman[65511]: 2024-05-21 03:19:47.448183773 +0000 UTC m=+0.018146053 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T03:19:47.839 INFO:journalctl@ceph.mds.k.smithi039.stdout:May 21 03:19:47 smithi039 podman[65511]: 2024-05-21 03:19:47.755087094 +0000 UTC m=+0.325049347 container init 8ad1a58936f916c735955ffffd45f65a57c02138f9d1c8bf83ca0af704b06f8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-21T03:19:47.839 INFO:journalctl@ceph.mds.k.smithi039.stdout:May 21 03:19:47 smithi039 podman[65511]: 2024-05-21 03:19:47.757806868 +0000 UTC m=+0.327769128 container start 8ad1a58936f916c735955ffffd45f65a57c02138f9d1c8bf83ca0af704b06f8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-k, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-05-21T03:19:47.839 INFO:journalctl@ceph.mds.k.smithi039.stdout:May 21 03:19:47 smithi039 ceph-mds[65530]: -- 172.21.15.39:0/1786283106 <== mon.2 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ca462b5a40 con 0x55ca46288800 2024-05-21T03:19:47.839 INFO:journalctl@ceph.mds.k.smithi039.stdout:May 21 03:19:47 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-k[65526]: starting mds.k at 2024-05-21T03:19:47.840 INFO:journalctl@ceph.mds.k.smithi039.stdout:May 21 03:19:47 smithi039 bash[65511]: 8ad1a58936f916c735955ffffd45f65a57c02138f9d1c8bf83ca0af704b06f8c 2024-05-21T03:19:48.012 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:19:48.269 INFO:journalctl@ceph.mds.k.smithi039.stdout:May 21 03:19:47 smithi039 systemd[1]: Started Ceph mds.k for c9ca94be-171f-11ef-bc9a-c7b262605968. 2024-05-21T03:19:49.166 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:48 smithi027 ceph-mon[32179]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:49.166 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:48 smithi027 ceph-mon[32179]: mds.? [v2:172.21.15.39:6832/1102873763,v1:172.21.15.39:6833/1102873763] up:boot 2024-05-21T03:19:49.167 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:48 smithi027 ceph-mon[32179]: fsmap 2 up:standby 2024-05-21T03:19:49.167 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:48 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T03:19:49.167 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:48 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:49.167 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:48 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:49.167 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:48 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:49.167 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:48 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:19:49.167 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:48 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:19:49.167 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:48 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:49.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:48 smithi039 ceph-mon[36429]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:49.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:48 smithi039 ceph-mon[36429]: mds.? [v2:172.21.15.39:6832/1102873763,v1:172.21.15.39:6833/1102873763] up:boot 2024-05-21T03:19:49.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:48 smithi039 ceph-mon[36429]: fsmap 2 up:standby 2024-05-21T03:19:49.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:48 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T03:19:49.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:48 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:49.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:48 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:49.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:48 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:49.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:48 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:19:49.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:48 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:19:49.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:48 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:49.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:48 smithi165 ceph-mon[36269]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:49.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:48 smithi165 ceph-mon[36269]: mds.? [v2:172.21.15.39:6832/1102873763,v1:172.21.15.39:6833/1102873763] up:boot 2024-05-21T03:19:49.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:48 smithi165 ceph-mon[36269]: fsmap 2 up:standby 2024-05-21T03:19:49.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:48 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T03:19:49.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:48 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:49.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:48 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:49.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:48 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:49.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:48 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:19:49.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:48 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:19:49.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:48 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:49.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.600+0000 7fb214117700 1 -- 172.21.15.27:0/2007965616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb20c102430 msgr2=0x7fb20c1028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:49.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.600+0000 7fb214117700 1 --2- 172.21.15.27:0/2007965616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb20c102430 0x7fb20c1028b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc009990 tx=0x7fb1fc009ca0 comp rx=0 tx=0).stop 2024-05-21T03:19:49.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.600+0000 7fb214117700 1 -- 172.21.15.27:0/2007965616 shutdown_connections 2024-05-21T03:19:49.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.600+0000 7fb214117700 1 --2- 172.21.15.27:0/2007965616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb20c102df0 0x7fb20c10f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:49.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.600+0000 7fb214117700 1 --2- 172.21.15.27:0/2007965616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb20c108510 0x7fb20c1088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:49.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.600+0000 7fb214117700 1 --2- 172.21.15.27:0/2007965616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb20c102430 0x7fb20c1028b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:49.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.600+0000 7fb214117700 1 -- 172.21.15.27:0/2007965616 >> 172.21.15.27:0/2007965616 conn(0x7fb20c0793b0 msgr2=0x7fb20c0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:19:49.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.600+0000 7fb214117700 1 -- 172.21.15.27:0/2007965616 shutdown_connections 2024-05-21T03:19:49.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.600+0000 7fb214117700 1 -- 172.21.15.27:0/2007965616 wait complete. 2024-05-21T03:19:49.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.601+0000 7fb214117700 1 Processor -- start 2024-05-21T03:19:49.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.601+0000 7fb214117700 1 -- start start 2024-05-21T03:19:49.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.601+0000 7fb214117700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb20c102430 0x7fb20c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:49.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.601+0000 7fb214117700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb20c102df0 0x7fb20c19af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:49.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.601+0000 7fb214117700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb20c108510 0x7fb20c1a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:49.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.601+0000 7fb214117700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c1125c0 con 0x7fb20c108510 2024-05-21T03:19:49.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.601+0000 7fb214117700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c112440 con 0x7fb20c102430 2024-05-21T03:19:49.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.601+0000 7fb214117700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c112740 con 0x7fb20c102df0 2024-05-21T03:19:49.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.601+0000 7fb2126b4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb20c108510 0x7fb20c1a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:49.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.601+0000 7fb2126b4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb20c108510 0x7fb20c1a5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58742/0 (socket says 172.21.15.27:58742) 2024-05-21T03:19:49.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.601+0000 7fb2126b4700 1 -- 172.21.15.27:0/3652003033 learned_addr learned my addr 172.21.15.27:0/3652003033 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:19:49.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.602+0000 7fb211eb3700 1 --2- 172.21.15.27:0/3652003033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb20c102430 0x7fb20c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:49.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.602+0000 7fb2116b2700 1 --2- 172.21.15.27:0/3652003033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb20c102df0 0x7fb20c19af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:49.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.602+0000 7fb2126b4700 1 -- 172.21.15.27:0/3652003033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb20c102df0 msgr2=0x7fb20c19af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:49.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.602+0000 7fb2126b4700 1 --2- 172.21.15.27:0/3652003033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb20c102df0 0x7fb20c19af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:49.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.602+0000 7fb2126b4700 1 -- 172.21.15.27:0/3652003033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb20c102430 msgr2=0x7fb20c19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:49.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.602+0000 7fb2126b4700 1 --2- 172.21.15.27:0/3652003033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb20c102430 0x7fb20c19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:49.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.602+0000 7fb2126b4700 1 -- 172.21.15.27:0/3652003033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb20c1a6000 con 0x7fb20c108510 2024-05-21T03:19:49.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.602+0000 7fb2126b4700 1 --2- 172.21.15.27:0/3652003033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb20c108510 0x7fb20c1a5ac0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb20400c900 tx=0x7fb20400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:19:49.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.602+0000 7fb202ffd700 1 -- 172.21.15.27:0/3652003033 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2040078d0 con 0x7fb20c108510 2024-05-21T03:19:49.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.602+0000 7fb214117700 1 -- 172.21.15.27:0/3652003033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb20c1a6230 con 0x7fb20c108510 2024-05-21T03:19:49.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.602+0000 7fb202ffd700 1 -- 172.21.15.27:0/3652003033 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb20400f450 con 0x7fb20c108510 2024-05-21T03:19:49.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.602+0000 7fb202ffd700 1 -- 172.21.15.27:0/3652003033 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb204018620 con 0x7fb20c108510 2024-05-21T03:19:49.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.602+0000 7fb214117700 1 -- 172.21.15.27:0/3652003033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb20c1a66f0 con 0x7fb20c108510 2024-05-21T03:19:49.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.604+0000 7fb214117700 1 -- 172.21.15.27:0/3652003033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb20c06a230 con 0x7fb20c108510 2024-05-21T03:19:49.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.606+0000 7fb202ffd700 1 -- 172.21.15.27:0/3652003033 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb204007a70 con 0x7fb20c108510 2024-05-21T03:19:49.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.607+0000 7fb202ffd700 1 --2- 172.21.15.27:0/3652003033 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb1f80b2880 0x7fb1f80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:49.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.607+0000 7fb211eb3700 1 --2- 172.21.15.27:0/3652003033 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb1f80b2880 0x7fb1f80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:49.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.607+0000 7fb202ffd700 1 -- 172.21.15.27:0/3652003033 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fb2040e1ce0 con 0x7fb20c108510 2024-05-21T03:19:49.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.607+0000 7fb211eb3700 1 --2- 172.21.15.27:0/3652003033 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb1f80b2880 0x7fb1f80b4d40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb208009550 tx=0x7fb2080091f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:19:49.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.615+0000 7fb202ffd700 1 -- 172.21.15.27:0/3652003033 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb204014030 con 0x7fb20c108510 2024-05-21T03:19:49.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.883+0000 7fb214117700 1 -- 172.21.15.27:0/3652003033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb20c02e560 con 0x7fb20c108510 2024-05-21T03:19:49.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.883+0000 7fb202ffd700 1 -- 172.21.15.27:0/3652003033 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7fb2040a3e20 con 0x7fb20c108510 2024-05-21T03:19:49.884 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:19:49.884 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-21T03:19:49.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.886+0000 7fb214117700 1 -- 172.21.15.27:0/3652003033 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb1f80b2880 msgr2=0x7fb1f80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:49.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.886+0000 7fb214117700 1 --2- 172.21.15.27:0/3652003033 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb1f80b2880 0x7fb1f80b4d40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb208009550 tx=0x7fb2080091f0 comp rx=0 tx=0).stop 2024-05-21T03:19:49.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.886+0000 7fb214117700 1 -- 172.21.15.27:0/3652003033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb20c108510 msgr2=0x7fb20c1a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:49.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.886+0000 7fb214117700 1 --2- 172.21.15.27:0/3652003033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb20c108510 0x7fb20c1a5ac0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb20400c900 tx=0x7fb20400cc10 comp rx=0 tx=0).stop 2024-05-21T03:19:49.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.886+0000 7fb214117700 1 -- 172.21.15.27:0/3652003033 shutdown_connections 2024-05-21T03:19:49.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.886+0000 7fb214117700 1 --2- 172.21.15.27:0/3652003033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb20c108510 0x7fb20c1a5ac0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:49.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.886+0000 7fb214117700 1 --2- 172.21.15.27:0/3652003033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb20c102430 0x7fb20c19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:49.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.886+0000 7fb214117700 1 --2- 172.21.15.27:0/3652003033 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb1f80b2880 0x7fb1f80b4d40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:49.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.886+0000 7fb214117700 1 --2- 172.21.15.27:0/3652003033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb20c102df0 0x7fb20c19af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:49.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.886+0000 7fb214117700 1 -- 172.21.15.27:0/3652003033 >> 172.21.15.27:0/3652003033 conn(0x7fb20c0793b0 msgr2=0x7fb20c1005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:19:49.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.886+0000 7fb214117700 1 -- 172.21.15.27:0/3652003033 shutdown_connections 2024-05-21T03:19:49.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:49.886+0000 7fb214117700 1 -- 172.21.15.27:0/3652003033 wait complete. 2024-05-21T03:19:49.887 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 3 2024-05-21T03:19:50.168 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:49 smithi027 ceph-mon[32179]: Deploying daemon mds.j on smithi027 2024-05-21T03:19:50.177 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:49 smithi039 ceph-mon[36429]: Deploying daemon mds.j on smithi027 2024-05-21T03:19:50.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:49 smithi165 ceph-mon[36269]: Deploying daemon mds.j on smithi027 2024-05-21T03:19:51.136 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:50 smithi027 ceph-mon[32179]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:51.136 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:50 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3652003033' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:19:51.141 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:50 smithi039 ceph-mon[36429]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:51.141 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:50 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/3652003033' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:19:51.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:50 smithi165 ceph-mon[36269]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:51.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:50 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/3652003033' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:19:51.927 INFO:journalctl@ceph.mds.j.smithi027.stdout:May 21 03:19:51 smithi027 podman[70535]: 2024-05-21 03:19:51.806876518 +0000 UTC m=+0.133031262 container create f9ede6e050edc8d10e24e5a84003cd2e431fc04cea41f4af245c863e00cd792b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-21T03:19:52.427 INFO:journalctl@ceph.mds.j.smithi027.stdout:May 21 03:19:52 smithi027 podman[70535]: 2024-05-21 03:19:52.043192222 +0000 UTC m=+0.369346973 container init f9ede6e050edc8d10e24e5a84003cd2e431fc04cea41f4af245c863e00cd792b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-j, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2) 2024-05-21T03:19:52.427 INFO:journalctl@ceph.mds.j.smithi027.stdout:May 21 03:19:52 smithi027 podman[70535]: 2024-05-21 03:19:52.046262796 +0000 UTC m=+0.372417551 container start f9ede6e050edc8d10e24e5a84003cd2e431fc04cea41f4af245c863e00cd792b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-j, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD) 2024-05-21T03:19:52.427 INFO:journalctl@ceph.mds.j.smithi027.stdout:May 21 03:19:52 smithi027 ceph-mds[70554]: -- 172.21.15.27:0/152569384 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55657d4e5a40 con 0x55657d4b8c00 2024-05-21T03:19:52.427 INFO:journalctl@ceph.mds.j.smithi027.stdout:May 21 03:19:52 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-j[70550]: starting mds.j at 2024-05-21T03:19:52.427 INFO:journalctl@ceph.mds.j.smithi027.stdout:May 21 03:19:52 smithi027 bash[70535]: f9ede6e050edc8d10e24e5a84003cd2e431fc04cea41f4af245c863e00cd792b 2024-05-21T03:19:52.427 INFO:journalctl@ceph.mds.j.smithi027.stdout:May 21 03:19:52 smithi027 systemd[1]: Started Ceph mds.j for c9ca94be-171f-11ef-bc9a-c7b262605968. 2024-05-21T03:19:53.171 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:52 smithi165 ceph-mon[36269]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:53.171 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:52 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:53.171 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:52 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:53.171 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:52 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:53.171 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:52 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:19:53.171 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:52 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:19:53.171 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:52 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:53.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:52 smithi027 ceph-mon[32179]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:53.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:52 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:53.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:52 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:53.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:52 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:53.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:52 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:19:53.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:52 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:19:53.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:52 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:53.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:52 smithi039 ceph-mon[36429]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:53.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:52 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:53.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:52 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:53.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:52 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:53.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:52 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:19:53.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:52 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:19:53.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:52 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:53.364 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:19:53.947 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:19:54.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:53 smithi165 ceph-mon[36269]: Deploying daemon mds.d on smithi027 2024-05-21T03:19:54.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:53 smithi165 ceph-mon[36269]: mds.? [v2:172.21.15.27:6834/148854869,v1:172.21.15.27:6835/148854869] up:boot 2024-05-21T03:19:54.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:53 smithi165 ceph-mon[36269]: fsmap 3 up:standby 2024-05-21T03:19:54.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:53 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-21T03:19:54.070 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:53 smithi027 ceph-mon[32179]: Deploying daemon mds.d on smithi027 2024-05-21T03:19:54.071 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:53 smithi027 ceph-mon[32179]: mds.? [v2:172.21.15.27:6834/148854869,v1:172.21.15.27:6835/148854869] up:boot 2024-05-21T03:19:54.071 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:53 smithi027 ceph-mon[32179]: fsmap 3 up:standby 2024-05-21T03:19:54.071 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:53 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-21T03:19:54.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:53 smithi039 ceph-mon[36429]: Deploying daemon mds.d on smithi027 2024-05-21T03:19:54.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:53 smithi039 ceph-mon[36429]: mds.? [v2:172.21.15.27:6834/148854869,v1:172.21.15.27:6835/148854869] up:boot 2024-05-21T03:19:54.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:53 smithi039 ceph-mon[36429]: fsmap 3 up:standby 2024-05-21T03:19:54.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:53 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-21T03:19:55.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:54 smithi027 ceph-mon[32179]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:55.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:54 smithi039 ceph-mon[36429]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:55.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:54 smithi165 ceph-mon[36269]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:55.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.811+0000 7f3526635700 1 -- 172.21.15.27:0/1445253906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3520102460 msgr2=0x7f35201028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:55.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.811+0000 7f3526635700 1 --2- 172.21.15.27:0/1445253906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3520102460 0x7f35201028e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3508009990 tx=0x7f3508009ca0 comp rx=0 tx=0).stop 2024-05-21T03:19:55.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.811+0000 7f3526635700 1 -- 172.21.15.27:0/1445253906 shutdown_connections 2024-05-21T03:19:55.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.811+0000 7f3526635700 1 --2- 172.21.15.27:0/1445253906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3520102e20 0x7f352010f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:55.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.811+0000 7f3526635700 1 --2- 172.21.15.27:0/1445253906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35201084f0 0x7f35201088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:55.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.811+0000 7f3526635700 1 --2- 172.21.15.27:0/1445253906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3520102460 0x7f35201028e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:55.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.811+0000 7f3526635700 1 -- 172.21.15.27:0/1445253906 >> 172.21.15.27:0/1445253906 conn(0x7f35200794e0 msgr2=0x7f35200798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:19:55.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.811+0000 7f3526635700 1 -- 172.21.15.27:0/1445253906 shutdown_connections 2024-05-21T03:19:55.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.811+0000 7f3526635700 1 -- 172.21.15.27:0/1445253906 wait complete. 2024-05-21T03:19:55.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.812+0000 7f3526635700 1 Processor -- start 2024-05-21T03:19:55.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.812+0000 7f3526635700 1 -- start start 2024-05-21T03:19:55.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.812+0000 7f3526635700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3520102460 0x7f3520199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:55.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.812+0000 7f3526635700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3520102e20 0x7f352019a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:55.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.812+0000 7f3526635700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35201084f0 0x7f352019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:55.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.812+0000 7f3526635700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3520112750 con 0x7f3520102460 2024-05-21T03:19:55.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.812+0000 7f3526635700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f35201125d0 con 0x7f35201084f0 2024-05-21T03:19:55.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.812+0000 7f3526635700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f35201128d0 con 0x7f3520102e20 2024-05-21T03:19:55.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.812+0000 7f351ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3520102460 0x7f3520199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:55.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.812+0000 7f351ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3520102460 0x7f3520199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42826/0 (socket says 172.21.15.27:42826) 2024-05-21T03:19:55.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.812+0000 7f351ffff700 1 -- 172.21.15.27:0/3689990574 learned_addr learned my addr 172.21.15.27:0/3689990574 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:19:55.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.813+0000 7f351f7fe700 1 --2- 172.21.15.27:0/3689990574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3520102e20 0x7f352019a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:55.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.813+0000 7f3524bd2700 1 --2- 172.21.15.27:0/3689990574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35201084f0 0x7f352019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:55.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.813+0000 7f351ffff700 1 -- 172.21.15.27:0/3689990574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3520102e20 msgr2=0x7f352019a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:55.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.813+0000 7f351ffff700 1 --2- 172.21.15.27:0/3689990574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3520102e20 0x7f352019a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:55.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.813+0000 7f351ffff700 1 -- 172.21.15.27:0/3689990574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35201084f0 msgr2=0x7f352019e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:55.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.813+0000 7f351ffff700 1 --2- 172.21.15.27:0/3689990574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35201084f0 0x7f352019e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:55.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.813+0000 7f351ffff700 1 -- 172.21.15.27:0/3689990574 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352019ee40 con 0x7f3520102460 2024-05-21T03:19:55.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.813+0000 7f351f7fe700 1 --2- 172.21.15.27:0/3689990574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3520102e20 0x7f352019a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:19:55.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.813+0000 7f351ffff700 1 --2- 172.21.15.27:0/3689990574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3520102460 0x7f3520199e30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f351000b530 tx=0x7f351000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:19:55.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.813+0000 7f351d7fa700 1 -- 172.21.15.27:0/3689990574 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510015020 con 0x7f3520102460 2024-05-21T03:19:55.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.813+0000 7f351d7fa700 1 -- 172.21.15.27:0/3689990574 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3510011ce0 con 0x7f3520102460 2024-05-21T03:19:55.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.813+0000 7f351d7fa700 1 -- 172.21.15.27:0/3689990574 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510010d20 con 0x7f3520102460 2024-05-21T03:19:55.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.813+0000 7f3526635700 1 -- 172.21.15.27:0/3689990574 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f352019f130 con 0x7f3520102460 2024-05-21T03:19:55.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.813+0000 7f3526635700 1 -- 172.21.15.27:0/3689990574 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35201a69b0 con 0x7f3520102460 2024-05-21T03:19:55.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.814+0000 7f3526635700 1 -- 172.21.15.27:0/3689990574 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f352006a230 con 0x7f3520102460 2024-05-21T03:19:55.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.817+0000 7f351d7fa700 1 -- 172.21.15.27:0/3689990574 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3510018020 con 0x7f3520102460 2024-05-21T03:19:55.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.817+0000 7f351d7fa700 1 --2- 172.21.15.27:0/3689990574 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f350c0b2900 0x7f350c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:19:55.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.817+0000 7f351f7fe700 1 --2- 172.21.15.27:0/3689990574 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f350c0b2900 0x7f350c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:19:55.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.817+0000 7f351d7fa700 1 -- 172.21.15.27:0/3689990574 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f35100e1aa0 con 0x7f3520102460 2024-05-21T03:19:55.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.818+0000 7f351f7fe700 1 --2- 172.21.15.27:0/3689990574 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f350c0b2900 0x7f350c0b4dc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f350800b5c0 tx=0x7f3508005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:19:55.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:55.821+0000 7f351d7fa700 1 -- 172.21.15.27:0/3689990574 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f351001f020 con 0x7f3520102460 2024-05-21T03:19:56.094 INFO:journalctl@ceph.mds.d.smithi027.stdout:May 21 03:19:56 smithi027 systemd[1]: Started Ceph mds.d for c9ca94be-171f-11ef-bc9a-c7b262605968. 2024-05-21T03:19:56.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:56.093+0000 7f3526635700 1 -- 172.21.15.27:0/3689990574 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f352019b190 con 0x7f3520102460 2024-05-21T03:19:56.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:56.094+0000 7f351d7fa700 1 -- 172.21.15.27:0/3689990574 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7f35100a9c60 con 0x7f3520102460 2024-05-21T03:19:56.095 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:19:56.095 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/148854869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":148854869},{"type":"v1","addr":"172.21.15.27:6835","nonce":148854869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-21T03:19:56.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:56.098+0000 7f3526635700 1 -- 172.21.15.27:0/3689990574 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f350c0b2900 msgr2=0x7f350c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:56.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:56.098+0000 7f3526635700 1 --2- 172.21.15.27:0/3689990574 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f350c0b2900 0x7f350c0b4dc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f350800b5c0 tx=0x7f3508005fb0 comp rx=0 tx=0).stop 2024-05-21T03:19:56.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:56.098+0000 7f3526635700 1 -- 172.21.15.27:0/3689990574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3520102460 msgr2=0x7f3520199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:19:56.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:56.098+0000 7f3526635700 1 --2- 172.21.15.27:0/3689990574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3520102460 0x7f3520199e30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f351000b530 tx=0x7f351000b8f0 comp rx=0 tx=0).stop 2024-05-21T03:19:56.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:56.098+0000 7f3526635700 1 -- 172.21.15.27:0/3689990574 shutdown_connections 2024-05-21T03:19:56.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:56.098+0000 7f3526635700 1 --2- 172.21.15.27:0/3689990574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35201084f0 0x7f352019e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:56.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:56.098+0000 7f3526635700 1 --2- 172.21.15.27:0/3689990574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3520102460 0x7f3520199e30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:56.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:56.098+0000 7f3526635700 1 --2- 172.21.15.27:0/3689990574 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f350c0b2900 0x7f350c0b4dc0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:56.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:56.098+0000 7f3526635700 1 --2- 172.21.15.27:0/3689990574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3520102e20 0x7f352019a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:19:56.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:56.098+0000 7f3526635700 1 -- 172.21.15.27:0/3689990574 >> 172.21.15.27:0/3689990574 conn(0x7f35200794e0 msgr2=0x7f352006bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:19:56.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:56.098+0000 7f3526635700 1 -- 172.21.15.27:0/3689990574 shutdown_connections 2024-05-21T03:19:56.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:19:56.098+0000 7f3526635700 1 -- 172.21.15.27:0/3689990574 wait complete. 2024-05-21T03:19:56.100 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 4 2024-05-21T03:19:57.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:56 smithi039 ceph-mon[36429]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:57.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:56 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:57.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:56 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:57.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:56 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:57.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:56 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:19:57.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:56 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:19:57.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:56 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:57.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:56 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/3689990574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:19:57.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:56 smithi027 ceph-mon[32179]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:57.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:56 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:57.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:56 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:57.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:56 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:57.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:56 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:19:57.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:56 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:19:57.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:56 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:57.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:56 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3689990574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:19:57.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:56 smithi165 ceph-mon[36269]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:57.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:56 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:57.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:56 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:57.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:56 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:19:57.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:56 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:19:57.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:56 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:19:57.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:56 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:19:57.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:56 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/3689990574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:19:58.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:57 smithi027 ceph-mon[32179]: Deploying daemon mds.e on smithi039 2024-05-21T03:19:58.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:57 smithi027 ceph-mon[32179]: mds.? [v2:172.21.15.27:6836/4145544730,v1:172.21.15.27:6837/4145544730] up:boot 2024-05-21T03:19:58.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:57 smithi027 ceph-mon[32179]: fsmap 4 up:standby 2024-05-21T03:19:58.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:57 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-21T03:19:58.236 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:57 smithi039 ceph-mon[36429]: Deploying daemon mds.e on smithi039 2024-05-21T03:19:58.236 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:57 smithi039 ceph-mon[36429]: mds.? [v2:172.21.15.27:6836/4145544730,v1:172.21.15.27:6837/4145544730] up:boot 2024-05-21T03:19:58.236 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:57 smithi039 ceph-mon[36429]: fsmap 4 up:standby 2024-05-21T03:19:58.236 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:57 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-21T03:19:58.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:57 smithi165 ceph-mon[36269]: Deploying daemon mds.e on smithi039 2024-05-21T03:19:58.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:57 smithi165 ceph-mon[36269]: mds.? [v2:172.21.15.27:6836/4145544730,v1:172.21.15.27:6837/4145544730] up:boot 2024-05-21T03:19:58.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:57 smithi165 ceph-mon[36269]: fsmap 4 up:standby 2024-05-21T03:19:58.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:57 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-21T03:19:58.737 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:19:58.769 INFO:journalctl@ceph.mds.e.smithi039.stdout:May 21 03:19:58 smithi039 podman[65825]: 2024-05-21 03:19:58.622603332 +0000 UTC m=+0.017779836 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T03:19:59.003 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:19:59.125 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:19:58 smithi027 ceph-mon[32179]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:59.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:19:58 smithi039 ceph-mon[36429]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:19:59.270 INFO:journalctl@ceph.mds.e.smithi039.stdout:May 21 03:19:58 smithi039 podman[65825]: 2024-05-21 03:19:58.928505114 +0000 UTC m=+0.323681598 container init 67ae61586fc6f90be192e7fed616f34c03ab3e557b49659a63b592d2fd093478 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_CLEAN=True) 2024-05-21T03:19:59.270 INFO:journalctl@ceph.mds.e.smithi039.stdout:May 21 03:19:58 smithi039 podman[65825]: 2024-05-21 03:19:58.931425468 +0000 UTC m=+0.326601961 container start 67ae61586fc6f90be192e7fed616f34c03ab3e557b49659a63b592d2fd093478 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T03:19:59.270 INFO:journalctl@ceph.mds.e.smithi039.stdout:May 21 03:19:58 smithi039 ceph-mds[65844]: -- 172.21.15.39:0/2226667573 <== mon.2 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f7f39a3a40 con 0x55f7f3976800 2024-05-21T03:19:59.270 INFO:journalctl@ceph.mds.e.smithi039.stdout:May 21 03:19:58 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-e[65840]: starting mds.e at 2024-05-21T03:19:59.270 INFO:journalctl@ceph.mds.e.smithi039.stdout:May 21 03:19:59 smithi039 bash[65825]: 67ae61586fc6f90be192e7fed616f34c03ab3e557b49659a63b592d2fd093478 2024-05-21T03:19:59.270 INFO:journalctl@ceph.mds.e.smithi039.stdout:May 21 03:19:59 smithi039 systemd[1]: Started Ceph mds.e for c9ca94be-171f-11ef-bc9a-c7b262605968. 2024-05-21T03:19:59.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:19:58 smithi165 ceph-mon[36269]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:00.319 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:00 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:00.319 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:00 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:00.319 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:00 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:00.319 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:00 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:20:00.319 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:00 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:20:00.319 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:00 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:20:00.319 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:00 smithi165 ceph-mon[36269]: Deploying daemon mds.f on smithi165 2024-05-21T03:20:00.319 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:00 smithi165 ceph-mon[36269]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:00.319 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:00 smithi165 ceph-mon[36269]: mds.? [v2:172.21.15.39:6834/3497521608,v1:172.21.15.39:6835/3497521608] up:boot 2024-05-21T03:20:00.319 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:00 smithi165 ceph-mon[36269]: fsmap 5 up:standby 2024-05-21T03:20:00.319 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:00 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T03:20:00.319 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:00 smithi165 ceph-mon[36269]: overall HEALTH_OK 2024-05-21T03:20:00.345 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:00 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:00.345 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:00 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:00.345 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:00 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:00.345 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:00 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:20:00.345 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:00 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:20:00.345 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:00 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:20:00.345 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:00 smithi027 ceph-mon[32179]: Deploying daemon mds.f on smithi165 2024-05-21T03:20:00.345 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:00 smithi027 ceph-mon[32179]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:00.345 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:00 smithi027 ceph-mon[32179]: mds.? [v2:172.21.15.39:6834/3497521608,v1:172.21.15.39:6835/3497521608] up:boot 2024-05-21T03:20:00.345 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:00 smithi027 ceph-mon[32179]: fsmap 5 up:standby 2024-05-21T03:20:00.345 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:00 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T03:20:00.346 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:00 smithi027 ceph-mon[32179]: overall HEALTH_OK 2024-05-21T03:20:00.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:00 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:00.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:00 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:00.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:00 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:00.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:00 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:20:00.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:00 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:20:00.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:00 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:20:00.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:00 smithi039 ceph-mon[36429]: Deploying daemon mds.f on smithi165 2024-05-21T03:20:00.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:00 smithi039 ceph-mon[36429]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:00.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:00 smithi039 ceph-mon[36429]: mds.? [v2:172.21.15.39:6834/3497521608,v1:172.21.15.39:6835/3497521608] up:boot 2024-05-21T03:20:00.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:00 smithi039 ceph-mon[36429]: fsmap 5 up:standby 2024-05-21T03:20:00.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:00 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T03:20:00.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:00 smithi039 ceph-mon[36429]: overall HEALTH_OK 2024-05-21T03:20:00.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.777+0000 7f1632f3f700 1 -- 172.21.15.27:0/67377229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f162c103820 msgr2=0x7f162c103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:00.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.777+0000 7f1632f3f700 1 --2- 172.21.15.27:0/67377229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f162c103820 0x7f162c103c00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f161c0099c0 tx=0x7f161c009cd0 comp rx=0 tx=0).stop 2024-05-21T03:20:00.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.777+0000 7f1632f3f700 1 -- 172.21.15.27:0/67377229 shutdown_connections 2024-05-21T03:20:00.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.777+0000 7f1632f3f700 1 --2- 172.21.15.27:0/67377229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f162c108910 0x7f162c110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:00.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.777+0000 7f1632f3f700 1 --2- 172.21.15.27:0/67377229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f162c103820 0x7f162c103c00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:00.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.777+0000 7f1632f3f700 1 --2- 172.21.15.27:0/67377229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f162c1041d0 0x7f162c1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:00.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.777+0000 7f1632f3f700 1 -- 172.21.15.27:0/67377229 >> 172.21.15.27:0/67377229 conn(0x7f162c07a6b0 msgr2=0x7f162c07aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:00.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.777+0000 7f1632f3f700 1 -- 172.21.15.27:0/67377229 shutdown_connections 2024-05-21T03:20:00.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.777+0000 7f1632f3f700 1 -- 172.21.15.27:0/67377229 wait complete. 2024-05-21T03:20:00.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.777+0000 7f1632f3f700 1 Processor -- start 2024-05-21T03:20:00.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.777+0000 7f1632f3f700 1 -- start start 2024-05-21T03:20:00.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.778+0000 7f1632f3f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f162c103820 0x7f162c19be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:00.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.778+0000 7f1632f3f700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f162c1041d0 0x7f162c19c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:00.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.778+0000 7f1632f3f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f162c108910 0x7f162c1a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:00.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.778+0000 7f1632f3f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f162c113750 con 0x7f162c103820 2024-05-21T03:20:00.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.778+0000 7f1632f3f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f162c1135d0 con 0x7f162c108910 2024-05-21T03:20:00.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.778+0000 7f1632f3f700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f162c1138d0 con 0x7f162c1041d0 2024-05-21T03:20:00.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.778+0000 7f1630cdb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f162c103820 0x7f162c19be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:00.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.778+0000 7f1630cdb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f162c103820 0x7f162c19be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42868/0 (socket says 172.21.15.27:42868) 2024-05-21T03:20:00.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.778+0000 7f1630cdb700 1 -- 172.21.15.27:0/678482196 learned_addr learned my addr 172.21.15.27:0/678482196 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:20:00.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.778+0000 7f162bfff700 1 --2- 172.21.15.27:0/678482196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f162c1041d0 0x7f162c19c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:00.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.778+0000 7f16314dc700 1 --2- 172.21.15.27:0/678482196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f162c108910 0x7f162c1a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:00.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.778+0000 7f1630cdb700 1 -- 172.21.15.27:0/678482196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f162c1041d0 msgr2=0x7f162c19c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:00.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.778+0000 7f1630cdb700 1 --2- 172.21.15.27:0/678482196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f162c1041d0 0x7f162c19c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:00.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.778+0000 7f1630cdb700 1 -- 172.21.15.27:0/678482196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f162c108910 msgr2=0x7f162c1a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:00.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.778+0000 7f1630cdb700 1 --2- 172.21.15.27:0/678482196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f162c108910 0x7f162c1a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:00.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.778+0000 7f1630cdb700 1 -- 172.21.15.27:0/678482196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162c1a0e30 con 0x7f162c103820 2024-05-21T03:20:00.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.778+0000 7f1630cdb700 1 --2- 172.21.15.27:0/678482196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f162c103820 0x7f162c19be20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f161c00b580 tx=0x7f161c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:00.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.778+0000 7f1629ffb700 1 -- 172.21.15.27:0/678482196 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c027070 con 0x7f162c103820 2024-05-21T03:20:00.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.779+0000 7f1632f3f700 1 -- 172.21.15.27:0/678482196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f162c1a10c0 con 0x7f162c103820 2024-05-21T03:20:00.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.779+0000 7f1629ffb700 1 -- 172.21.15.27:0/678482196 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f161c004020 con 0x7f162c103820 2024-05-21T03:20:00.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.779+0000 7f1629ffb700 1 -- 172.21.15.27:0/678482196 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c022e30 con 0x7f162c103820 2024-05-21T03:20:00.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.779+0000 7f1632f3f700 1 -- 172.21.15.27:0/678482196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f162c1a89a0 con 0x7f162c103820 2024-05-21T03:20:00.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.780+0000 7f1632f3f700 1 -- 172.21.15.27:0/678482196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f162c06a230 con 0x7f162c103820 2024-05-21T03:20:00.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.781+0000 7f1629ffb700 1 -- 172.21.15.27:0/678482196 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f161c022590 con 0x7f162c103820 2024-05-21T03:20:00.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.782+0000 7f1629ffb700 1 --2- 172.21.15.27:0/678482196 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f16180b2880 0x7f16180b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:00.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.782+0000 7f162bfff700 1 --2- 172.21.15.27:0/678482196 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f16180b2880 0x7f16180b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:00.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.782+0000 7f1629ffb700 1 -- 172.21.15.27:0/678482196 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f161c0e2f40 con 0x7f162c103820 2024-05-21T03:20:00.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.782+0000 7f162bfff700 1 --2- 172.21.15.27:0/678482196 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f16180b2880 0x7f16180b4d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1614006fd0 tx=0x7f1614008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:00.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:00.785+0000 7f1629ffb700 1 -- 172.21.15.27:0/678482196 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f161c030070 con 0x7f162c103820 2024-05-21T03:20:01.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:01.046+0000 7f1632f3f700 1 -- 172.21.15.27:0/678482196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f162c1a8b70 con 0x7f162c103820 2024-05-21T03:20:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:01.047+0000 7f1629ffb700 1 -- 172.21.15.27:0/678482196 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7f161c0a6070 con 0x7f162c103820 2024-05-21T03:20:01.047 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:20:01.047 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/148854869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":148854869},{"type":"v1","addr":"172.21.15.27:6835","nonce":148854869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/4145544730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":4145544730},{"type":"v1","addr":"172.21.15.27:6837","nonce":4145544730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3497521608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3497521608},{"type":"v1","addr":"172.21.15.39:6835","nonce":3497521608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-21T03:20:01.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:01.049+0000 7f1632f3f700 1 -- 172.21.15.27:0/678482196 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f16180b2880 msgr2=0x7f16180b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:01.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:01.049+0000 7f1632f3f700 1 --2- 172.21.15.27:0/678482196 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f16180b2880 0x7f16180b4d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1614006fd0 tx=0x7f1614008040 comp rx=0 tx=0).stop 2024-05-21T03:20:01.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:01.049+0000 7f1632f3f700 1 -- 172.21.15.27:0/678482196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f162c103820 msgr2=0x7f162c19be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:01.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:01.049+0000 7f1632f3f700 1 --2- 172.21.15.27:0/678482196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f162c103820 0x7f162c19be20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f161c00b580 tx=0x7f161c0047b0 comp rx=0 tx=0).stop 2024-05-21T03:20:01.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:01.049+0000 7f1632f3f700 1 -- 172.21.15.27:0/678482196 shutdown_connections 2024-05-21T03:20:01.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:01.049+0000 7f1632f3f700 1 --2- 172.21.15.27:0/678482196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f162c108910 0x7f162c1a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:01.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:01.049+0000 7f1632f3f700 1 --2- 172.21.15.27:0/678482196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f162c103820 0x7f162c19be20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:01.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:01.049+0000 7f1632f3f700 1 --2- 172.21.15.27:0/678482196 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f16180b2880 0x7f16180b4d40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:01.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:01.049+0000 7f1632f3f700 1 --2- 172.21.15.27:0/678482196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f162c1041d0 0x7f162c19c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:01.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:01.049+0000 7f1632f3f700 1 -- 172.21.15.27:0/678482196 >> 172.21.15.27:0/678482196 conn(0x7f162c07a6b0 msgr2=0x7f162c109a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:01.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:01.049+0000 7f1632f3f700 1 -- 172.21.15.27:0/678482196 shutdown_connections 2024-05-21T03:20:01.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:01.049+0000 7f1632f3f700 1 -- 172.21.15.27:0/678482196 wait complete. 2024-05-21T03:20:01.051 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 6 2024-05-21T03:20:01.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:01 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/678482196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:20:01.271 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 21 03:20:01 smithi165 systemd[1]: Starting Ceph mds.f for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:20:01.349 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:01 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/678482196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:20:01.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:01 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/678482196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:20:01.639 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 21 03:20:01 smithi165 podman[66277]: 2024-05-21 03:20:01.283163685 +0000 UTC m=+0.017095430 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T03:20:01.639 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 21 03:20:01 smithi165 podman[66277]: 2024-05-21 03:20:01.387685835 +0000 UTC m=+0.121617569 container create 5588f1b9598aa03b19200bfcf9293c037ba916fd450585db3a17a99ed87df461 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-f, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD) 2024-05-21T03:20:01.639 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 21 03:20:01 smithi165 podman[66277]: 2024-05-21 03:20:01.564958399 +0000 UTC m=+0.298890147 container init 5588f1b9598aa03b19200bfcf9293c037ba916fd450585db3a17a99ed87df461 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-f, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-21T03:20:01.639 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 21 03:20:01 smithi165 podman[66277]: 2024-05-21 03:20:01.567625342 +0000 UTC m=+0.301557077 container start 5588f1b9598aa03b19200bfcf9293c037ba916fd450585db3a17a99ed87df461 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-f, org.label-schema.build-date=20240514, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-21T03:20:01.640 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 21 03:20:01 smithi165 ceph-mds[66296]: -- 172.21.15.165:0/40877886 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556c56f19a40 con 0x556c56eecc00 2024-05-21T03:20:01.640 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 21 03:20:01 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-f[66292]: starting mds.f at 2024-05-21T03:20:01.640 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 21 03:20:01 smithi165 bash[66277]: 5588f1b9598aa03b19200bfcf9293c037ba916fd450585db3a17a99ed87df461 2024-05-21T03:20:02.021 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 21 03:20:01 smithi165 systemd[1]: Started Ceph mds.f for c9ca94be-171f-11ef-bc9a-c7b262605968. 2024-05-21T03:20:03.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:02 smithi039 ceph-mon[36429]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:03.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:02 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:03.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:02 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:03.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:02 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:03.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:02 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:20:03.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:02 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:20:03.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:02 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:20:03.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:02 smithi039 ceph-mon[36429]: Deploying daemon mds.c on smithi165 2024-05-21T03:20:03.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:02 smithi039 ceph-mon[36429]: mds.? [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] up:boot 2024-05-21T03:20:03.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:02 smithi039 ceph-mon[36429]: fsmap 6 up:standby 2024-05-21T03:20:03.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:02 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T03:20:03.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:02 smithi165 ceph-mon[36269]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:03.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:02 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:03.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:02 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:03.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:02 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:03.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:02 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:20:03.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:02 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:20:03.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:02 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:20:03.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:02 smithi165 ceph-mon[36269]: Deploying daemon mds.c on smithi165 2024-05-21T03:20:03.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:02 smithi165 ceph-mon[36269]: mds.? [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] up:boot 2024-05-21T03:20:03.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:02 smithi165 ceph-mon[36269]: fsmap 6 up:standby 2024-05-21T03:20:03.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:02 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T03:20:03.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:02 smithi027 ceph-mon[32179]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:03.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:02 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:03.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:02 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:03.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:02 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:03.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:02 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:20:03.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:02 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:20:03.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:02 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:20:03.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:02 smithi027 ceph-mon[32179]: Deploying daemon mds.c on smithi165 2024-05-21T03:20:03.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:02 smithi027 ceph-mon[32179]: mds.? [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] up:boot 2024-05-21T03:20:03.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:02 smithi027 ceph-mon[32179]: fsmap 6 up:standby 2024-05-21T03:20:03.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:02 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T03:20:03.662 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:20:03.927 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:20:04.021 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 21 03:20:03 smithi165 systemd[1]: Starting Ceph mds.c for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:20:04.335 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 21 03:20:04 smithi165 podman[66577]: 2024-05-21 03:20:04.065496215 +0000 UTC m=+0.105629352 container create f306b4b0d890f9a4be97dafd9b2a73f3f4a43f0c8740802a183e8ef2d41afc37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-c, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-21T03:20:04.335 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 21 03:20:04 smithi165 podman[66577]: 2024-05-21 03:20:03.977184606 +0000 UTC m=+0.017317754 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T03:20:04.335 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 21 03:20:04 smithi165 podman[66577]: 2024-05-21 03:20:04.309865352 +0000 UTC m=+0.349998483 container init f306b4b0d890f9a4be97dafd9b2a73f3f4a43f0c8740802a183e8ef2d41afc37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-21T03:20:04.335 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 21 03:20:04 smithi165 podman[66577]: 2024-05-21 03:20:04.312385339 +0000 UTC m=+0.352518474 container start f306b4b0d890f9a4be97dafd9b2a73f3f4a43f0c8740802a183e8ef2d41afc37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-c, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T03:20:04.694 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 21 03:20:04 smithi165 ceph-mds[66597]: -- 172.21.15.165:0/1947393112 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56051fd3fa40 con 0x56051fd12c00 2024-05-21T03:20:04.694 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 21 03:20:04 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-c[66593]: starting mds.c at 2024-05-21T03:20:04.694 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 21 03:20:04 smithi165 bash[66577]: f306b4b0d890f9a4be97dafd9b2a73f3f4a43f0c8740802a183e8ef2d41afc37 2024-05-21T03:20:04.694 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 21 03:20:04 smithi165 systemd[1]: Started Ceph mds.c for c9ca94be-171f-11ef-bc9a-c7b262605968. 2024-05-21T03:20:04.956 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:04 smithi039 ceph-mon[36429]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:04.956 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:04 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:04.956 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:04 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:04.956 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:04 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:04.956 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:04 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:20:04.956 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:04 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:20:04.956 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:04 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:20:05.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:04 smithi165 ceph-mon[36269]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:05.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:04 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:05.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:04 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:05.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:04 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:05.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:04 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:20:05.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:04 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:20:05.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:04 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:20:05.031 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:04 smithi027 ceph-mon[32179]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:05.031 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:04 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:05.031 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:04 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:05.031 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:04 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:05.031 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:04 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:20:05.031 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:04 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:20:05.032 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:04 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:20:05.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.548+0000 7f82797e5700 1 -- 172.21.15.27:0/2725800898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8274103590 msgr2=0x7f8274103a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:05.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.548+0000 7f82797e5700 1 --2- 172.21.15.27:0/2725800898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8274103590 0x7f8274103a10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f825c009990 tx=0x7f825c009ca0 comp rx=0 tx=0).stop 2024-05-21T03:20:05.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.548+0000 7f82797e5700 1 -- 172.21.15.27:0/2725800898 shutdown_connections 2024-05-21T03:20:05.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.548+0000 7f82797e5700 1 --2- 172.21.15.27:0/2725800898 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8274103f50 0x7f8274110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:05.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.548+0000 7f82797e5700 1 --2- 172.21.15.27:0/2725800898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82741096a0 0x7f8274109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:05.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.548+0000 7f82797e5700 1 --2- 172.21.15.27:0/2725800898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8274103590 0x7f8274103a10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:05.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.548+0000 7f82797e5700 1 -- 172.21.15.27:0/2725800898 >> 172.21.15.27:0/2725800898 conn(0x7f827407a6f0 msgr2=0x7f827407ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:05.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.548+0000 7f82797e5700 1 -- 172.21.15.27:0/2725800898 shutdown_connections 2024-05-21T03:20:05.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.548+0000 7f82797e5700 1 -- 172.21.15.27:0/2725800898 wait complete. 2024-05-21T03:20:05.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.549+0000 7f82797e5700 1 Processor -- start 2024-05-21T03:20:05.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.549+0000 7f82797e5700 1 -- start start 2024-05-21T03:20:05.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.549+0000 7f82797e5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8274103590 0x7f8274075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:05.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.549+0000 7f82797e5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8274103f50 0x7f8274075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:05.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.549+0000 7f82797e5700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82741096a0 0x7f82740762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:05.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.549+0000 7f82797e5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8274113770 con 0x7f8274103f50 2024-05-21T03:20:05.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.549+0000 7f82797e5700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f82741135f0 con 0x7f8274103590 2024-05-21T03:20:05.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.549+0000 7f82797e5700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f82741138f0 con 0x7f82741096a0 2024-05-21T03:20:05.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.549+0000 7f82727fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8274103f50 0x7f8274075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:05.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.549+0000 7f82727fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8274103f50 0x7f8274075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33824/0 (socket says 172.21.15.27:33824) 2024-05-21T03:20:05.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.549+0000 7f82727fc700 1 -- 172.21.15.27:0/2612945748 learned_addr learned my addr 172.21.15.27:0/2612945748 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:20:05.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.549+0000 7f8272ffd700 1 --2- 172.21.15.27:0/2612945748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8274103590 0x7f8274075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:05.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.549+0000 7f82737fe700 1 --2- 172.21.15.27:0/2612945748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82741096a0 0x7f82740762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:05.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.550+0000 7f82727fc700 1 -- 172.21.15.27:0/2612945748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82741096a0 msgr2=0x7f82740762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:05.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.550+0000 7f82727fc700 1 --2- 172.21.15.27:0/2612945748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82741096a0 0x7f82740762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:05.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.550+0000 7f82727fc700 1 -- 172.21.15.27:0/2612945748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8274103590 msgr2=0x7f8274075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:05.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.550+0000 7f82727fc700 1 --2- 172.21.15.27:0/2612945748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8274103590 0x7f8274075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:05.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.550+0000 7f82727fc700 1 -- 172.21.15.27:0/2612945748 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82741a8130 con 0x7f8274103f50 2024-05-21T03:20:05.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.550+0000 7f82737fe700 1 --2- 172.21.15.27:0/2612945748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82741096a0 0x7f82740762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:20:05.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.550+0000 7f82727fc700 1 --2- 172.21.15.27:0/2612945748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8274103f50 0x7f8274075d80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f825c000c00 tx=0x7f825c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:05.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.550+0000 7f826bfff700 1 -- 172.21.15.27:0/2612945748 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c027070 con 0x7f8274103f50 2024-05-21T03:20:05.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.550+0000 7f82797e5700 1 -- 172.21.15.27:0/2612945748 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82741a83c0 con 0x7f8274103f50 2024-05-21T03:20:05.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.550+0000 7f826bfff700 1 -- 172.21.15.27:0/2612945748 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f825c003f20 con 0x7f8274103f50 2024-05-21T03:20:05.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.550+0000 7f826bfff700 1 -- 172.21.15.27:0/2612945748 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c022cf0 con 0x7f8274103f50 2024-05-21T03:20:05.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.550+0000 7f82797e5700 1 -- 172.21.15.27:0/2612945748 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82741a8820 con 0x7f8274103f50 2024-05-21T03:20:05.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.551+0000 7f82797e5700 1 -- 172.21.15.27:0/2612945748 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f827406a230 con 0x7f8274103f50 2024-05-21T03:20:05.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.553+0000 7f826bfff700 1 -- 172.21.15.27:0/2612945748 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f825c02a070 con 0x7f8274103f50 2024-05-21T03:20:05.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.553+0000 7f826bfff700 1 --2- 172.21.15.27:0/2612945748 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f82600b2880 0x7f82600b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:05.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.553+0000 7f8272ffd700 1 --2- 172.21.15.27:0/2612945748 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f82600b2880 0x7f82600b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:05.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.553+0000 7f826bfff700 1 -- 172.21.15.27:0/2612945748 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f825c0e2e60 con 0x7f8274103f50 2024-05-21T03:20:05.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.554+0000 7f8272ffd700 1 --2- 172.21.15.27:0/2612945748 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f82600b2880 0x7f82600b4d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f826400a400 tx=0x7f826400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:05.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.557+0000 7f826bfff700 1 -- 172.21.15.27:0/2612945748 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f825c013070 con 0x7f8274103f50 2024-05-21T03:20:05.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.826+0000 7f82797e5700 1 -- 172.21.15.27:0/2612945748 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f827402e560 con 0x7f8274103f50 2024-05-21T03:20:05.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.826+0000 7f826bfff700 1 -- 172.21.15.27:0/2612945748 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6346 (secure 0 0 0) 0x7f825c0ab0b0 con 0x7f8274103f50 2024-05-21T03:20:05.827 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:20:05.827 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/148854869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":148854869},{"type":"v1","addr":"172.21.15.27:6835","nonce":148854869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/4145544730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":4145544730},{"type":"v1","addr":"172.21.15.27:6837","nonce":4145544730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3497521608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3497521608},{"type":"v1","addr":"172.21.15.39:6835","nonce":3497521608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3576910218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3576910218},{"type":"v1","addr":"172.21.15.165:6835","nonce":3576910218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/691356139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":691356139},{"type":"v1","addr":"172.21.15.165:6837","nonce":691356139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-21T03:20:05.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.829+0000 7f82797e5700 1 -- 172.21.15.27:0/2612945748 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f82600b2880 msgr2=0x7f82600b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:05.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.829+0000 7f82797e5700 1 --2- 172.21.15.27:0/2612945748 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f82600b2880 0x7f82600b4d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f826400a400 tx=0x7f826400d040 comp rx=0 tx=0).stop 2024-05-21T03:20:05.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.829+0000 7f82797e5700 1 -- 172.21.15.27:0/2612945748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8274103f50 msgr2=0x7f8274075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:05.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.829+0000 7f82797e5700 1 --2- 172.21.15.27:0/2612945748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8274103f50 0x7f8274075d80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f825c000c00 tx=0x7f825c0047b0 comp rx=0 tx=0).stop 2024-05-21T03:20:05.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.829+0000 7f82797e5700 1 -- 172.21.15.27:0/2612945748 shutdown_connections 2024-05-21T03:20:05.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.829+0000 7f82797e5700 1 --2- 172.21.15.27:0/2612945748 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f82600b2880 0x7f82600b4d40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:05.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.829+0000 7f82797e5700 1 --2- 172.21.15.27:0/2612945748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82741096a0 0x7f82740762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:05.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.829+0000 7f82797e5700 1 --2- 172.21.15.27:0/2612945748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8274103590 0x7f8274075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:05.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.829+0000 7f82797e5700 1 --2- 172.21.15.27:0/2612945748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8274103f50 0x7f8274075d80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:05.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.829+0000 7f82797e5700 1 -- 172.21.15.27:0/2612945748 >> 172.21.15.27:0/2612945748 conn(0x7f827407a6f0 msgr2=0x7f8274102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:05.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.829+0000 7f82797e5700 1 -- 172.21.15.27:0/2612945748 shutdown_connections 2024-05-21T03:20:05.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:05.829+0000 7f82797e5700 1 -- 172.21.15.27:0/2612945748 wait complete. 2024-05-21T03:20:05.831 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 8 2024-05-21T03:20:06.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:05 smithi039 ceph-mon[36429]: Deploying daemon mds.h on smithi039 2024-05-21T03:20:06.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:05 smithi039 ceph-mon[36429]: mds.? [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] up:boot 2024-05-21T03:20:06.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:05 smithi039 ceph-mon[36429]: fsmap 7 up:standby 2024-05-21T03:20:06.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:05 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T03:20:06.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:05 smithi165 ceph-mon[36269]: Deploying daemon mds.h on smithi039 2024-05-21T03:20:06.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:05 smithi165 ceph-mon[36269]: mds.? [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] up:boot 2024-05-21T03:20:06.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:05 smithi165 ceph-mon[36269]: fsmap 7 up:standby 2024-05-21T03:20:06.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:05 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T03:20:06.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:05 smithi027 ceph-mon[32179]: Deploying daemon mds.h on smithi039 2024-05-21T03:20:06.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:05 smithi027 ceph-mon[32179]: mds.? [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] up:boot 2024-05-21T03:20:06.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:05 smithi027 ceph-mon[32179]: fsmap 7 up:standby 2024-05-21T03:20:06.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:05 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T03:20:06.959 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:06 smithi039 ceph-mon[36429]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:06.959 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:06 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/2612945748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:20:06.959 INFO:journalctl@ceph.mds.h.smithi039.stdout:May 21 03:20:06 smithi039 systemd[1]: Starting Ceph mds.h for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:20:07.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:06 smithi165 ceph-mon[36269]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:07.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:06 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/2612945748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:20:07.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:06 smithi027 ceph-mon[32179]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:07.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:06 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2612945748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:20:07.218 INFO:journalctl@ceph.mds.h.smithi039.stdout:May 21 03:20:06 smithi039 podman[66126]: 2024-05-21 03:20:06.862147577 +0000 UTC m=+0.017777739 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T03:20:07.218 INFO:journalctl@ceph.mds.h.smithi039.stdout:May 21 03:20:06 smithi039 podman[66126]: 2024-05-21 03:20:06.982681175 +0000 UTC m=+0.138311317 container create 4156b82453d026441ad60d265fe7b6f21bf2a9e9c8a9d2812c9dd9eee457960f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-h, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5) 2024-05-21T03:20:07.218 INFO:journalctl@ceph.mds.h.smithi039.stdout:May 21 03:20:07 smithi039 podman[66126]: 2024-05-21 03:20:07.218756528 +0000 UTC m=+0.374386666 container init 4156b82453d026441ad60d265fe7b6f21bf2a9e9c8a9d2812c9dd9eee457960f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T03:20:07.521 INFO:journalctl@ceph.mds.h.smithi039.stdout:May 21 03:20:07 smithi039 podman[66126]: 2024-05-21 03:20:07.221423569 +0000 UTC m=+0.377053718 container start 4156b82453d026441ad60d265fe7b6f21bf2a9e9c8a9d2812c9dd9eee457960f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-h, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea) 2024-05-21T03:20:07.521 INFO:journalctl@ceph.mds.h.smithi039.stdout:May 21 03:20:07 smithi039 ceph-mds[66147]: -- 172.21.15.39:0/3846201253 <== mon.2 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fd2f9e3a40 con 0x55fd2f9b7000 2024-05-21T03:20:07.522 INFO:journalctl@ceph.mds.h.smithi039.stdout:May 21 03:20:07 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-h[66143]: starting mds.h at 2024-05-21T03:20:07.522 INFO:journalctl@ceph.mds.h.smithi039.stdout:May 21 03:20:07 smithi039 bash[66126]: 4156b82453d026441ad60d265fe7b6f21bf2a9e9c8a9d2812c9dd9eee457960f 2024-05-21T03:20:07.522 INFO:journalctl@ceph.mds.h.smithi039.stdout:May 21 03:20:07 smithi039 systemd[1]: Started Ceph mds.h for c9ca94be-171f-11ef-bc9a-c7b262605968. 2024-05-21T03:20:08.437 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:20:08.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:08 smithi027 ceph-mon[32179]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:08.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:08 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:08.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:08 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:08.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:08 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:08.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:08 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:20:08.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:08 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:20:08.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:08 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:20:08.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:08 smithi027 ceph-mon[32179]: Deploying daemon mds.i on smithi165 2024-05-21T03:20:08.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:08 smithi027 ceph-mon[32179]: mds.? [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] up:boot 2024-05-21T03:20:08.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:08 smithi027 ceph-mon[32179]: fsmap 8 up:standby 2024-05-21T03:20:08.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:08 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T03:20:08.700 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:20:08.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:08 smithi039 ceph-mon[36429]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:08.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:08 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:08.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:08 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:08.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:08 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:08.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:08 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:20:08.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:08 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:20:08.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:08 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:20:08.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:08 smithi039 ceph-mon[36429]: Deploying daemon mds.i on smithi165 2024-05-21T03:20:08.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:08 smithi039 ceph-mon[36429]: mds.? [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] up:boot 2024-05-21T03:20:08.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:08 smithi039 ceph-mon[36429]: fsmap 8 up:standby 2024-05-21T03:20:08.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:08 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T03:20:08.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:08 smithi165 ceph-mon[36269]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:08.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:08 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:08.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:08 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:08.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:08 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:08.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:08 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:20:08.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:08 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:20:08.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:08 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:20:08.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:08 smithi165 ceph-mon[36269]: Deploying daemon mds.i on smithi165 2024-05-21T03:20:08.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:08 smithi165 ceph-mon[36269]: mds.? [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] up:boot 2024-05-21T03:20:08.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:08 smithi165 ceph-mon[36269]: fsmap 8 up:standby 2024-05-21T03:20:08.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:08 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T03:20:09.521 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 21 03:20:09 smithi165 systemd[1]: Starting Ceph mds.i for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:20:09.963 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 21 03:20:09 smithi165 podman[66880]: 2024-05-21 03:20:09.566003523 +0000 UTC m=+0.017188311 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T03:20:09.963 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 21 03:20:09 smithi165 podman[66880]: 2024-05-21 03:20:09.669829054 +0000 UTC m=+0.121013839 container create 9929b3291d14931ef3dca492fa2245988a9c893f8cb5f8cb82529d9c6ab3db22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-i, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T03:20:09.964 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 21 03:20:09 smithi165 podman[66880]: 2024-05-21 03:20:09.872790774 +0000 UTC m=+0.323975559 container init 9929b3291d14931ef3dca492fa2245988a9c893f8cb5f8cb82529d9c6ab3db22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-i, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514) 2024-05-21T03:20:09.964 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 21 03:20:09 smithi165 podman[66880]: 2024-05-21 03:20:09.875298762 +0000 UTC m=+0.326483557 container start 9929b3291d14931ef3dca492fa2245988a9c893f8cb5f8cb82529d9c6ab3db22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-21T03:20:09.964 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 21 03:20:09 smithi165 ceph-mds[66899]: -- 172.21.15.165:0/1874315892 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563c10f45a40 con 0x563c10f19000 2024-05-21T03:20:09.964 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 21 03:20:09 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-i[66895]: starting mds.i at 2024-05-21T03:20:09.964 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 21 03:20:09 smithi165 bash[66880]: 9929b3291d14931ef3dca492fa2245988a9c893f8cb5f8cb82529d9c6ab3db22 2024-05-21T03:20:10.271 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 21 03:20:09 smithi165 systemd[1]: Started Ceph mds.i for c9ca94be-171f-11ef-bc9a-c7b262605968. 2024-05-21T03:20:10.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.351+0000 7f5c724cd700 1 -- 172.21.15.27:0/3051751401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c6c102df0 msgr2=0x7f5c6c10f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:10.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.351+0000 7f5c724cd700 1 --2- 172.21.15.27:0/3051751401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c6c102df0 0x7f5c6c10f2e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5c6400b0a0 tx=0x7f5c6400b3b0 comp rx=0 tx=0).stop 2024-05-21T03:20:10.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.351+0000 7f5c724cd700 1 -- 172.21.15.27:0/3051751401 shutdown_connections 2024-05-21T03:20:10.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.351+0000 7f5c724cd700 1 --2- 172.21.15.27:0/3051751401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c6c102df0 0x7f5c6c10f2e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:10.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.351+0000 7f5c724cd700 1 --2- 172.21.15.27:0/3051751401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c6c108510 0x7f5c6c1088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:10.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.351+0000 7f5c724cd700 1 --2- 172.21.15.27:0/3051751401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c6c102430 0x7f5c6c1028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:10.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.351+0000 7f5c724cd700 1 -- 172.21.15.27:0/3051751401 >> 172.21.15.27:0/3051751401 conn(0x7f5c6c0793b0 msgr2=0x7f5c6c0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:10.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.351+0000 7f5c724cd700 1 -- 172.21.15.27:0/3051751401 shutdown_connections 2024-05-21T03:20:10.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.351+0000 7f5c724cd700 1 -- 172.21.15.27:0/3051751401 wait complete. 2024-05-21T03:20:10.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.352+0000 7f5c724cd700 1 Processor -- start 2024-05-21T03:20:10.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.352+0000 7f5c724cd700 1 -- start start 2024-05-21T03:20:10.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.352+0000 7f5c724cd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c6c102430 0x7f5c6c19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:10.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.352+0000 7f5c724cd700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c6c102df0 0x7f5c6c19af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:10.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.352+0000 7f5c724cd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c6c108510 0x7f5c6c1a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:10.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.352+0000 7f5c724cd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6c1125e0 con 0x7f5c6c102430 2024-05-21T03:20:10.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.352+0000 7f5c724cd700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6c112460 con 0x7f5c6c108510 2024-05-21T03:20:10.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.352+0000 7f5c724cd700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6c112760 con 0x7f5c6c102df0 2024-05-21T03:20:10.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.352+0000 7f5c6bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c6c102430 0x7f5c6c19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:10.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.353+0000 7f5c6bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c6c102430 0x7f5c6c19a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33848/0 (socket says 172.21.15.27:33848) 2024-05-21T03:20:10.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.353+0000 7f5c6bfff700 1 -- 172.21.15.27:0/692588802 learned_addr learned my addr 172.21.15.27:0/692588802 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:20:10.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.353+0000 7f5c6b7fe700 1 --2- 172.21.15.27:0/692588802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c6c102df0 0x7f5c6c19af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:10.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.353+0000 7f5c70a6a700 1 --2- 172.21.15.27:0/692588802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c6c108510 0x7f5c6c1a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:10.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.353+0000 7f5c6bfff700 1 -- 172.21.15.27:0/692588802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c6c102df0 msgr2=0x7f5c6c19af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:10.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.353+0000 7f5c6bfff700 1 --2- 172.21.15.27:0/692588802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c6c102df0 0x7f5c6c19af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:10.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.353+0000 7f5c6bfff700 1 -- 172.21.15.27:0/692588802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c6c108510 msgr2=0x7f5c6c1a5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:10.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.353+0000 7f5c6bfff700 1 --2- 172.21.15.27:0/692588802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c6c108510 0x7f5c6c1a5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:10.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.353+0000 7f5c6bfff700 1 -- 172.21.15.27:0/692588802 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c6c1a5f80 con 0x7f5c6c102430 2024-05-21T03:20:10.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.353+0000 7f5c6bfff700 1 --2- 172.21.15.27:0/692588802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c6c102430 0x7f5c6c19a9c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00d980 tx=0x7f5c5c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:10.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.353+0000 7f5c697fa700 1 -- 172.21.15.27:0/692588802 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c017070 con 0x7f5c6c102430 2024-05-21T03:20:10.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.353+0000 7f5c697fa700 1 -- 172.21.15.27:0/692588802 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c5c005020 con 0x7f5c6c102430 2024-05-21T03:20:10.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.353+0000 7f5c697fa700 1 -- 172.21.15.27:0/692588802 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c00fa10 con 0x7f5c6c102430 2024-05-21T03:20:10.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.353+0000 7f5c724cd700 1 -- 172.21.15.27:0/692588802 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c6c1a61b0 con 0x7f5c6c102430 2024-05-21T03:20:10.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.353+0000 7f5c724cd700 1 -- 172.21.15.27:0/692588802 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c6c1a6670 con 0x7f5c6c102430 2024-05-21T03:20:10.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.354+0000 7f5c724cd700 1 -- 172.21.15.27:0/692588802 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c6c10a9a0 con 0x7f5c6c102430 2024-05-21T03:20:10.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.356+0000 7f5c697fa700 1 -- 172.21.15.27:0/692588802 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5c5c004a30 con 0x7f5c6c102430 2024-05-21T03:20:10.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.357+0000 7f5c697fa700 1 --2- 172.21.15.27:0/692588802 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f5c540b2880 0x7f5c540b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:10.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.357+0000 7f5c6b7fe700 1 --2- 172.21.15.27:0/692588802 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f5c540b2880 0x7f5c540b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:10.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.357+0000 7f5c697fa700 1 -- 172.21.15.27:0/692588802 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f5c5c0e2090 con 0x7f5c6c102430 2024-05-21T03:20:10.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.357+0000 7f5c6b7fe700 1 --2- 172.21.15.27:0/692588802 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f5c540b2880 0x7f5c540b4d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c079a00 tx=0x7f5c60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:10.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.361+0000 7f5c697fa700 1 -- 172.21.15.27:0/692588802 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5c5c020020 con 0x7f5c6c102430 2024-05-21T03:20:10.426 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:10 smithi027 ceph-mon[32179]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:10.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:10 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:10.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:10 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:10.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:10 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:10.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:10 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:20:10.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:10 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:20:10.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:10 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:20:10.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.638+0000 7f5c724cd700 1 -- 172.21.15.27:0/692588802 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c6c103b50 con 0x7f5c6c102430 2024-05-21T03:20:10.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.638+0000 7f5c697fa700 1 -- 172.21.15.27:0/692588802 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8001 (secure 0 0 0) 0x7f5c5c0aa1b0 con 0x7f5c6c102430 2024-05-21T03:20:10.638 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:20:10.638 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/148854869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":148854869},{"type":"v1","addr":"172.21.15.27:6835","nonce":148854869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/4145544730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":4145544730},{"type":"v1","addr":"172.21.15.27:6837","nonce":4145544730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3497521608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3497521608},{"type":"v1","addr":"172.21.15.39:6835","nonce":3497521608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3576910218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3576910218},{"type":"v1","addr":"172.21.15.165:6835","nonce":3576910218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3673846852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3673846852},{"type":"v1","addr":"172.21.15.39:6837","nonce":3673846852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/691356139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":691356139},{"type":"v1","addr":"172.21.15.165:6837","nonce":691356139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6839/4158170814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4158170814},{"type":"v1","addr":"172.21.15.165:6839","nonce":4158170814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-21T03:20:10.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.640+0000 7f5c724cd700 1 -- 172.21.15.27:0/692588802 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f5c540b2880 msgr2=0x7f5c540b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:10.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.640+0000 7f5c724cd700 1 --2- 172.21.15.27:0/692588802 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f5c540b2880 0x7f5c540b4d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c079a00 tx=0x7f5c60009260 comp rx=0 tx=0).stop 2024-05-21T03:20:10.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.640+0000 7f5c724cd700 1 -- 172.21.15.27:0/692588802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c6c102430 msgr2=0x7f5c6c19a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:10.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.640+0000 7f5c724cd700 1 --2- 172.21.15.27:0/692588802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c6c102430 0x7f5c6c19a9c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00d980 tx=0x7f5c5c00dc90 comp rx=0 tx=0).stop 2024-05-21T03:20:10.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.640+0000 7f5c724cd700 1 -- 172.21.15.27:0/692588802 shutdown_connections 2024-05-21T03:20:10.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.640+0000 7f5c724cd700 1 --2- 172.21.15.27:0/692588802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c6c108510 0x7f5c6c1a5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:10.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.640+0000 7f5c724cd700 1 --2- 172.21.15.27:0/692588802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c6c102430 0x7f5c6c19a9c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:10.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.640+0000 7f5c724cd700 1 --2- 172.21.15.27:0/692588802 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f5c540b2880 0x7f5c540b4d40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:10.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.640+0000 7f5c724cd700 1 --2- 172.21.15.27:0/692588802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c6c102df0 0x7f5c6c19af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:10.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.640+0000 7f5c724cd700 1 -- 172.21.15.27:0/692588802 >> 172.21.15.27:0/692588802 conn(0x7f5c6c0793b0 msgr2=0x7f5c6c100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:10.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.641+0000 7f5c724cd700 1 -- 172.21.15.27:0/692588802 shutdown_connections 2024-05-21T03:20:10.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:10.641+0000 7f5c724cd700 1 -- 172.21.15.27:0/692588802 wait complete. 2024-05-21T03:20:10.641 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 10 2024-05-21T03:20:10.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:10 smithi039 ceph-mon[36429]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:10.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:10 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:10.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:10 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:10.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:10 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:10.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:10 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:20:10.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:10 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:20:10.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:10 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:20:10.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:10 smithi165 ceph-mon[36269]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:10.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:10 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:10.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:10 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:10.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:10 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:10.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:10 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:20:10.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:10 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:20:10.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:10 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:20:11.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:11 smithi027 ceph-mon[32179]: Deploying daemon mds.g on smithi027 2024-05-21T03:20:11.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:11 smithi027 ceph-mon[32179]: mds.? [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] up:boot 2024-05-21T03:20:11.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:11 smithi027 ceph-mon[32179]: fsmap 9 up:standby 2024-05-21T03:20:11.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:11 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T03:20:11.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:11 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/692588802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:20:11.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:11 smithi039 ceph-mon[36429]: Deploying daemon mds.g on smithi027 2024-05-21T03:20:11.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:11 smithi039 ceph-mon[36429]: mds.? [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] up:boot 2024-05-21T03:20:11.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:11 smithi039 ceph-mon[36429]: fsmap 9 up:standby 2024-05-21T03:20:11.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:11 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T03:20:11.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:11 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/692588802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:20:11.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:11 smithi165 ceph-mon[36269]: Deploying daemon mds.g on smithi027 2024-05-21T03:20:11.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:11 smithi165 ceph-mon[36269]: mds.? [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] up:boot 2024-05-21T03:20:11.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:11 smithi165 ceph-mon[36269]: fsmap 9 up:standby 2024-05-21T03:20:11.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:11 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T03:20:11.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:11 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/692588802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:20:12.541 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:12 smithi027 ceph-mon[32179]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:12.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:12 smithi039 ceph-mon[36429]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:12.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:12 smithi165 ceph-mon[36269]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:12.927 INFO:journalctl@ceph.mds.g.smithi027.stdout:May 21 03:20:12 smithi027 systemd[1]: Starting Ceph mds.g for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:20:13.210 INFO:journalctl@ceph.mds.g.smithi027.stdout:May 21 03:20:12 smithi027 podman[71786]: 2024-05-21 03:20:12.855029456 +0000 UTC m=+0.017731010 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T03:20:13.210 INFO:journalctl@ceph.mds.g.smithi027.stdout:May 21 03:20:12 smithi027 podman[71786]: 2024-05-21 03:20:12.972333281 +0000 UTC m=+0.135034832 container create 1fe1fed96868b998b051e877ff95aca649b808c2266acf5b605e105024fa44d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-g, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-21T03:20:13.210 INFO:journalctl@ceph.mds.g.smithi027.stdout:May 21 03:20:13 smithi027 podman[71786]: 2024-05-21 03:20:13.182979141 +0000 UTC m=+0.345680693 container init 1fe1fed96868b998b051e877ff95aca649b808c2266acf5b605e105024fa44d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-g, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-21T03:20:13.210 INFO:journalctl@ceph.mds.g.smithi027.stdout:May 21 03:20:13 smithi027 podman[71786]: 2024-05-21 03:20:13.185947623 +0000 UTC m=+0.348649179 container start 1fe1fed96868b998b051e877ff95aca649b808c2266acf5b605e105024fa44d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T03:20:13.401 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:20:13.656 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:20:13.676 INFO:journalctl@ceph.mds.g.smithi027.stdout:May 21 03:20:13 smithi027 ceph-mds[71805]: -- 172.21.15.27:0/2968674020 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5617b66a1a40 con 0x5617b6674c00 2024-05-21T03:20:13.676 INFO:journalctl@ceph.mds.g.smithi027.stdout:May 21 03:20:13 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-g[71801]: starting mds.g at 2024-05-21T03:20:13.677 INFO:journalctl@ceph.mds.g.smithi027.stdout:May 21 03:20:13 smithi027 bash[71786]: 1fe1fed96868b998b051e877ff95aca649b808c2266acf5b605e105024fa44d2 2024-05-21T03:20:13.677 INFO:journalctl@ceph.mds.g.smithi027.stdout:May 21 03:20:13 smithi027 systemd[1]: Started Ceph mds.g for c9ca94be-171f-11ef-bc9a-c7b262605968. 2024-05-21T03:20:14.666 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:14 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:14.667 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:14 smithi027 ceph-mon[32179]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:14.667 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:14 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:14.667 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:14 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:14.667 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:14 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:20:14.667 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:14 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:20:14.667 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:14 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:20:14.667 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:14 smithi027 ceph-mon[32179]: Deploying daemon mds.a on smithi027 2024-05-21T03:20:14.667 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:14 smithi027 ceph-mon[32179]: mds.? [v2:172.21.15.27:6838/745486764,v1:172.21.15.27:6839/745486764] up:boot 2024-05-21T03:20:14.667 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:14 smithi027 ceph-mon[32179]: fsmap 10 up:standby 2024-05-21T03:20:14.667 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:14 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-21T03:20:14.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:14 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:14.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:14 smithi039 ceph-mon[36429]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:14.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:14 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:14.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:14 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:14.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:14 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:20:14.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:14 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:20:14.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:14 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:20:14.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:14 smithi039 ceph-mon[36429]: Deploying daemon mds.a on smithi027 2024-05-21T03:20:14.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:14 smithi039 ceph-mon[36429]: mds.? [v2:172.21.15.27:6838/745486764,v1:172.21.15.27:6839/745486764] up:boot 2024-05-21T03:20:14.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:14 smithi039 ceph-mon[36429]: fsmap 10 up:standby 2024-05-21T03:20:14.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:14 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-21T03:20:14.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:14 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:14.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:14 smithi165 ceph-mon[36269]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:14.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:14 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:14.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:14 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:14.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:14 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:20:14.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:14 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:20:14.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:14 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:20:14.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:14 smithi165 ceph-mon[36269]: Deploying daemon mds.a on smithi027 2024-05-21T03:20:14.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:14 smithi165 ceph-mon[36269]: mds.? [v2:172.21.15.27:6838/745486764,v1:172.21.15.27:6839/745486764] up:boot 2024-05-21T03:20:14.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:14 smithi165 ceph-mon[36269]: fsmap 10 up:standby 2024-05-21T03:20:14.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:14 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-21T03:20:15.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.828+0000 7f5d8dd10700 1 -- 172.21.15.27:0/3007297080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d88108510 msgr2=0x7f5d881088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:15.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.828+0000 7f5d8dd10700 1 --2- 172.21.15.27:0/3007297080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d88108510 0x7f5d881088f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5d78009990 tx=0x7f5d78009ca0 comp rx=0 tx=0).stop 2024-05-21T03:20:15.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.828+0000 7f5d8dd10700 1 -- 172.21.15.27:0/3007297080 shutdown_connections 2024-05-21T03:20:15.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.828+0000 7f5d8dd10700 1 --2- 172.21.15.27:0/3007297080 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d88102df0 0x7f5d8810f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:15.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.828+0000 7f5d8dd10700 1 --2- 172.21.15.27:0/3007297080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d88108510 0x7f5d881088f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:15.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.828+0000 7f5d8dd10700 1 --2- 172.21.15.27:0/3007297080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d88102430 0x7f5d881028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:15.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.828+0000 7f5d8dd10700 1 -- 172.21.15.27:0/3007297080 >> 172.21.15.27:0/3007297080 conn(0x7f5d880793b0 msgr2=0x7f5d880797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:15.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.828+0000 7f5d8dd10700 1 -- 172.21.15.27:0/3007297080 shutdown_connections 2024-05-21T03:20:15.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.828+0000 7f5d8dd10700 1 -- 172.21.15.27:0/3007297080 wait complete. 2024-05-21T03:20:15.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.829+0000 7f5d8dd10700 1 Processor -- start 2024-05-21T03:20:15.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.829+0000 7f5d8dd10700 1 -- start start 2024-05-21T03:20:15.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.829+0000 7f5d8dd10700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d88102430 0x7f5d8819f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:15.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.829+0000 7f5d8dd10700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d88102df0 0x7f5d8819fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:15.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.829+0000 7f5d8dd10700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d88108510 0x7f5d88199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:15.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.830+0000 7f5d8dd10700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5d88112630 con 0x7f5d88108510 2024-05-21T03:20:15.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.830+0000 7f5d8dd10700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5d881124b0 con 0x7f5d88102430 2024-05-21T03:20:15.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.830+0000 7f5d8dd10700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5d881127b0 con 0x7f5d88102df0 2024-05-21T03:20:15.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.830+0000 7f5d87fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d88108510 0x7f5d88199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:15.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.830+0000 7f5d87fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d88108510 0x7f5d88199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57614/0 (socket says 172.21.15.27:57614) 2024-05-21T03:20:15.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.830+0000 7f5d87fff700 1 -- 172.21.15.27:0/2561260435 learned_addr learned my addr 172.21.15.27:0/2561260435 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:20:15.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.830+0000 7f5d86ffd700 1 --2- 172.21.15.27:0/2561260435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d88102df0 0x7f5d8819fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:15.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.830+0000 7f5d877fe700 1 --2- 172.21.15.27:0/2561260435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d88102430 0x7f5d8819f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:15.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.830+0000 7f5d87fff700 1 -- 172.21.15.27:0/2561260435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d88102df0 msgr2=0x7f5d8819fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:15.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.830+0000 7f5d87fff700 1 --2- 172.21.15.27:0/2561260435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d88102df0 0x7f5d8819fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:15.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.830+0000 7f5d87fff700 1 -- 172.21.15.27:0/2561260435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d88102430 msgr2=0x7f5d8819f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:15.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.830+0000 7f5d87fff700 1 --2- 172.21.15.27:0/2561260435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d88102430 0x7f5d8819f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:15.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.830+0000 7f5d87fff700 1 -- 172.21.15.27:0/2561260435 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d8819a1a0 con 0x7f5d88108510 2024-05-21T03:20:15.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.830+0000 7f5d877fe700 1 --2- 172.21.15.27:0/2561260435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d88102430 0x7f5d8819f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:20:15.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.830+0000 7f5d86ffd700 1 --2- 172.21.15.27:0/2561260435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d88102df0 0x7f5d8819fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:20:15.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.830+0000 7f5d87fff700 1 --2- 172.21.15.27:0/2561260435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d88108510 0x7f5d88199a40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5d80007ee0 tx=0x7f5d8000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:15.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.830+0000 7f5d84ff9700 1 -- 172.21.15.27:0/2561260435 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8000dbd0 con 0x7f5d88108510 2024-05-21T03:20:15.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.830+0000 7f5d8dd10700 1 -- 172.21.15.27:0/2561260435 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d8819a490 con 0x7f5d88108510 2024-05-21T03:20:15.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.830+0000 7f5d84ff9700 1 -- 172.21.15.27:0/2561260435 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d8000dd70 con 0x7f5d88108510 2024-05-21T03:20:15.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.830+0000 7f5d84ff9700 1 -- 172.21.15.27:0/2561260435 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d80005990 con 0x7f5d88108510 2024-05-21T03:20:15.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.830+0000 7f5d8dd10700 1 -- 172.21.15.27:0/2561260435 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d881a6750 con 0x7f5d88108510 2024-05-21T03:20:15.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.831+0000 7f5d8dd10700 1 -- 172.21.15.27:0/2561260435 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d8806a230 con 0x7f5d88108510 2024-05-21T03:20:15.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.833+0000 7f5d84ff9700 1 -- 172.21.15.27:0/2561260435 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5d8001a020 con 0x7f5d88108510 2024-05-21T03:20:15.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.833+0000 7f5d84ff9700 1 --2- 172.21.15.27:0/2561260435 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f5d700b2880 0x7f5d700b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:15.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.833+0000 7f5d877fe700 1 --2- 172.21.15.27:0/2561260435 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f5d700b2880 0x7f5d700b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:15.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.834+0000 7f5d84ff9700 1 -- 172.21.15.27:0/2561260435 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f5d800e1bf0 con 0x7f5d88108510 2024-05-21T03:20:15.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.834+0000 7f5d877fe700 1 --2- 172.21.15.27:0/2561260435 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f5d700b2880 0x7f5d700b4d40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5d7800b5c0 tx=0x7f5d78005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:15.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:15.837+0000 7f5d84ff9700 1 -- 172.21.15.27:0/2561260435 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5d80010070 con 0x7f5d88108510 2024-05-21T03:20:16.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:16.098+0000 7f5d8dd10700 1 -- 172.21.15.27:0/2561260435 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d881a6950 con 0x7f5d88108510 2024-05-21T03:20:16.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:16.099+0000 7f5d84ff9700 1 -- 172.21.15.27:0/2561260435 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8824 (secure 0 0 0) 0x7f5d800a9db0 con 0x7f5d88108510 2024-05-21T03:20:16.103 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:20:16.103 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/148854869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":148854869},{"type":"v1","addr":"172.21.15.27:6835","nonce":148854869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/4145544730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":4145544730},{"type":"v1","addr":"172.21.15.27:6837","nonce":4145544730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/745486764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":745486764},{"type":"v1","addr":"172.21.15.27:6839","nonce":745486764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3497521608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3497521608},{"type":"v1","addr":"172.21.15.39:6835","nonce":3497521608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3576910218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3576910218},{"type":"v1","addr":"172.21.15.165:6835","nonce":3576910218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3673846852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3673846852},{"type":"v1","addr":"172.21.15.39:6837","nonce":3673846852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/691356139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":691356139},{"type":"v1","addr":"172.21.15.165:6837","nonce":691356139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6839/4158170814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4158170814},{"type":"v1","addr":"172.21.15.165:6839","nonce":4158170814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-21T03:20:16.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:16.101+0000 7f5d8dd10700 1 -- 172.21.15.27:0/2561260435 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f5d700b2880 msgr2=0x7f5d700b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:16.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:16.101+0000 7f5d8dd10700 1 --2- 172.21.15.27:0/2561260435 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f5d700b2880 0x7f5d700b4d40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5d7800b5c0 tx=0x7f5d78005fb0 comp rx=0 tx=0).stop 2024-05-21T03:20:16.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:16.101+0000 7f5d8dd10700 1 -- 172.21.15.27:0/2561260435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d88108510 msgr2=0x7f5d88199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:16.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:16.101+0000 7f5d8dd10700 1 --2- 172.21.15.27:0/2561260435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d88108510 0x7f5d88199a40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5d80007ee0 tx=0x7f5d8000d3b0 comp rx=0 tx=0).stop 2024-05-21T03:20:16.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:16.101+0000 7f5d8dd10700 1 -- 172.21.15.27:0/2561260435 shutdown_connections 2024-05-21T03:20:16.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:16.101+0000 7f5d8dd10700 1 --2- 172.21.15.27:0/2561260435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d88108510 0x7f5d88199a40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:16.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:16.101+0000 7f5d8dd10700 1 --2- 172.21.15.27:0/2561260435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d88102430 0x7f5d8819f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:16.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:16.101+0000 7f5d8dd10700 1 --2- 172.21.15.27:0/2561260435 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f5d700b2880 0x7f5d700b4d40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:16.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:16.101+0000 7f5d8dd10700 1 --2- 172.21.15.27:0/2561260435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d88102df0 0x7f5d8819fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:16.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:16.101+0000 7f5d8dd10700 1 -- 172.21.15.27:0/2561260435 >> 172.21.15.27:0/2561260435 conn(0x7f5d880793b0 msgr2=0x7f5d8806bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:16.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:16.101+0000 7f5d8dd10700 1 -- 172.21.15.27:0/2561260435 shutdown_connections 2024-05-21T03:20:16.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:16.101+0000 7f5d8dd10700 1 -- 172.21.15.27:0/2561260435 wait complete. 2024-05-21T03:20:16.106 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 11 2024-05-21T03:20:16.550 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:16 smithi027 ceph-mon[32179]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:16.550 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:16 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2561260435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:20:16.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:16 smithi039 ceph-mon[36429]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:16.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:16 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/2561260435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:20:16.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:16 smithi165 ceph-mon[36269]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:16.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:16 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/2561260435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:20:16.905 INFO:journalctl@ceph.mds.a.smithi027.stdout:May 21 03:20:16 smithi027 systemd[1]: Starting Ceph mds.a for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:20:17.176 INFO:journalctl@ceph.mds.a.smithi027.stdout:May 21 03:20:17 smithi027 podman[72243]: 2024-05-21 03:20:17.014876711 +0000 UTC m=+0.017831868 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T03:20:17.177 INFO:journalctl@ceph.mds.a.smithi027.stdout:May 21 03:20:17 smithi027 podman[72243]: 2024-05-21 03:20:17.149231091 +0000 UTC m=+0.152186269 container create 8230c3edc1621e10369cc1392f409c5070d11c6b48f99be11244b3b2fabd414a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-21T03:20:17.677 INFO:journalctl@ceph.mds.a.smithi027.stdout:May 21 03:20:17 smithi027 podman[72243]: 2024-05-21 03:20:17.402172948 +0000 UTC m=+0.405128096 container init 8230c3edc1621e10369cc1392f409c5070d11c6b48f99be11244b3b2fabd414a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-21T03:20:17.677 INFO:journalctl@ceph.mds.a.smithi027.stdout:May 21 03:20:17 smithi027 podman[72243]: 2024-05-21 03:20:17.405107639 +0000 UTC m=+0.408062804 container start 8230c3edc1621e10369cc1392f409c5070d11c6b48f99be11244b3b2fabd414a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-a, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-21T03:20:17.677 INFO:journalctl@ceph.mds.a.smithi027.stdout:May 21 03:20:17 smithi027 ceph-mds[72262]: -- 172.21.15.27:0/2050066967 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eb2c751a40 con 0x55eb2c724800 2024-05-21T03:20:17.677 INFO:journalctl@ceph.mds.a.smithi027.stdout:May 21 03:20:17 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-a[72258]: starting mds.a at 2024-05-21T03:20:17.677 INFO:journalctl@ceph.mds.a.smithi027.stdout:May 21 03:20:17 smithi027 bash[72243]: 8230c3edc1621e10369cc1392f409c5070d11c6b48f99be11244b3b2fabd414a 2024-05-21T03:20:17.677 INFO:journalctl@ceph.mds.a.smithi027.stdout:May 21 03:20:17 smithi027 systemd[1]: Started Ceph mds.a for c9ca94be-171f-11ef-bc9a-c7b262605968. 2024-05-21T03:20:18.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:18 smithi027 ceph-mon[32179]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:18.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:18 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:18.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:18 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:18.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:18 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:18.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:18 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:20:18.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:18 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:20:18.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:18 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:20:18.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:18 smithi027 ceph-mon[32179]: Deploying daemon mds.b on smithi039 2024-05-21T03:20:18.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:18 smithi027 ceph-mon[32179]: mds.? [v2:172.21.15.27:6840/3982752351,v1:172.21.15.27:6841/3982752351] up:boot 2024-05-21T03:20:18.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:18 smithi027 ceph-mon[32179]: fsmap 11 up:standby 2024-05-21T03:20:18.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:18 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T03:20:18.939 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:20:19.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:18 smithi039 ceph-mon[36429]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:19.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:18 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:19.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:18 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:19.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:18 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:19.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:18 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:20:19.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:18 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:20:19.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:18 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:20:19.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:18 smithi039 ceph-mon[36429]: Deploying daemon mds.b on smithi039 2024-05-21T03:20:19.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:18 smithi039 ceph-mon[36429]: mds.? [v2:172.21.15.27:6840/3982752351,v1:172.21.15.27:6841/3982752351] up:boot 2024-05-21T03:20:19.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:18 smithi039 ceph-mon[36429]: fsmap 11 up:standby 2024-05-21T03:20:19.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:18 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T03:20:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:18 smithi165 ceph-mon[36269]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:18 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:18 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:18 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:18 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T03:20:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:18 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T03:20:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:18 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:20:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:18 smithi165 ceph-mon[36269]: Deploying daemon mds.b on smithi039 2024-05-21T03:20:19.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:18 smithi165 ceph-mon[36269]: mds.? [v2:172.21.15.27:6840/3982752351,v1:172.21.15.27:6841/3982752351] up:boot 2024-05-21T03:20:19.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:18 smithi165 ceph-mon[36269]: fsmap 11 up:standby 2024-05-21T03:20:19.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:18 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T03:20:19.206 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:20:19.965 INFO:journalctl@ceph.mds.b.smithi039.stdout:May 21 03:20:19 smithi039 systemd[1]: Starting Ceph mds.b for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:20:20.232 INFO:journalctl@ceph.mds.b.smithi039.stdout:May 21 03:20:19 smithi039 podman[66431]: 2024-05-21 03:20:19.867956177 +0000 UTC m=+0.017972575 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T03:20:20.232 INFO:journalctl@ceph.mds.b.smithi039.stdout:May 21 03:20:19 smithi039 podman[66431]: 2024-05-21 03:20:19.986653431 +0000 UTC m=+0.136669882 container create 1f3a14b16cf9f44eb54e8dadb9525bcdba1315938f03d8bfabd9ed6bdcf81faf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-21T03:20:20.232 INFO:journalctl@ceph.mds.b.smithi039.stdout:May 21 03:20:20 smithi039 podman[66431]: 2024-05-21 03:20:20.20619684 +0000 UTC m=+0.356213239 container init 1f3a14b16cf9f44eb54e8dadb9525bcdba1315938f03d8bfabd9ed6bdcf81faf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-b, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-21T03:20:20.232 INFO:journalctl@ceph.mds.b.smithi039.stdout:May 21 03:20:20 smithi039 podman[66431]: 2024-05-21 03:20:20.208989127 +0000 UTC m=+0.359005536 container start 1f3a14b16cf9f44eb54e8dadb9525bcdba1315938f03d8bfabd9ed6bdcf81faf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True) 2024-05-21T03:20:20.519 INFO:journalctl@ceph.mds.b.smithi039.stdout:May 21 03:20:20 smithi039 ceph-mds[66450]: -- 172.21.15.39:0/2138596855 <== mon.2 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a213fbba40 con 0x55a213f8f000 2024-05-21T03:20:20.519 INFO:journalctl@ceph.mds.b.smithi039.stdout:May 21 03:20:20 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-b[66446]: starting mds.b at 2024-05-21T03:20:20.519 INFO:journalctl@ceph.mds.b.smithi039.stdout:May 21 03:20:20 smithi039 bash[66431]: 1f3a14b16cf9f44eb54e8dadb9525bcdba1315938f03d8bfabd9ed6bdcf81faf 2024-05-21T03:20:20.519 INFO:journalctl@ceph.mds.b.smithi039.stdout:May 21 03:20:20 smithi039 systemd[1]: Started Ceph mds.b for c9ca94be-171f-11ef-bc9a-c7b262605968. 2024-05-21T03:20:20.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:20 smithi027 ceph-mon[32179]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:20.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:20 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:20.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:20 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:20.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:20 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:20.615 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:20 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:20.616 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:20 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:20.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.854+0000 7f0050b4d700 1 -- 172.21.15.27:0/3684728025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f004c108510 msgr2=0x7f004c1088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:20.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.854+0000 7f0050b4d700 1 --2- 172.21.15.27:0/3684728025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f004c108510 0x7f004c1088f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0034009990 tx=0x7f0034009ca0 comp rx=0 tx=0).stop 2024-05-21T03:20:20.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.854+0000 7f0050b4d700 1 -- 172.21.15.27:0/3684728025 shutdown_connections 2024-05-21T03:20:20.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.854+0000 7f0050b4d700 1 --2- 172.21.15.27:0/3684728025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f004c102df0 0x7f004c10f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:20.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.854+0000 7f0050b4d700 1 --2- 172.21.15.27:0/3684728025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f004c108510 0x7f004c1088f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:20.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.854+0000 7f0050b4d700 1 --2- 172.21.15.27:0/3684728025 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f004c102430 0x7f004c1028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:20.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.854+0000 7f0050b4d700 1 -- 172.21.15.27:0/3684728025 >> 172.21.15.27:0/3684728025 conn(0x7f004c0793b0 msgr2=0x7f004c0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:20.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.854+0000 7f0050b4d700 1 -- 172.21.15.27:0/3684728025 shutdown_connections 2024-05-21T03:20:20.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.854+0000 7f0050b4d700 1 -- 172.21.15.27:0/3684728025 wait complete. 2024-05-21T03:20:20.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.855+0000 7f0050b4d700 1 Processor -- start 2024-05-21T03:20:20.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.856+0000 7f0050b4d700 1 -- start start 2024-05-21T03:20:20.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.856+0000 7f0050b4d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f004c102430 0x7f004c19f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:20.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.856+0000 7f0050b4d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f004c102df0 0x7f004c19fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:20.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.856+0000 7f0050b4d700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f004c108510 0x7f004c199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:20.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.856+0000 7f0050b4d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f004c1125c0 con 0x7f004c102df0 2024-05-21T03:20:20.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.856+0000 7f0050b4d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f004c112440 con 0x7f004c102430 2024-05-21T03:20:20.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.856+0000 7f0050b4d700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f004c112740 con 0x7f004c108510 2024-05-21T03:20:20.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.856+0000 7f0049d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f004c102df0 0x7f004c19fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:20.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.856+0000 7f0049d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f004c102df0 0x7f004c19fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57664/0 (socket says 172.21.15.27:57664) 2024-05-21T03:20:20.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.856+0000 7f0049d9b700 1 -- 172.21.15.27:0/2667080624 learned_addr learned my addr 172.21.15.27:0/2667080624 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:20:20.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.856+0000 7f004ad9d700 1 --2- 172.21.15.27:0/2667080624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f004c108510 0x7f004c199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:20.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.856+0000 7f004a59c700 1 --2- 172.21.15.27:0/2667080624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f004c102430 0x7f004c19f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:20.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.856+0000 7f0049d9b700 1 -- 172.21.15.27:0/2667080624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f004c108510 msgr2=0x7f004c199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:20.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.856+0000 7f0049d9b700 1 --2- 172.21.15.27:0/2667080624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f004c108510 0x7f004c199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:20.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.856+0000 7f0049d9b700 1 -- 172.21.15.27:0/2667080624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f004c102430 msgr2=0x7f004c19f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:20.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.856+0000 7f0049d9b700 1 --2- 172.21.15.27:0/2667080624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f004c102430 0x7f004c19f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:20.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.856+0000 7f0049d9b700 1 -- 172.21.15.27:0/2667080624 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f004c19a230 con 0x7f004c102df0 2024-05-21T03:20:20.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.856+0000 7f004ad9d700 1 --2- 172.21.15.27:0/2667080624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f004c108510 0x7f004c199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:20:20.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.857+0000 7f0049d9b700 1 --2- 172.21.15.27:0/2667080624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f004c102df0 0x7f004c19fe00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f003c00d980 tx=0x7f003c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:20.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.857+0000 7f00437fe700 1 -- 172.21.15.27:0/2667080624 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c016070 con 0x7f004c102df0 2024-05-21T03:20:20.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.857+0000 7f0050b4d700 1 -- 172.21.15.27:0/2667080624 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f004c19a520 con 0x7f004c102df0 2024-05-21T03:20:20.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.857+0000 7f00437fe700 1 -- 172.21.15.27:0/2667080624 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f003c004830 con 0x7f004c102df0 2024-05-21T03:20:20.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.857+0000 7f00437fe700 1 -- 172.21.15.27:0/2667080624 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c005540 con 0x7f004c102df0 2024-05-21T03:20:20.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.857+0000 7f0050b4d700 1 -- 172.21.15.27:0/2667080624 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f004c103fa0 con 0x7f004c102df0 2024-05-21T03:20:20.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.858+0000 7f0050b4d700 1 -- 172.21.15.27:0/2667080624 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f004c06a230 con 0x7f004c102df0 2024-05-21T03:20:20.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.859+0000 7f00437fe700 1 -- 172.21.15.27:0/2667080624 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f003c004220 con 0x7f004c102df0 2024-05-21T03:20:20.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.860+0000 7f00437fe700 1 --2- 172.21.15.27:0/2667080624 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f00380b2880 0x7f00380b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:20.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.861+0000 7f004a59c700 1 --2- 172.21.15.27:0/2667080624 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f00380b2880 0x7f00380b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:20.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.861+0000 7f00437fe700 1 -- 172.21.15.27:0/2667080624 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f003c0e2090 con 0x7f004c102df0 2024-05-21T03:20:20.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.861+0000 7f004a59c700 1 --2- 172.21.15.27:0/2667080624 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f00380b2880 0x7f00380b4d40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f004c100b00 tx=0x7f0034005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:20.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:20.864+0000 7f00437fe700 1 -- 172.21.15.27:0/2667080624 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f003c01f020 con 0x7f004c102df0 2024-05-21T03:20:21.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:20 smithi039 ceph-mon[36429]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:21.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:20 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:21.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:20 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:21.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:20 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:21.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:20 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:21.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:20 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:21.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:20 smithi165 ceph-mon[36269]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:21.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:20 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:21.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:20 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:21.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:20 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:21.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:20 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:21.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:20 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:21.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:21.127+0000 7f0050b4d700 1 -- 172.21.15.27:0/2667080624 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f004c19afb0 con 0x7f004c102df0 2024-05-21T03:20:21.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:21.127+0000 7f00437fe700 1 -- 172.21.15.27:0/2667080624 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10473 (secure 0 0 0) 0x7f003c0aa250 con 0x7f004c102df0 2024-05-21T03:20:21.128 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:20:21.128 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/148854869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":148854869},{"type":"v1","addr":"172.21.15.27:6835","nonce":148854869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/4145544730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":4145544730},{"type":"v1","addr":"172.21.15.27:6837","nonce":4145544730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/745486764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":745486764},{"type":"v1","addr":"172.21.15.27:6839","nonce":745486764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/3982752351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3982752351},{"type":"v1","addr":"172.21.15.27:6841","nonce":3982752351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3497521608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3497521608},{"type":"v1","addr":"172.21.15.39:6835","nonce":3497521608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3576910218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3576910218},{"type":"v1","addr":"172.21.15.165:6835","nonce":3576910218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3673846852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3673846852},{"type":"v1","addr":"172.21.15.39:6837","nonce":3673846852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/691356139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":691356139},{"type":"v1","addr":"172.21.15.165:6837","nonce":691356139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6839/4158170814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4158170814},{"type":"v1","addr":"172.21.15.165:6839","nonce":4158170814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/287607819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":287607819},{"type":"v1","addr":"172.21.15.39:6839","nonce":287607819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-21T03:20:21.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:21.131+0000 7f0050b4d700 1 -- 172.21.15.27:0/2667080624 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f00380b2880 msgr2=0x7f00380b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:21.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:21.131+0000 7f0050b4d700 1 --2- 172.21.15.27:0/2667080624 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f00380b2880 0x7f00380b4d40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f004c100b00 tx=0x7f0034005960 comp rx=0 tx=0).stop 2024-05-21T03:20:21.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:21.131+0000 7f0050b4d700 1 -- 172.21.15.27:0/2667080624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f004c102df0 msgr2=0x7f004c19fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:21.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:21.131+0000 7f0050b4d700 1 --2- 172.21.15.27:0/2667080624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f004c102df0 0x7f004c19fe00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f003c00d980 tx=0x7f003c00dc90 comp rx=0 tx=0).stop 2024-05-21T03:20:21.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:21.131+0000 7f0050b4d700 1 -- 172.21.15.27:0/2667080624 shutdown_connections 2024-05-21T03:20:21.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:21.131+0000 7f0050b4d700 1 --2- 172.21.15.27:0/2667080624 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f00380b2880 0x7f00380b4d40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:21.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:21.131+0000 7f0050b4d700 1 --2- 172.21.15.27:0/2667080624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f004c108510 0x7f004c199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:21.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:21.131+0000 7f0050b4d700 1 --2- 172.21.15.27:0/2667080624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f004c102430 0x7f004c19f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:21.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:21.131+0000 7f0050b4d700 1 --2- 172.21.15.27:0/2667080624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f004c102df0 0x7f004c19fe00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:21.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:21.131+0000 7f0050b4d700 1 -- 172.21.15.27:0/2667080624 >> 172.21.15.27:0/2667080624 conn(0x7f004c0793b0 msgr2=0x7f004c06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:21.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:21.131+0000 7f0050b4d700 1 -- 172.21.15.27:0/2667080624 shutdown_connections 2024-05-21T03:20:21.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:21.131+0000 7f0050b4d700 1 -- 172.21.15.27:0/2667080624 wait complete. 2024-05-21T03:20:21.133 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 13 2024-05-21T03:20:21.682 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:21 smithi165 ceph-mon[36269]: mds.? [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] up:boot 2024-05-21T03:20:21.682 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:21 smithi165 ceph-mon[36269]: fsmap 12 up:standby 2024-05-21T03:20:21.682 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:21 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T03:20:21.682 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:21 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:20:21.683 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:21 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/2667080624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:20:21.694 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:21 smithi039 ceph-mon[36429]: mds.? [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] up:boot 2024-05-21T03:20:21.694 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:21 smithi039 ceph-mon[36429]: fsmap 12 up:standby 2024-05-21T03:20:21.694 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:21 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T03:20:21.694 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:21 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:20:21.694 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:21 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/2667080624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:20:21.724 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:21 smithi027 ceph-mon[32179]: mds.? [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] up:boot 2024-05-21T03:20:21.724 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:21 smithi027 ceph-mon[32179]: fsmap 12 up:standby 2024-05-21T03:20:21.724 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:21 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T03:20:21.724 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:21 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:20:21.724 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:21 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2667080624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:20:21.753 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-21T03:20:22.667 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:20:22.897 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:22 smithi027 ceph-mon[32179]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:22.897 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:22 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:22.897 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:22 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:22.897 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:22 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:22.898 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:22 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:22.966 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:22 smithi039 ceph-mon[36429]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:22.966 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:22 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:22.966 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:22 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:22.967 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:22 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:22.967 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:22 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:23.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:22 smithi165 ceph-mon[36269]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:23.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:22 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:23.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:22 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:23.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:22 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:23.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:22 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:24.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.226+0000 7fe79f52d700 1 -- 172.21.15.27:0/1319892966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe79810ff90 msgr2=0x7fe798112380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:24.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.226+0000 7fe79f52d700 1 --2- 172.21.15.27:0/1319892966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe79810ff90 0x7fe798112380 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe79000b0a0 tx=0x7fe79000b3b0 comp rx=0 tx=0).stop 2024-05-21T03:20:24.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.226+0000 7fe79f52d700 1 -- 172.21.15.27:0/1319892966 shutdown_connections 2024-05-21T03:20:24.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.226+0000 7fe79f52d700 1 --2- 172.21.15.27:0/1319892966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe79810ff90 0x7fe798112380 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:24.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.226+0000 7fe79f52d700 1 --2- 172.21.15.27:0/1319892966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe79806bcf0 0x7fe79806c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:24.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.226+0000 7fe79f52d700 1 --2- 172.21.15.27:0/1319892966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe79806c610 0x7fe79810fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:24.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.226+0000 7fe79f52d700 1 -- 172.21.15.27:0/1319892966 >> 172.21.15.27:0/1319892966 conn(0x7fe7980794e0 msgr2=0x7fe7980798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:24.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.226+0000 7fe79f52d700 1 -- 172.21.15.27:0/1319892966 shutdown_connections 2024-05-21T03:20:24.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.226+0000 7fe79f52d700 1 -- 172.21.15.27:0/1319892966 wait complete. 2024-05-21T03:20:24.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.227+0000 7fe79f52d700 1 Processor -- start 2024-05-21T03:20:24.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.227+0000 7fe79f52d700 1 -- start start 2024-05-21T03:20:24.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.227+0000 7fe79f52d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe79806bcf0 0x7fe79819be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:24.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.227+0000 7fe79f52d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe79806c610 0x7fe79819c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:24.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.227+0000 7fe79f52d700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe79810ff90 0x7fe7981a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:24.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.227+0000 7fe79f52d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe7981159a0 con 0x7fe79806bcf0 2024-05-21T03:20:24.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.227+0000 7fe79f52d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe798115820 con 0x7fe79806c610 2024-05-21T03:20:24.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.227+0000 7fe79f52d700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe798115b20 con 0x7fe79810ff90 2024-05-21T03:20:24.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.227+0000 7fe79d2c9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe79806bcf0 0x7fe79819be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:24.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.227+0000 7fe79d2c9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe79806bcf0 0x7fe79819be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56012/0 (socket says 172.21.15.27:56012) 2024-05-21T03:20:24.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.227+0000 7fe79d2c9700 1 -- 172.21.15.27:0/604161923 learned_addr learned my addr 172.21.15.27:0/604161923 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:20:24.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.227+0000 7fe79cac8700 1 --2- 172.21.15.27:0/604161923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe79806c610 0x7fe79819c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:24.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.227+0000 7fe79daca700 1 --2- 172.21.15.27:0/604161923 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe79810ff90 0x7fe7981a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:24.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.228+0000 7fe79d2c9700 1 -- 172.21.15.27:0/604161923 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe79810ff90 msgr2=0x7fe7981a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:24.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.228+0000 7fe79d2c9700 1 --2- 172.21.15.27:0/604161923 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe79810ff90 0x7fe7981a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:24.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.228+0000 7fe79d2c9700 1 -- 172.21.15.27:0/604161923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe79806c610 msgr2=0x7fe79819c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:24.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.228+0000 7fe79d2c9700 1 --2- 172.21.15.27:0/604161923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe79806c610 0x7fe79819c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:24.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.228+0000 7fe79d2c9700 1 -- 172.21.15.27:0/604161923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7981a0e40 con 0x7fe79806bcf0 2024-05-21T03:20:24.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.228+0000 7fe79cac8700 1 --2- 172.21.15.27:0/604161923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe79806c610 0x7fe79819c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:20:24.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.228+0000 7fe79d2c9700 1 --2- 172.21.15.27:0/604161923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe79806bcf0 0x7fe79819be30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe79400d950 tx=0x7fe79400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:24.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.228+0000 7fe78e7fc700 1 -- 172.21.15.27:0/604161923 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794016070 con 0x7fe79806bcf0 2024-05-21T03:20:24.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.228+0000 7fe79f52d700 1 -- 172.21.15.27:0/604161923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7981a1130 con 0x7fe79806bcf0 2024-05-21T03:20:24.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.228+0000 7fe78e7fc700 1 -- 172.21.15.27:0/604161923 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe79400fd40 con 0x7fe79806bcf0 2024-05-21T03:20:24.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.228+0000 7fe78e7fc700 1 -- 172.21.15.27:0/604161923 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79400e3b0 con 0x7fe79806bcf0 2024-05-21T03:20:24.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.228+0000 7fe79f52d700 1 -- 172.21.15.27:0/604161923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7981a89b0 con 0x7fe79806bcf0 2024-05-21T03:20:24.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.229+0000 7fe79f52d700 1 -- 172.21.15.27:0/604161923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe79806a230 con 0x7fe79806bcf0 2024-05-21T03:20:24.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.230+0000 7fe78e7fc700 1 -- 172.21.15.27:0/604161923 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe794019070 con 0x7fe79806bcf0 2024-05-21T03:20:24.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.231+0000 7fe78e7fc700 1 --2- 172.21.15.27:0/604161923 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe7840b2880 0x7fe7840b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:24.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.232+0000 7fe79cac8700 1 --2- 172.21.15.27:0/604161923 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe7840b2880 0x7fe7840b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:24.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.232+0000 7fe78e7fc700 1 -- 172.21.15.27:0/604161923 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fe7940e1860 con 0x7fe79806bcf0 2024-05-21T03:20:24.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.232+0000 7fe79cac8700 1 --2- 172.21.15.27:0/604161923 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe7840b2880 0x7fe7840b4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe798101eb0 tx=0x7fe788009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:24.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.236+0000 7fe78e7fc700 1 -- 172.21.15.27:0/604161923 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe7940a9a20 con 0x7fe79806bcf0 2024-05-21T03:20:24.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.504+0000 7fe79f52d700 1 -- 172.21.15.27:0/604161923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe79819d190 con 0x7fe79806bcf0 2024-05-21T03:20:24.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.505+0000 7fe78e7fc700 1 -- 172.21.15.27:0/604161923 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v73) ==== 84+0+65 (secure 0 0 0) 0x7fe7940a9320 con 0x7fe79806bcf0 2024-05-21T03:20:24.506 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:20:24.506 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-05-21T03:20:24.506 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:20:24.506 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2024-05-21T03:20:24.506 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2024-05-21T03:20:24.506 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:20:24.506 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-05-21T03:20:24.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.509+0000 7fe79f52d700 1 -- 172.21.15.27:0/604161923 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe7840b2880 msgr2=0x7fe7840b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:24.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.509+0000 7fe79f52d700 1 --2- 172.21.15.27:0/604161923 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe7840b2880 0x7fe7840b4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe798101eb0 tx=0x7fe788009190 comp rx=0 tx=0).stop 2024-05-21T03:20:24.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.509+0000 7fe79f52d700 1 -- 172.21.15.27:0/604161923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe79806bcf0 msgr2=0x7fe79819be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:24.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.509+0000 7fe79f52d700 1 --2- 172.21.15.27:0/604161923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe79806bcf0 0x7fe79819be30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe79400d950 tx=0x7fe79400dc60 comp rx=0 tx=0).stop 2024-05-21T03:20:24.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.509+0000 7fe79f52d700 1 -- 172.21.15.27:0/604161923 shutdown_connections 2024-05-21T03:20:24.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.509+0000 7fe79f52d700 1 --2- 172.21.15.27:0/604161923 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe7840b2880 0x7fe7840b4d40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:24.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.509+0000 7fe79f52d700 1 --2- 172.21.15.27:0/604161923 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe79810ff90 0x7fe7981a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:24.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.509+0000 7fe79f52d700 1 --2- 172.21.15.27:0/604161923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe79806bcf0 0x7fe79819be30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:24.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.509+0000 7fe79f52d700 1 --2- 172.21.15.27:0/604161923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe79806c610 0x7fe79819c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:24.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.509+0000 7fe79f52d700 1 -- 172.21.15.27:0/604161923 >> 172.21.15.27:0/604161923 conn(0x7fe7980794e0 msgr2=0x7fe7981114d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:24.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.509+0000 7fe79f52d700 1 -- 172.21.15.27:0/604161923 shutdown_connections 2024-05-21T03:20:24.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:24.509+0000 7fe79f52d700 1 -- 172.21.15.27:0/604161923 wait complete. 2024-05-21T03:20:24.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:24 smithi027 ceph-mon[32179]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:24.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:24 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:24.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:24 smithi039 ceph-mon[36429]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:24.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:24 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:24.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:24 smithi165 ceph-mon[36269]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:24.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:24 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:25.356 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-21T03:20:25.356 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-21T03:20:25.612 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:20:25.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:25 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:25.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:25 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:25.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:25 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/604161923' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-21T03:20:25.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:25 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:20:25.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:25 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:20:25.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:25 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:25.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:25 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:20:25.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:25 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:25.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:25 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:25.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:25 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/604161923' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-21T03:20:25.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:25 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:20:25.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:25 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:20:25.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:25 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:25.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:25 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:20:25.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:25 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:25.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:25 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:25.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:25 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/604161923' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-21T03:20:25.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:25 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:20:25.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:25 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:20:25.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:25 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:25.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:25 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:20:26.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:26 smithi027 ceph-mon[32179]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:26.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:26 smithi039 ceph-mon[36429]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:26.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:26 smithi165 ceph-mon[36269]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:27.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.269+0000 7fc3c01c4700 1 -- 172.21.15.27:0/1638007757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3b8109680 msgr2=0x7fc3b8109a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:27.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.269+0000 7fc3c01c4700 1 --2- 172.21.15.27:0/1638007757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3b8109680 0x7fc3b8109a60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc3b40099c0 tx=0x7fc3b4009cd0 comp rx=0 tx=0).stop 2024-05-21T03:20:27.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.269+0000 7fc3c01c4700 1 -- 172.21.15.27:0/1638007757 shutdown_connections 2024-05-21T03:20:27.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.269+0000 7fc3c01c4700 1 --2- 172.21.15.27:0/1638007757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3b8103fa0 0x7fc3b8110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:27.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.269+0000 7fc3c01c4700 1 --2- 172.21.15.27:0/1638007757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3b8109680 0x7fc3b8109a60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:27.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.269+0000 7fc3c01c4700 1 --2- 172.21.15.27:0/1638007757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3b81035e0 0x7fc3b8103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:27.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.269+0000 7fc3c01c4700 1 -- 172.21.15.27:0/1638007757 >> 172.21.15.27:0/1638007757 conn(0x7fc3b807a6f0 msgr2=0x7fc3b807ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:27.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.269+0000 7fc3c01c4700 1 -- 172.21.15.27:0/1638007757 shutdown_connections 2024-05-21T03:20:27.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.269+0000 7fc3c01c4700 1 -- 172.21.15.27:0/1638007757 wait complete. 2024-05-21T03:20:27.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.270+0000 7fc3c01c4700 1 Processor -- start 2024-05-21T03:20:27.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.270+0000 7fc3c01c4700 1 -- start start 2024-05-21T03:20:27.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.270+0000 7fc3c01c4700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3b81035e0 0x7fc3b819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:27.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.270+0000 7fc3c01c4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3b8103fa0 0x7fc3b819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:27.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.270+0000 7fc3c01c4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3b8109680 0x7fc3b81999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:27.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.270+0000 7fc3c01c4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b81137d0 con 0x7fc3b8109680 2024-05-21T03:20:27.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.270+0000 7fc3c01c4700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b8113650 con 0x7fc3b8103fa0 2024-05-21T03:20:27.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.270+0000 7fc3c01c4700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b8113950 con 0x7fc3b81035e0 2024-05-21T03:20:27.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.270+0000 7fc3be761700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3b8109680 0x7fc3b81999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:27.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.270+0000 7fc3be761700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3b8109680 0x7fc3b81999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56032/0 (socket says 172.21.15.27:56032) 2024-05-21T03:20:27.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.270+0000 7fc3be761700 1 -- 172.21.15.27:0/3691809424 learned_addr learned my addr 172.21.15.27:0/3691809424 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:20:27.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.271+0000 7fc3bdf60700 1 --2- 172.21.15.27:0/3691809424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3b81035e0 0x7fc3b819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:27.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.271+0000 7fc3bd75f700 1 --2- 172.21.15.27:0/3691809424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3b8103fa0 0x7fc3b819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:27.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.271+0000 7fc3be761700 1 -- 172.21.15.27:0/3691809424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3b81035e0 msgr2=0x7fc3b819f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:27.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.271+0000 7fc3be761700 1 --2- 172.21.15.27:0/3691809424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3b81035e0 0x7fc3b819f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:27.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.271+0000 7fc3be761700 1 -- 172.21.15.27:0/3691809424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3b8103fa0 msgr2=0x7fc3b819fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:27.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.271+0000 7fc3be761700 1 --2- 172.21.15.27:0/3691809424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3b8103fa0 0x7fc3b819fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:27.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.271+0000 7fc3be761700 1 -- 172.21.15.27:0/3691809424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b819a110 con 0x7fc3b8109680 2024-05-21T03:20:27.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.271+0000 7fc3be761700 1 --2- 172.21.15.27:0/3691809424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3b8109680 0x7fc3b81999b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0007d70 tx=0x7fc3b000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:27.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.271+0000 7fc3aeffd700 1 -- 172.21.15.27:0/3691809424 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b0016020 con 0x7fc3b8109680 2024-05-21T03:20:27.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.271+0000 7fc3aeffd700 1 -- 172.21.15.27:0/3691809424 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3b000ad40 con 0x7fc3b8109680 2024-05-21T03:20:27.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.271+0000 7fc3c01c4700 1 -- 172.21.15.27:0/3691809424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b819a400 con 0x7fc3b8109680 2024-05-21T03:20:27.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.271+0000 7fc3aeffd700 1 -- 172.21.15.27:0/3691809424 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b00115e0 con 0x7fc3b8109680 2024-05-21T03:20:27.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.271+0000 7fc3c01c4700 1 -- 172.21.15.27:0/3691809424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3b81a66c0 con 0x7fc3b8109680 2024-05-21T03:20:27.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.272+0000 7fc3c01c4700 1 -- 172.21.15.27:0/3691809424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3b810bb10 con 0x7fc3b8109680 2024-05-21T03:20:27.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.274+0000 7fc3aeffd700 1 -- 172.21.15.27:0/3691809424 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc3b0019020 con 0x7fc3b8109680 2024-05-21T03:20:27.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.274+0000 7fc3aeffd700 1 --2- 172.21.15.27:0/3691809424 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fc3a40b2900 0x7fc3a40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:27.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.274+0000 7fc3bdf60700 1 --2- 172.21.15.27:0/3691809424 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fc3a40b2900 0x7fc3a40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:27.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.274+0000 7fc3aeffd700 1 -- 172.21.15.27:0/3691809424 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fc3b00e2100 con 0x7fc3b8109680 2024-05-21T03:20:27.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.275+0000 7fc3bdf60700 1 --2- 172.21.15.27:0/3691809424 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fc3a40b2900 0x7fc3a40b4dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4000c00 tx=0x7fc3b401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:27.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.278+0000 7fc3aeffd700 1 -- 172.21.15.27:0/3691809424 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc3b0020020 con 0x7fc3b8109680 2024-05-21T03:20:27.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:27.501+0000 7fc3c01c4700 1 -- 172.21.15.27:0/3691809424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc3b819aee0 con 0x7fc3b8109680 2024-05-21T03:20:28.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:28.082+0000 7fc3aeffd700 1 -- 172.21.15.27:0/3691809424 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v74) ==== 138+0+0 (secure 0 0 0) 0x7fc3b00aa2c0 con 0x7fc3b8109680 2024-05-21T03:20:28.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:28.086+0000 7fc3c01c4700 1 -- 172.21.15.27:0/3691809424 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fc3a40b2900 msgr2=0x7fc3a40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:28.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:28.086+0000 7fc3c01c4700 1 --2- 172.21.15.27:0/3691809424 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fc3a40b2900 0x7fc3a40b4dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4000c00 tx=0x7fc3b401a040 comp rx=0 tx=0).stop 2024-05-21T03:20:28.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:28.086+0000 7fc3c01c4700 1 -- 172.21.15.27:0/3691809424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3b8109680 msgr2=0x7fc3b81999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:28.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:28.086+0000 7fc3c01c4700 1 --2- 172.21.15.27:0/3691809424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3b8109680 0x7fc3b81999b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0007d70 tx=0x7fc3b000a340 comp rx=0 tx=0).stop 2024-05-21T03:20:28.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:28.086+0000 7fc3c01c4700 1 -- 172.21.15.27:0/3691809424 shutdown_connections 2024-05-21T03:20:28.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:28.086+0000 7fc3c01c4700 1 --2- 172.21.15.27:0/3691809424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3b8109680 0x7fc3b81999b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:28.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:28.086+0000 7fc3c01c4700 1 --2- 172.21.15.27:0/3691809424 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fc3a40b2900 0x7fc3a40b4dc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:28.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:28.086+0000 7fc3c01c4700 1 --2- 172.21.15.27:0/3691809424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3b81035e0 0x7fc3b819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:28.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:28.086+0000 7fc3c01c4700 1 --2- 172.21.15.27:0/3691809424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3b8103fa0 0x7fc3b819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:28.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:28.086+0000 7fc3c01c4700 1 -- 172.21.15.27:0/3691809424 >> 172.21.15.27:0/3691809424 conn(0x7fc3b807a6f0 msgr2=0x7fc3b8102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:28.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:28.086+0000 7fc3c01c4700 1 -- 172.21.15.27:0/3691809424 shutdown_connections 2024-05-21T03:20:28.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:28.086+0000 7fc3c01c4700 1 -- 172.21.15.27:0/3691809424 wait complete. 2024-05-21T03:20:28.087 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2024-05-21T03:20:28.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:27 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:28.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:27 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:28.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:27 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:28.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:27 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:28.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:27 smithi039 ceph-mon[36429]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:28.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:27 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/3691809424' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-21T03:20:28.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:27 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:28.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:27 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:28.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:27 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:28.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:27 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:28.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:27 smithi165 ceph-mon[36269]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:28.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:27 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/3691809424' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-21T03:20:28.411 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:27 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:28.411 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:27 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:28.411 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:27 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:28.411 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:27 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:28.411 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:27 smithi027 ceph-mon[32179]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:28.411 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:27 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3691809424' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-21T03:20:28.665 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-21T03:20:28.928 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:20:29.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:29 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/3691809424' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-21T03:20:29.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:29 smithi039 ceph-mon[36429]: osdmap e74: 12 total, 12 up, 12 in 2024-05-21T03:20:29.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:29 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:29.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:29 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:29.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:29 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:20:29.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:29 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:20:29.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:29 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:29.389 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:29 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3691809424' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-21T03:20:29.389 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:29 smithi027 ceph-mon[32179]: osdmap e74: 12 total, 12 up, 12 in 2024-05-21T03:20:29.389 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:29 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:29.389 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:29 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:29.389 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:29 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:20:29.389 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:29 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:20:29.389 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:29 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:29.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:29 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/3691809424' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-21T03:20:29.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:29 smithi165 ceph-mon[36269]: osdmap e74: 12 total, 12 up, 12 in 2024-05-21T03:20:29.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:29 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:29.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:29 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:29.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:29 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:20:29.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:29 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:20:29.481 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:29 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:20:30.334 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:30 smithi027 ceph-mon[32179]: osdmap e75: 12 total, 12 up, 12 in 2024-05-21T03:20:30.335 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:30 smithi027 ceph-mon[32179]: pgmap v283: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:30.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:30 smithi039 ceph-mon[36429]: osdmap e75: 12 total, 12 up, 12 in 2024-05-21T03:20:30.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:30 smithi039 ceph-mon[36429]: pgmap v283: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:30.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:30 smithi165 ceph-mon[36269]: osdmap e75: 12 total, 12 up, 12 in 2024-05-21T03:20:30.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:30 smithi165 ceph-mon[36269]: pgmap v283: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:30.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.601+0000 7fb2b0181700 1 -- 172.21.15.27:0/1709109191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2a8108520 msgr2=0x7fb2a8108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:30.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.601+0000 7fb2b0181700 1 --2- 172.21.15.27:0/1709109191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2a8108520 0x7fb2a8108900 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb29800b0a0 tx=0x7fb29800b3b0 comp rx=0 tx=0).stop 2024-05-21T03:20:30.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.601+0000 7fb2b0181700 1 -- 172.21.15.27:0/1709109191 shutdown_connections 2024-05-21T03:20:30.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.601+0000 7fb2b0181700 1 --2- 172.21.15.27:0/1709109191 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2a8102e00 0x7fb2a810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:30.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.601+0000 7fb2b0181700 1 --2- 172.21.15.27:0/1709109191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2a8108520 0x7fb2a8108900 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:30.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.601+0000 7fb2b0181700 1 --2- 172.21.15.27:0/1709109191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2a8102440 0x7fb2a81028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:30.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.601+0000 7fb2b0181700 1 -- 172.21.15.27:0/1709109191 >> 172.21.15.27:0/1709109191 conn(0x7fb2a80793f0 msgr2=0x7fb2a8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:30.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.601+0000 7fb2b0181700 1 -- 172.21.15.27:0/1709109191 shutdown_connections 2024-05-21T03:20:30.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.601+0000 7fb2b0181700 1 -- 172.21.15.27:0/1709109191 wait complete. 2024-05-21T03:20:30.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.602+0000 7fb2b0181700 1 Processor -- start 2024-05-21T03:20:30.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.602+0000 7fb2b0181700 1 -- start start 2024-05-21T03:20:30.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.602+0000 7fb2b0181700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2a8102440 0x7fb2a819f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:30.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.602+0000 7fb2b0181700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2a8102e00 0x7fb2a819fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:30.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.602+0000 7fb2b0181700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2a8108520 0x7fb2a8199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:30.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.602+0000 7fb2b0181700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a8112640 con 0x7fb2a8102440 2024-05-21T03:20:30.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.602+0000 7fb2b0181700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a81124c0 con 0x7fb2a8102e00 2024-05-21T03:20:30.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.602+0000 7fb2b0181700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a81127c0 con 0x7fb2a8108520 2024-05-21T03:20:30.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.603+0000 7fb2adf1d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2a8102440 0x7fb2a819f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:30.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.603+0000 7fb2adf1d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2a8102440 0x7fb2a819f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56056/0 (socket says 172.21.15.27:56056) 2024-05-21T03:20:30.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.603+0000 7fb2adf1d700 1 -- 172.21.15.27:0/759292394 learned_addr learned my addr 172.21.15.27:0/759292394 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:20:30.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.603+0000 7fb2ae71e700 1 --2- 172.21.15.27:0/759292394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2a8108520 0x7fb2a8199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:30.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.603+0000 7fb2ad71c700 1 --2- 172.21.15.27:0/759292394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2a8102e00 0x7fb2a819fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:30.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.603+0000 7fb2adf1d700 1 -- 172.21.15.27:0/759292394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2a8108520 msgr2=0x7fb2a8199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:30.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.603+0000 7fb2adf1d700 1 --2- 172.21.15.27:0/759292394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2a8108520 0x7fb2a8199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:30.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.603+0000 7fb2adf1d700 1 -- 172.21.15.27:0/759292394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2a8102e00 msgr2=0x7fb2a819fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:30.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.603+0000 7fb2adf1d700 1 --2- 172.21.15.27:0/759292394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2a8102e00 0x7fb2a819fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:30.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.603+0000 7fb2adf1d700 1 -- 172.21.15.27:0/759292394 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a819a1b0 con 0x7fb2a8102440 2024-05-21T03:20:30.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.603+0000 7fb2adf1d700 1 --2- 172.21.15.27:0/759292394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2a8102440 0x7fb2a819f890 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb2980062a0 tx=0x7fb298004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:30.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.603+0000 7fb2a6ffd700 1 -- 172.21.15.27:0/759292394 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb298021070 con 0x7fb2a8102440 2024-05-21T03:20:30.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.603+0000 7fb2a6ffd700 1 -- 172.21.15.27:0/759292394 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb298007c40 con 0x7fb2a8102440 2024-05-21T03:20:30.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.603+0000 7fb2b0181700 1 -- 172.21.15.27:0/759292394 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a819a440 con 0x7fb2a8102440 2024-05-21T03:20:30.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.603+0000 7fb2a6ffd700 1 -- 172.21.15.27:0/759292394 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb298004660 con 0x7fb2a8102440 2024-05-21T03:20:30.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.603+0000 7fb2b0181700 1 -- 172.21.15.27:0/759292394 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a81a6760 con 0x7fb2a8102440 2024-05-21T03:20:30.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.604+0000 7fb2b0181700 1 -- 172.21.15.27:0/759292394 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2a806a230 con 0x7fb2a8102440 2024-05-21T03:20:30.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.606+0000 7fb2a6ffd700 1 -- 172.21.15.27:0/759292394 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb298003a00 con 0x7fb2a8102440 2024-05-21T03:20:30.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.607+0000 7fb2a6ffd700 1 --2- 172.21.15.27:0/759292394 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb2900b2880 0x7fb2900b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:30.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.607+0000 7fb2ad71c700 1 --2- 172.21.15.27:0/759292394 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb2900b2880 0x7fb2900b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:30.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.607+0000 7fb2a6ffd700 1 -- 172.21.15.27:0/759292394 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7497+0+0 (secure 0 0 0) 0x7fb29800f020 con 0x7fb2a8102440 2024-05-21T03:20:30.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.608+0000 7fb2ad71c700 1 --2- 172.21.15.27:0/759292394 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb2900b2880 0x7fb2900b4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8079980 tx=0x7fb29c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:30.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.611+0000 7fb2a6ffd700 1 -- 172.21.15.27:0/759292394 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb29802a020 con 0x7fb2a8102440 2024-05-21T03:20:30.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:30.825+0000 7fb2b0181700 1 -- 172.21.15.27:0/759292394 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb2a81a6960 con 0x7fb2a8102440 2024-05-21T03:20:31.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:31.105+0000 7fb2a6ffd700 1 -- 172.21.15.27:0/759292394 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v77) ==== 170+0+0 (secure 0 0 0) 0x7fb2980ab8c0 con 0x7fb2a8102440 2024-05-21T03:20:31.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:31.108+0000 7fb2b0181700 1 -- 172.21.15.27:0/759292394 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb2900b2880 msgr2=0x7fb2900b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:31.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:31.108+0000 7fb2b0181700 1 --2- 172.21.15.27:0/759292394 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb2900b2880 0x7fb2900b4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8079980 tx=0x7fb29c0073d0 comp rx=0 tx=0).stop 2024-05-21T03:20:31.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:31.108+0000 7fb2b0181700 1 -- 172.21.15.27:0/759292394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2a8102440 msgr2=0x7fb2a819f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:31.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:31.108+0000 7fb2b0181700 1 --2- 172.21.15.27:0/759292394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2a8102440 0x7fb2a819f890 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb2980062a0 tx=0x7fb298004e60 comp rx=0 tx=0).stop 2024-05-21T03:20:31.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:31.108+0000 7fb2b0181700 1 -- 172.21.15.27:0/759292394 shutdown_connections 2024-05-21T03:20:31.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:31.108+0000 7fb2b0181700 1 --2- 172.21.15.27:0/759292394 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb2900b2880 0x7fb2900b4d40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:31.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:31.108+0000 7fb2b0181700 1 --2- 172.21.15.27:0/759292394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2a8108520 0x7fb2a8199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:31.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:31.108+0000 7fb2b0181700 1 --2- 172.21.15.27:0/759292394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2a8102440 0x7fb2a819f890 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:31.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:31.108+0000 7fb2b0181700 1 --2- 172.21.15.27:0/759292394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2a8102e00 0x7fb2a819fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:31.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:31.108+0000 7fb2b0181700 1 -- 172.21.15.27:0/759292394 >> 172.21.15.27:0/759292394 conn(0x7fb2a80793f0 msgr2=0x7fb2a806b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:31.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:31.109+0000 7fb2b0181700 1 -- 172.21.15.27:0/759292394 shutdown_connections 2024-05-21T03:20:31.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:31.109+0000 7fb2b0181700 1 -- 172.21.15.27:0/759292394 wait complete. 2024-05-21T03:20:31.110 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2024-05-21T03:20:31.416 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:31 smithi027 ceph-mon[32179]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-21T03:20:31.417 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:31 smithi027 ceph-mon[32179]: osdmap e76: 12 total, 12 up, 12 in 2024-05-21T03:20:31.417 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:31 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/759292394' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-21T03:20:31.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:31 smithi039 ceph-mon[36429]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-21T03:20:31.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:31 smithi039 ceph-mon[36429]: osdmap e76: 12 total, 12 up, 12 in 2024-05-21T03:20:31.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:31 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/759292394' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-21T03:20:31.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:31 smithi165 ceph-mon[36269]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-21T03:20:31.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:31 smithi165 ceph-mon[36269]: osdmap e76: 12 total, 12 up, 12 in 2024-05-21T03:20:31.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:31 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/759292394' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-21T03:20:31.928 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-21T03:20:32.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:32 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/759292394' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-21T03:20:32.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:32 smithi027 ceph-mon[32179]: osdmap e77: 12 total, 12 up, 12 in 2024-05-21T03:20:32.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:32 smithi027 ceph-mon[32179]: pgmap v286: 97 pgs: 12 creating+peering, 16 active+clean, 69 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:32.201 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:20:32.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:32 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/759292394' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-21T03:20:32.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:32 smithi039 ceph-mon[36429]: osdmap e77: 12 total, 12 up, 12 in 2024-05-21T03:20:32.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:32 smithi039 ceph-mon[36429]: pgmap v286: 97 pgs: 12 creating+peering, 16 active+clean, 69 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:32.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:32 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/759292394' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-21T03:20:32.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:32 smithi165 ceph-mon[36269]: osdmap e77: 12 total, 12 up, 12 in 2024-05-21T03:20:32.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:32 smithi165 ceph-mon[36269]: pgmap v286: 97 pgs: 12 creating+peering, 16 active+clean, 69 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:33.248 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:33 smithi027 ceph-mon[32179]: osdmap e78: 12 total, 12 up, 12 in 2024-05-21T03:20:33.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:33 smithi039 ceph-mon[36429]: osdmap e78: 12 total, 12 up, 12 in 2024-05-21T03:20:33.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:33 smithi165 ceph-mon[36269]: osdmap e78: 12 total, 12 up, 12 in 2024-05-21T03:20:33.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.784+0000 7f7bdb0a8700 1 -- 172.21.15.27:0/1978436403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bd4102440 msgr2=0x7f7bd41028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:33.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.784+0000 7f7bdb0a8700 1 --2- 172.21.15.27:0/1978436403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bd4102440 0x7f7bd41028c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0009990 tx=0x7f7bc0009ca0 comp rx=0 tx=0).stop 2024-05-21T03:20:33.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.784+0000 7f7bdb0a8700 1 -- 172.21.15.27:0/1978436403 shutdown_connections 2024-05-21T03:20:33.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.784+0000 7f7bdb0a8700 1 --2- 172.21.15.27:0/1978436403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bd4102e00 0x7f7bd410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:33.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.784+0000 7f7bdb0a8700 1 --2- 172.21.15.27:0/1978436403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bd4108520 0x7f7bd4108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:33.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.784+0000 7f7bdb0a8700 1 --2- 172.21.15.27:0/1978436403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bd4102440 0x7f7bd41028c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:33.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.784+0000 7f7bdb0a8700 1 -- 172.21.15.27:0/1978436403 >> 172.21.15.27:0/1978436403 conn(0x7f7bd40793f0 msgr2=0x7f7bd4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:33.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.784+0000 7f7bdb0a8700 1 -- 172.21.15.27:0/1978436403 shutdown_connections 2024-05-21T03:20:33.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.784+0000 7f7bdb0a8700 1 -- 172.21.15.27:0/1978436403 wait complete. 2024-05-21T03:20:33.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.784+0000 7f7bdb0a8700 1 Processor -- start 2024-05-21T03:20:33.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.785+0000 7f7bdb0a8700 1 -- start start 2024-05-21T03:20:33.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.785+0000 7f7bdb0a8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bd4102440 0x7f7bd419f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:33.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.785+0000 7f7bdb0a8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bd4102e00 0x7f7bd419fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:33.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.785+0000 7f7bdb0a8700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bd4108520 0x7f7bd4199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:33.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.785+0000 7f7bdb0a8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd4112620 con 0x7f7bd4102440 2024-05-21T03:20:33.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.785+0000 7f7bdb0a8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd41124a0 con 0x7f7bd4102e00 2024-05-21T03:20:33.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.785+0000 7f7bdb0a8700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd41127a0 con 0x7f7bd4108520 2024-05-21T03:20:33.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.785+0000 7f7bd8e44700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bd4102440 0x7f7bd419f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:33.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.785+0000 7f7bd8e44700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bd4102440 0x7f7bd419f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49094/0 (socket says 172.21.15.27:49094) 2024-05-21T03:20:33.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.785+0000 7f7bd8e44700 1 -- 172.21.15.27:0/3000287571 learned_addr learned my addr 172.21.15.27:0/3000287571 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:20:33.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.785+0000 7f7bd3fff700 1 --2- 172.21.15.27:0/3000287571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bd4102e00 0x7f7bd419fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:33.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.785+0000 7f7bd9645700 1 --2- 172.21.15.27:0/3000287571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bd4108520 0x7f7bd4199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:33.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.785+0000 7f7bd8e44700 1 -- 172.21.15.27:0/3000287571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bd4108520 msgr2=0x7f7bd4199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:33.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.785+0000 7f7bd8e44700 1 --2- 172.21.15.27:0/3000287571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bd4108520 0x7f7bd4199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:33.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.785+0000 7f7bd8e44700 1 -- 172.21.15.27:0/3000287571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bd4102e00 msgr2=0x7f7bd419fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:33.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.785+0000 7f7bd8e44700 1 --2- 172.21.15.27:0/3000287571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bd4102e00 0x7f7bd419fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:33.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.785+0000 7f7bd8e44700 1 -- 172.21.15.27:0/3000287571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd419a1e0 con 0x7f7bd4102440 2024-05-21T03:20:33.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.785+0000 7f7bd8e44700 1 --2- 172.21.15.27:0/3000287571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bd4102440 0x7f7bd419f8c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800d950 tx=0x7f7bc800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:33.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.786+0000 7f7bd1ffb700 1 -- 172.21.15.27:0/3000287571 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8016070 con 0x7f7bd4102440 2024-05-21T03:20:33.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.786+0000 7f7bdb0a8700 1 -- 172.21.15.27:0/3000287571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bd419a4d0 con 0x7f7bd4102440 2024-05-21T03:20:33.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.786+0000 7f7bd1ffb700 1 -- 172.21.15.27:0/3000287571 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bc800fd40 con 0x7f7bd4102440 2024-05-21T03:20:33.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.786+0000 7f7bd1ffb700 1 -- 172.21.15.27:0/3000287571 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc800e3b0 con 0x7f7bd4102440 2024-05-21T03:20:33.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.786+0000 7f7bdb0a8700 1 -- 172.21.15.27:0/3000287571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bd41a6790 con 0x7f7bd4102440 2024-05-21T03:20:33.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.787+0000 7f7bdb0a8700 1 -- 172.21.15.27:0/3000287571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bd406a230 con 0x7f7bd4102440 2024-05-21T03:20:33.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.788+0000 7f7bd1ffb700 1 -- 172.21.15.27:0/3000287571 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7bc8019070 con 0x7f7bd4102440 2024-05-21T03:20:33.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.789+0000 7f7bd1ffb700 1 --2- 172.21.15.27:0/3000287571 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f7bc40b2880 0x7f7bc40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:33.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.789+0000 7f7bd3fff700 1 --2- 172.21.15.27:0/3000287571 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f7bc40b2880 0x7f7bc40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:33.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.789+0000 7f7bd1ffb700 1 -- 172.21.15.27:0/3000287571 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7879+0+0 (secure 0 0 0) 0x7f7bc80e1a20 con 0x7f7bd4102440 2024-05-21T03:20:33.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.789+0000 7f7bd3fff700 1 --2- 172.21.15.27:0/3000287571 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f7bc40b2880 0x7f7bc40b4d40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4079bb0 tx=0x7f7bc0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:33.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:33.793+0000 7f7bd1ffb700 1 -- 172.21.15.27:0/3000287571 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7bc80a98f0 con 0x7f7bd4102440 2024-05-21T03:20:34.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:34.054+0000 7f7bdb0a8700 1 -- 172.21.15.27:0/3000287571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7bd402e560 con 0x7f7bd4102440 2024-05-21T03:20:34.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:34.259+0000 7f7bd1ffb700 1 -- 172.21.15.27:0/3000287571 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f7bc80a91f0 con 0x7f7bd4102440 2024-05-21T03:20:34.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:34.262+0000 7f7bdb0a8700 1 -- 172.21.15.27:0/3000287571 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f7bc40b2880 msgr2=0x7f7bc40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:34.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:34.262+0000 7f7bdb0a8700 1 --2- 172.21.15.27:0/3000287571 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f7bc40b2880 0x7f7bc40b4d40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4079bb0 tx=0x7f7bc0005fd0 comp rx=0 tx=0).stop 2024-05-21T03:20:34.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:34.262+0000 7f7bdb0a8700 1 -- 172.21.15.27:0/3000287571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bd4102440 msgr2=0x7f7bd419f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:34.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:34.262+0000 7f7bdb0a8700 1 --2- 172.21.15.27:0/3000287571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bd4102440 0x7f7bd419f8c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800d950 tx=0x7f7bc800dc60 comp rx=0 tx=0).stop 2024-05-21T03:20:34.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:34.262+0000 7f7bdb0a8700 1 -- 172.21.15.27:0/3000287571 shutdown_connections 2024-05-21T03:20:34.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:34.262+0000 7f7bdb0a8700 1 --2- 172.21.15.27:0/3000287571 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f7bc40b2880 0x7f7bc40b4d40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:34.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:34.262+0000 7f7bdb0a8700 1 --2- 172.21.15.27:0/3000287571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bd4108520 0x7f7bd4199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:34.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:34.262+0000 7f7bdb0a8700 1 --2- 172.21.15.27:0/3000287571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bd4102440 0x7f7bd419f8c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:34.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:34.262+0000 7f7bdb0a8700 1 --2- 172.21.15.27:0/3000287571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bd4102e00 0x7f7bd419fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:34.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:34.262+0000 7f7bdb0a8700 1 -- 172.21.15.27:0/3000287571 >> 172.21.15.27:0/3000287571 conn(0x7f7bd40793f0 msgr2=0x7f7bd406b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:34.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:34.262+0000 7f7bdb0a8700 1 -- 172.21.15.27:0/3000287571 shutdown_connections 2024-05-21T03:20:34.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:34.262+0000 7f7bdb0a8700 1 -- 172.21.15.27:0/3000287571 wait complete. 2024-05-21T03:20:34.264 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 2 and data pool 3 2024-05-21T03:20:34.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:34 smithi027 ceph-mon[32179]: osdmap e79: 12 total, 12 up, 12 in 2024-05-21T03:20:34.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:34 smithi027 ceph-mon[32179]: pgmap v289: 97 pgs: 26 creating+peering, 37 active+clean, 34 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:34.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:34 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3000287571' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-21T03:20:34.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:34 smithi027 ceph-mon[32179]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-21T03:20:34.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:34 smithi039 ceph-mon[36429]: osdmap e79: 12 total, 12 up, 12 in 2024-05-21T03:20:34.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:34 smithi039 ceph-mon[36429]: pgmap v289: 97 pgs: 26 creating+peering, 37 active+clean, 34 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:34.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:34 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/3000287571' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-21T03:20:34.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:34 smithi039 ceph-mon[36429]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-21T03:20:34.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:34 smithi165 ceph-mon[36269]: osdmap e79: 12 total, 12 up, 12 in 2024-05-21T03:20:34.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:34 smithi165 ceph-mon[36269]: pgmap v289: 97 pgs: 26 creating+peering, 37 active+clean, 34 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:34.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:34 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/3000287571' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-21T03:20:34.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:34 smithi165 ceph-mon[36269]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-21T03:20:34.958 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-21T03:20:35.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:35 smithi027 ceph-mon[32179]: osdmap e80: 12 total, 12 up, 12 in 2024-05-21T03:20:35.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:35 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3000287571' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-21T03:20:35.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:35 smithi027 ceph-mon[32179]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-21T03:20:35.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:35 smithi027 ceph-mon[32179]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-21T03:20:35.219 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:20:35.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:35 smithi039 ceph-mon[36429]: osdmap e80: 12 total, 12 up, 12 in 2024-05-21T03:20:35.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:35 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/3000287571' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-21T03:20:35.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:35 smithi039 ceph-mon[36429]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-21T03:20:35.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:35 smithi039 ceph-mon[36429]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-21T03:20:35.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:35 smithi165 ceph-mon[36269]: osdmap e80: 12 total, 12 up, 12 in 2024-05-21T03:20:35.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:35 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/3000287571' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-21T03:20:35.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:35 smithi165 ceph-mon[36269]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-21T03:20:35.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:35 smithi165 ceph-mon[36269]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-21T03:20:36.019 INFO:journalctl@ceph.mds.b.smithi039.stdout:May 21 03:20:35 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-b[66446]: 2024-05-21T03:20:35.589+0000 7fbbf5f0f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-21T03:20:36.304 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:36 smithi027 ceph-mon[32179]: mds.? [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] up:active 2024-05-21T03:20:36.304 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:36 smithi027 ceph-mon[32179]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T03:20:36.304 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:36 smithi027 ceph-mon[32179]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T03:20:36.304 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:36 smithi027 ceph-mon[32179]: pgmap v291: 97 pgs: 23 creating+peering, 60 active+clean, 14 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:36.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:36 smithi039 ceph-mon[36429]: mds.? [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] up:active 2024-05-21T03:20:36.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:36 smithi039 ceph-mon[36429]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T03:20:36.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:36 smithi039 ceph-mon[36429]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T03:20:36.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:36 smithi039 ceph-mon[36429]: pgmap v291: 97 pgs: 23 creating+peering, 60 active+clean, 14 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:36.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:36 smithi165 ceph-mon[36269]: mds.? [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] up:active 2024-05-21T03:20:36.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:36 smithi165 ceph-mon[36269]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T03:20:36.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:36 smithi165 ceph-mon[36269]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T03:20:36.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:36 smithi165 ceph-mon[36269]: pgmap v291: 97 pgs: 23 creating+peering, 60 active+clean, 14 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:20:36.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.864+0000 7f24e4333700 1 -- 172.21.15.27:0/3163593127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24dc06bf10 msgr2=0x7f24dc06c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:36.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.864+0000 7f24e4333700 1 --2- 172.21.15.27:0/3163593127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24dc06bf10 0x7f24dc06c2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f24d80099c0 tx=0x7f24d8009cd0 comp rx=0 tx=0).stop 2024-05-21T03:20:36.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.864+0000 7f24e4333700 1 -- 172.21.15.27:0/3163593127 shutdown_connections 2024-05-21T03:20:36.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.864+0000 7f24e4333700 1 --2- 172.21.15.27:0/3163593127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24dc108860 0x7f24dc110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:36.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.864+0000 7f24e4333700 1 --2- 172.21.15.27:0/3163593127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24dc06bf10 0x7f24dc06c2f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:36.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.864+0000 7f24e4333700 1 --2- 172.21.15.27:0/3163593127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24dc06c8c0 0x7f24dc108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:36.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.864+0000 7f24e4333700 1 -- 172.21.15.27:0/3163593127 >> 172.21.15.27:0/3163593127 conn(0x7f24dc078490 msgr2=0x7f24dc0788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:36.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.864+0000 7f24e4333700 1 -- 172.21.15.27:0/3163593127 shutdown_connections 2024-05-21T03:20:36.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.864+0000 7f24e4333700 1 -- 172.21.15.27:0/3163593127 wait complete. 2024-05-21T03:20:36.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.865+0000 7f24e4333700 1 Processor -- start 2024-05-21T03:20:36.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.865+0000 7f24e4333700 1 -- start start 2024-05-21T03:20:36.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.865+0000 7f24e4333700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24dc06bf10 0x7f24dc19fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:36.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.865+0000 7f24e4333700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24dc06c8c0 0x7f24dc19ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:36.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.865+0000 7f24e4333700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24dc108860 0x7f24dc199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:36.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.865+0000 7f24e4333700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc1137b0 con 0x7f24dc06bf10 2024-05-21T03:20:36.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.865+0000 7f24e4333700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc113630 con 0x7f24dc108860 2024-05-21T03:20:36.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.865+0000 7f24e4333700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc113930 con 0x7f24dc06c8c0 2024-05-21T03:20:36.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.865+0000 7f24e20cf700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24dc06bf10 0x7f24dc19fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:36.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.865+0000 7f24e20cf700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24dc06bf10 0x7f24dc19fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49110/0 (socket says 172.21.15.27:49110) 2024-05-21T03:20:36.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.865+0000 7f24e20cf700 1 -- 172.21.15.27:0/938121508 learned_addr learned my addr 172.21.15.27:0/938121508 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:20:36.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.866+0000 7f24e18ce700 1 --2- 172.21.15.27:0/938121508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24dc06c8c0 0x7f24dc19ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:36.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.866+0000 7f24e28d0700 1 --2- 172.21.15.27:0/938121508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24dc108860 0x7f24dc199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:36.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.866+0000 7f24e20cf700 1 -- 172.21.15.27:0/938121508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24dc06c8c0 msgr2=0x7f24dc19ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:36.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.866+0000 7f24e20cf700 1 --2- 172.21.15.27:0/938121508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24dc06c8c0 0x7f24dc19ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:36.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.866+0000 7f24e20cf700 1 -- 172.21.15.27:0/938121508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24dc108860 msgr2=0x7f24dc199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:36.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.866+0000 7f24e20cf700 1 --2- 172.21.15.27:0/938121508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24dc108860 0x7f24dc199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:36.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.866+0000 7f24e20cf700 1 -- 172.21.15.27:0/938121508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24dc19a410 con 0x7f24dc06bf10 2024-05-21T03:20:36.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.866+0000 7f24e28d0700 1 --2- 172.21.15.27:0/938121508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24dc108860 0x7f24dc199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:20:36.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.866+0000 7f24e20cf700 1 --2- 172.21.15.27:0/938121508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24dc06bf10 0x7f24dc19fa60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f24d800b580 tx=0x7f24d80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:36.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.866+0000 7f24d37fe700 1 -- 172.21.15.27:0/938121508 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8027070 con 0x7f24dc06bf10 2024-05-21T03:20:36.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.866+0000 7f24e4333700 1 -- 172.21.15.27:0/938121508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24dc19a6a0 con 0x7f24dc06bf10 2024-05-21T03:20:36.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.866+0000 7f24d37fe700 1 -- 172.21.15.27:0/938121508 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24d8004020 con 0x7f24dc06bf10 2024-05-21T03:20:36.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.866+0000 7f24d37fe700 1 -- 172.21.15.27:0/938121508 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8022e30 con 0x7f24dc06bf10 2024-05-21T03:20:36.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.866+0000 7f24e4333700 1 -- 172.21.15.27:0/938121508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24dc1a69e0 con 0x7f24dc06bf10 2024-05-21T03:20:36.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.867+0000 7f24e4333700 1 -- 172.21.15.27:0/938121508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24dc19aa00 con 0x7f24dc06bf10 2024-05-21T03:20:36.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.869+0000 7f24d37fe700 1 -- 172.21.15.27:0/938121508 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f24d8022590 con 0x7f24dc06bf10 2024-05-21T03:20:36.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.869+0000 7f24d37fe700 1 --2- 172.21.15.27:0/938121508 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f24c80b2900 0x7f24c80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:36.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.870+0000 7f24e18ce700 1 --2- 172.21.15.27:0/938121508 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f24c80b2900 0x7f24c80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:36.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.870+0000 7f24d37fe700 1 -- 172.21.15.27:0/938121508 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f24d80e2be0 con 0x7f24dc06bf10 2024-05-21T03:20:36.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.870+0000 7f24e18ce700 1 --2- 172.21.15.27:0/938121508 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f24c80b2900 0x7f24c80b4dc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f24cc005950 tx=0x7f24cc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:36.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:36.874+0000 7f24d37fe700 1 -- 172.21.15.27:0/938121508 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f24d8013070 con 0x7f24dc06bf10 2024-05-21T03:20:37.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:37.095+0000 7f24e4333700 1 -- 172.21.15.27:0/938121508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24dc06a230 con 0x7f24dc06bf10 2024-05-21T03:20:37.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:37.097+0000 7f24d37fe700 1 -- 172.21.15.27:0/938121508 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22157 (secure 0 0 0) 0x7f24d80aaae0 con 0x7f24dc06bf10 2024-05-21T03:20:37.097 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:20:37.097 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":80,"fsid":"c9ca94be-171f-11ef-bc9a-c7b262605968","created":"2024-05-21T03:12:18.217236+0000","modified":"2024-05-21T03:20:34.056022+0000","last_up_change":"2024-05-21T03:19:30.382150+0000","last_in_change":"2024-05-21T03:19:09.058249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T03:15:41.340194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T03:20:27.507346+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T03:20:30.830169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c745c09f-9a76-4d05-af75-c7554f6d5e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6803","nonce":1399996449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6805","nonce":1399996449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6809","nonce":1399996449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6807","nonce":1399996449}]},"public_addr":"172.21.15.27:6803/1399996449","cluster_addr":"172.21.15.27:6805/1399996449","heartbeat_back_addr":"172.21.15.27:6809/1399996449","heartbeat_front_addr":"172.21.15.27:6807/1399996449","state":["exists","up"]},{"osd":1,"uuid":"8b73aa00-4364-438f-9722-53caa4e9bec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6801","nonce":1776318847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6803","nonce":1776318847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6807","nonce":1776318847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6805","nonce":1776318847}]},"public_addr":"172.21.15.39:6801/1776318847","cluster_addr":"172.21.15.39:6803/1776318847","heartbeat_back_addr":"172.21.15.39:6807/1776318847","heartbeat_front_addr":"172.21.15.39:6805/1776318847","state":["exists","up"]},{"osd":2,"uuid":"0822043e-7d3c-4e98-9dc0-94b98b4d562a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6801","nonce":703322625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6803","nonce":703322625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6807","nonce":703322625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6805","nonce":703322625}]},"public_addr":"172.21.15.165:6801/703322625","cluster_addr":"172.21.15.165:6803/703322625","heartbeat_back_addr":"172.21.15.165:6807/703322625","heartbeat_front_addr":"172.21.15.165:6805/703322625","state":["exists","up"]},{"osd":3,"uuid":"c563b1b2-efcb-4f8c-8121-c1938f83131e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6811","nonce":1933403135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6813","nonce":1933403135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6817","nonce":1933403135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6815","nonce":1933403135}]},"public_addr":"172.21.15.27:6811/1933403135","cluster_addr":"172.21.15.27:6813/1933403135","heartbeat_back_addr":"172.21.15.27:6817/1933403135","heartbeat_front_addr":"172.21.15.27:6815/1933403135","state":["exists","up"]},{"osd":4,"uuid":"0c793b87-b045-4b4a-89e7-87ec36deeb0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6809","nonce":155579331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6811","nonce":155579331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6815","nonce":155579331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6813","nonce":155579331}]},"public_addr":"172.21.15.39:6809/155579331","cluster_addr":"172.21.15.39:6811/155579331","heartbeat_back_addr":"172.21.15.39:6815/155579331","heartbeat_front_addr":"172.21.15.39:6813/155579331","state":["exists","up"]},{"osd":5,"uuid":"367eeb5a-9d75-4644-bf46-317c03f33d8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6809","nonce":386850244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6811","nonce":386850244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6815","nonce":386850244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6813","nonce":386850244}]},"public_addr":"172.21.15.165:6809/386850244","cluster_addr":"172.21.15.165:6811/386850244","heartbeat_back_addr":"172.21.15.165:6815/386850244","heartbeat_front_addr":"172.21.15.165:6813/386850244","state":["exists","up"]},{"osd":6,"uuid":"e000167f-24fd-4115-83f0-8afbfa8d8991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6819","nonce":3531376679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6821","nonce":3531376679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6825","nonce":3531376679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6823","nonce":3531376679}]},"public_addr":"172.21.15.27:6819/3531376679","cluster_addr":"172.21.15.27:6821/3531376679","heartbeat_back_addr":"172.21.15.27:6825/3531376679","heartbeat_front_addr":"172.21.15.27:6823/3531376679","state":["exists","up"]},{"osd":7,"uuid":"4b8d486b-868e-4f33-98a1-b99686adeaaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6817","nonce":1550035852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6819","nonce":1550035852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6823","nonce":1550035852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6821","nonce":1550035852}]},"public_addr":"172.21.15.39:6817/1550035852","cluster_addr":"172.21.15.39:6819/1550035852","heartbeat_back_addr":"172.21.15.39:6823/1550035852","heartbeat_front_addr":"172.21.15.39:6821/1550035852","state":["exists","up"]},{"osd":8,"uuid":"2259f35c-2edd-4331-9a63-8c0c1acbdc71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6817","nonce":3184704971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6819","nonce":3184704971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6823","nonce":3184704971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6821","nonce":3184704971}]},"public_addr":"172.21.15.165:6817/3184704971","cluster_addr":"172.21.15.165:6819/3184704971","heartbeat_back_addr":"172.21.15.165:6823/3184704971","heartbeat_front_addr":"172.21.15.165:6821/3184704971","state":["exists","up"]},{"osd":9,"uuid":"10307ecc-8270-4be2-b62b-9708e09bb207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6827","nonce":2800048793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6829","nonce":2800048793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6833","nonce":2800048793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6831","nonce":2800048793}]},"public_addr":"172.21.15.27:6827/2800048793","cluster_addr":"172.21.15.27:6829/2800048793","heartbeat_back_addr":"172.21.15.27:6833/2800048793","heartbeat_front_addr":"172.21.15.27:6831/2800048793","state":["exists","up"]},{"osd":10,"uuid":"dfe1b8ca-ff8a-4b7f-ade5-ad4f00cf60cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6825","nonce":363500811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6827","nonce":363500811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6831","nonce":363500811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6829","nonce":363500811}]},"public_addr":"172.21.15.39:6825/363500811","cluster_addr":"172.21.15.39:6827/363500811","heartbeat_back_addr":"172.21.15.39:6831/363500811","heartbeat_front_addr":"172.21.15.39:6829/363500811","state":["exists","up"]},{"osd":11,"uuid":"f9531145-5d09-4dbd-a050-3159ee09db24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6825","nonce":2274332243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6827","nonce":2274332243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6831","nonce":2274332243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6829","nonce":2274332243}]},"public_addr":"172.21.15.165:6825/2274332243","cluster_addr":"172.21.15.165:6827/2274332243","heartbeat_back_addr":"172.21.15.165:6831/2274332243","heartbeat_front_addr":"172.21.15.165:6829/2274332243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:14:47.077860+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:15:13.272879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:15:38.046651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:03.376483+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:29.179280+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:53.391261+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:17:18.472829+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:17:44.349925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:18:08.652193+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:18:35.282669+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:19:01.637151+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:19:27.461566+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6800/1185518061":"2024-05-22T03:13:13.249220+0000","172.21.15.27:6801/1185518061":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/4194431212":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/1890032101":"2024-05-22T03:13:13.249220+0000","172.21.15.27:6800/175229478":"2024-05-22T03:12:45.739195+0000","172.21.15.27:0/999325706":"2024-05-22T03:12:45.739195+0000","172.21.15.27:6801/175229478":"2024-05-22T03:12:45.739195+0000","172.21.15.27:0/4184128736":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/3000197506":"2024-05-22T03:12:45.739195+0000","172.21.15.27:0/972677319":"2024-05-22T03:12:45.739195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T03:20:37.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:37.099+0000 7f24e4333700 1 -- 172.21.15.27:0/938121508 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f24c80b2900 msgr2=0x7f24c80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:37.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:37.099+0000 7f24e4333700 1 --2- 172.21.15.27:0/938121508 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f24c80b2900 0x7f24c80b4dc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f24cc005950 tx=0x7f24cc0058e0 comp rx=0 tx=0).stop 2024-05-21T03:20:37.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:37.099+0000 7f24e4333700 1 -- 172.21.15.27:0/938121508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24dc06bf10 msgr2=0x7f24dc19fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:37.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:37.099+0000 7f24e4333700 1 --2- 172.21.15.27:0/938121508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24dc06bf10 0x7f24dc19fa60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f24d800b580 tx=0x7f24d80047b0 comp rx=0 tx=0).stop 2024-05-21T03:20:37.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:37.099+0000 7f24e4333700 1 -- 172.21.15.27:0/938121508 shutdown_connections 2024-05-21T03:20:37.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:37.099+0000 7f24e4333700 1 --2- 172.21.15.27:0/938121508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24dc108860 0x7f24dc199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:37.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:37.099+0000 7f24e4333700 1 --2- 172.21.15.27:0/938121508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24dc06bf10 0x7f24dc19fa60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:37.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:37.099+0000 7f24e4333700 1 --2- 172.21.15.27:0/938121508 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f24c80b2900 0x7f24c80b4dc0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:37.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:37.099+0000 7f24e4333700 1 --2- 172.21.15.27:0/938121508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24dc06c8c0 0x7f24dc19ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:37.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:37.099+0000 7f24e4333700 1 -- 172.21.15.27:0/938121508 >> 172.21.15.27:0/938121508 conn(0x7f24dc078490 msgr2=0x7f24dc1005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:37.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:37.099+0000 7f24e4333700 1 -- 172.21.15.27:0/938121508 shutdown_connections 2024-05-21T03:20:37.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:37.099+0000 7f24e4333700 1 -- 172.21.15.27:0/938121508 wait complete. 2024-05-21T03:20:37.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:37 smithi039 ceph-mon[36429]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-21T03:20:37.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:37 smithi039 ceph-mon[36429]: Cluster is now healthy 2024-05-21T03:20:37.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:37 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/938121508' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:20:37.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:37 smithi165 ceph-mon[36269]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-21T03:20:37.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:37 smithi165 ceph-mon[36269]: Cluster is now healthy 2024-05-21T03:20:37.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:37 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/938121508' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:20:37.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:37 smithi027 ceph-mon[32179]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-21T03:20:37.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:37 smithi027 ceph-mon[32179]: Cluster is now healthy 2024-05-21T03:20:37.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:37 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/938121508' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:20:37.742 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-21T03:20:37.996 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:20:38.425 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:38 smithi027 ceph-mon[32179]: pgmap v292: 97 pgs: 97 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-21T03:20:38.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:38 smithi039 ceph-mon[36429]: pgmap v292: 97 pgs: 97 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-21T03:20:38.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:38 smithi165 ceph-mon[36269]: pgmap v292: 97 pgs: 97 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-21T03:20:39.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.586+0000 7fa623639700 1 -- 172.21.15.27:0/2872910899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa61c1084f0 msgr2=0x7fa61c1088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:39.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.586+0000 7fa623639700 1 --2- 172.21.15.27:0/2872910899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa61c1084f0 0x7fa61c1088d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa618009990 tx=0x7fa618009ca0 comp rx=0 tx=0).stop 2024-05-21T03:20:39.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.586+0000 7fa623639700 1 -- 172.21.15.27:0/2872910899 shutdown_connections 2024-05-21T03:20:39.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.586+0000 7fa623639700 1 --2- 172.21.15.27:0/2872910899 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa61c102e20 0x7fa61c10f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:39.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.586+0000 7fa623639700 1 --2- 172.21.15.27:0/2872910899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa61c1084f0 0x7fa61c1088d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:39.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.586+0000 7fa623639700 1 --2- 172.21.15.27:0/2872910899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa61c102460 0x7fa61c1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:39.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.586+0000 7fa623639700 1 -- 172.21.15.27:0/2872910899 >> 172.21.15.27:0/2872910899 conn(0x7fa61c0794e0 msgr2=0x7fa61c0798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:39.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.586+0000 7fa623639700 1 -- 172.21.15.27:0/2872910899 shutdown_connections 2024-05-21T03:20:39.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.586+0000 7fa623639700 1 -- 172.21.15.27:0/2872910899 wait complete. 2024-05-21T03:20:39.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.587+0000 7fa623639700 1 Processor -- start 2024-05-21T03:20:39.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.587+0000 7fa623639700 1 -- start start 2024-05-21T03:20:39.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.587+0000 7fa623639700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa61c102460 0x7fa61c0ff490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:39.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.587+0000 7fa623639700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa61c102e20 0x7fa61c0ff9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:39.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.587+0000 7fa623639700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa61c1084f0 0x7fa61c0fff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:39.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.587+0000 7fa623639700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c1126a0 con 0x7fa61c1084f0 2024-05-21T03:20:39.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.587+0000 7fa623639700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c112520 con 0x7fa61c102460 2024-05-21T03:20:39.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.587+0000 7fa623639700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c112820 con 0x7fa61c102e20 2024-05-21T03:20:39.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.587+0000 7fa621bd6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa61c1084f0 0x7fa61c0fff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:39.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.587+0000 7fa621bd6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa61c1084f0 0x7fa61c0fff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49138/0 (socket says 172.21.15.27:49138) 2024-05-21T03:20:39.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.587+0000 7fa621bd6700 1 -- 172.21.15.27:0/70764 learned_addr learned my addr 172.21.15.27:0/70764 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:20:39.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.587+0000 7fa620bd4700 1 --2- 172.21.15.27:0/70764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa61c102e20 0x7fa61c0ff9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:39.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.588+0000 7fa6213d5700 1 --2- 172.21.15.27:0/70764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa61c102460 0x7fa61c0ff490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:39.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.588+0000 7fa621bd6700 1 -- 172.21.15.27:0/70764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa61c102e20 msgr2=0x7fa61c0ff9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:39.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.588+0000 7fa621bd6700 1 --2- 172.21.15.27:0/70764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa61c102e20 0x7fa61c0ff9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:39.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.588+0000 7fa621bd6700 1 -- 172.21.15.27:0/70764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa61c102460 msgr2=0x7fa61c0ff490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:39.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.588+0000 7fa621bd6700 1 --2- 172.21.15.27:0/70764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa61c102460 0x7fa61c0ff490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:39.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.588+0000 7fa621bd6700 1 -- 172.21.15.27:0/70764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61c1a8200 con 0x7fa61c1084f0 2024-05-21T03:20:39.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.588+0000 7fa621bd6700 1 --2- 172.21.15.27:0/70764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa61c1084f0 0x7fa61c0fff10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa61400ada0 tx=0x7fa61400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:39.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.588+0000 7fa6127fc700 1 -- 172.21.15.27:0/70764 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa614017020 con 0x7fa61c1084f0 2024-05-21T03:20:39.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.588+0000 7fa6127fc700 1 -- 172.21.15.27:0/70764 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa61400ce30 con 0x7fa61c1084f0 2024-05-21T03:20:39.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.588+0000 7fa623639700 1 -- 172.21.15.27:0/70764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa61c1a84f0 con 0x7fa61c1084f0 2024-05-21T03:20:39.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.588+0000 7fa6127fc700 1 -- 172.21.15.27:0/70764 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa614005350 con 0x7fa61c1084f0 2024-05-21T03:20:39.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.588+0000 7fa623639700 1 -- 172.21.15.27:0/70764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa61c1a8a30 con 0x7fa61c1084f0 2024-05-21T03:20:39.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.589+0000 7fa623639700 1 -- 172.21.15.27:0/70764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa61c06a230 con 0x7fa61c1084f0 2024-05-21T03:20:39.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.590+0000 7fa6127fc700 1 -- 172.21.15.27:0/70764 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fa61400c980 con 0x7fa61c1084f0 2024-05-21T03:20:39.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.591+0000 7fa6127fc700 1 --2- 172.21.15.27:0/70764 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fa6080b2880 0x7fa6080b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:39.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.591+0000 7fa6213d5700 1 --2- 172.21.15.27:0/70764 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fa6080b2880 0x7fa6080b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:39.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.591+0000 7fa6127fc700 1 -- 172.21.15.27:0/70764 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fa6140e1d90 con 0x7fa61c1084f0 2024-05-21T03:20:39.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.592+0000 7fa6213d5700 1 --2- 172.21.15.27:0/70764 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fa6080b2880 0x7fa6080b4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa618006010 tx=0x7fa61801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:39.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.595+0000 7fa6127fc700 1 -- 172.21.15.27:0/70764 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa614021020 con 0x7fa61c1084f0 2024-05-21T03:20:39.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.811+0000 7fa623639700 1 -- 172.21.15.27:0/70764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa61c02e720 con 0x7fa61c1084f0 2024-05-21T03:20:39.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.812+0000 7fa6127fc700 1 -- 172.21.15.27:0/70764 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22157 (secure 0 0 0) 0x7fa61400a1f0 con 0x7fa61c1084f0 2024-05-21T03:20:39.813 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:20:39.813 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":80,"fsid":"c9ca94be-171f-11ef-bc9a-c7b262605968","created":"2024-05-21T03:12:18.217236+0000","modified":"2024-05-21T03:20:34.056022+0000","last_up_change":"2024-05-21T03:19:30.382150+0000","last_in_change":"2024-05-21T03:19:09.058249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T03:15:41.340194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T03:20:27.507346+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T03:20:30.830169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c745c09f-9a76-4d05-af75-c7554f6d5e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6803","nonce":1399996449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6805","nonce":1399996449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6809","nonce":1399996449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6807","nonce":1399996449}]},"public_addr":"172.21.15.27:6803/1399996449","cluster_addr":"172.21.15.27:6805/1399996449","heartbeat_back_addr":"172.21.15.27:6809/1399996449","heartbeat_front_addr":"172.21.15.27:6807/1399996449","state":["exists","up"]},{"osd":1,"uuid":"8b73aa00-4364-438f-9722-53caa4e9bec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6801","nonce":1776318847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6803","nonce":1776318847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6807","nonce":1776318847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6805","nonce":1776318847}]},"public_addr":"172.21.15.39:6801/1776318847","cluster_addr":"172.21.15.39:6803/1776318847","heartbeat_back_addr":"172.21.15.39:6807/1776318847","heartbeat_front_addr":"172.21.15.39:6805/1776318847","state":["exists","up"]},{"osd":2,"uuid":"0822043e-7d3c-4e98-9dc0-94b98b4d562a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6801","nonce":703322625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6803","nonce":703322625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6807","nonce":703322625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6805","nonce":703322625}]},"public_addr":"172.21.15.165:6801/703322625","cluster_addr":"172.21.15.165:6803/703322625","heartbeat_back_addr":"172.21.15.165:6807/703322625","heartbeat_front_addr":"172.21.15.165:6805/703322625","state":["exists","up"]},{"osd":3,"uuid":"c563b1b2-efcb-4f8c-8121-c1938f83131e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6811","nonce":1933403135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6813","nonce":1933403135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6817","nonce":1933403135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6815","nonce":1933403135}]},"public_addr":"172.21.15.27:6811/1933403135","cluster_addr":"172.21.15.27:6813/1933403135","heartbeat_back_addr":"172.21.15.27:6817/1933403135","heartbeat_front_addr":"172.21.15.27:6815/1933403135","state":["exists","up"]},{"osd":4,"uuid":"0c793b87-b045-4b4a-89e7-87ec36deeb0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6809","nonce":155579331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6811","nonce":155579331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6815","nonce":155579331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6813","nonce":155579331}]},"public_addr":"172.21.15.39:6809/155579331","cluster_addr":"172.21.15.39:6811/155579331","heartbeat_back_addr":"172.21.15.39:6815/155579331","heartbeat_front_addr":"172.21.15.39:6813/155579331","state":["exists","up"]},{"osd":5,"uuid":"367eeb5a-9d75-4644-bf46-317c03f33d8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6809","nonce":386850244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6811","nonce":386850244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6815","nonce":386850244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6813","nonce":386850244}]},"public_addr":"172.21.15.165:6809/386850244","cluster_addr":"172.21.15.165:6811/386850244","heartbeat_back_addr":"172.21.15.165:6815/386850244","heartbeat_front_addr":"172.21.15.165:6813/386850244","state":["exists","up"]},{"osd":6,"uuid":"e000167f-24fd-4115-83f0-8afbfa8d8991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6819","nonce":3531376679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6821","nonce":3531376679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6825","nonce":3531376679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6823","nonce":3531376679}]},"public_addr":"172.21.15.27:6819/3531376679","cluster_addr":"172.21.15.27:6821/3531376679","heartbeat_back_addr":"172.21.15.27:6825/3531376679","heartbeat_front_addr":"172.21.15.27:6823/3531376679","state":["exists","up"]},{"osd":7,"uuid":"4b8d486b-868e-4f33-98a1-b99686adeaaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6817","nonce":1550035852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6819","nonce":1550035852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6823","nonce":1550035852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6821","nonce":1550035852}]},"public_addr":"172.21.15.39:6817/1550035852","cluster_addr":"172.21.15.39:6819/1550035852","heartbeat_back_addr":"172.21.15.39:6823/1550035852","heartbeat_front_addr":"172.21.15.39:6821/1550035852","state":["exists","up"]},{"osd":8,"uuid":"2259f35c-2edd-4331-9a63-8c0c1acbdc71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6817","nonce":3184704971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6819","nonce":3184704971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6823","nonce":3184704971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6821","nonce":3184704971}]},"public_addr":"172.21.15.165:6817/3184704971","cluster_addr":"172.21.15.165:6819/3184704971","heartbeat_back_addr":"172.21.15.165:6823/3184704971","heartbeat_front_addr":"172.21.15.165:6821/3184704971","state":["exists","up"]},{"osd":9,"uuid":"10307ecc-8270-4be2-b62b-9708e09bb207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6827","nonce":2800048793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6829","nonce":2800048793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6833","nonce":2800048793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6831","nonce":2800048793}]},"public_addr":"172.21.15.27:6827/2800048793","cluster_addr":"172.21.15.27:6829/2800048793","heartbeat_back_addr":"172.21.15.27:6833/2800048793","heartbeat_front_addr":"172.21.15.27:6831/2800048793","state":["exists","up"]},{"osd":10,"uuid":"dfe1b8ca-ff8a-4b7f-ade5-ad4f00cf60cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6825","nonce":363500811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6827","nonce":363500811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6831","nonce":363500811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6829","nonce":363500811}]},"public_addr":"172.21.15.39:6825/363500811","cluster_addr":"172.21.15.39:6827/363500811","heartbeat_back_addr":"172.21.15.39:6831/363500811","heartbeat_front_addr":"172.21.15.39:6829/363500811","state":["exists","up"]},{"osd":11,"uuid":"f9531145-5d09-4dbd-a050-3159ee09db24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6825","nonce":2274332243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6827","nonce":2274332243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6831","nonce":2274332243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6829","nonce":2274332243}]},"public_addr":"172.21.15.165:6825/2274332243","cluster_addr":"172.21.15.165:6827/2274332243","heartbeat_back_addr":"172.21.15.165:6831/2274332243","heartbeat_front_addr":"172.21.15.165:6829/2274332243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:14:47.077860+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:15:13.272879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:15:38.046651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:03.376483+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:29.179280+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:53.391261+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:17:18.472829+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:17:44.349925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:18:08.652193+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:18:35.282669+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:19:01.637151+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:19:27.461566+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6800/1185518061":"2024-05-22T03:13:13.249220+0000","172.21.15.27:6801/1185518061":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/4194431212":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/1890032101":"2024-05-22T03:13:13.249220+0000","172.21.15.27:6800/175229478":"2024-05-22T03:12:45.739195+0000","172.21.15.27:0/999325706":"2024-05-22T03:12:45.739195+0000","172.21.15.27:6801/175229478":"2024-05-22T03:12:45.739195+0000","172.21.15.27:0/4184128736":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/3000197506":"2024-05-22T03:12:45.739195+0000","172.21.15.27:0/972677319":"2024-05-22T03:12:45.739195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T03:20:39.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.815+0000 7fa623639700 1 -- 172.21.15.27:0/70764 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fa6080b2880 msgr2=0x7fa6080b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:39.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.815+0000 7fa623639700 1 --2- 172.21.15.27:0/70764 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fa6080b2880 0x7fa6080b4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa618006010 tx=0x7fa61801a040 comp rx=0 tx=0).stop 2024-05-21T03:20:39.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.815+0000 7fa623639700 1 -- 172.21.15.27:0/70764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa61c1084f0 msgr2=0x7fa61c0fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:39.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.815+0000 7fa623639700 1 --2- 172.21.15.27:0/70764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa61c1084f0 0x7fa61c0fff10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa61400ada0 tx=0x7fa61400c330 comp rx=0 tx=0).stop 2024-05-21T03:20:39.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.815+0000 7fa623639700 1 -- 172.21.15.27:0/70764 shutdown_connections 2024-05-21T03:20:39.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.815+0000 7fa623639700 1 --2- 172.21.15.27:0/70764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa61c1084f0 0x7fa61c0fff10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:39.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.815+0000 7fa623639700 1 --2- 172.21.15.27:0/70764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa61c102460 0x7fa61c0ff490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:39.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.815+0000 7fa623639700 1 --2- 172.21.15.27:0/70764 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fa6080b2880 0x7fa6080b4d40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:39.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.815+0000 7fa623639700 1 --2- 172.21.15.27:0/70764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa61c102e20 0x7fa61c0ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:39.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.815+0000 7fa623639700 1 -- 172.21.15.27:0/70764 >> 172.21.15.27:0/70764 conn(0x7fa61c0794e0 msgr2=0x7fa61c06c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:39.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.815+0000 7fa623639700 1 -- 172.21.15.27:0/70764 shutdown_connections 2024-05-21T03:20:39.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:39.815+0000 7fa623639700 1 -- 172.21.15.27:0/70764 wait complete. 2024-05-21T03:20:40.540 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-21T03:20:40.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:40 smithi027 ceph-mon[32179]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-05-21T03:20:40.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:40 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/70764' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:20:40.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:40 smithi039 ceph-mon[36429]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-05-21T03:20:40.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:40 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/70764' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:20:40.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:40 smithi165 ceph-mon[36269]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-05-21T03:20:40.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:40 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/70764' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:20:40.798 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:20:42.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.508+0000 7faa3a57b700 1 -- 172.21.15.27:0/3468304507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa34109720 msgr2=0x7faa34109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:42.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.508+0000 7faa3a57b700 1 --2- 172.21.15.27:0/3468304507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa34109720 0x7faa34109b00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7faa24009990 tx=0x7faa24009ca0 comp rx=0 tx=0).stop 2024-05-21T03:20:42.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.508+0000 7faa3a57b700 1 -- 172.21.15.27:0/3468304507 shutdown_connections 2024-05-21T03:20:42.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.508+0000 7faa3a57b700 1 --2- 172.21.15.27:0/3468304507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa34102930 0x7faa34110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:42.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.508+0000 7faa3a57b700 1 --2- 172.21.15.27:0/3468304507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa34109720 0x7faa34109b00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:42.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.508+0000 7faa3a57b700 1 --2- 172.21.15.27:0/3468304507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa34101f70 0x7faa341023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:42.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.508+0000 7faa3a57b700 1 -- 172.21.15.27:0/3468304507 >> 172.21.15.27:0/3468304507 conn(0x7faa340776b0 msgr2=0x7faa34077ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:42.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.508+0000 7faa3a57b700 1 -- 172.21.15.27:0/3468304507 shutdown_connections 2024-05-21T03:20:42.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.508+0000 7faa3a57b700 1 -- 172.21.15.27:0/3468304507 wait complete. 2024-05-21T03:20:42.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.509+0000 7faa3a57b700 1 Processor -- start 2024-05-21T03:20:42.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.509+0000 7faa3a57b700 1 -- start start 2024-05-21T03:20:42.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.509+0000 7faa3a57b700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa34101f70 0x7faa3419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:42.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.509+0000 7faa3a57b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa34102930 0x7faa3419fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:42.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.509+0000 7faa3a57b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa34109720 0x7faa34199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:42.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.509+0000 7faa3a57b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faa34113750 con 0x7faa34109720 2024-05-21T03:20:42.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.509+0000 7faa3a57b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7faa341135d0 con 0x7faa34102930 2024-05-21T03:20:42.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.509+0000 7faa3a57b700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7faa341138d0 con 0x7faa34101f70 2024-05-21T03:20:42.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.509+0000 7faa38b18700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa34109720 0x7faa34199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:42.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.509+0000 7faa38b18700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa34109720 0x7faa34199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49150/0 (socket says 172.21.15.27:49150) 2024-05-21T03:20:42.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.509+0000 7faa38b18700 1 -- 172.21.15.27:0/462578519 learned_addr learned my addr 172.21.15.27:0/462578519 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:20:42.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.510+0000 7faa33fff700 1 --2- 172.21.15.27:0/462578519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa34101f70 0x7faa3419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:42.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.510+0000 7faa2bfff700 1 --2- 172.21.15.27:0/462578519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa34102930 0x7faa3419fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:42.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.510+0000 7faa38b18700 1 -- 172.21.15.27:0/462578519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa34101f70 msgr2=0x7faa3419f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:42.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.510+0000 7faa38b18700 1 --2- 172.21.15.27:0/462578519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa34101f70 0x7faa3419f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:42.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.510+0000 7faa38b18700 1 -- 172.21.15.27:0/462578519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa34102930 msgr2=0x7faa3419fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:42.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.510+0000 7faa38b18700 1 --2- 172.21.15.27:0/462578519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa34102930 0x7faa3419fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:42.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.510+0000 7faa38b18700 1 -- 172.21.15.27:0/462578519 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa3419a170 con 0x7faa34109720 2024-05-21T03:20:42.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.510+0000 7faa33fff700 1 --2- 172.21.15.27:0/462578519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa34101f70 0x7faa3419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:20:42.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.510+0000 7faa2bfff700 1 --2- 172.21.15.27:0/462578519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa34102930 0x7faa3419fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:20:42.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.510+0000 7faa38b18700 1 --2- 172.21.15.27:0/462578519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa34109720 0x7faa34199a10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00ada0 tx=0x7faa2c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:42.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.510+0000 7faa31ffb700 1 -- 172.21.15.27:0/462578519 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2c017020 con 0x7faa34109720 2024-05-21T03:20:42.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.510+0000 7faa3a57b700 1 -- 172.21.15.27:0/462578519 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa3419a460 con 0x7faa34109720 2024-05-21T03:20:42.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.510+0000 7faa31ffb700 1 -- 172.21.15.27:0/462578519 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa2c00ce30 con 0x7faa34109720 2024-05-21T03:20:42.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.510+0000 7faa31ffb700 1 -- 172.21.15.27:0/462578519 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2c0054a0 con 0x7faa34109720 2024-05-21T03:20:42.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.510+0000 7faa3a57b700 1 -- 172.21.15.27:0/462578519 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa341a6720 con 0x7faa34109720 2024-05-21T03:20:42.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.511+0000 7faa3a57b700 1 -- 172.21.15.27:0/462578519 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa3410bb60 con 0x7faa34109720 2024-05-21T03:20:42.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.513+0000 7faa31ffb700 1 -- 172.21.15.27:0/462578519 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7faa2c00c980 con 0x7faa34109720 2024-05-21T03:20:42.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.513+0000 7faa31ffb700 1 --2- 172.21.15.27:0/462578519 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7faa200b2880 0x7faa200b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:42.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.513+0000 7faa33fff700 1 --2- 172.21.15.27:0/462578519 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7faa200b2880 0x7faa200b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:42.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.514+0000 7faa31ffb700 1 -- 172.21.15.27:0/462578519 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7faa2c0e18b0 con 0x7faa34109720 2024-05-21T03:20:42.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.514+0000 7faa33fff700 1 --2- 172.21.15.27:0/462578519 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7faa200b2880 0x7faa200b4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faa24000c00 tx=0x7faa24005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:42.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.518+0000 7faa31ffb700 1 -- 172.21.15.27:0/462578519 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faa2c021020 con 0x7faa34109720 2024-05-21T03:20:42.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:42 smithi027 ceph-mon[32179]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-21T03:20:42.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:42 smithi039 ceph-mon[36429]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-21T03:20:42.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:42 smithi165 ceph-mon[36269]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-21T03:20:42.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:42.777+0000 7faa3a57b700 1 -- 172.21.15.27:0/462578519 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7faa3419af40 con 0x7faa34109720 2024-05-21T03:20:43.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:43.406+0000 7faa31ffb700 1 -- 172.21.15.27:0/462578519 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7faa2c0a97b0 con 0x7faa34109720 2024-05-21T03:20:43.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:43.409+0000 7faa3a57b700 1 -- 172.21.15.27:0/462578519 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7faa200b2880 msgr2=0x7faa200b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:43.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:43.409+0000 7faa3a57b700 1 --2- 172.21.15.27:0/462578519 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7faa200b2880 0x7faa200b4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faa24000c00 tx=0x7faa24005fb0 comp rx=0 tx=0).stop 2024-05-21T03:20:43.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:43.409+0000 7faa3a57b700 1 -- 172.21.15.27:0/462578519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa34109720 msgr2=0x7faa34199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:43.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:43.409+0000 7faa3a57b700 1 --2- 172.21.15.27:0/462578519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa34109720 0x7faa34199a10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00ada0 tx=0x7faa2c00c330 comp rx=0 tx=0).stop 2024-05-21T03:20:43.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:43.409+0000 7faa3a57b700 1 -- 172.21.15.27:0/462578519 shutdown_connections 2024-05-21T03:20:43.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:43.409+0000 7faa3a57b700 1 --2- 172.21.15.27:0/462578519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa34109720 0x7faa34199a10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:43.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:43.409+0000 7faa3a57b700 1 --2- 172.21.15.27:0/462578519 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7faa200b2880 0x7faa200b4d40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:43.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:43.409+0000 7faa3a57b700 1 --2- 172.21.15.27:0/462578519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa34101f70 0x7faa3419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:43.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:43.409+0000 7faa3a57b700 1 --2- 172.21.15.27:0/462578519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa34102930 0x7faa3419fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:43.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:43.409+0000 7faa3a57b700 1 -- 172.21.15.27:0/462578519 >> 172.21.15.27:0/462578519 conn(0x7faa340776b0 msgr2=0x7faa34100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:43.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:43.409+0000 7faa3a57b700 1 -- 172.21.15.27:0/462578519 shutdown_connections 2024-05-21T03:20:43.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:43.409+0000 7faa3a57b700 1 -- 172.21.15.27:0/462578519 wait complete. 2024-05-21T03:20:43.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:43 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/462578519' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-21T03:20:43.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:43 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/462578519' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-21T03:20:43.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:43 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/462578519' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-21T03:20:44.111 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-21T03:20:44.111 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-05-21T03:20:44.367 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:20:44.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:44 smithi027 ceph-mon[32179]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-21T03:20:44.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:44 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/462578519' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-21T03:20:44.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:44 smithi027 ceph-mon[32179]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T03:20:44.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:44 smithi039 ceph-mon[36429]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-21T03:20:44.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:44 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/462578519' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-21T03:20:44.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:44 smithi039 ceph-mon[36429]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T03:20:44.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:44 smithi165 ceph-mon[36269]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-21T03:20:44.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:44 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/462578519' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-21T03:20:44.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:44 smithi165 ceph-mon[36269]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T03:20:46.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.055+0000 7febb5024700 1 -- 172.21.15.27:0/397968341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febb0107000 msgr2=0x7febb010eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:46.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.055+0000 7febb5024700 1 --2- 172.21.15.27:0/397968341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febb0107000 0x7febb010eb40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7feba800b0a0 tx=0x7feba800b3b0 comp rx=0 tx=0).stop 2024-05-21T03:20:46.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.055+0000 7febb5024700 1 -- 172.21.15.27:0/397968341 shutdown_connections 2024-05-21T03:20:46.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.055+0000 7febb5024700 1 --2- 172.21.15.27:0/397968341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febb0107000 0x7febb010eb40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:46.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.055+0000 7febb5024700 1 --2- 172.21.15.27:0/397968341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7febb0102160 0x7febb0102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:46.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.055+0000 7febb5024700 1 --2- 172.21.15.27:0/397968341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febb0102b10 0x7febb01068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:46.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.055+0000 7febb5024700 1 -- 172.21.15.27:0/397968341 >> 172.21.15.27:0/397968341 conn(0x7febb0077780 msgr2=0x7febb0077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:46.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.055+0000 7febb5024700 1 -- 172.21.15.27:0/397968341 shutdown_connections 2024-05-21T03:20:46.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.055+0000 7febb5024700 1 -- 172.21.15.27:0/397968341 wait complete. 2024-05-21T03:20:46.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.055+0000 7febb5024700 1 Processor -- start 2024-05-21T03:20:46.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.056+0000 7febb5024700 1 -- start start 2024-05-21T03:20:46.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.056+0000 7febb5024700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febb0102160 0x7febb019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:46.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.056+0000 7febb5024700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febb0102b10 0x7febb019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:46.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.056+0000 7febb5024700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7febb0107000 0x7febb0199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:46.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.056+0000 7febb5024700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7febb01126c0 con 0x7febb0102b10 2024-05-21T03:20:46.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.056+0000 7febb5024700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7febb0112540 con 0x7febb0107000 2024-05-21T03:20:46.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.056+0000 7febb5024700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7febb0112840 con 0x7febb0102160 2024-05-21T03:20:46.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.056+0000 7febae59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febb0102b10 0x7febb019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:46.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.056+0000 7febae59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febb0102b10 0x7febb019fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35008/0 (socket says 172.21.15.27:35008) 2024-05-21T03:20:46.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.056+0000 7febae59c700 1 -- 172.21.15.27:0/3751012365 learned_addr learned my addr 172.21.15.27:0/3751012365 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:20:46.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.056+0000 7febaf59e700 1 --2- 172.21.15.27:0/3751012365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7febb0107000 0x7febb0199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:46.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.056+0000 7febaed9d700 1 --2- 172.21.15.27:0/3751012365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febb0102160 0x7febb019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:46.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.056+0000 7febae59c700 1 -- 172.21.15.27:0/3751012365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febb0102160 msgr2=0x7febb019f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:46.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.056+0000 7febae59c700 1 --2- 172.21.15.27:0/3751012365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febb0102160 0x7febb019f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:46.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.056+0000 7febae59c700 1 -- 172.21.15.27:0/3751012365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7febb0107000 msgr2=0x7febb0199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:46.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.056+0000 7febae59c700 1 --2- 172.21.15.27:0/3751012365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7febb0107000 0x7febb0199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:46.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.056+0000 7febae59c700 1 -- 172.21.15.27:0/3751012365 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb019a070 con 0x7febb0102b10 2024-05-21T03:20:46.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.057+0000 7febae59c700 1 --2- 172.21.15.27:0/3751012365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febb0102b10 0x7febb019fc90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7feba400c8c0 tx=0x7feba400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:46.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.057+0000 7feb97fff700 1 -- 172.21.15.27:0/3751012365 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba4004220 con 0x7febb0102b10 2024-05-21T03:20:46.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.057+0000 7feb97fff700 1 -- 172.21.15.27:0/3751012365 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feba40043c0 con 0x7febb0102b10 2024-05-21T03:20:46.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.057+0000 7febb5024700 1 -- 172.21.15.27:0/3751012365 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febb019a360 con 0x7febb0102b10 2024-05-21T03:20:46.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.057+0000 7feb97fff700 1 -- 172.21.15.27:0/3751012365 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba4003680 con 0x7febb0102b10 2024-05-21T03:20:46.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.057+0000 7febb5024700 1 -- 172.21.15.27:0/3751012365 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febb0103830 con 0x7febb0102b10 2024-05-21T03:20:46.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.058+0000 7febb5024700 1 -- 172.21.15.27:0/3751012365 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febb006a230 con 0x7febb0102b10 2024-05-21T03:20:46.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.059+0000 7feb97fff700 1 -- 172.21.15.27:0/3751012365 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7feba4004750 con 0x7febb0102b10 2024-05-21T03:20:46.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.060+0000 7feb97fff700 1 --2- 172.21.15.27:0/3751012365 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7feb980b2880 0x7feb980b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:46.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.060+0000 7febaed9d700 1 --2- 172.21.15.27:0/3751012365 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7feb980b2880 0x7feb980b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:46.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.060+0000 7feb97fff700 1 -- 172.21.15.27:0/3751012365 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7feba40e1fa0 con 0x7febb0102b10 2024-05-21T03:20:46.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.060+0000 7febaed9d700 1 --2- 172.21.15.27:0/3751012365 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7feb980b2880 0x7feb980b4d40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7febb019aa50 tx=0x7feba000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:46.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.063+0000 7feb97fff700 1 -- 172.21.15.27:0/3751012365 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feba4014030 con 0x7febb0102b10 2024-05-21T03:20:46.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.325+0000 7febb5024700 1 -- 172.21.15.27:0/3751012365 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7febb002e560 con 0x7febb0102b10 2024-05-21T03:20:46.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.411+0000 7feb97fff700 1 -- 172.21.15.27:0/3751012365 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7feba40a9e10 con 0x7febb0102b10 2024-05-21T03:20:46.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.413+0000 7febb5024700 1 -- 172.21.15.27:0/3751012365 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7feb980b2880 msgr2=0x7feb980b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:46.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.413+0000 7febb5024700 1 --2- 172.21.15.27:0/3751012365 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7feb980b2880 0x7feb980b4d40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7febb019aa50 tx=0x7feba000a380 comp rx=0 tx=0).stop 2024-05-21T03:20:46.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.413+0000 7febb5024700 1 -- 172.21.15.27:0/3751012365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febb0102b10 msgr2=0x7febb019fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:46.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.413+0000 7febb5024700 1 --2- 172.21.15.27:0/3751012365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febb0102b10 0x7febb019fc90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7feba400c8c0 tx=0x7feba400cbd0 comp rx=0 tx=0).stop 2024-05-21T03:20:46.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.413+0000 7febb5024700 1 -- 172.21.15.27:0/3751012365 shutdown_connections 2024-05-21T03:20:46.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.413+0000 7febb5024700 1 --2- 172.21.15.27:0/3751012365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7febb0107000 0x7febb0199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:46.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.413+0000 7febb5024700 1 --2- 172.21.15.27:0/3751012365 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7feb980b2880 0x7feb980b4d40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:46.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.413+0000 7febb5024700 1 --2- 172.21.15.27:0/3751012365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febb0102160 0x7febb019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:46.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.413+0000 7febb5024700 1 --2- 172.21.15.27:0/3751012365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febb0102b10 0x7febb019fc90 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:46.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.413+0000 7febb5024700 1 -- 172.21.15.27:0/3751012365 >> 172.21.15.27:0/3751012365 conn(0x7febb0077780 msgr2=0x7febb010f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:46.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.413+0000 7febb5024700 1 -- 172.21.15.27:0/3751012365 shutdown_connections 2024-05-21T03:20:46.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:46.413+0000 7febb5024700 1 -- 172.21.15.27:0/3751012365 wait complete. 2024-05-21T03:20:46.624 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:46 smithi027 ceph-mon[32179]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-05-21T03:20:46.624 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:46 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3751012365' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-21T03:20:46.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:46 smithi039 ceph-mon[36429]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-05-21T03:20:46.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:46 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/3751012365' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-21T03:20:46.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:46 smithi165 ceph-mon[36269]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-05-21T03:20:46.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:46 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/3751012365' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-21T03:20:47.030 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-21T03:20:47.281 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:20:47.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:47 smithi027 ceph-mon[32179]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-21T03:20:47.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:47 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3751012365' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-21T03:20:47.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:47 smithi027 ceph-mon[32179]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-21T03:20:47.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:47 smithi027 ceph-mon[32179]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T03:20:47.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:47 smithi027 ceph-mon[32179]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-21T03:20:47.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:47 smithi027 ceph-mon[32179]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-21T03:20:47.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:47 smithi039 ceph-mon[36429]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-21T03:20:47.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:47 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/3751012365' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-21T03:20:47.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:47 smithi039 ceph-mon[36429]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-21T03:20:47.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:47 smithi039 ceph-mon[36429]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T03:20:47.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:47 smithi039 ceph-mon[36429]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-21T03:20:47.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:47 smithi039 ceph-mon[36429]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-21T03:20:47.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:47 smithi165 ceph-mon[36269]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-21T03:20:47.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:47 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/3751012365' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-21T03:20:47.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:47 smithi165 ceph-mon[36269]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-21T03:20:47.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:47 smithi165 ceph-mon[36269]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T03:20:47.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:47 smithi165 ceph-mon[36269]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-21T03:20:47.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:47 smithi165 ceph-mon[36269]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-21T03:20:48.583 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:48 smithi027 ceph-mon[32179]: pgmap v297: 97 pgs: 97 active+clean; 580 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 4 op/s 2024-05-21T03:20:48.583 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:48 smithi027 ceph-mon[32179]: mds.? [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] up:active 2024-05-21T03:20:48.583 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:48 smithi027 ceph-mon[32179]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-21T03:20:48.583 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:48 smithi027 ceph-mon[32179]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-21T03:20:48.583 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:48 smithi027 ceph-mon[32179]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-05-21T03:20:48.584 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:48 smithi027 ceph-mon[32179]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-21T03:20:48.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:48 smithi039 ceph-mon[36429]: pgmap v297: 97 pgs: 97 active+clean; 580 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 4 op/s 2024-05-21T03:20:48.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:48 smithi039 ceph-mon[36429]: mds.? [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] up:active 2024-05-21T03:20:48.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:48 smithi039 ceph-mon[36429]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-21T03:20:48.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:48 smithi039 ceph-mon[36429]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-21T03:20:48.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:48 smithi039 ceph-mon[36429]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-05-21T03:20:48.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:48 smithi039 ceph-mon[36429]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-21T03:20:48.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:48 smithi165 ceph-mon[36269]: pgmap v297: 97 pgs: 97 active+clean; 580 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 4 op/s 2024-05-21T03:20:48.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:48 smithi165 ceph-mon[36269]: mds.? [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] up:active 2024-05-21T03:20:48.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:48 smithi165 ceph-mon[36269]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-21T03:20:48.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:48 smithi165 ceph-mon[36269]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-21T03:20:48.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:48 smithi165 ceph-mon[36269]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-05-21T03:20:48.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:48 smithi165 ceph-mon[36269]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-21T03:20:48.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.860+0000 7f527272d700 1 -- 172.21.15.27:0/1040320376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f526c1041a0 msgr2=0x7f526c1081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:48.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.860+0000 7f527272d700 1 --2- 172.21.15.27:0/1040320376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f526c1041a0 0x7f526c1081c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f52600099c0 tx=0x7f5260009cd0 comp rx=0 tx=0).stop 2024-05-21T03:20:48.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.860+0000 7f527272d700 1 -- 172.21.15.27:0/1040320376 shutdown_connections 2024-05-21T03:20:48.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.860+0000 7f527272d700 1 --2- 172.21.15.27:0/1040320376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f526c1088f0 0x7f526c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:48.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.860+0000 7f527272d700 1 --2- 172.21.15.27:0/1040320376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f526c1037f0 0x7f526c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:48.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.860+0000 7f527272d700 1 --2- 172.21.15.27:0/1040320376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f526c1041a0 0x7f526c1081c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:48.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.860+0000 7f527272d700 1 -- 172.21.15.27:0/1040320376 >> 172.21.15.27:0/1040320376 conn(0x7f526c07a6a0 msgr2=0x7f526c07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:48.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.860+0000 7f527272d700 1 -- 172.21.15.27:0/1040320376 shutdown_connections 2024-05-21T03:20:48.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.860+0000 7f527272d700 1 -- 172.21.15.27:0/1040320376 wait complete. 2024-05-21T03:20:48.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.861+0000 7f527272d700 1 Processor -- start 2024-05-21T03:20:48.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.861+0000 7f527272d700 1 -- start start 2024-05-21T03:20:48.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.861+0000 7f527272d700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f526c1037f0 0x7f526c19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:48.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.861+0000 7f527272d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f526c1041a0 0x7f526c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:48.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.861+0000 7f527272d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f526c1088f0 0x7f526c199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:48.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.861+0000 7f527272d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f526c1136d0 con 0x7f526c1041a0 2024-05-21T03:20:48.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.861+0000 7f527272d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f526c113550 con 0x7f526c1088f0 2024-05-21T03:20:48.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.861+0000 7f527272d700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f526c113850 con 0x7f526c1037f0 2024-05-21T03:20:48.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.861+0000 7f526b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f526c1041a0 0x7f526c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:48.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.862+0000 7f526b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f526c1041a0 0x7f526c19fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35016/0 (socket says 172.21.15.27:35016) 2024-05-21T03:20:48.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.862+0000 7f526b7fe700 1 -- 172.21.15.27:0/2872226932 learned_addr learned my addr 172.21.15.27:0/2872226932 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:20:48.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.862+0000 7f5270cca700 1 --2- 172.21.15.27:0/2872226932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f526c1088f0 0x7f526c199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:48.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.862+0000 7f526bfff700 1 --2- 172.21.15.27:0/2872226932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f526c1037f0 0x7f526c19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:48.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.862+0000 7f526b7fe700 1 -- 172.21.15.27:0/2872226932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f526c1037f0 msgr2=0x7f526c19f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:48.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.862+0000 7f526b7fe700 1 --2- 172.21.15.27:0/2872226932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f526c1037f0 0x7f526c19f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:48.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.862+0000 7f526b7fe700 1 -- 172.21.15.27:0/2872226932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f526c1088f0 msgr2=0x7f526c199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:48.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.862+0000 7f526b7fe700 1 --2- 172.21.15.27:0/2872226932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f526c1088f0 0x7f526c199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:48.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.862+0000 7f526b7fe700 1 -- 172.21.15.27:0/2872226932 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f526c19a0b0 con 0x7f526c1041a0 2024-05-21T03:20:48.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.862+0000 7f526b7fe700 1 --2- 172.21.15.27:0/2872226932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f526c1041a0 0x7f526c19fcd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f526000b580 tx=0x7f52600047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:48.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.862+0000 7f52697fa700 1 -- 172.21.15.27:0/2872226932 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5260027070 con 0x7f526c1041a0 2024-05-21T03:20:48.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.862+0000 7f527272d700 1 -- 172.21.15.27:0/2872226932 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f526c19a340 con 0x7f526c1041a0 2024-05-21T03:20:48.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.862+0000 7f52697fa700 1 -- 172.21.15.27:0/2872226932 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5260004020 con 0x7f526c1041a0 2024-05-21T03:20:48.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.862+0000 7f52697fa700 1 -- 172.21.15.27:0/2872226932 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5260022e10 con 0x7f526c1041a0 2024-05-21T03:20:48.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.862+0000 7f527272d700 1 -- 172.21.15.27:0/2872226932 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f526c1a6660 con 0x7f526c1041a0 2024-05-21T03:20:48.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.864+0000 7f527272d700 1 -- 172.21.15.27:0/2872226932 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f526c06a230 con 0x7f526c1041a0 2024-05-21T03:20:48.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.865+0000 7f52697fa700 1 -- 172.21.15.27:0/2872226932 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f526002a070 con 0x7f526c1041a0 2024-05-21T03:20:48.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.865+0000 7f52697fa700 1 --2- 172.21.15.27:0/2872226932 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f52540b2880 0x7f52540b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:48.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.866+0000 7f526bfff700 1 --2- 172.21.15.27:0/2872226932 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f52540b2880 0x7f52540b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:48.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.866+0000 7f52697fa700 1 -- 172.21.15.27:0/2872226932 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f52600e2db0 con 0x7f526c1041a0 2024-05-21T03:20:48.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.866+0000 7f526bfff700 1 --2- 172.21.15.27:0/2872226932 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f52540b2880 0x7f52540b4d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f526c19aa90 tx=0x7f525c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:48.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:48.870+0000 7f52697fa700 1 -- 172.21.15.27:0/2872226932 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5260013070 con 0x7f526c1041a0 2024-05-21T03:20:49.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:49.132+0000 7f527272d700 1 -- 172.21.15.27:0/2872226932 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f526c02e560 con 0x7f526c1041a0 2024-05-21T03:20:49.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:49.454+0000 7f52697fa700 1 -- 172.21.15.27:0/2872226932 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7f52600aacb0 con 0x7f526c1041a0 2024-05-21T03:20:49.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:49.456+0000 7f527272d700 1 -- 172.21.15.27:0/2872226932 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f52540b2880 msgr2=0x7f52540b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:49.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:49.456+0000 7f527272d700 1 --2- 172.21.15.27:0/2872226932 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f52540b2880 0x7f52540b4d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f526c19aa90 tx=0x7f525c00a380 comp rx=0 tx=0).stop 2024-05-21T03:20:49.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:49.456+0000 7f527272d700 1 -- 172.21.15.27:0/2872226932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f526c1041a0 msgr2=0x7f526c19fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:49.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:49.456+0000 7f527272d700 1 --2- 172.21.15.27:0/2872226932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f526c1041a0 0x7f526c19fcd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f526000b580 tx=0x7f52600047b0 comp rx=0 tx=0).stop 2024-05-21T03:20:49.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:49.457+0000 7f527272d700 1 -- 172.21.15.27:0/2872226932 shutdown_connections 2024-05-21T03:20:49.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:49.457+0000 7f527272d700 1 --2- 172.21.15.27:0/2872226932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f526c1088f0 0x7f526c199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:49.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:49.457+0000 7f527272d700 1 --2- 172.21.15.27:0/2872226932 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f52540b2880 0x7f52540b4d40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:49.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:49.457+0000 7f527272d700 1 --2- 172.21.15.27:0/2872226932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f526c1037f0 0x7f526c19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:49.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:49.457+0000 7f527272d700 1 --2- 172.21.15.27:0/2872226932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f526c1041a0 0x7f526c19fcd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:49.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:49.457+0000 7f527272d700 1 -- 172.21.15.27:0/2872226932 >> 172.21.15.27:0/2872226932 conn(0x7f526c07a6a0 msgr2=0x7f526c1015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:49.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:49.457+0000 7f527272d700 1 -- 172.21.15.27:0/2872226932 shutdown_connections 2024-05-21T03:20:49.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:49.457+0000 7f527272d700 1 -- 172.21.15.27:0/2872226932 wait complete. 2024-05-21T03:20:49.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:49 smithi039 ceph-mon[36429]: mds.? [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] up:active 2024-05-21T03:20:49.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:49 smithi039 ceph-mon[36429]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-21T03:20:49.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:49 smithi039 ceph-mon[36429]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-05-21T03:20:49.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:49 smithi039 ceph-mon[36429]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-05-21T03:20:49.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:49 smithi039 ceph-mon[36429]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-05-21T03:20:49.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:49 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/2872226932' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-21T03:20:49.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:49 smithi165 ceph-mon[36269]: mds.? [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] up:active 2024-05-21T03:20:49.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:49 smithi165 ceph-mon[36269]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-21T03:20:49.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:49 smithi165 ceph-mon[36269]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-05-21T03:20:49.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:49 smithi165 ceph-mon[36269]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-05-21T03:20:49.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:49 smithi165 ceph-mon[36269]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-05-21T03:20:49.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:49 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/2872226932' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-21T03:20:49.820 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:49 smithi027 ceph-mon[32179]: mds.? [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] up:active 2024-05-21T03:20:49.820 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:49 smithi027 ceph-mon[32179]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-21T03:20:49.821 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:49 smithi027 ceph-mon[32179]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-05-21T03:20:49.821 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:49 smithi027 ceph-mon[32179]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-05-21T03:20:49.821 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:49 smithi027 ceph-mon[32179]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-05-21T03:20:49.821 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:49 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2872226932' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-21T03:20:50.247 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-21T03:20:50.506 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:20:50.622 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:50 smithi027 ceph-mon[32179]: pgmap v298: 97 pgs: 97 active+clean; 580 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-05-21T03:20:50.622 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:50 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2872226932' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-21T03:20:50.622 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:50 smithi027 ceph-mon[32179]: mds.? [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] up:active 2024-05-21T03:20:50.622 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:50 smithi027 ceph-mon[32179]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-21T03:20:50.622 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:50 smithi027 ceph-mon[32179]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-21T03:20:50.622 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:50 smithi027 ceph-mon[32179]: Cluster is now healthy 2024-05-21T03:20:50.622 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:50 smithi027 ceph-mon[32179]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-05-21T03:20:50.622 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:50 smithi027 ceph-mon[32179]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-21T03:20:50.623 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:50 smithi027 ceph-mon[32179]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-05-21T03:20:50.769 INFO:journalctl@ceph.mds.b.smithi039.stdout:May 21 03:20:50 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-b[66446]: 2024-05-21T03:20:50.591+0000 7fbbf5f0f700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-05-21T03:20:50.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:50 smithi039 ceph-mon[36429]: pgmap v298: 97 pgs: 97 active+clean; 580 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-05-21T03:20:50.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:50 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/2872226932' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-21T03:20:50.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:50 smithi039 ceph-mon[36429]: mds.? [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] up:active 2024-05-21T03:20:50.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:50 smithi039 ceph-mon[36429]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-21T03:20:50.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:50 smithi039 ceph-mon[36429]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-21T03:20:50.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:50 smithi039 ceph-mon[36429]: Cluster is now healthy 2024-05-21T03:20:50.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:50 smithi039 ceph-mon[36429]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-05-21T03:20:50.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:50 smithi039 ceph-mon[36429]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-21T03:20:50.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:50 smithi039 ceph-mon[36429]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-05-21T03:20:50.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:50 smithi165 ceph-mon[36269]: pgmap v298: 97 pgs: 97 active+clean; 580 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-05-21T03:20:50.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:50 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/2872226932' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-21T03:20:50.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:50 smithi165 ceph-mon[36269]: mds.? [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] up:active 2024-05-21T03:20:50.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:50 smithi165 ceph-mon[36269]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-21T03:20:50.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:50 smithi165 ceph-mon[36269]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-21T03:20:50.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:50 smithi165 ceph-mon[36269]: Cluster is now healthy 2024-05-21T03:20:50.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:50 smithi165 ceph-mon[36269]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-05-21T03:20:50.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:50 smithi165 ceph-mon[36269]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-21T03:20:50.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:50 smithi165 ceph-mon[36269]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-05-21T03:20:51.732 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:51 smithi027 ceph-mon[32179]: mds.? [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] up:active 2024-05-21T03:20:51.732 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:51 smithi027 ceph-mon[32179]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-21T03:20:51.732 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:51 smithi027 ceph-mon[32179]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:20:51.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:51 smithi039 ceph-mon[36429]: mds.? [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] up:active 2024-05-21T03:20:51.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:51 smithi039 ceph-mon[36429]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-21T03:20:51.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:51 smithi039 ceph-mon[36429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:20:51.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:51 smithi165 ceph-mon[36269]: mds.? [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] up:active 2024-05-21T03:20:51.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:51 smithi165 ceph-mon[36269]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-21T03:20:51.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:51 smithi165 ceph-mon[36269]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:20:52.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.194+0000 7fd96d2eb700 1 -- 172.21.15.27:0/2787601542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9681088f0 msgr2=0x7fd968110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:52.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.194+0000 7fd96d2eb700 1 --2- 172.21.15.27:0/2787601542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9681088f0 0x7fd968110470 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd96000b0d0 tx=0x7fd96000b3e0 comp rx=0 tx=0).stop 2024-05-21T03:20:52.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.194+0000 7fd96d2eb700 1 -- 172.21.15.27:0/2787601542 shutdown_connections 2024-05-21T03:20:52.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.194+0000 7fd96d2eb700 1 --2- 172.21.15.27:0/2787601542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9681088f0 0x7fd968110470 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:52.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.194+0000 7fd96d2eb700 1 --2- 172.21.15.27:0/2787601542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9681037f0 0x7fd968103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:52.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.194+0000 7fd96d2eb700 1 --2- 172.21.15.27:0/2787601542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9681041a0 0x7fd9681081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:52.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.194+0000 7fd96d2eb700 1 -- 172.21.15.27:0/2787601542 >> 172.21.15.27:0/2787601542 conn(0x7fd96807a6a0 msgr2=0x7fd96807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:52.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.194+0000 7fd96d2eb700 1 -- 172.21.15.27:0/2787601542 shutdown_connections 2024-05-21T03:20:52.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.194+0000 7fd96d2eb700 1 -- 172.21.15.27:0/2787601542 wait complete. 2024-05-21T03:20:52.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.195+0000 7fd96d2eb700 1 Processor -- start 2024-05-21T03:20:52.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.195+0000 7fd96d2eb700 1 -- start start 2024-05-21T03:20:52.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.195+0000 7fd96d2eb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9681037f0 0x7fd96819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:52.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.195+0000 7fd96d2eb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9681041a0 0x7fd96819fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:52.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.195+0000 7fd96d2eb700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9681088f0 0x7fd9681998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:52.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.195+0000 7fd96d2eb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd9681136e0 con 0x7fd9681037f0 2024-05-21T03:20:52.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.195+0000 7fd96d2eb700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd968113560 con 0x7fd9681041a0 2024-05-21T03:20:52.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.195+0000 7fd96d2eb700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd968113860 con 0x7fd9681088f0 2024-05-21T03:20:52.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.195+0000 7fd966ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9681037f0 0x7fd96819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:52.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.195+0000 7fd966ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9681037f0 0x7fd96819f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35036/0 (socket says 172.21.15.27:35036) 2024-05-21T03:20:52.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.195+0000 7fd966ffd700 1 -- 172.21.15.27:0/2062017270 learned_addr learned my addr 172.21.15.27:0/2062017270 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:20:52.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.196+0000 7fd966ffd700 1 -- 172.21.15.27:0/2062017270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9681088f0 msgr2=0x7fd9681998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:52.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.196+0000 7fd9667fc700 1 --2- 172.21.15.27:0/2062017270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9681041a0 0x7fd96819fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:52.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.196+0000 7fd9677fe700 1 --2- 172.21.15.27:0/2062017270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9681088f0 0x7fd9681998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:52.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.196+0000 7fd966ffd700 1 --2- 172.21.15.27:0/2062017270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9681088f0 0x7fd9681998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:52.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.196+0000 7fd966ffd700 1 -- 172.21.15.27:0/2062017270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9681041a0 msgr2=0x7fd96819fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:52.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.196+0000 7fd966ffd700 1 --2- 172.21.15.27:0/2062017270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9681041a0 0x7fd96819fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:52.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.196+0000 7fd966ffd700 1 -- 172.21.15.27:0/2062017270 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd96819a020 con 0x7fd9681037f0 2024-05-21T03:20:52.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.196+0000 7fd9677fe700 1 --2- 172.21.15.27:0/2062017270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9681088f0 0x7fd9681998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:20:52.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.196+0000 7fd966ffd700 1 --2- 172.21.15.27:0/2062017270 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9681037f0 0x7fd96819f700 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd95800c970 tx=0x7fd95800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:52.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.196+0000 7fd94ffff700 1 -- 172.21.15.27:0/2062017270 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd958007aa0 con 0x7fd9681037f0 2024-05-21T03:20:52.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.196+0000 7fd94ffff700 1 -- 172.21.15.27:0/2062017270 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd958004ce0 con 0x7fd9681037f0 2024-05-21T03:20:52.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.196+0000 7fd96d2eb700 1 -- 172.21.15.27:0/2062017270 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd96819a310 con 0x7fd9681037f0 2024-05-21T03:20:52.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.196+0000 7fd94ffff700 1 -- 172.21.15.27:0/2062017270 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9580055c0 con 0x7fd9681037f0 2024-05-21T03:20:52.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.196+0000 7fd96d2eb700 1 -- 172.21.15.27:0/2062017270 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9681a65d0 con 0x7fd9681037f0 2024-05-21T03:20:52.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.197+0000 7fd96d2eb700 1 -- 172.21.15.27:0/2062017270 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd96806a230 con 0x7fd9681037f0 2024-05-21T03:20:52.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.199+0000 7fd94ffff700 1 -- 172.21.15.27:0/2062017270 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd958007c40 con 0x7fd9681037f0 2024-05-21T03:20:52.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.200+0000 7fd94ffff700 1 --2- 172.21.15.27:0/2062017270 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd9500b2880 0x7fd9500b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:52.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.200+0000 7fd9667fc700 1 --2- 172.21.15.27:0/2062017270 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd9500b2880 0x7fd9500b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:52.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.200+0000 7fd94ffff700 1 -- 172.21.15.27:0/2062017270 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fd9580e2200 con 0x7fd9681037f0 2024-05-21T03:20:52.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.200+0000 7fd9667fc700 1 --2- 172.21.15.27:0/2062017270 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd9500b2880 0x7fd9500b4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd9681030c0 tx=0x7fd95c005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:52.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.203+0000 7fd94ffff700 1 -- 172.21.15.27:0/2062017270 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd9580aa190 con 0x7fd9681037f0 2024-05-21T03:20:52.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.467+0000 7fd96d2eb700 1 -- 172.21.15.27:0/2062017270 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fd96802e560 con 0x7fd9681037f0 2024-05-21T03:20:52.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.482+0000 7fd94ffff700 1 -- 172.21.15.27:0/2062017270 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v28) ==== 115+0+0 (secure 0 0 0) 0x7fd9580a5020 con 0x7fd9681037f0 2024-05-21T03:20:52.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.484+0000 7fd96d2eb700 1 -- 172.21.15.27:0/2062017270 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd9500b2880 msgr2=0x7fd9500b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:52.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.484+0000 7fd96d2eb700 1 --2- 172.21.15.27:0/2062017270 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd9500b2880 0x7fd9500b4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd9681030c0 tx=0x7fd95c005d50 comp rx=0 tx=0).stop 2024-05-21T03:20:52.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.484+0000 7fd96d2eb700 1 -- 172.21.15.27:0/2062017270 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9681037f0 msgr2=0x7fd96819f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:52.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.484+0000 7fd96d2eb700 1 --2- 172.21.15.27:0/2062017270 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9681037f0 0x7fd96819f700 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd95800c970 tx=0x7fd95800cd30 comp rx=0 tx=0).stop 2024-05-21T03:20:52.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.484+0000 7fd96d2eb700 1 -- 172.21.15.27:0/2062017270 shutdown_connections 2024-05-21T03:20:52.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.484+0000 7fd96d2eb700 1 --2- 172.21.15.27:0/2062017270 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd9500b2880 0x7fd9500b4d40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:52.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.484+0000 7fd96d2eb700 1 --2- 172.21.15.27:0/2062017270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9681088f0 0x7fd9681998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:52.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.484+0000 7fd96d2eb700 1 --2- 172.21.15.27:0/2062017270 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9681037f0 0x7fd96819f700 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:52.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.484+0000 7fd96d2eb700 1 --2- 172.21.15.27:0/2062017270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9681041a0 0x7fd96819fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:52.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.484+0000 7fd96d2eb700 1 -- 172.21.15.27:0/2062017270 >> 172.21.15.27:0/2062017270 conn(0x7fd96807a6a0 msgr2=0x7fd968101670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:52.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.485+0000 7fd96d2eb700 1 -- 172.21.15.27:0/2062017270 shutdown_connections 2024-05-21T03:20:52.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:52.485+0000 7fd96d2eb700 1 -- 172.21.15.27:0/2062017270 wait complete. 2024-05-21T03:20:52.743 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:52 smithi027 ceph-mon[32179]: pgmap v299: 97 pgs: 97 active+clean; 582 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-05-21T03:20:52.744 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:52 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2062017270' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-21T03:20:52.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:52 smithi039 ceph-mon[36429]: pgmap v299: 97 pgs: 97 active+clean; 582 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-05-21T03:20:52.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:52 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/2062017270' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-21T03:20:52.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:52 smithi165 ceph-mon[36269]: pgmap v299: 97 pgs: 97 active+clean; 582 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-05-21T03:20:52.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:52 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/2062017270' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-21T03:20:53.124 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-05-21T03:20:53.124 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-21T03:20:53.374 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:20:53.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:53 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/2062017270' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-21T03:20:53.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:53 smithi039 ceph-mon[36429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:20:53.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:53 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/2062017270' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-21T03:20:53.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:53 smithi165 ceph-mon[36269]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:20:53.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:53 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2062017270' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-21T03:20:53.817 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:53 smithi027 ceph-mon[32179]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:20:54.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:54 smithi039 ceph-mon[36429]: pgmap v300: 97 pgs: 97 active+clean; 584 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 4.0 KiB/s wr, 16 op/s 2024-05-21T03:20:54.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:54 smithi165 ceph-mon[36269]: pgmap v300: 97 pgs: 97 active+clean; 584 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 4.0 KiB/s wr, 16 op/s 2024-05-21T03:20:54.823 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:54 smithi027 ceph-mon[32179]: pgmap v300: 97 pgs: 97 active+clean; 584 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 4.0 KiB/s wr, 16 op/s 2024-05-21T03:20:55.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.089+0000 7f0c80d31700 1 -- 172.21.15.27:0/3091740934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c7c1088f0 msgr2=0x7f0c7c110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:55.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.089+0000 7f0c80d31700 1 --2- 172.21.15.27:0/3091740934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c7c1088f0 0x7f0c7c110470 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400b0d0 tx=0x7f0c7400b3e0 comp rx=0 tx=0).stop 2024-05-21T03:20:55.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.089+0000 7f0c80d31700 1 -- 172.21.15.27:0/3091740934 shutdown_connections 2024-05-21T03:20:55.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.089+0000 7f0c80d31700 1 --2- 172.21.15.27:0/3091740934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c7c1088f0 0x7f0c7c110470 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:55.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.089+0000 7f0c80d31700 1 --2- 172.21.15.27:0/3091740934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c7c1037f0 0x7f0c7c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:55.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.089+0000 7f0c80d31700 1 --2- 172.21.15.27:0/3091740934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c7c1041a0 0x7f0c7c1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:55.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.089+0000 7f0c80d31700 1 -- 172.21.15.27:0/3091740934 >> 172.21.15.27:0/3091740934 conn(0x7f0c7c07a6a0 msgr2=0x7f0c7c07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:55.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.089+0000 7f0c80d31700 1 -- 172.21.15.27:0/3091740934 shutdown_connections 2024-05-21T03:20:55.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.089+0000 7f0c80d31700 1 -- 172.21.15.27:0/3091740934 wait complete. 2024-05-21T03:20:55.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.090+0000 7f0c80d31700 1 Processor -- start 2024-05-21T03:20:55.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.090+0000 7f0c80d31700 1 -- start start 2024-05-21T03:20:55.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.090+0000 7f0c80d31700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c7c1037f0 0x7f0c7c075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:55.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.090+0000 7f0c80d31700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c7c1041a0 0x7f0c7c075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:55.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.090+0000 7f0c80d31700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c7c1088f0 0x7f0c7c0762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:55.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.090+0000 7f0c80d31700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7c1136c0 con 0x7f0c7c1037f0 2024-05-21T03:20:55.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.090+0000 7f0c80d31700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7c113540 con 0x7f0c7c1088f0 2024-05-21T03:20:55.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.090+0000 7f0c80d31700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7c113840 con 0x7f0c7c1041a0 2024-05-21T03:20:55.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.090+0000 7f0c7a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c7c1037f0 0x7f0c7c075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:55.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.090+0000 7f0c7a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c7c1037f0 0x7f0c7c075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36866/0 (socket says 172.21.15.27:36866) 2024-05-21T03:20:55.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.090+0000 7f0c7a59c700 1 -- 172.21.15.27:0/48511095 learned_addr learned my addr 172.21.15.27:0/48511095 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:20:55.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.090+0000 7f0c79d9b700 1 --2- 172.21.15.27:0/48511095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c7c1041a0 0x7f0c7c075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:55.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.091+0000 7f0c7ad9d700 1 --2- 172.21.15.27:0/48511095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c7c1088f0 0x7f0c7c0762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:55.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.091+0000 7f0c7a59c700 1 -- 172.21.15.27:0/48511095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c7c1041a0 msgr2=0x7f0c7c075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:55.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.091+0000 7f0c7a59c700 1 --2- 172.21.15.27:0/48511095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c7c1041a0 0x7f0c7c075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:55.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.091+0000 7f0c7a59c700 1 -- 172.21.15.27:0/48511095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c7c1088f0 msgr2=0x7f0c7c0762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:55.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.091+0000 7f0c7a59c700 1 --2- 172.21.15.27:0/48511095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c7c1088f0 0x7f0c7c0762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:55.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.091+0000 7f0c7a59c700 1 -- 172.21.15.27:0/48511095 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c7c1a8070 con 0x7f0c7c1037f0 2024-05-21T03:20:55.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.091+0000 7f0c7ad9d700 1 --2- 172.21.15.27:0/48511095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c7c1088f0 0x7f0c7c0762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:20:55.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.091+0000 7f0c7a59c700 1 --2- 172.21.15.27:0/48511095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c7c1037f0 0x7f0c7c075840 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c00c9b0 tx=0x7f0c6c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:55.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.091+0000 7f0c6b7fe700 1 -- 172.21.15.27:0/48511095 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c007b00 con 0x7f0c7c1037f0 2024-05-21T03:20:55.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.091+0000 7f0c6b7fe700 1 -- 172.21.15.27:0/48511095 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c6c007ca0 con 0x7f0c7c1037f0 2024-05-21T03:20:55.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.091+0000 7f0c80d31700 1 -- 172.21.15.27:0/48511095 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c7c1a8360 con 0x7f0c7c1037f0 2024-05-21T03:20:55.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.091+0000 7f0c6b7fe700 1 -- 172.21.15.27:0/48511095 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c018660 con 0x7f0c7c1037f0 2024-05-21T03:20:55.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.091+0000 7f0c80d31700 1 -- 172.21.15.27:0/48511095 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c7c1a88a0 con 0x7f0c7c1037f0 2024-05-21T03:20:55.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.092+0000 7f0c80d31700 1 -- 172.21.15.27:0/48511095 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c7c06a230 con 0x7f0c7c1037f0 2024-05-21T03:20:55.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.094+0000 7f0c6b7fe700 1 -- 172.21.15.27:0/48511095 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0c6c00f450 con 0x7f0c7c1037f0 2024-05-21T03:20:55.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.094+0000 7f0c6b7fe700 1 --2- 172.21.15.27:0/48511095 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f0c640b2880 0x7f0c640b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:55.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.095+0000 7f0c79d9b700 1 --2- 172.21.15.27:0/48511095 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f0c640b2880 0x7f0c640b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:55.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.095+0000 7f0c6b7fe700 1 -- 172.21.15.27:0/48511095 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f0c6c0e2350 con 0x7f0c7c1037f0 2024-05-21T03:20:55.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.095+0000 7f0c79d9b700 1 --2- 172.21.15.27:0/48511095 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f0c640b2880 0x7f0c640b4d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c108170 tx=0x7f0c700073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:55.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.098+0000 7f0c6b7fe700 1 -- 172.21.15.27:0/48511095 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0c6c014030 con 0x7f0c7c1037f0 2024-05-21T03:20:55.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.343+0000 7f0c80d31700 1 -- 172.21.15.27:0/48511095 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f0c7c063980 con 0x7f0c640b2880 2024-05-21T03:20:55.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.376+0000 7f0c6b7fe700 1 -- 172.21.15.27:0/48511095 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0c7c063980 con 0x7f0c640b2880 2024-05-21T03:20:55.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.378+0000 7f0c80d31700 1 -- 172.21.15.27:0/48511095 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f0c640b2880 msgr2=0x7f0c640b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:55.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.378+0000 7f0c80d31700 1 --2- 172.21.15.27:0/48511095 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f0c640b2880 0x7f0c640b4d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c108170 tx=0x7f0c700073d0 comp rx=0 tx=0).stop 2024-05-21T03:20:55.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.378+0000 7f0c80d31700 1 -- 172.21.15.27:0/48511095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c7c1037f0 msgr2=0x7f0c7c075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:55.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.378+0000 7f0c80d31700 1 --2- 172.21.15.27:0/48511095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c7c1037f0 0x7f0c7c075840 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c00c9b0 tx=0x7f0c6c00ccc0 comp rx=0 tx=0).stop 2024-05-21T03:20:55.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.378+0000 7f0c80d31700 1 -- 172.21.15.27:0/48511095 shutdown_connections 2024-05-21T03:20:55.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.378+0000 7f0c80d31700 1 --2- 172.21.15.27:0/48511095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c7c1088f0 0x7f0c7c0762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:55.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.378+0000 7f0c80d31700 1 --2- 172.21.15.27:0/48511095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c7c1037f0 0x7f0c7c075840 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:55.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.378+0000 7f0c80d31700 1 --2- 172.21.15.27:0/48511095 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f0c640b2880 0x7f0c640b4d40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:55.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.378+0000 7f0c80d31700 1 --2- 172.21.15.27:0/48511095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c7c1041a0 0x7f0c7c075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:55.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.378+0000 7f0c80d31700 1 -- 172.21.15.27:0/48511095 >> 172.21.15.27:0/48511095 conn(0x7f0c7c07a6a0 msgr2=0x7f0c7c101540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:55.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.378+0000 7f0c80d31700 1 -- 172.21.15.27:0/48511095 shutdown_connections 2024-05-21T03:20:55.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:55.378+0000 7f0c80d31700 1 -- 172.21.15.27:0/48511095 wait complete. 2024-05-21T03:20:55.677 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:20:55 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:20:55.348+0000 7f5ab5cf1700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T03:20:55.677 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:20:55 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:20:55.348+0000 7f5ab5cf1700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T03:20:55.677 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:20:55 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:20:55.348+0000 7f5ab5cf1700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T03:20:55.677 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:20:55 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:20:55.348+0000 7f5ab5cf1700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T03:20:55.677 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:20:55 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:20:55.348+0000 7f5ab5cf1700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T03:20:56.130 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-21T03:20:56.383 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:20:56.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:56 smithi039 ceph-mon[36429]: pgmap v301: 97 pgs: 97 active+clean; 584 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-21T03:20:56.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:56 smithi039 ceph-mon[36429]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:20:56.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:56 smithi039 ceph-mon[36429]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-21T03:20:56.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:56 smithi165 ceph-mon[36269]: pgmap v301: 97 pgs: 97 active+clean; 584 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-21T03:20:56.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:56 smithi165 ceph-mon[36269]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:20:56.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:56 smithi165 ceph-mon[36269]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-21T03:20:56.828 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:56 smithi027 ceph-mon[32179]: pgmap v301: 97 pgs: 97 active+clean; 584 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-21T03:20:56.829 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:56 smithi027 ceph-mon[32179]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:20:56.829 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:56 smithi027 ceph-mon[32179]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-21T03:20:58.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.147+0000 7f77391e5700 1 -- 172.21.15.27:0/694991089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77341095b0 msgr2=0x7f7734109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:58.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.148+0000 7f77391e5700 1 --2- 172.21.15.27:0/694991089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77341095b0 0x7f7734109990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7724009990 tx=0x7f7724009ca0 comp rx=0 tx=0).stop 2024-05-21T03:20:58.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.148+0000 7f77391e5700 1 -- 172.21.15.27:0/694991089 shutdown_connections 2024-05-21T03:20:58.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.148+0000 7f77391e5700 1 --2- 172.21.15.27:0/694991089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f773406bbe0 0x7f77341103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:58.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.148+0000 7f77391e5700 1 --2- 172.21.15.27:0/694991089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77341095b0 0x7f7734109990 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:58.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.148+0000 7f77391e5700 1 --2- 172.21.15.27:0/694991089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f773406b220 0x7f773406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:58.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.148+0000 7f77391e5700 1 -- 172.21.15.27:0/694991089 >> 172.21.15.27:0/694991089 conn(0x7f77340793f0 msgr2=0x7f7734079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:58.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.148+0000 7f77391e5700 1 -- 172.21.15.27:0/694991089 shutdown_connections 2024-05-21T03:20:58.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.148+0000 7f77391e5700 1 -- 172.21.15.27:0/694991089 wait complete. 2024-05-21T03:20:58.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.148+0000 7f77391e5700 1 Processor -- start 2024-05-21T03:20:58.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.149+0000 7f77391e5700 1 -- start start 2024-05-21T03:20:58.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.149+0000 7f77391e5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f773406b220 0x7f7734102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:58.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.149+0000 7f77391e5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f773406bbe0 0x7f7734102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:58.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.149+0000 7f77391e5700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77341095b0 0x7f7734102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:58.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.149+0000 7f77391e5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7734113740 con 0x7f773406b220 2024-05-21T03:20:58.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.149+0000 7f77391e5700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f77341135c0 con 0x7f773406bbe0 2024-05-21T03:20:58.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.149+0000 7f77391e5700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f77341138c0 con 0x7f77341095b0 2024-05-21T03:20:58.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.149+0000 7f7732d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f773406b220 0x7f7734102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:58.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.149+0000 7f7732d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f773406b220 0x7f7734102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36908/0 (socket says 172.21.15.27:36908) 2024-05-21T03:20:58.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.149+0000 7f7732d9d700 1 -- 172.21.15.27:0/1940579674 learned_addr learned my addr 172.21.15.27:0/1940579674 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:20:58.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.149+0000 7f773259c700 1 --2- 172.21.15.27:0/1940579674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f773406bbe0 0x7f7734102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:58.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.149+0000 7f773359e700 1 --2- 172.21.15.27:0/1940579674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77341095b0 0x7f7734102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:58.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.150+0000 7f7732d9d700 1 -- 172.21.15.27:0/1940579674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77341095b0 msgr2=0x7f7734102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:58.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.150+0000 7f7732d9d700 1 --2- 172.21.15.27:0/1940579674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77341095b0 0x7f7734102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:58.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.150+0000 7f7732d9d700 1 -- 172.21.15.27:0/1940579674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f773406bbe0 msgr2=0x7f7734102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:58.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.150+0000 7f7732d9d700 1 --2- 172.21.15.27:0/1940579674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f773406bbe0 0x7f7734102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:58.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.150+0000 7f7732d9d700 1 -- 172.21.15.27:0/1940579674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77341a8100 con 0x7f773406b220 2024-05-21T03:20:58.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.150+0000 7f773359e700 1 --2- 172.21.15.27:0/1940579674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77341095b0 0x7f7734102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:20:58.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.150+0000 7f7732d9d700 1 --2- 172.21.15.27:0/1940579674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f773406b220 0x7f7734102510 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7724000c00 tx=0x7f77240047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:58.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.150+0000 7f771bfff700 1 -- 172.21.15.27:0/1940579674 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7724027070 con 0x7f773406b220 2024-05-21T03:20:58.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.150+0000 7f77391e5700 1 -- 172.21.15.27:0/1940579674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77341a8390 con 0x7f773406b220 2024-05-21T03:20:58.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.150+0000 7f771bfff700 1 -- 172.21.15.27:0/1940579674 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77240040e0 con 0x7f773406b220 2024-05-21T03:20:58.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.150+0000 7f77391e5700 1 -- 172.21.15.27:0/1940579674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77341a8840 con 0x7f773406b220 2024-05-21T03:20:58.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.150+0000 7f771bfff700 1 -- 172.21.15.27:0/1940579674 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772400f460 con 0x7f773406b220 2024-05-21T03:20:58.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.151+0000 7f77391e5700 1 -- 172.21.15.27:0/1940579674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f773410ba90 con 0x7f773406b220 2024-05-21T03:20:58.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.152+0000 7f771bfff700 1 -- 172.21.15.27:0/1940579674 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f772402a070 con 0x7f773406b220 2024-05-21T03:20:58.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.153+0000 7f771bfff700 1 --2- 172.21.15.27:0/1940579674 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f771c0b2990 0x7f771c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:20:58.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.153+0000 7f773259c700 1 --2- 172.21.15.27:0/1940579674 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f771c0b2990 0x7f771c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:20:58.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.153+0000 7f771bfff700 1 -- 172.21.15.27:0/1940579674 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f77240e3580 con 0x7f773406b220 2024-05-21T03:20:58.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.154+0000 7f773259c700 1 --2- 172.21.15.27:0/1940579674 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f771c0b2990 0x7f771c0b4e50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f77340799f0 tx=0x7f772800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:20:58.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.157+0000 7f771bfff700 1 -- 172.21.15.27:0/1940579674 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7724031070 con 0x7f773406b220 2024-05-21T03:20:58.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.405+0000 7f77391e5700 1 -- 172.21.15.27:0/1940579674 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7734063980 con 0x7f771c0b2990 2024-05-21T03:20:58.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.438+0000 7f771bfff700 1 -- 172.21.15.27:0/1940579674 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7734063980 con 0x7f771c0b2990 2024-05-21T03:20:58.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.441+0000 7f77391e5700 1 -- 172.21.15.27:0/1940579674 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f771c0b2990 msgr2=0x7f771c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:58.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.441+0000 7f77391e5700 1 --2- 172.21.15.27:0/1940579674 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f771c0b2990 0x7f771c0b4e50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f77340799f0 tx=0x7f772800a380 comp rx=0 tx=0).stop 2024-05-21T03:20:58.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.441+0000 7f77391e5700 1 -- 172.21.15.27:0/1940579674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f773406b220 msgr2=0x7f7734102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:20:58.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.441+0000 7f77391e5700 1 --2- 172.21.15.27:0/1940579674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f773406b220 0x7f7734102510 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7724000c00 tx=0x7f77240047b0 comp rx=0 tx=0).stop 2024-05-21T03:20:58.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.441+0000 7f77391e5700 1 -- 172.21.15.27:0/1940579674 shutdown_connections 2024-05-21T03:20:58.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.441+0000 7f77391e5700 1 --2- 172.21.15.27:0/1940579674 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f771c0b2990 0x7f771c0b4e50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:58.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.441+0000 7f77391e5700 1 --2- 172.21.15.27:0/1940579674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77341095b0 0x7f7734102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:58.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.441+0000 7f77391e5700 1 --2- 172.21.15.27:0/1940579674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f773406b220 0x7f7734102510 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:58.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.441+0000 7f77391e5700 1 --2- 172.21.15.27:0/1940579674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f773406bbe0 0x7f7734102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:20:58.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.441+0000 7f77391e5700 1 -- 172.21.15.27:0/1940579674 >> 172.21.15.27:0/1940579674 conn(0x7f77340793f0 msgr2=0x7f7734100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:20:58.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.441+0000 7f77391e5700 1 -- 172.21.15.27:0/1940579674 shutdown_connections 2024-05-21T03:20:58.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:20:58.441+0000 7f77391e5700 1 -- 172.21.15.27:0/1940579674 wait complete. 2024-05-21T03:20:58.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:58 smithi027 ceph-mon[32179]: pgmap v302: 97 pgs: 97 active+clean; 586 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.2 KiB/s wr, 20 op/s 2024-05-21T03:20:58.768 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:58 smithi039 ceph-mon[36429]: pgmap v302: 97 pgs: 97 active+clean; 586 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.2 KiB/s wr, 20 op/s 2024-05-21T03:20:58.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:58 smithi165 ceph-mon[36269]: pgmap v302: 97 pgs: 97 active+clean; 586 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.2 KiB/s wr, 20 op/s 2024-05-21T03:20:59.115 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-21T03:20:59.371 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:20:59.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:20:59 smithi039 ceph-mon[36429]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:20:59.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:20:59 smithi165 ceph-mon[36269]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:20:59.835 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:20:59 smithi027 ceph-mon[32179]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:21:00.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:00 smithi039 ceph-mon[36429]: pgmap v303: 97 pgs: 97 active+clean; 606 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.6 KiB/s wr, 22 op/s 2024-05-21T03:21:00.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:00 smithi165 ceph-mon[36269]: pgmap v303: 97 pgs: 97 active+clean; 606 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.6 KiB/s wr, 22 op/s 2024-05-21T03:21:00.828 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:00 smithi027 ceph-mon[32179]: pgmap v303: 97 pgs: 97 active+clean; 606 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.6 KiB/s wr, 22 op/s 2024-05-21T03:21:01.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.095+0000 7fe6d9738700 1 -- 172.21.15.27:0/111468094 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6d41088f0 msgr2=0x7fe6d4110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:01.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.095+0000 7fe6d9738700 1 --2- 172.21.15.27:0/111468094 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6d41088f0 0x7fe6d4110470 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc00b0d0 tx=0x7fe6cc00b3e0 comp rx=0 tx=0).stop 2024-05-21T03:21:01.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.095+0000 7fe6d9738700 1 -- 172.21.15.27:0/111468094 shutdown_connections 2024-05-21T03:21:01.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.095+0000 7fe6d9738700 1 --2- 172.21.15.27:0/111468094 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6d41088f0 0x7fe6d4110470 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:01.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.095+0000 7fe6d9738700 1 --2- 172.21.15.27:0/111468094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6d41037f0 0x7fe6d4103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:01.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.095+0000 7fe6d9738700 1 --2- 172.21.15.27:0/111468094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6d41041a0 0x7fe6d41081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:01.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.095+0000 7fe6d9738700 1 -- 172.21.15.27:0/111468094 >> 172.21.15.27:0/111468094 conn(0x7fe6d407a6a0 msgr2=0x7fe6d407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:01.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.095+0000 7fe6d9738700 1 -- 172.21.15.27:0/111468094 shutdown_connections 2024-05-21T03:21:01.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.095+0000 7fe6d9738700 1 -- 172.21.15.27:0/111468094 wait complete. 2024-05-21T03:21:01.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.096+0000 7fe6d9738700 1 Processor -- start 2024-05-21T03:21:01.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.096+0000 7fe6d9738700 1 -- start start 2024-05-21T03:21:01.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.096+0000 7fe6d9738700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6d41037f0 0x7fe6d4077070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:01.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.096+0000 7fe6d9738700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6d41041a0 0x7fe6d4072100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:01.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.096+0000 7fe6d9738700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6d41088f0 0x7fe6d4072640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:01.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.096+0000 7fe6d9738700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d41136e0 con 0x7fe6d41088f0 2024-05-21T03:21:01.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.096+0000 7fe6d9738700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d4113560 con 0x7fe6d41037f0 2024-05-21T03:21:01.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.096+0000 7fe6d9738700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d4113860 con 0x7fe6d41041a0 2024-05-21T03:21:01.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.096+0000 7fe6d2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6d41037f0 0x7fe6d4077070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:01.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.096+0000 7fe6d2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6d41037f0 0x7fe6d4077070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.27:51202/0 (socket says 172.21.15.27:51202) 2024-05-21T03:21:01.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.096+0000 7fe6d2ffd700 1 -- 172.21.15.27:0/1224335417 learned_addr learned my addr 172.21.15.27:0/1224335417 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:21:01.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.096+0000 7fe6d37fe700 1 --2- 172.21.15.27:0/1224335417 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6d41088f0 0x7fe6d4072640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:01.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.096+0000 7fe6d27fc700 1 --2- 172.21.15.27:0/1224335417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6d41041a0 0x7fe6d4072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:01.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.097+0000 7fe6d2ffd700 1 -- 172.21.15.27:0/1224335417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6d41041a0 msgr2=0x7fe6d4072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:01.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.097+0000 7fe6d2ffd700 1 --2- 172.21.15.27:0/1224335417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6d41041a0 0x7fe6d4072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:01.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.097+0000 7fe6d2ffd700 1 -- 172.21.15.27:0/1224335417 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6d41088f0 msgr2=0x7fe6d4072640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:01.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.097+0000 7fe6d2ffd700 1 --2- 172.21.15.27:0/1224335417 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6d41088f0 0x7fe6d4072640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:01.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.097+0000 7fe6d2ffd700 1 -- 172.21.15.27:0/1224335417 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d4072d70 con 0x7fe6d41037f0 2024-05-21T03:21:01.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.097+0000 7fe6d37fe700 1 --2- 172.21.15.27:0/1224335417 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6d41088f0 0x7fe6d4072640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:21:01.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.097+0000 7fe6d2ffd700 1 --2- 172.21.15.27:0/1224335417 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6d41037f0 0x7fe6d4077070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400d980 tx=0x7fe6c400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:01.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.097+0000 7fe6bbfff700 1 -- 172.21.15.27:0/1224335417 <== mon.2 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4017070 con 0x7fe6d41037f0 2024-05-21T03:21:01.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.097+0000 7fe6d9738700 1 -- 172.21.15.27:0/1224335417 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d41b0c00 con 0x7fe6d41037f0 2024-05-21T03:21:01.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.097+0000 7fe6bbfff700 1 -- 172.21.15.27:0/1224335417 <== mon.2 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6c4005020 con 0x7fe6d41037f0 2024-05-21T03:21:01.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.097+0000 7fe6d9738700 1 -- 172.21.15.27:0/1224335417 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d41b1050 con 0x7fe6d41037f0 2024-05-21T03:21:01.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.097+0000 7fe6bbfff700 1 -- 172.21.15.27:0/1224335417 <== mon.2 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c400fa10 con 0x7fe6d41037f0 2024-05-21T03:21:01.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.098+0000 7fe6d9738700 1 -- 172.21.15.27:0/1224335417 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6d410bac0 con 0x7fe6d41037f0 2024-05-21T03:21:01.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.100+0000 7fe6bbfff700 1 -- 172.21.15.27:0/1224335417 <== mon.2 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe6c4004a30 con 0x7fe6d41037f0 2024-05-21T03:21:01.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.101+0000 7fe6bbfff700 1 --2- 172.21.15.27:0/1224335417 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe6bc0b2990 0x7fe6bc0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:01.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.101+0000 7fe6d27fc700 1 --2- 172.21.15.27:0/1224335417 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe6bc0b2990 0x7fe6bc0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:01.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.101+0000 7fe6bbfff700 1 -- 172.21.15.27:0/1224335417 <== mon.2 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fe6c40e24e0 con 0x7fe6d41037f0 2024-05-21T03:21:01.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.101+0000 7fe6d27fc700 1 --2- 172.21.15.27:0/1224335417 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe6bc0b2990 0x7fe6bc0b4e50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe6d40738c0 tx=0x7fe6c800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:01.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.104+0000 7fe6bbfff700 1 -- 172.21.15.27:0/1224335417 <== mon.2 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe6c4020020 con 0x7fe6d41037f0 2024-05-21T03:21:01.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.354+0000 7fe6d9738700 1 -- 172.21.15.27:0/1224335417 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fe6d4063980 con 0x7fe6bc0b2990 2024-05-21T03:21:01.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.361+0000 7fe6bbfff700 1 -- 172.21.15.27:0/1224335417 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fe6d4063980 con 0x7fe6bc0b2990 2024-05-21T03:21:01.363 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/qa/sv_0/c93d203f-ed8b-4038-8f04-4b8657b28c90 2024-05-21T03:21:01.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.364+0000 7fe6d9738700 1 -- 172.21.15.27:0/1224335417 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe6bc0b2990 msgr2=0x7fe6bc0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:01.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.364+0000 7fe6d9738700 1 --2- 172.21.15.27:0/1224335417 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe6bc0b2990 0x7fe6bc0b4e50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe6d40738c0 tx=0x7fe6c800a300 comp rx=0 tx=0).stop 2024-05-21T03:21:01.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.364+0000 7fe6d9738700 1 -- 172.21.15.27:0/1224335417 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6d41037f0 msgr2=0x7fe6d4077070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:01.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.364+0000 7fe6d9738700 1 --2- 172.21.15.27:0/1224335417 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6d41037f0 0x7fe6d4077070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400d980 tx=0x7fe6c400dc90 comp rx=0 tx=0).stop 2024-05-21T03:21:01.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.364+0000 7fe6d9738700 1 -- 172.21.15.27:0/1224335417 shutdown_connections 2024-05-21T03:21:01.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.364+0000 7fe6d9738700 1 --2- 172.21.15.27:0/1224335417 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6d41088f0 0x7fe6d4072640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:01.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.364+0000 7fe6d9738700 1 --2- 172.21.15.27:0/1224335417 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6d41037f0 0x7fe6d4077070 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:01.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.364+0000 7fe6d9738700 1 --2- 172.21.15.27:0/1224335417 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe6bc0b2990 0x7fe6bc0b4e50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:01.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.364+0000 7fe6d9738700 1 --2- 172.21.15.27:0/1224335417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6d41041a0 0x7fe6d4072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:01.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.364+0000 7fe6d9738700 1 -- 172.21.15.27:0/1224335417 >> 172.21.15.27:0/1224335417 conn(0x7fe6d407a6a0 msgr2=0x7fe6d4101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:01.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.364+0000 7fe6d9738700 1 -- 172.21.15.27:0/1224335417 shutdown_connections 2024-05-21T03:21:01.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:01.364+0000 7fe6d9738700 1 -- 172.21.15.27:0/1224335417 wait complete. 2024-05-21T03:21:02.059 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-21T03:21:02.315 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:21:02.667 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:02 smithi027 ceph-mon[32179]: pgmap v304: 97 pgs: 97 active+clean; 606 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.4 KiB/s wr, 21 op/s 2024-05-21T03:21:02.667 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:02 smithi027 ceph-mon[32179]: from='client.24524 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:21:02.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:02 smithi039 ceph-mon[36429]: pgmap v304: 97 pgs: 97 active+clean; 606 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.4 KiB/s wr, 21 op/s 2024-05-21T03:21:02.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:02 smithi039 ceph-mon[36429]: from='client.24524 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:21:02.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:02 smithi165 ceph-mon[36269]: pgmap v304: 97 pgs: 97 active+clean; 606 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.4 KiB/s wr, 21 op/s 2024-05-21T03:21:02.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:02 smithi165 ceph-mon[36269]: from='client.24524 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:21:04.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.166+0000 7f6985f08700 1 -- 172.21.15.27:0/4289350704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69801084d0 msgr2=0x7f69801088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:04.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.166+0000 7f6985f08700 1 --2- 172.21.15.27:0/4289350704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69801084d0 0x7f69801088b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f697000b0a0 tx=0x7f697000b3b0 comp rx=0 tx=0).stop 2024-05-21T03:21:04.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.166+0000 7f6985f08700 1 -- 172.21.15.27:0/4289350704 shutdown_connections 2024-05-21T03:21:04.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.166+0000 7f6985f08700 1 --2- 172.21.15.27:0/4289350704 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6980102e90 0x7f698010f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:04.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.166+0000 7f6985f08700 1 --2- 172.21.15.27:0/4289350704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69801084d0 0x7f69801088b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.166+0000 7f6985f08700 1 --2- 172.21.15.27:0/4289350704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69801024d0 0x7f6980102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.166+0000 7f6985f08700 1 -- 172.21.15.27:0/4289350704 >> 172.21.15.27:0/4289350704 conn(0x7f6980078490 msgr2=0x7f69800788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.166+0000 7f6985f08700 1 -- 172.21.15.27:0/4289350704 shutdown_connections 2024-05-21T03:21:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.166+0000 7f6985f08700 1 -- 172.21.15.27:0/4289350704 wait complete. 2024-05-21T03:21:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.167+0000 7f6985f08700 1 Processor -- start 2024-05-21T03:21:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.167+0000 7f6985f08700 1 -- start start 2024-05-21T03:21:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.167+0000 7f6985f08700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69801024d0 0x7f698019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.167+0000 7f6985f08700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6980102e90 0x7f698019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.167+0000 7f6985f08700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69801084d0 0x7f6980199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.167+0000 7f6985f08700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f69801125e0 con 0x7f69801024d0 2024-05-21T03:21:04.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.167+0000 7f6985f08700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6980112460 con 0x7f69801084d0 2024-05-21T03:21:04.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.167+0000 7f6985f08700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6980112760 con 0x7f6980102e90 2024-05-21T03:21:04.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.168+0000 7f697f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69801024d0 0x7f698019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:04.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.168+0000 7f697f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69801024d0 0x7f698019f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48554/0 (socket says 172.21.15.27:48554) 2024-05-21T03:21:04.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.168+0000 7f697f7fe700 1 -- 172.21.15.27:0/1288243855 learned_addr learned my addr 172.21.15.27:0/1288243855 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:21:04.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.168+0000 7f697effd700 1 --2- 172.21.15.27:0/1288243855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6980102e90 0x7f698019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:04.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.168+0000 7f697ffff700 1 --2- 172.21.15.27:0/1288243855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69801084d0 0x7f6980199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:04.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.168+0000 7f697f7fe700 1 -- 172.21.15.27:0/1288243855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6980102e90 msgr2=0x7f698019fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:04.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.168+0000 7f697f7fe700 1 --2- 172.21.15.27:0/1288243855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6980102e90 0x7f698019fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:04.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.168+0000 7f697f7fe700 1 -- 172.21.15.27:0/1288243855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69801084d0 msgr2=0x7f6980199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:04.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.168+0000 7f697f7fe700 1 --2- 172.21.15.27:0/1288243855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69801084d0 0x7f6980199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:04.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.168+0000 7f697f7fe700 1 -- 172.21.15.27:0/1288243855 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698019a0d0 con 0x7f69801024d0 2024-05-21T03:21:04.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.168+0000 7f697f7fe700 1 --2- 172.21.15.27:0/1288243855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69801024d0 0x7f698019f7b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6970006070 tx=0x7f6970004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:04.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.168+0000 7f697cff9700 1 -- 172.21.15.27:0/1288243855 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970021070 con 0x7f69801024d0 2024-05-21T03:21:04.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.168+0000 7f697cff9700 1 -- 172.21.15.27:0/1288243855 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6970003c00 con 0x7f69801024d0 2024-05-21T03:21:04.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.168+0000 7f697cff9700 1 -- 172.21.15.27:0/1288243855 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69700047b0 con 0x7f69801024d0 2024-05-21T03:21:04.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.168+0000 7f6985f08700 1 -- 172.21.15.27:0/1288243855 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f698019a360 con 0x7f69801024d0 2024-05-21T03:21:04.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.168+0000 7f6985f08700 1 -- 172.21.15.27:0/1288243855 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69801a6680 con 0x7f69801024d0 2024-05-21T03:21:04.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.169+0000 7f6985f08700 1 -- 172.21.15.27:0/1288243855 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6980103b80 con 0x7f69801024d0 2024-05-21T03:21:04.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.171+0000 7f697cff9700 1 -- 172.21.15.27:0/1288243855 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6970007a00 con 0x7f69801024d0 2024-05-21T03:21:04.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.172+0000 7f697cff9700 1 --2- 172.21.15.27:0/1288243855 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f69680b2990 0x7f69680b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:04.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.172+0000 7f697effd700 1 --2- 172.21.15.27:0/1288243855 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f69680b2990 0x7f69680b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:04.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.172+0000 7f697cff9700 1 -- 172.21.15.27:0/1288243855 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f697000f020 con 0x7f69801024d0 2024-05-21T03:21:04.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.172+0000 7f697effd700 1 --2- 172.21.15.27:0/1288243855 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f69680b2990 0x7f69680b4e50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6980078ae0 tx=0x7f697400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:04.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.175+0000 7f697cff9700 1 -- 172.21.15.27:0/1288243855 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f697002a020 con 0x7f69801024d0 2024-05-21T03:21:04.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.422+0000 7f6985f08700 1 -- 172.21.15.27:0/1288243855 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6980063980 con 0x7f69680b2990 2024-05-21T03:21:04.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.450+0000 7f697cff9700 1 -- 172.21.15.27:0/1288243855 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6980063980 con 0x7f69680b2990 2024-05-21T03:21:04.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.453+0000 7f6985f08700 1 -- 172.21.15.27:0/1288243855 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f69680b2990 msgr2=0x7f69680b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:04.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.453+0000 7f6985f08700 1 --2- 172.21.15.27:0/1288243855 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f69680b2990 0x7f69680b4e50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6980078ae0 tx=0x7f697400a300 comp rx=0 tx=0).stop 2024-05-21T03:21:04.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.453+0000 7f6985f08700 1 -- 172.21.15.27:0/1288243855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69801024d0 msgr2=0x7f698019f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:04.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.453+0000 7f6985f08700 1 --2- 172.21.15.27:0/1288243855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69801024d0 0x7f698019f7b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6970006070 tx=0x7f6970004e40 comp rx=0 tx=0).stop 2024-05-21T03:21:04.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.453+0000 7f6985f08700 1 -- 172.21.15.27:0/1288243855 shutdown_connections 2024-05-21T03:21:04.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.453+0000 7f6985f08700 1 --2- 172.21.15.27:0/1288243855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69801084d0 0x7f6980199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:04.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.453+0000 7f6985f08700 1 --2- 172.21.15.27:0/1288243855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69801024d0 0x7f698019f7b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:04.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.453+0000 7f6985f08700 1 --2- 172.21.15.27:0/1288243855 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f69680b2990 0x7f69680b4e50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:04.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.453+0000 7f6985f08700 1 --2- 172.21.15.27:0/1288243855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6980102e90 0x7f698019fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:04.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.453+0000 7f6985f08700 1 -- 172.21.15.27:0/1288243855 >> 172.21.15.27:0/1288243855 conn(0x7f6980078490 msgr2=0x7f698006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:04.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.453+0000 7f6985f08700 1 -- 172.21.15.27:0/1288243855 shutdown_connections 2024-05-21T03:21:04.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:04.453+0000 7f6985f08700 1 -- 172.21.15.27:0/1288243855 wait complete. 2024-05-21T03:21:04.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:04 smithi027 ceph-mon[32179]: pgmap v305: 97 pgs: 97 active+clean; 606 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.7 KiB/s wr, 15 op/s 2024-05-21T03:21:04.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:04 smithi039 ceph-mon[36429]: pgmap v305: 97 pgs: 97 active+clean; 606 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.7 KiB/s wr, 15 op/s 2024-05-21T03:21:04.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:04 smithi165 ceph-mon[36269]: pgmap v305: 97 pgs: 97 active+clean; 606 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.7 KiB/s wr, 15 op/s 2024-05-21T03:21:05.160 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-21T03:21:05.416 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:21:05.808 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:05 smithi027 ceph-mon[32179]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:21:06.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:05 smithi039 ceph-mon[36429]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:21:06.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:05 smithi165 ceph-mon[36269]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:21:06.820 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:06 smithi027 ceph-mon[32179]: pgmap v306: 97 pgs: 97 active+clean; 610 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-05-21T03:21:07.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:06 smithi165 ceph-mon[36269]: pgmap v306: 97 pgs: 97 active+clean; 610 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-05-21T03:21:07.022 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:06 smithi039 ceph-mon[36429]: pgmap v306: 97 pgs: 97 active+clean; 610 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-05-21T03:21:07.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.083+0000 7f13b073c700 1 -- 172.21.15.27:0/2573348886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13a81035e0 msgr2=0x7f13a8103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:07.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.083+0000 7f13b073c700 1 --2- 172.21.15.27:0/2573348886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13a81035e0 0x7f13a8103a60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f13980099c0 tx=0x7f1398009cd0 comp rx=0 tx=0).stop 2024-05-21T03:21:07.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.083+0000 7f13b073c700 1 -- 172.21.15.27:0/2573348886 shutdown_connections 2024-05-21T03:21:07.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.083+0000 7f13b073c700 1 --2- 172.21.15.27:0/2573348886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13a8103fa0 0x7f13a8110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:07.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.083+0000 7f13b073c700 1 --2- 172.21.15.27:0/2573348886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13a8109680 0x7f13a8109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:07.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.083+0000 7f13b073c700 1 --2- 172.21.15.27:0/2573348886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13a81035e0 0x7f13a8103a60 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:07.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.083+0000 7f13b073c700 1 -- 172.21.15.27:0/2573348886 >> 172.21.15.27:0/2573348886 conn(0x7f13a807a6f0 msgr2=0x7f13a807ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:07.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.083+0000 7f13b073c700 1 -- 172.21.15.27:0/2573348886 shutdown_connections 2024-05-21T03:21:07.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.083+0000 7f13b073c700 1 -- 172.21.15.27:0/2573348886 wait complete. 2024-05-21T03:21:07.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.084+0000 7f13b073c700 1 Processor -- start 2024-05-21T03:21:07.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.084+0000 7f13b073c700 1 -- start start 2024-05-21T03:21:07.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.084+0000 7f13b073c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13a81035e0 0x7f13a819f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:07.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.084+0000 7f13b073c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13a8103fa0 0x7f13a819fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:07.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.084+0000 7f13b073c700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13a8109680 0x7f13a8199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:07.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.084+0000 7f13b073c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f13a81137e0 con 0x7f13a81035e0 2024-05-21T03:21:07.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.084+0000 7f13b073c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f13a8113660 con 0x7f13a8103fa0 2024-05-21T03:21:07.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.084+0000 7f13b073c700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f13a8113960 con 0x7f13a8109680 2024-05-21T03:21:07.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.084+0000 7f13ae4d8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13a81035e0 0x7f13a819f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:07.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.084+0000 7f13ae4d8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13a81035e0 0x7f13a819f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48580/0 (socket says 172.21.15.27:48580) 2024-05-21T03:21:07.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.085+0000 7f13ae4d8700 1 -- 172.21.15.27:0/1845641649 learned_addr learned my addr 172.21.15.27:0/1845641649 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:21:07.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.085+0000 7f13aecd9700 1 --2- 172.21.15.27:0/1845641649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13a8109680 0x7f13a8199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:07.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.085+0000 7f13ae4d8700 1 -- 172.21.15.27:0/1845641649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13a8109680 msgr2=0x7f13a8199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:07.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.085+0000 7f13adcd7700 1 --2- 172.21.15.27:0/1845641649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13a8103fa0 0x7f13a819fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:07.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.085+0000 7f13ae4d8700 1 --2- 172.21.15.27:0/1845641649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13a8109680 0x7f13a8199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:07.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.085+0000 7f13ae4d8700 1 -- 172.21.15.27:0/1845641649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13a8103fa0 msgr2=0x7f13a819fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:07.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.085+0000 7f13ae4d8700 1 --2- 172.21.15.27:0/1845641649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13a8103fa0 0x7f13a819fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:07.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.085+0000 7f13ae4d8700 1 -- 172.21.15.27:0/1845641649 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a819a1c0 con 0x7f13a81035e0 2024-05-21T03:21:07.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.085+0000 7f13aecd9700 1 --2- 172.21.15.27:0/1845641649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13a8109680 0x7f13a8199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:21:07.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.085+0000 7f13adcd7700 1 --2- 172.21.15.27:0/1845641649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13a8103fa0 0x7f13a819fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:21:07.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.085+0000 7f13ae4d8700 1 --2- 172.21.15.27:0/1845641649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13a81035e0 0x7f13a819f8a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f13a400c970 tx=0x7f13a400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:07.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.085+0000 7f139f7fe700 1 -- 172.21.15.27:0/1845641649 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a4007aa0 con 0x7f13a81035e0 2024-05-21T03:21:07.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.085+0000 7f139f7fe700 1 -- 172.21.15.27:0/1845641649 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13a4004ce0 con 0x7f13a81035e0 2024-05-21T03:21:07.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.085+0000 7f13b073c700 1 -- 172.21.15.27:0/1845641649 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a819a4b0 con 0x7f13a81035e0 2024-05-21T03:21:07.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.085+0000 7f139f7fe700 1 -- 172.21.15.27:0/1845641649 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a40055c0 con 0x7f13a81035e0 2024-05-21T03:21:07.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.085+0000 7f13b073c700 1 -- 172.21.15.27:0/1845641649 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a81a6770 con 0x7f13a81035e0 2024-05-21T03:21:07.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.086+0000 7f13b073c700 1 -- 172.21.15.27:0/1845641649 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13a8104cf0 con 0x7f13a81035e0 2024-05-21T03:21:07.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.088+0000 7f139f7fe700 1 -- 172.21.15.27:0/1845641649 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f13a4007c40 con 0x7f13a81035e0 2024-05-21T03:21:07.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.088+0000 7f139f7fe700 1 --2- 172.21.15.27:0/1845641649 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f13940b2990 0x7f13940b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:07.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.089+0000 7f13adcd7700 1 --2- 172.21.15.27:0/1845641649 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f13940b2990 0x7f13940b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:07.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.089+0000 7f139f7fe700 1 -- 172.21.15.27:0/1845641649 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f13a40e21d0 con 0x7f13a81035e0 2024-05-21T03:21:07.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.089+0000 7f13adcd7700 1 --2- 172.21.15.27:0/1845641649 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f13940b2990 0x7f13940b4e50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1398003800 tx=0x7f1398009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:07.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.092+0000 7f139f7fe700 1 -- 172.21.15.27:0/1845641649 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f13a40aa060 con 0x7f13a81035e0 2024-05-21T03:21:07.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.343+0000 7f13b073c700 1 -- 172.21.15.27:0/1845641649 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f13a8063980 con 0x7f13940b2990 2024-05-21T03:21:07.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.348+0000 7f139f7fe700 1 -- 172.21.15.27:0/1845641649 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f13a8063980 con 0x7f13940b2990 2024-05-21T03:21:07.348 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/qa/sv_1/3bb4c492-edff-4f68-974a-5fef1e70bd24 2024-05-21T03:21:07.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.350+0000 7f13b073c700 1 -- 172.21.15.27:0/1845641649 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f13940b2990 msgr2=0x7f13940b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:07.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.350+0000 7f13b073c700 1 --2- 172.21.15.27:0/1845641649 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f13940b2990 0x7f13940b4e50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1398003800 tx=0x7f1398009f10 comp rx=0 tx=0).stop 2024-05-21T03:21:07.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.350+0000 7f13b073c700 1 -- 172.21.15.27:0/1845641649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13a81035e0 msgr2=0x7f13a819f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:07.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.350+0000 7f13b073c700 1 --2- 172.21.15.27:0/1845641649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13a81035e0 0x7f13a819f8a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f13a400c970 tx=0x7f13a400cd30 comp rx=0 tx=0).stop 2024-05-21T03:21:07.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.350+0000 7f13b073c700 1 -- 172.21.15.27:0/1845641649 shutdown_connections 2024-05-21T03:21:07.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.350+0000 7f13b073c700 1 --2- 172.21.15.27:0/1845641649 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f13940b2990 0x7f13940b4e50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:07.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.350+0000 7f13b073c700 1 --2- 172.21.15.27:0/1845641649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13a8109680 0x7f13a8199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:07.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.350+0000 7f13b073c700 1 --2- 172.21.15.27:0/1845641649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13a81035e0 0x7f13a819f8a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:07.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.350+0000 7f13b073c700 1 --2- 172.21.15.27:0/1845641649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13a8103fa0 0x7f13a819fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:07.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.350+0000 7f13b073c700 1 -- 172.21.15.27:0/1845641649 >> 172.21.15.27:0/1845641649 conn(0x7f13a807a6f0 msgr2=0x7f13a8102270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:07.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.350+0000 7f13b073c700 1 -- 172.21.15.27:0/1845641649 shutdown_connections 2024-05-21T03:21:07.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:07.350+0000 7f13b073c700 1 -- 172.21.15.27:0/1845641649 wait complete. 2024-05-21T03:21:08.018 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:21:08.273 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:21:08.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:08 smithi027 ceph-mon[32179]: pgmap v307: 97 pgs: 97 active+clean; 610 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-05-21T03:21:08.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:08 smithi027 ceph-mon[32179]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:21:09.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:08 smithi039 ceph-mon[36429]: pgmap v307: 97 pgs: 97 active+clean; 610 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-05-21T03:21:09.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:08 smithi039 ceph-mon[36429]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:21:09.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:08 smithi165 ceph-mon[36269]: pgmap v307: 97 pgs: 97 active+clean; 610 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-05-21T03:21:09.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:08 smithi165 ceph-mon[36269]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:21:09.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.935+0000 7fe34b33b700 1 -- 172.21.15.27:0/3434152747 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3441041d0 msgr2=0x7fe3441081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:09.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.935+0000 7fe34b33b700 1 --2- 172.21.15.27:0/3434152747 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3441041d0 0x7fe3441081e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe3340099c0 tx=0x7fe334009cd0 comp rx=0 tx=0).stop 2024-05-21T03:21:09.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.935+0000 7fe34b33b700 1 -- 172.21.15.27:0/3434152747 shutdown_connections 2024-05-21T03:21:09.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.935+0000 7fe34b33b700 1 --2- 172.21.15.27:0/3434152747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe344108910 0x7fe344110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:09.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.935+0000 7fe34b33b700 1 --2- 172.21.15.27:0/3434152747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe344103820 0x7fe344103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:09.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.935+0000 7fe34b33b700 1 --2- 172.21.15.27:0/3434152747 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3441041d0 0x7fe3441081e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:09.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.935+0000 7fe34b33b700 1 -- 172.21.15.27:0/3434152747 >> 172.21.15.27:0/3434152747 conn(0x7fe34407a6b0 msgr2=0x7fe34407aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:09.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.935+0000 7fe34b33b700 1 -- 172.21.15.27:0/3434152747 shutdown_connections 2024-05-21T03:21:09.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.935+0000 7fe34b33b700 1 -- 172.21.15.27:0/3434152747 wait complete. 2024-05-21T03:21:09.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.936+0000 7fe34b33b700 1 Processor -- start 2024-05-21T03:21:09.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.936+0000 7fe34b33b700 1 -- start start 2024-05-21T03:21:09.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.936+0000 7fe34b33b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe344103820 0x7fe34419be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:09.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.936+0000 7fe34b33b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3441041d0 0x7fe34419c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.936+0000 7fe34b33b700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe344108910 0x7fe3441a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.936+0000 7fe34b33b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe3441137b0 con 0x7fe344103820 2024-05-21T03:21:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.936+0000 7fe34b33b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe344113630 con 0x7fe3441041d0 2024-05-21T03:21:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.936+0000 7fe34b33b700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe344113930 con 0x7fe344108910 2024-05-21T03:21:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.936+0000 7fe3490d7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe344103820 0x7fe34419be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.936+0000 7fe3490d7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe344103820 0x7fe34419be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48604/0 (socket says 172.21.15.27:48604) 2024-05-21T03:21:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.936+0000 7fe3490d7700 1 -- 172.21.15.27:0/1710748956 learned_addr learned my addr 172.21.15.27:0/1710748956 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:21:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.937+0000 7fe3488d6700 1 --2- 172.21.15.27:0/1710748956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3441041d0 0x7fe34419c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.937+0000 7fe3490d7700 1 -- 172.21.15.27:0/1710748956 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe344108910 msgr2=0x7fe3441a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.937+0000 7fe3498d8700 1 --2- 172.21.15.27:0/1710748956 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe344108910 0x7fe3441a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.937+0000 7fe3490d7700 1 --2- 172.21.15.27:0/1710748956 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe344108910 0x7fe3441a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.937+0000 7fe3490d7700 1 -- 172.21.15.27:0/1710748956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3441041d0 msgr2=0x7fe34419c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.937+0000 7fe3490d7700 1 --2- 172.21.15.27:0/1710748956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3441041d0 0x7fe34419c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.937+0000 7fe3490d7700 1 -- 172.21.15.27:0/1710748956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3441a0e50 con 0x7fe344103820 2024-05-21T03:21:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.937+0000 7fe3488d6700 1 --2- 172.21.15.27:0/1710748956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3441041d0 0x7fe34419c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:21:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.937+0000 7fe3498d8700 1 --2- 172.21.15.27:0/1710748956 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe344108910 0x7fe3441a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:21:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.937+0000 7fe3490d7700 1 --2- 172.21.15.27:0/1710748956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe344103820 0x7fe34419be90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe34000d980 tx=0x7fe34000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.937+0000 7fe33a7fc700 1 -- 172.21.15.27:0/1710748956 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe340016070 con 0x7fe344103820 2024-05-21T03:21:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.937+0000 7fe34b33b700 1 -- 172.21.15.27:0/1710748956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3441a1140 con 0x7fe344103820 2024-05-21T03:21:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.937+0000 7fe33a7fc700 1 -- 172.21.15.27:0/1710748956 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3400044d0 con 0x7fe344103820 2024-05-21T03:21:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.937+0000 7fe33a7fc700 1 -- 172.21.15.27:0/1710748956 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe340004d00 con 0x7fe344103820 2024-05-21T03:21:09.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.937+0000 7fe34b33b700 1 -- 172.21.15.27:0/1710748956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3441a89c0 con 0x7fe344103820 2024-05-21T03:21:09.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.938+0000 7fe34b33b700 1 -- 172.21.15.27:0/1710748956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe34406a230 con 0x7fe344103820 2024-05-21T03:21:09.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.939+0000 7fe33a7fc700 1 -- 172.21.15.27:0/1710748956 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe340004020 con 0x7fe344103820 2024-05-21T03:21:09.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.940+0000 7fe33a7fc700 1 --2- 172.21.15.27:0/1710748956 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe3300b2910 0x7fe3300b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:09.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.940+0000 7fe3488d6700 1 --2- 172.21.15.27:0/1710748956 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe3300b2910 0x7fe3300b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:09.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.940+0000 7fe33a7fc700 1 -- 172.21.15.27:0/1710748956 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fe3400e2190 con 0x7fe344103820 2024-05-21T03:21:09.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.941+0000 7fe3488d6700 1 --2- 172.21.15.27:0/1710748956 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe3300b2910 0x7fe3300b4dd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe344103350 tx=0x7fe334005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:09.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:09.945+0000 7fe33a7fc700 1 -- 172.21.15.27:0/1710748956 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe34001f020 con 0x7fe344103820 2024-05-21T03:21:10.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:10.216+0000 7fe34b33b700 1 -- 172.21.15.27:0/1710748956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3441a8bf0 con 0x7fe344103820 2024-05-21T03:21:10.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:10.217+0000 7fe33a7fc700 1 -- 172.21.15.27:0/1710748956 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11924 (secure 0 0 0) 0x7fe3400a9f00 con 0x7fe344103820 2024-05-21T03:21:10.217 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:21:10.217 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/148854869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":148854869},{"type":"v1","addr":"172.21.15.27:6835","nonce":148854869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/4145544730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":4145544730},{"type":"v1","addr":"172.21.15.27:6837","nonce":4145544730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T03:20:34.055985+0000","modified":"2024-05-21T03:20:52.467919+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24472,"mds_2":24457,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/745486764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":745486764},{"type":"v1","addr":"172.21.15.27:6839","nonce":745486764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6841/3982752351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3982752351},{"type":"v1","addr":"172.21.15.27:6841","nonce":3982752351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/3497521608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3497521608},{"type":"v1","addr":"172.21.15.39:6835","nonce":3497521608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.165:6835/3576910218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3576910218},{"type":"v1","addr":"172.21.15.165:6835","nonce":3576910218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/3673846852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3673846852},{"type":"v1","addr":"172.21.15.39:6837","nonce":3673846852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6837/691356139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":691356139},{"type":"v1","addr":"172.21.15.165:6837","nonce":691356139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6839/4158170814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4158170814},{"type":"v1","addr":"172.21.15.165:6839","nonce":4158170814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6839/287607819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":287607819},{"type":"v1","addr":"172.21.15.39:6839","nonce":287607819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24451,24472,24452,24457,24476]},"id":1}]} 2024-05-21T03:21:10.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:10.219+0000 7fe34b33b700 1 -- 172.21.15.27:0/1710748956 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe3300b2910 msgr2=0x7fe3300b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:10.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:10.219+0000 7fe34b33b700 1 --2- 172.21.15.27:0/1710748956 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe3300b2910 0x7fe3300b4dd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe344103350 tx=0x7fe334005fb0 comp rx=0 tx=0).stop 2024-05-21T03:21:10.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:10.219+0000 7fe34b33b700 1 -- 172.21.15.27:0/1710748956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe344103820 msgr2=0x7fe34419be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:10.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:10.219+0000 7fe34b33b700 1 --2- 172.21.15.27:0/1710748956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe344103820 0x7fe34419be90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe34000d980 tx=0x7fe34000dd40 comp rx=0 tx=0).stop 2024-05-21T03:21:10.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:10.219+0000 7fe34b33b700 1 -- 172.21.15.27:0/1710748956 shutdown_connections 2024-05-21T03:21:10.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:10.219+0000 7fe34b33b700 1 --2- 172.21.15.27:0/1710748956 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe3300b2910 0x7fe3300b4dd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:10.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:10.219+0000 7fe34b33b700 1 --2- 172.21.15.27:0/1710748956 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe344108910 0x7fe3441a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:10.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:10.219+0000 7fe34b33b700 1 --2- 172.21.15.27:0/1710748956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe344103820 0x7fe34419be90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:10.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:10.219+0000 7fe34b33b700 1 --2- 172.21.15.27:0/1710748956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3441041d0 0x7fe34419c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:10.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:10.219+0000 7fe34b33b700 1 -- 172.21.15.27:0/1710748956 >> 172.21.15.27:0/1710748956 conn(0x7fe34407a6b0 msgr2=0x7fe3441019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:10.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:10.219+0000 7fe34b33b700 1 -- 172.21.15.27:0/1710748956 shutdown_connections 2024-05-21T03:21:10.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:10.219+0000 7fe34b33b700 1 -- 172.21.15.27:0/1710748956 wait complete. 2024-05-21T03:21:10.221 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 28 2024-05-21T03:21:10.791 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:10 smithi027 ceph-mon[32179]: pgmap v308: 97 pgs: 97 active+clean; 633 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 4.9 KiB/s wr, 13 op/s 2024-05-21T03:21:10.791 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:10 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/1710748956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:21:10.821 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-21T03:21:11.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:10 smithi039 ceph-mon[36429]: pgmap v308: 97 pgs: 97 active+clean; 633 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 4.9 KiB/s wr, 13 op/s 2024-05-21T03:21:11.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:10 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/1710748956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:21:11.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:10 smithi165 ceph-mon[36269]: pgmap v308: 97 pgs: 97 active+clean; 633 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 4.9 KiB/s wr, 13 op/s 2024-05-21T03:21:11.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:10 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/1710748956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:21:11.071 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:21:12.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.692+0000 7f616a239700 1 -- 172.21.15.27:0/259572544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6164105730 msgr2=0x7f6164105bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:12.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.692+0000 7f616a239700 1 --2- 172.21.15.27:0/259572544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6164105730 0x7f6164105bb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f614c009990 tx=0x7f614c009ca0 comp rx=0 tx=0).stop 2024-05-21T03:21:12.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.692+0000 7f616a239700 1 -- 172.21.15.27:0/259572544 shutdown_connections 2024-05-21T03:21:12.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.692+0000 7f616a239700 1 --2- 172.21.15.27:0/259572544 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61641060f0 0x7f6164112600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:12.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.692+0000 7f616a239700 1 --2- 172.21.15.27:0/259572544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f616410b730 0x7f616410bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:12.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.692+0000 7f616a239700 1 --2- 172.21.15.27:0/259572544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6164105730 0x7f6164105bb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:12.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.692+0000 7f616a239700 1 -- 172.21.15.27:0/259572544 >> 172.21.15.27:0/259572544 conn(0x7f6164101220 msgr2=0x7f6164103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:12.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.692+0000 7f616a239700 1 -- 172.21.15.27:0/259572544 shutdown_connections 2024-05-21T03:21:12.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.692+0000 7f616a239700 1 -- 172.21.15.27:0/259572544 wait complete. 2024-05-21T03:21:12.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.692+0000 7f616a239700 1 Processor -- start 2024-05-21T03:21:12.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.693+0000 7f616a239700 1 -- start start 2024-05-21T03:21:12.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.693+0000 7f616a239700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6164105730 0x7f616419bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:12.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.693+0000 7f616a239700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61641060f0 0x7f616419c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:12.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.693+0000 7f616a239700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f616410b730 0x7f61641a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:12.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.693+0000 7f616a239700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6164115920 con 0x7f6164105730 2024-05-21T03:21:12.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.693+0000 7f616a239700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f61641157a0 con 0x7f616410b730 2024-05-21T03:21:12.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.693+0000 7f616a239700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6164115aa0 con 0x7f61641060f0 2024-05-21T03:21:12.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.693+0000 7f61637fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6164105730 0x7f616419bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:12.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.693+0000 7f61637fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6164105730 0x7f616419bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44786/0 (socket says 172.21.15.27:44786) 2024-05-21T03:21:12.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.693+0000 7f61637fe700 1 -- 172.21.15.27:0/4169675681 learned_addr learned my addr 172.21.15.27:0/4169675681 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:21:12.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.693+0000 7f6163fff700 1 --2- 172.21.15.27:0/4169675681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f616410b730 0x7f61641a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:12.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.693+0000 7f6162ffd700 1 --2- 172.21.15.27:0/4169675681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61641060f0 0x7f616419c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:12.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.693+0000 7f61637fe700 1 -- 172.21.15.27:0/4169675681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61641060f0 msgr2=0x7f616419c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:12.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.693+0000 7f61637fe700 1 --2- 172.21.15.27:0/4169675681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61641060f0 0x7f616419c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:12.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.693+0000 7f61637fe700 1 -- 172.21.15.27:0/4169675681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f616410b730 msgr2=0x7f61641a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:12.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.693+0000 7f61637fe700 1 --2- 172.21.15.27:0/4169675681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f616410b730 0x7f61641a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:12.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.693+0000 7f61637fe700 1 -- 172.21.15.27:0/4169675681 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61641a0d70 con 0x7f6164105730 2024-05-21T03:21:12.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.694+0000 7f61637fe700 1 --2- 172.21.15.27:0/4169675681 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6164105730 0x7f616419bdb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f615400c9b0 tx=0x7f615400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:12.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.694+0000 7f6160ff9700 1 -- 172.21.15.27:0/4169675681 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6154007b00 con 0x7f6164105730 2024-05-21T03:21:12.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.694+0000 7f6160ff9700 1 -- 172.21.15.27:0/4169675681 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f615400ce50 con 0x7f6164105730 2024-05-21T03:21:12.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.694+0000 7f616a239700 1 -- 172.21.15.27:0/4169675681 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61641a1060 con 0x7f6164105730 2024-05-21T03:21:12.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.694+0000 7f6160ff9700 1 -- 172.21.15.27:0/4169675681 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6154018660 con 0x7f6164105730 2024-05-21T03:21:12.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.694+0000 7f616a239700 1 -- 172.21.15.27:0/4169675681 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61641a88e0 con 0x7f6164105730 2024-05-21T03:21:12.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.695+0000 7f616a239700 1 -- 172.21.15.27:0/4169675681 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6164106e70 con 0x7f6164105730 2024-05-21T03:21:12.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.696+0000 7f6160ff9700 1 -- 172.21.15.27:0/4169675681 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f615400f450 con 0x7f6164105730 2024-05-21T03:21:12.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.697+0000 7f6160ff9700 1 --2- 172.21.15.27:0/4169675681 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f61500b2910 0x7f61500b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:12.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.697+0000 7f6162ffd700 1 --2- 172.21.15.27:0/4169675681 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f61500b2910 0x7f61500b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:12.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.697+0000 7f6160ff9700 1 -- 172.21.15.27:0/4169675681 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f61540e20a0 con 0x7f6164105730 2024-05-21T03:21:12.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.697+0000 7f6162ffd700 1 --2- 172.21.15.27:0/4169675681 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f61500b2910 0x7f61500b4dd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f614c0098e0 tx=0x7f614c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:12.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.701+0000 7f6160ff9700 1 -- 172.21.15.27:0/4169675681 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6154014030 con 0x7f6164105730 2024-05-21T03:21:12.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.921+0000 7f616a239700 1 -- 172.21.15.27:0/4169675681 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f616419d0c0 con 0x7f6164105730 2024-05-21T03:21:12.922 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:12 smithi027 ceph-mon[32179]: pgmap v309: 97 pgs: 97 active+clean; 633 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-05-21T03:21:12.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.923+0000 7f6160ff9700 1 -- 172.21.15.27:0/4169675681 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22157 (secure 0 0 0) 0x7f61540a9e10 con 0x7f6164105730 2024-05-21T03:21:12.924 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:21:12.924 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":80,"fsid":"c9ca94be-171f-11ef-bc9a-c7b262605968","created":"2024-05-21T03:12:18.217236+0000","modified":"2024-05-21T03:20:34.056022+0000","last_up_change":"2024-05-21T03:19:30.382150+0000","last_in_change":"2024-05-21T03:19:09.058249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T03:15:41.340194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T03:20:27.507346+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T03:20:30.830169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c745c09f-9a76-4d05-af75-c7554f6d5e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6803","nonce":1399996449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6805","nonce":1399996449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6809","nonce":1399996449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6807","nonce":1399996449}]},"public_addr":"172.21.15.27:6803/1399996449","cluster_addr":"172.21.15.27:6805/1399996449","heartbeat_back_addr":"172.21.15.27:6809/1399996449","heartbeat_front_addr":"172.21.15.27:6807/1399996449","state":["exists","up"]},{"osd":1,"uuid":"8b73aa00-4364-438f-9722-53caa4e9bec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6801","nonce":1776318847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6803","nonce":1776318847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6807","nonce":1776318847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6805","nonce":1776318847}]},"public_addr":"172.21.15.39:6801/1776318847","cluster_addr":"172.21.15.39:6803/1776318847","heartbeat_back_addr":"172.21.15.39:6807/1776318847","heartbeat_front_addr":"172.21.15.39:6805/1776318847","state":["exists","up"]},{"osd":2,"uuid":"0822043e-7d3c-4e98-9dc0-94b98b4d562a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6801","nonce":703322625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6803","nonce":703322625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6807","nonce":703322625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6805","nonce":703322625}]},"public_addr":"172.21.15.165:6801/703322625","cluster_addr":"172.21.15.165:6803/703322625","heartbeat_back_addr":"172.21.15.165:6807/703322625","heartbeat_front_addr":"172.21.15.165:6805/703322625","state":["exists","up"]},{"osd":3,"uuid":"c563b1b2-efcb-4f8c-8121-c1938f83131e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6811","nonce":1933403135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6813","nonce":1933403135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6817","nonce":1933403135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6815","nonce":1933403135}]},"public_addr":"172.21.15.27:6811/1933403135","cluster_addr":"172.21.15.27:6813/1933403135","heartbeat_back_addr":"172.21.15.27:6817/1933403135","heartbeat_front_addr":"172.21.15.27:6815/1933403135","state":["exists","up"]},{"osd":4,"uuid":"0c793b87-b045-4b4a-89e7-87ec36deeb0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6809","nonce":155579331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6811","nonce":155579331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6815","nonce":155579331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6813","nonce":155579331}]},"public_addr":"172.21.15.39:6809/155579331","cluster_addr":"172.21.15.39:6811/155579331","heartbeat_back_addr":"172.21.15.39:6815/155579331","heartbeat_front_addr":"172.21.15.39:6813/155579331","state":["exists","up"]},{"osd":5,"uuid":"367eeb5a-9d75-4644-bf46-317c03f33d8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6809","nonce":386850244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6811","nonce":386850244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6815","nonce":386850244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6813","nonce":386850244}]},"public_addr":"172.21.15.165:6809/386850244","cluster_addr":"172.21.15.165:6811/386850244","heartbeat_back_addr":"172.21.15.165:6815/386850244","heartbeat_front_addr":"172.21.15.165:6813/386850244","state":["exists","up"]},{"osd":6,"uuid":"e000167f-24fd-4115-83f0-8afbfa8d8991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6819","nonce":3531376679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6821","nonce":3531376679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6825","nonce":3531376679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6823","nonce":3531376679}]},"public_addr":"172.21.15.27:6819/3531376679","cluster_addr":"172.21.15.27:6821/3531376679","heartbeat_back_addr":"172.21.15.27:6825/3531376679","heartbeat_front_addr":"172.21.15.27:6823/3531376679","state":["exists","up"]},{"osd":7,"uuid":"4b8d486b-868e-4f33-98a1-b99686adeaaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6817","nonce":1550035852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6819","nonce":1550035852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6823","nonce":1550035852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6821","nonce":1550035852}]},"public_addr":"172.21.15.39:6817/1550035852","cluster_addr":"172.21.15.39:6819/1550035852","heartbeat_back_addr":"172.21.15.39:6823/1550035852","heartbeat_front_addr":"172.21.15.39:6821/1550035852","state":["exists","up"]},{"osd":8,"uuid":"2259f35c-2edd-4331-9a63-8c0c1acbdc71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6817","nonce":3184704971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6819","nonce":3184704971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6823","nonce":3184704971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6821","nonce":3184704971}]},"public_addr":"172.21.15.165:6817/3184704971","cluster_addr":"172.21.15.165:6819/3184704971","heartbeat_back_addr":"172.21.15.165:6823/3184704971","heartbeat_front_addr":"172.21.15.165:6821/3184704971","state":["exists","up"]},{"osd":9,"uuid":"10307ecc-8270-4be2-b62b-9708e09bb207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6827","nonce":2800048793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6829","nonce":2800048793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6833","nonce":2800048793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6831","nonce":2800048793}]},"public_addr":"172.21.15.27:6827/2800048793","cluster_addr":"172.21.15.27:6829/2800048793","heartbeat_back_addr":"172.21.15.27:6833/2800048793","heartbeat_front_addr":"172.21.15.27:6831/2800048793","state":["exists","up"]},{"osd":10,"uuid":"dfe1b8ca-ff8a-4b7f-ade5-ad4f00cf60cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6825","nonce":363500811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6827","nonce":363500811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6831","nonce":363500811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6829","nonce":363500811}]},"public_addr":"172.21.15.39:6825/363500811","cluster_addr":"172.21.15.39:6827/363500811","heartbeat_back_addr":"172.21.15.39:6831/363500811","heartbeat_front_addr":"172.21.15.39:6829/363500811","state":["exists","up"]},{"osd":11,"uuid":"f9531145-5d09-4dbd-a050-3159ee09db24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6825","nonce":2274332243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6827","nonce":2274332243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6831","nonce":2274332243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6829","nonce":2274332243}]},"public_addr":"172.21.15.165:6825/2274332243","cluster_addr":"172.21.15.165:6827/2274332243","heartbeat_back_addr":"172.21.15.165:6831/2274332243","heartbeat_front_addr":"172.21.15.165:6829/2274332243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:14:47.077860+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:15:13.272879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:15:38.046651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:03.376483+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:29.179280+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:53.391261+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:17:18.472829+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:17:44.349925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:18:08.652193+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:18:35.282669+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:19:01.637151+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:19:27.461566+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6800/1185518061":"2024-05-22T03:13:13.249220+0000","172.21.15.27:6801/1185518061":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/4194431212":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/1890032101":"2024-05-22T03:13:13.249220+0000","172.21.15.27:6800/175229478":"2024-05-22T03:12:45.739195+0000","172.21.15.27:0/999325706":"2024-05-22T03:12:45.739195+0000","172.21.15.27:6801/175229478":"2024-05-22T03:12:45.739195+0000","172.21.15.27:0/4184128736":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/3000197506":"2024-05-22T03:12:45.739195+0000","172.21.15.27:0/972677319":"2024-05-22T03:12:45.739195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T03:21:12.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.926+0000 7f616a239700 1 -- 172.21.15.27:0/4169675681 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f61500b2910 msgr2=0x7f61500b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:12.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.926+0000 7f616a239700 1 --2- 172.21.15.27:0/4169675681 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f61500b2910 0x7f61500b4dd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f614c0098e0 tx=0x7f614c005960 comp rx=0 tx=0).stop 2024-05-21T03:21:12.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.926+0000 7f616a239700 1 -- 172.21.15.27:0/4169675681 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6164105730 msgr2=0x7f616419bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:12.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.926+0000 7f616a239700 1 --2- 172.21.15.27:0/4169675681 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6164105730 0x7f616419bdb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f615400c9b0 tx=0x7f615400ccc0 comp rx=0 tx=0).stop 2024-05-21T03:21:12.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.926+0000 7f616a239700 1 -- 172.21.15.27:0/4169675681 shutdown_connections 2024-05-21T03:21:12.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.926+0000 7f616a239700 1 --2- 172.21.15.27:0/4169675681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f616410b730 0x7f61641a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:12.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.926+0000 7f616a239700 1 --2- 172.21.15.27:0/4169675681 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6164105730 0x7f616419bdb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:12.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.926+0000 7f616a239700 1 --2- 172.21.15.27:0/4169675681 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f61500b2910 0x7f61500b4dd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:12.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.926+0000 7f616a239700 1 --2- 172.21.15.27:0/4169675681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61641060f0 0x7f616419c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:12.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.926+0000 7f616a239700 1 -- 172.21.15.27:0/4169675681 >> 172.21.15.27:0/4169675681 conn(0x7f6164101220 msgr2=0x7f6164101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:12.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.926+0000 7f616a239700 1 -- 172.21.15.27:0/4169675681 shutdown_connections 2024-05-21T03:21:12.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:12.926+0000 7f616a239700 1 -- 172.21.15.27:0/4169675681 wait complete. 2024-05-21T03:21:13.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:12 smithi039 ceph-mon[36429]: pgmap v309: 97 pgs: 97 active+clean; 633 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-05-21T03:21:13.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:12 smithi165 ceph-mon[36269]: pgmap v309: 97 pgs: 97 active+clean; 633 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-05-21T03:21:13.589 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:13 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/4169675681' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:21:13.589 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:13 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-05-21T03:21:13.589 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:13 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-05-21T03:21:13.589 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:13 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-05-21T03:21:13.589 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:13 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 8]}]: dispatch 2024-05-21T03:21:13.589 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:13 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-05-21T03:21:13.589 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:13 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]: dispatch 2024-05-21T03:21:13.589 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:13 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-05-21T03:21:13.589 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:13 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-05-21T03:21:13.590 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:13 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-05-21T03:21:13.590 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:13 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2024-05-21T03:21:13.617 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-21T03:21:13.618 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-21T03:21:13.868 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:21:14.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:13 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/4169675681' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:21:14.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:13 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-05-21T03:21:14.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:13 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-05-21T03:21:14.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:13 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-05-21T03:21:14.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:13 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 8]}]: dispatch 2024-05-21T03:21:14.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:13 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-05-21T03:21:14.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:13 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]: dispatch 2024-05-21T03:21:14.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:13 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-05-21T03:21:14.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:13 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-05-21T03:21:14.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:13 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-05-21T03:21:14.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:13 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2024-05-21T03:21:14.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:13 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/4169675681' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:21:14.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:13 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-05-21T03:21:14.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:13 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-05-21T03:21:14.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:13 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-05-21T03:21:14.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:13 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 8]}]: dispatch 2024-05-21T03:21:14.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:13 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-05-21T03:21:14.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:13 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]: dispatch 2024-05-21T03:21:14.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:13 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-05-21T03:21:14.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:13 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-05-21T03:21:14.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:13 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-05-21T03:21:14.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:13 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2024-05-21T03:21:14.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:14 smithi027 ceph-mon[32179]: pgmap v310: 97 pgs: 97 active+clean; 633 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-05-21T03:21:14.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:14 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-05-21T03:21:14.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:14 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-05-21T03:21:14.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:14 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-05-21T03:21:14.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:14 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 8]}]': finished 2024-05-21T03:21:14.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:14 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-05-21T03:21:14.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:14 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]': finished 2024-05-21T03:21:14.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:14 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-05-21T03:21:14.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:14 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-05-21T03:21:14.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:14 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-05-21T03:21:14.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:14 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2024-05-21T03:21:14.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:14 smithi027 ceph-mon[32179]: osdmap e81: 12 total, 12 up, 12 in 2024-05-21T03:21:15.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:14 smithi039 ceph-mon[36429]: pgmap v310: 97 pgs: 97 active+clean; 633 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-05-21T03:21:15.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:14 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-05-21T03:21:15.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:14 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-05-21T03:21:15.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:14 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-05-21T03:21:15.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:14 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 8]}]': finished 2024-05-21T03:21:15.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:14 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-05-21T03:21:15.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:14 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]': finished 2024-05-21T03:21:15.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:14 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-05-21T03:21:15.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:14 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-05-21T03:21:15.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:14 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-05-21T03:21:15.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:14 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2024-05-21T03:21:15.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:14 smithi039 ceph-mon[36429]: osdmap e81: 12 total, 12 up, 12 in 2024-05-21T03:21:15.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:14 smithi165 ceph-mon[36269]: pgmap v310: 97 pgs: 97 active+clean; 633 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-05-21T03:21:15.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:14 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-05-21T03:21:15.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:14 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-05-21T03:21:15.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:14 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-05-21T03:21:15.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:14 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 8]}]': finished 2024-05-21T03:21:15.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:14 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-05-21T03:21:15.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:14 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]': finished 2024-05-21T03:21:15.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:14 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-05-21T03:21:15.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:14 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-05-21T03:21:15.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:14 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-05-21T03:21:15.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:14 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2024-05-21T03:21:15.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:14 smithi165 ceph-mon[36269]: osdmap e81: 12 total, 12 up, 12 in 2024-05-21T03:21:15.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.455+0000 7fd026f21700 1 -- 172.21.15.27:0/2565645783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0201037f0 msgr2=0x7fd020103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:15.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.455+0000 7fd026f21700 1 --2- 172.21.15.27:0/2565645783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0201037f0 0x7fd020103bd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd01000b0d0 tx=0x7fd01000b3e0 comp rx=0 tx=0).stop 2024-05-21T03:21:15.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.455+0000 7fd026f21700 1 -- 172.21.15.27:0/2565645783 shutdown_connections 2024-05-21T03:21:15.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.455+0000 7fd026f21700 1 --2- 172.21.15.27:0/2565645783 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0201088f0 0x7fd020110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:15.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.455+0000 7fd026f21700 1 --2- 172.21.15.27:0/2565645783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0201037f0 0x7fd020103bd0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:15.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.455+0000 7fd026f21700 1 --2- 172.21.15.27:0/2565645783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0201041a0 0x7fd0201081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:15.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.455+0000 7fd026f21700 1 -- 172.21.15.27:0/2565645783 >> 172.21.15.27:0/2565645783 conn(0x7fd02007a6a0 msgr2=0x7fd02007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:15.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.455+0000 7fd026f21700 1 -- 172.21.15.27:0/2565645783 shutdown_connections 2024-05-21T03:21:15.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.455+0000 7fd026f21700 1 -- 172.21.15.27:0/2565645783 wait complete. 2024-05-21T03:21:15.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.455+0000 7fd026f21700 1 Processor -- start 2024-05-21T03:21:15.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.455+0000 7fd026f21700 1 -- start start 2024-05-21T03:21:15.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.456+0000 7fd026f21700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0201037f0 0x7fd020075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:15.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.456+0000 7fd026f21700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0201041a0 0x7fd020075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:15.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.456+0000 7fd026f21700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0201088f0 0x7fd0200762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:15.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.456+0000 7fd026f21700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd0201137a0 con 0x7fd0201041a0 2024-05-21T03:21:15.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.456+0000 7fd026f21700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd020113620 con 0x7fd0201037f0 2024-05-21T03:21:15.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.456+0000 7fd026f21700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd020113920 con 0x7fd0201088f0 2024-05-21T03:21:15.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.456+0000 7fd01ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0201041a0 0x7fd020075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:15.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.456+0000 7fd01ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0201041a0 0x7fd020075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44814/0 (socket says 172.21.15.27:44814) 2024-05-21T03:21:15.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.456+0000 7fd01ffff700 1 -- 172.21.15.27:0/885532381 learned_addr learned my addr 172.21.15.27:0/885532381 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:21:15.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.456+0000 7fd0254be700 1 --2- 172.21.15.27:0/885532381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0201088f0 0x7fd0200762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:15.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.456+0000 7fd024cbd700 1 --2- 172.21.15.27:0/885532381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0201037f0 0x7fd020075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:15.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.456+0000 7fd01ffff700 1 -- 172.21.15.27:0/885532381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0201088f0 msgr2=0x7fd0200762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:15.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.456+0000 7fd01ffff700 1 --2- 172.21.15.27:0/885532381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0201088f0 0x7fd0200762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:15.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.456+0000 7fd01ffff700 1 -- 172.21.15.27:0/885532381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0201037f0 msgr2=0x7fd020075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:15.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.456+0000 7fd01ffff700 1 --2- 172.21.15.27:0/885532381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0201037f0 0x7fd020075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:15.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.456+0000 7fd01ffff700 1 -- 172.21.15.27:0/885532381 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0201a81a0 con 0x7fd0201041a0 2024-05-21T03:21:15.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.456+0000 7fd01ffff700 1 --2- 172.21.15.27:0/885532381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0201041a0 0x7fd020075d80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd01400b530 tx=0x7fd01400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:15.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.456+0000 7fd01dffb700 1 -- 172.21.15.27:0/885532381 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd014015020 con 0x7fd0201041a0 2024-05-21T03:21:15.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.456+0000 7fd01dffb700 1 -- 172.21.15.27:0/885532381 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd014011ce0 con 0x7fd0201041a0 2024-05-21T03:21:15.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.456+0000 7fd026f21700 1 -- 172.21.15.27:0/885532381 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0201a8490 con 0x7fd0201041a0 2024-05-21T03:21:15.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.456+0000 7fd01dffb700 1 -- 172.21.15.27:0/885532381 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd014010d20 con 0x7fd0201041a0 2024-05-21T03:21:15.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.457+0000 7fd026f21700 1 -- 172.21.15.27:0/885532381 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0201a8a20 con 0x7fd0201041a0 2024-05-21T03:21:15.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.457+0000 7fd026f21700 1 -- 172.21.15.27:0/885532381 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd02010bb30 con 0x7fd0201041a0 2024-05-21T03:21:15.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.459+0000 7fd01dffb700 1 -- 172.21.15.27:0/885532381 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fd014018020 con 0x7fd0201041a0 2024-05-21T03:21:15.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.460+0000 7fd01dffb700 1 --2- 172.21.15.27:0/885532381 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd0080b2990 0x7fd0080b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:15.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.460+0000 7fd024cbd700 1 --2- 172.21.15.27:0/885532381 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd0080b2990 0x7fd0080b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:15.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.460+0000 7fd01dffb700 1 -- 172.21.15.27:0/885532381 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fd0140e1fe0 con 0x7fd0201041a0 2024-05-21T03:21:15.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.460+0000 7fd024cbd700 1 --2- 172.21.15.27:0/885532381 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd0080b2990 0x7fd0080b4e50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd01000d010 tx=0x7fd01000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:15.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.464+0000 7fd01dffb700 1 -- 172.21.15.27:0/885532381 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd01401f020 con 0x7fd0201041a0 2024-05-21T03:21:15.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.673+0000 7fd026f21700 1 -- 172.21.15.27:0/885532381 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd020063980 con 0x7fd0080b2990 2024-05-21T03:21:15.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.681+0000 7fd01dffb700 1 -- 172.21.15.27:0/885532381 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258629 (secure 0 0 0) 0x7fd020063980 con 0x7fd0080b2990 2024-05-21T03:21:15.686 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:21:15.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.684+0000 7fd026f21700 1 -- 172.21.15.27:0/885532381 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd0080b2990 msgr2=0x7fd0080b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:15.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.684+0000 7fd026f21700 1 --2- 172.21.15.27:0/885532381 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd0080b2990 0x7fd0080b4e50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd01000d010 tx=0x7fd01000b040 comp rx=0 tx=0).stop 2024-05-21T03:21:15.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.684+0000 7fd026f21700 1 -- 172.21.15.27:0/885532381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0201041a0 msgr2=0x7fd020075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:15.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.684+0000 7fd026f21700 1 --2- 172.21.15.27:0/885532381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0201041a0 0x7fd020075d80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd01400b530 tx=0x7fd01400b8f0 comp rx=0 tx=0).stop 2024-05-21T03:21:15.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.684+0000 7fd026f21700 1 -- 172.21.15.27:0/885532381 shutdown_connections 2024-05-21T03:21:15.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.684+0000 7fd026f21700 1 --2- 172.21.15.27:0/885532381 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd0080b2990 0x7fd0080b4e50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:15.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.684+0000 7fd026f21700 1 --2- 172.21.15.27:0/885532381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0201088f0 0x7fd0200762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:15.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.684+0000 7fd026f21700 1 --2- 172.21.15.27:0/885532381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0201037f0 0x7fd020075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:15.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.684+0000 7fd026f21700 1 --2- 172.21.15.27:0/885532381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0201041a0 0x7fd020075d80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:15.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.684+0000 7fd026f21700 1 -- 172.21.15.27:0/885532381 >> 172.21.15.27:0/885532381 conn(0x7fd02007a6a0 msgr2=0x7fd020101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:15.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.685+0000 7fd026f21700 1 -- 172.21.15.27:0/885532381 shutdown_connections 2024-05-21T03:21:15.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:15.685+0000 7fd026f21700 1 -- 172.21.15.27:0/885532381 wait complete. 2024-05-21T03:21:15.690 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-05-21T03:21:15.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:15 smithi027 ceph-mon[32179]: osdmap e82: 12 total, 12 up, 12 in 2024-05-21T03:21:16.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:15 smithi039 ceph-mon[36429]: osdmap e82: 12 total, 12 up, 12 in 2024-05-21T03:21:16.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:15 smithi165 ceph-mon[36269]: osdmap e82: 12 total, 12 up, 12 in 2024-05-21T03:21:16.342 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-05-21T03:21:15.344036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652564,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":349,"num_read_kb":274,"num_write":364,"num_write_kb":2279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":300,"ondisk_log_size":300,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357380,"kb_used_data":12376,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584444,"statfs":{"total":1151940427776,"available":1151574470656,"internally_reserved":0,"allocated":12673024,"data_stored":6959245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19139,"internal_metadata":353154365},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27422,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":73,"num_write":16,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002059"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.275996+0000","last_change":"2024-05-21T03:20:32.149961+0000","last_active":"2024-05-21T03:20:34.275996+0000","last_peered":"2024-05-21T03:20:34.275996+0000","last_clean":"2024-05-21T03:20:34.275996+0000","last_became_active":"2024-05-21T03:20:32.149552+0000","last_became_peered":"2024-05-21T03:20:32.149552+0000","last_unstale":"2024-05-21T03:20:34.275996+0000","last_undegraded":"2024-05-21T03:20:34.275996+0000","last_fullsized":"2024-05-21T03:20:34.275996+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:20:38.543551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T03:21:13.550802+0000","last_change":"2024-05-21T03:20:32.146413+0000","last_active":"2024-05-21T03:21:13.550802+0000","last_peered":"2024-05-21T03:21:13.550802+0000","last_clean":"2024-05-21T03:21:13.550802+0000","last_became_active":"2024-05-21T03:20:32.146006+0000","last_became_peered":"2024-05-21T03:20:32.146006+0000","last_unstale":"2024-05-21T03:21:13.550802+0000","last_undegraded":"2024-05-21T03:21:13.550802+0000","last_fullsized":"2024-05-21T03:21:13.550802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:05:12.888550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T03:21:13.551438+0000","last_change":"2024-05-21T03:20:32.153654+0000","last_active":"2024-05-21T03:21:13.551438+0000","last_peered":"2024-05-21T03:21:13.551438+0000","last_clean":"2024-05-21T03:21:13.551438+0000","last_became_active":"2024-05-21T03:20:32.153296+0000","last_became_peered":"2024-05-21T03:20:32.153296+0000","last_unstale":"2024-05-21T03:21:13.551438+0000","last_undegraded":"2024-05-21T03:21:13.551438+0000","last_fullsized":"2024-05-21T03:21:13.551438+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:16:09.423236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.273432+0000","last_change":"2024-05-21T03:20:32.138672+0000","last_active":"2024-05-21T03:20:34.273432+0000","last_peered":"2024-05-21T03:20:34.273432+0000","last_clean":"2024-05-21T03:20:34.273432+0000","last_became_active":"2024-05-21T03:20:32.138163+0000","last_became_peered":"2024-05-21T03:20:32.138163+0000","last_unstale":"2024-05-21T03:20:34.273432+0000","last_undegraded":"2024-05-21T03:20:34.273432+0000","last_fullsized":"2024-05-21T03:20:34.273432+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:26:55.012897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.272238+0000","last_change":"2024-05-21T03:20:32.153950+0000","last_active":"2024-05-21T03:20:34.272238+0000","last_peered":"2024-05-21T03:20:34.272238+0000","last_clean":"2024-05-21T03:20:34.272238+0000","last_became_active":"2024-05-21T03:20:32.153567+0000","last_became_peered":"2024-05-21T03:20:32.153567+0000","last_unstale":"2024-05-21T03:20:34.272238+0000","last_undegraded":"2024-05-21T03:20:34.272238+0000","last_fullsized":"2024-05-21T03:20:34.272238+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:28:34.621193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T03:21:13.553377+0000","last_change":"2024-05-21T03:20:32.140038+0000","last_active":"2024-05-21T03:21:13.553377+0000","last_peered":"2024-05-21T03:21:13.553377+0000","last_clean":"2024-05-21T03:21:13.553377+0000","last_became_active":"2024-05-21T03:20:32.139690+0000","last_became_peered":"2024-05-21T03:20:32.139690+0000","last_unstale":"2024-05-21T03:21:13.553377+0000","last_undegraded":"2024-05-21T03:21:13.553377+0000","last_fullsized":"2024-05-21T03:21:13.553377+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:31:24.270755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.275064+0000","last_change":"2024-05-21T03:20:32.143735+0000","last_active":"2024-05-21T03:20:34.275064+0000","last_peered":"2024-05-21T03:20:34.275064+0000","last_clean":"2024-05-21T03:20:34.275064+0000","last_became_active":"2024-05-21T03:20:32.143397+0000","last_became_peered":"2024-05-21T03:20:32.143397+0000","last_unstale":"2024-05-21T03:20:34.275064+0000","last_undegraded":"2024-05-21T03:20:34.275064+0000","last_fullsized":"2024-05-21T03:20:34.275064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:40:01.951608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.560831+0000","last_change":"2024-05-21T03:20:32.156999+0000","last_active":"2024-05-21T03:21:14.560831+0000","last_peered":"2024-05-21T03:21:14.560831+0000","last_clean":"2024-05-21T03:21:14.560831+0000","last_became_active":"2024-05-21T03:20:32.156677+0000","last_became_peered":"2024-05-21T03:20:32.156677+0000","last_unstale":"2024-05-21T03:21:14.560831+0000","last_undegraded":"2024-05-21T03:21:14.560831+0000","last_fullsized":"2024-05-21T03:21:14.560831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:33:24.225667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.134738+0000","last_change":"2024-05-21T03:20:32.145039+0000","last_active":"2024-05-21T03:20:34.134738+0000","last_peered":"2024-05-21T03:20:34.134738+0000","last_clean":"2024-05-21T03:20:34.134738+0000","last_became_active":"2024-05-21T03:20:32.140926+0000","last_became_peered":"2024-05-21T03:20:32.140926+0000","last_unstale":"2024-05-21T03:20:34.134738+0000","last_undegraded":"2024-05-21T03:20:34.134738+0000","last_fullsized":"2024-05-21T03:20:34.134738+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:11:27.049426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T03:21:13.554169+0000","last_change":"2024-05-21T03:20:32.145840+0000","last_active":"2024-05-21T03:21:13.554169+0000","last_peered":"2024-05-21T03:21:13.554169+0000","last_clean":"2024-05-21T03:21:13.554169+0000","last_became_active":"2024-05-21T03:20:32.145291+0000","last_became_peered":"2024-05-21T03:20:32.145291+0000","last_unstale":"2024-05-21T03:21:13.554169+0000","last_undegraded":"2024-05-21T03:21:13.554169+0000","last_fullsized":"2024-05-21T03:21:13.554169+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:22:27.696588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.276364+0000","last_change":"2024-05-21T03:20:32.142522+0000","last_active":"2024-05-21T03:20:34.276364+0000","last_peered":"2024-05-21T03:20:34.276364+0000","last_clean":"2024-05-21T03:20:34.276364+0000","last_became_active":"2024-05-21T03:20:32.142165+0000","last_became_peered":"2024-05-21T03:20:32.142165+0000","last_unstale":"2024-05-21T03:20:34.276364+0000","last_undegraded":"2024-05-21T03:20:34.276364+0000","last_fullsized":"2024-05-21T03:20:34.276364+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:58:39.085834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.273113+0000","last_change":"2024-05-21T03:20:32.142526+0000","last_active":"2024-05-21T03:20:34.273113+0000","last_peered":"2024-05-21T03:20:34.273113+0000","last_clean":"2024-05-21T03:20:34.273113+0000","last_became_active":"2024-05-21T03:20:32.141143+0000","last_became_peered":"2024-05-21T03:20:32.141143+0000","last_unstale":"2024-05-21T03:20:34.273113+0000","last_undegraded":"2024-05-21T03:20:34.273113+0000","last_fullsized":"2024-05-21T03:20:34.273113+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:35:11.336796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.270834+0000","last_change":"2024-05-21T03:20:32.135631+0000","last_active":"2024-05-21T03:20:34.270834+0000","last_peered":"2024-05-21T03:20:34.270834+0000","last_clean":"2024-05-21T03:20:34.270834+0000","last_became_active":"2024-05-21T03:20:32.135302+0000","last_became_peered":"2024-05-21T03:20:32.135302+0000","last_unstale":"2024-05-21T03:20:34.270834+0000","last_undegraded":"2024-05-21T03:20:34.270834+0000","last_fullsized":"2024-05-21T03:20:34.270834+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:05:55.000690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T03:21:13.550967+0000","last_change":"2024-05-21T03:20:32.146580+0000","last_active":"2024-05-21T03:21:13.550967+0000","last_peered":"2024-05-21T03:21:13.550967+0000","last_clean":"2024-05-21T03:21:13.550967+0000","last_became_active":"2024-05-21T03:20:32.145803+0000","last_became_peered":"2024-05-21T03:20:32.145803+0000","last_unstale":"2024-05-21T03:21:13.550967+0000","last_undegraded":"2024-05-21T03:21:13.550967+0000","last_fullsized":"2024-05-21T03:21:13.550967+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:34:09.108799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"peering","last_fresh":"2024-05-21T03:20:34.304094+0000","last_change":"2024-05-21T03:21:13.549865+0000","last_active":"2024-05-21T03:21:13.549865+0000","last_peered":"2024-05-21T03:20:34.304094+0000","last_clean":"2024-05-21T03:20:34.304094+0000","last_became_active":"2024-05-21T03:20:32.142338+0000","last_became_peered":"2024-05-21T03:20:32.142338+0000","last_unstale":"2024-05-21T03:20:34.304094+0000","last_undegraded":"2024-05-21T03:20:34.304094+0000","last_fullsized":"2024-05-21T03:20:34.304094+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4,6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.134081+0000","last_change":"2024-05-21T03:20:32.148944+0000","last_active":"2024-05-21T03:20:34.134081+0000","last_peered":"2024-05-21T03:20:34.134081+0000","last_clean":"2024-05-21T03:20:34.134081+0000","last_became_active":"2024-05-21T03:20:32.148502+0000","last_became_peered":"2024-05-21T03:20:32.148502+0000","last_unstale":"2024-05-21T03:20:34.134081+0000","last_undegraded":"2024-05-21T03:20:34.134081+0000","last_fullsized":"2024-05-21T03:20:34.134081+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:11:00.178105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.276965+0000","last_change":"2024-05-21T03:20:32.150609+0000","last_active":"2024-05-21T03:20:34.276965+0000","last_peered":"2024-05-21T03:20:34.276965+0000","last_clean":"2024-05-21T03:20:34.276965+0000","last_became_active":"2024-05-21T03:20:32.150314+0000","last_became_peered":"2024-05-21T03:20:32.150314+0000","last_unstale":"2024-05-21T03:20:34.276965+0000","last_undegraded":"2024-05-21T03:20:34.276965+0000","last_fullsized":"2024-05-21T03:20:34.276965+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:34:21.869304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.271753+0000","last_change":"2024-05-21T03:20:32.142472+0000","last_active":"2024-05-21T03:20:34.271753+0000","last_peered":"2024-05-21T03:20:34.271753+0000","last_clean":"2024-05-21T03:20:34.271753+0000","last_became_active":"2024-05-21T03:20:32.142161+0000","last_became_peered":"2024-05-21T03:20:32.142161+0000","last_unstale":"2024-05-21T03:20:34.271753+0000","last_undegraded":"2024-05-21T03:20:34.271753+0000","last_fullsized":"2024-05-21T03:20:34.271753+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:26:57.281655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.273932+0000","last_change":"2024-05-21T03:20:32.153987+0000","last_active":"2024-05-21T03:20:34.273932+0000","last_peered":"2024-05-21T03:20:34.273932+0000","last_clean":"2024-05-21T03:20:34.273932+0000","last_became_active":"2024-05-21T03:20:32.153617+0000","last_became_peered":"2024-05-21T03:20:32.153617+0000","last_unstale":"2024-05-21T03:20:34.273932+0000","last_undegraded":"2024-05-21T03:20:34.273932+0000","last_fullsized":"2024-05-21T03:20:34.273932+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:57:07.341758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":58,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:21:11.513921+0000","last_change":"2024-05-21T03:20:29.099928+0000","last_active":"2024-05-21T03:21:11.513921+0000","last_peered":"2024-05-21T03:21:11.513921+0000","last_clean":"2024-05-21T03:21:11.513921+0000","last_became_active":"2024-05-21T03:20:29.099592+0000","last_became_peered":"2024-05-21T03:20:29.099592+0000","last_unstale":"2024-05-21T03:21:11.513921+0000","last_undegraded":"2024-05-21T03:21:11.513921+0000","last_fullsized":"2024-05-21T03:21:11.513921+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:35:34.843325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561455+0000","last_change":"2024-05-21T03:20:32.154962+0000","last_active":"2024-05-21T03:21:14.561455+0000","last_peered":"2024-05-21T03:21:14.561455+0000","last_clean":"2024-05-21T03:21:14.561455+0000","last_became_active":"2024-05-21T03:20:32.154620+0000","last_became_peered":"2024-05-21T03:20:32.154620+0000","last_unstale":"2024-05-21T03:21:14.561455+0000","last_undegraded":"2024-05-21T03:21:14.561455+0000","last_fullsized":"2024-05-21T03:21:14.561455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:56:21.553272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561738+0000","last_change":"2024-05-21T03:20:29.097312+0000","last_active":"2024-05-21T03:21:14.561738+0000","last_peered":"2024-05-21T03:21:14.561738+0000","last_clean":"2024-05-21T03:21:14.561738+0000","last_became_active":"2024-05-21T03:20:29.096977+0000","last_became_peered":"2024-05-21T03:20:29.096977+0000","last_unstale":"2024-05-21T03:21:14.561738+0000","last_undegraded":"2024-05-21T03:21:14.561738+0000","last_fullsized":"2024-05-21T03:21:14.561738+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:28:45.467446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T03:21:13.550899+0000","last_change":"2024-05-21T03:20:32.150035+0000","last_active":"2024-05-21T03:21:13.550899+0000","last_peered":"2024-05-21T03:21:13.550899+0000","last_clean":"2024-05-21T03:21:13.550899+0000","last_became_active":"2024-05-21T03:20:32.149688+0000","last_became_peered":"2024-05-21T03:20:32.149688+0000","last_unstale":"2024-05-21T03:21:13.550899+0000","last_undegraded":"2024-05-21T03:21:13.550899+0000","last_fullsized":"2024-05-21T03:21:13.550899+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:27:00.410770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:50.480488+0000","last_change":"2024-05-21T03:20:29.105397+0000","last_active":"2024-05-21T03:20:50.480488+0000","last_peered":"2024-05-21T03:20:50.480488+0000","last_clean":"2024-05-21T03:20:50.480488+0000","last_became_active":"2024-05-21T03:20:29.105099+0000","last_became_peered":"2024-05-21T03:20:29.105099+0000","last_unstale":"2024-05-21T03:20:50.480488+0000","last_undegraded":"2024-05-21T03:20:50.480488+0000","last_fullsized":"2024-05-21T03:20:50.480488+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:12:12.273627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.134216+0000","last_change":"2024-05-21T03:20:32.150301+0000","last_active":"2024-05-21T03:20:34.134216+0000","last_peered":"2024-05-21T03:20:34.134216+0000","last_clean":"2024-05-21T03:20:34.134216+0000","last_became_active":"2024-05-21T03:20:32.148475+0000","last_became_peered":"2024-05-21T03:20:32.148475+0000","last_unstale":"2024-05-21T03:20:34.134216+0000","last_undegraded":"2024-05-21T03:20:34.134216+0000","last_fullsized":"2024-05-21T03:20:34.134216+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:59:30.339660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:49.470795+0000","last_change":"2024-05-21T03:20:29.098861+0000","last_active":"2024-05-21T03:20:49.470795+0000","last_peered":"2024-05-21T03:20:49.470795+0000","last_clean":"2024-05-21T03:20:49.470795+0000","last_became_active":"2024-05-21T03:20:29.097874+0000","last_became_peered":"2024-05-21T03:20:29.097874+0000","last_unstale":"2024-05-21T03:20:49.470795+0000","last_undegraded":"2024-05-21T03:20:49.470795+0000","last_fullsized":"2024-05-21T03:20:49.470795+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:25:36.227339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.134491+0000","last_change":"2024-05-21T03:20:32.142820+0000","last_active":"2024-05-21T03:20:34.134491+0000","last_peered":"2024-05-21T03:20:34.134491+0000","last_clean":"2024-05-21T03:20:34.134491+0000","last_became_active":"2024-05-21T03:20:32.142396+0000","last_became_peered":"2024-05-21T03:20:32.142396+0000","last_unstale":"2024-05-21T03:20:34.134491+0000","last_undegraded":"2024-05-21T03:20:34.134491+0000","last_fullsized":"2024-05-21T03:20:34.134491+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:30:14.452389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":51,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:21:10.513887+0000","last_change":"2024-05-21T03:20:29.103153+0000","last_active":"2024-05-21T03:21:10.513887+0000","last_peered":"2024-05-21T03:21:10.513887+0000","last_clean":"2024-05-21T03:21:10.513887+0000","last_became_active":"2024-05-21T03:20:29.102819+0000","last_became_peered":"2024-05-21T03:20:29.102819+0000","last_unstale":"2024-05-21T03:21:10.513887+0000","last_undegraded":"2024-05-21T03:21:10.513887+0000","last_fullsized":"2024-05-21T03:21:10.513887+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:47:49.843926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.271029+0000","last_change":"2024-05-21T03:20:32.142263+0000","last_active":"2024-05-21T03:20:34.271029+0000","last_peered":"2024-05-21T03:20:34.271029+0000","last_clean":"2024-05-21T03:20:34.271029+0000","last_became_active":"2024-05-21T03:20:32.141820+0000","last_became_peered":"2024-05-21T03:20:32.141820+0000","last_unstale":"2024-05-21T03:20:34.271029+0000","last_undegraded":"2024-05-21T03:20:34.271029+0000","last_fullsized":"2024-05-21T03:20:34.271029+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:45:20.021237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T03:21:13.553201+0000","last_change":"2024-05-21T03:20:29.100088+0000","last_active":"2024-05-21T03:21:13.553201+0000","last_peered":"2024-05-21T03:21:13.553201+0000","last_clean":"2024-05-21T03:21:13.553201+0000","last_became_active":"2024-05-21T03:20:29.097343+0000","last_became_peered":"2024-05-21T03:20:29.097343+0000","last_unstale":"2024-05-21T03:21:13.553201+0000","last_undegraded":"2024-05-21T03:21:13.553201+0000","last_fullsized":"2024-05-21T03:21:13.553201+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:00:35.682393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.134153+0000","last_change":"2024-05-21T03:20:32.153907+0000","last_active":"2024-05-21T03:20:34.134153+0000","last_peered":"2024-05-21T03:20:34.134153+0000","last_clean":"2024-05-21T03:20:34.134153+0000","last_became_active":"2024-05-21T03:20:32.153507+0000","last_became_peered":"2024-05-21T03:20:32.153507+0000","last_unstale":"2024-05-21T03:20:34.134153+0000","last_undegraded":"2024-05-21T03:20:34.134153+0000","last_fullsized":"2024-05-21T03:20:34.134153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:09:15.503685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:47.445753+0000","last_change":"2024-05-21T03:20:29.097945+0000","last_active":"2024-05-21T03:20:47.445753+0000","last_peered":"2024-05-21T03:20:47.445753+0000","last_clean":"2024-05-21T03:20:47.445753+0000","last_became_active":"2024-05-21T03:20:29.097624+0000","last_became_peered":"2024-05-21T03:20:29.097624+0000","last_unstale":"2024-05-21T03:20:47.445753+0000","last_undegraded":"2024-05-21T03:20:47.445753+0000","last_fullsized":"2024-05-21T03:20:47.445753+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:01:01.694893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.560719+0000","last_change":"2024-05-21T03:20:32.148955+0000","last_active":"2024-05-21T03:21:14.560719+0000","last_peered":"2024-05-21T03:21:14.560719+0000","last_clean":"2024-05-21T03:21:14.560719+0000","last_became_active":"2024-05-21T03:20:32.148631+0000","last_became_peered":"2024-05-21T03:20:32.148631+0000","last_unstale":"2024-05-21T03:21:14.560719+0000","last_undegraded":"2024-05-21T03:21:14.560719+0000","last_fullsized":"2024-05-21T03:21:14.560719+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:36:25.820580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:49.471530+0000","last_change":"2024-05-21T03:20:29.100902+0000","last_active":"2024-05-21T03:20:49.471530+0000","last_peered":"2024-05-21T03:20:49.471530+0000","last_clean":"2024-05-21T03:20:49.471530+0000","last_became_active":"2024-05-21T03:20:29.100570+0000","last_became_peered":"2024-05-21T03:20:29.100570+0000","last_unstale":"2024-05-21T03:20:49.471530+0000","last_undegraded":"2024-05-21T03:20:49.471530+0000","last_fullsized":"2024-05-21T03:20:49.471530+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:58:36.651338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.273110+0000","last_change":"2024-05-21T03:20:32.150724+0000","last_active":"2024-05-21T03:20:34.273110+0000","last_peered":"2024-05-21T03:20:34.273110+0000","last_clean":"2024-05-21T03:20:34.273110+0000","last_became_active":"2024-05-21T03:20:32.150371+0000","last_became_peered":"2024-05-21T03:20:32.150371+0000","last_unstale":"2024-05-21T03:20:34.273110+0000","last_undegraded":"2024-05-21T03:20:34.273110+0000","last_fullsized":"2024-05-21T03:20:34.273110+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:44:32.321192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":76,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T03:21:13.553954+0000","last_change":"2024-05-21T03:20:29.095045+0000","last_active":"2024-05-21T03:21:13.553954+0000","last_peered":"2024-05-21T03:21:13.553954+0000","last_clean":"2024-05-21T03:21:13.553954+0000","last_became_active":"2024-05-21T03:20:29.094552+0000","last_became_peered":"2024-05-21T03:20:29.094552+0000","last_unstale":"2024-05-21T03:21:13.553954+0000","last_undegraded":"2024-05-21T03:21:13.553954+0000","last_fullsized":"2024-05-21T03:21:13.553954+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:36:15.155125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.134572+0000","last_change":"2024-05-21T03:20:32.149813+0000","last_active":"2024-05-21T03:20:34.134572+0000","last_peered":"2024-05-21T03:20:34.134572+0000","last_clean":"2024-05-21T03:20:34.134572+0000","last_became_active":"2024-05-21T03:20:32.148422+0000","last_became_peered":"2024-05-21T03:20:32.148422+0000","last_unstale":"2024-05-21T03:20:34.134572+0000","last_undegraded":"2024-05-21T03:20:34.134572+0000","last_fullsized":"2024-05-21T03:20:34.134572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:55:35.115463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:49.469697+0000","last_change":"2024-05-21T03:20:29.101463+0000","last_active":"2024-05-21T03:20:49.469697+0000","last_peered":"2024-05-21T03:20:49.469697+0000","last_clean":"2024-05-21T03:20:49.469697+0000","last_became_active":"2024-05-21T03:20:29.101127+0000","last_became_peered":"2024-05-21T03:20:29.101127+0000","last_unstale":"2024-05-21T03:20:49.469697+0000","last_undegraded":"2024-05-21T03:20:49.469697+0000","last_fullsized":"2024-05-21T03:20:49.469697+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:58:17.408315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"peering","last_fresh":"2024-05-21T03:20:34.304165+0000","last_change":"2024-05-21T03:21:13.549888+0000","last_active":"2024-05-21T03:21:13.549888+0000","last_peered":"2024-05-21T03:20:34.304165+0000","last_clean":"2024-05-21T03:20:34.304165+0000","last_became_active":"2024-05-21T03:20:32.144900+0000","last_became_peered":"2024-05-21T03:20:32.144900+0000","last_unstale":"2024-05-21T03:20:34.304165+0000","last_undegraded":"2024-05-21T03:20:34.304165+0000","last_fullsized":"2024-05-21T03:20:34.304165+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5,8],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:48.455514+0000","last_change":"2024-05-21T03:20:29.098725+0000","last_active":"2024-05-21T03:20:48.455514+0000","last_peered":"2024-05-21T03:20:48.455514+0000","last_clean":"2024-05-21T03:20:48.455514+0000","last_became_active":"2024-05-21T03:20:29.098471+0000","last_became_peered":"2024-05-21T03:20:29.098471+0000","last_unstale":"2024-05-21T03:20:48.455514+0000","last_undegraded":"2024-05-21T03:20:48.455514+0000","last_fullsized":"2024-05-21T03:20:48.455514+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:17:11.059825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.134349+0000","last_change":"2024-05-21T03:20:32.152489+0000","last_active":"2024-05-21T03:20:34.134349+0000","last_peered":"2024-05-21T03:20:34.134349+0000","last_clean":"2024-05-21T03:20:34.134349+0000","last_became_active":"2024-05-21T03:20:32.151804+0000","last_became_peered":"2024-05-21T03:20:32.151804+0000","last_unstale":"2024-05-21T03:20:34.134349+0000","last_undegraded":"2024-05-21T03:20:34.134349+0000","last_fullsized":"2024-05-21T03:20:34.134349+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:27:44.640559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'28","reported_seq":108,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T03:21:13.553898+0000","last_change":"2024-05-21T03:20:29.099968+0000","last_active":"2024-05-21T03:21:13.553898+0000","last_peered":"2024-05-21T03:21:13.553898+0000","last_clean":"2024-05-21T03:21:13.553898+0000","last_became_active":"2024-05-21T03:20:29.099181+0000","last_became_peered":"2024-05-21T03:20:29.099181+0000","last_unstale":"2024-05-21T03:21:13.553898+0000","last_undegraded":"2024-05-21T03:21:13.553898+0000","last_fullsized":"2024-05-21T03:21:13.553898+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:35:33.934412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54367,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":56,"num_read_kb":80,"num_write":30,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.272012+0000","last_change":"2024-05-21T03:20:32.144024+0000","last_active":"2024-05-21T03:20:34.272012+0000","last_peered":"2024-05-21T03:20:34.272012+0000","last_clean":"2024-05-21T03:20:34.272012+0000","last_became_active":"2024-05-21T03:20:32.143668+0000","last_became_peered":"2024-05-21T03:20:32.143668+0000","last_unstale":"2024-05-21T03:20:34.272012+0000","last_undegraded":"2024-05-21T03:20:34.272012+0000","last_fullsized":"2024-05-21T03:20:34.272012+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:11:31.100802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:50.480792+0000","last_change":"2024-05-21T03:20:29.101770+0000","last_active":"2024-05-21T03:20:50.480792+0000","last_peered":"2024-05-21T03:20:50.480792+0000","last_clean":"2024-05-21T03:20:50.480792+0000","last_became_active":"2024-05-21T03:20:29.100407+0000","last_became_peered":"2024-05-21T03:20:29.100407+0000","last_unstale":"2024-05-21T03:20:50.480792+0000","last_undegraded":"2024-05-21T03:20:50.480792+0000","last_fullsized":"2024-05-21T03:20:50.480792+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:01:33.991246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.270509+0000","last_change":"2024-05-21T03:20:32.147479+0000","last_active":"2024-05-21T03:20:34.270509+0000","last_peered":"2024-05-21T03:20:34.270509+0000","last_clean":"2024-05-21T03:20:34.270509+0000","last_became_active":"2024-05-21T03:20:32.147160+0000","last_became_peered":"2024-05-21T03:20:32.147160+0000","last_unstale":"2024-05-21T03:20:34.270509+0000","last_undegraded":"2024-05-21T03:20:34.270509+0000","last_fullsized":"2024-05-21T03:20:34.270509+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:51:59.343380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:49.471824+0000","last_change":"2024-05-21T03:20:29.101875+0000","last_active":"2024-05-21T03:20:49.471824+0000","last_peered":"2024-05-21T03:20:49.471824+0000","last_clean":"2024-05-21T03:20:49.471824+0000","last_became_active":"2024-05-21T03:20:29.101468+0000","last_became_peered":"2024-05-21T03:20:29.101468+0000","last_unstale":"2024-05-21T03:20:49.471824+0000","last_undegraded":"2024-05-21T03:20:49.471824+0000","last_fullsized":"2024-05-21T03:20:49.471824+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:10:19.577836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.276098+0000","last_change":"2024-05-21T03:20:32.139002+0000","last_active":"2024-05-21T03:20:34.276098+0000","last_peered":"2024-05-21T03:20:34.276098+0000","last_clean":"2024-05-21T03:20:34.276098+0000","last_became_active":"2024-05-21T03:20:32.138619+0000","last_became_peered":"2024-05-21T03:20:32.138619+0000","last_unstale":"2024-05-21T03:20:34.276098+0000","last_undegraded":"2024-05-21T03:20:34.276098+0000","last_fullsized":"2024-05-21T03:20:34.276098+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:21:36.136283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":52,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:21:10.513166+0000","last_change":"2024-05-21T03:20:29.101532+0000","last_active":"2024-05-21T03:21:10.513166+0000","last_peered":"2024-05-21T03:21:10.513166+0000","last_clean":"2024-05-21T03:21:10.513166+0000","last_became_active":"2024-05-21T03:20:29.101206+0000","last_became_peered":"2024-05-21T03:20:29.101206+0000","last_unstale":"2024-05-21T03:21:10.513166+0000","last_undegraded":"2024-05-21T03:21:10.513166+0000","last_fullsized":"2024-05-21T03:21:10.513166+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:47:38.555995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.275268+0000","last_change":"2024-05-21T03:20:32.146866+0000","last_active":"2024-05-21T03:20:34.275268+0000","last_peered":"2024-05-21T03:20:34.275268+0000","last_clean":"2024-05-21T03:20:34.275268+0000","last_became_active":"2024-05-21T03:20:32.146463+0000","last_became_peered":"2024-05-21T03:20:32.146463+0000","last_unstale":"2024-05-21T03:20:34.275268+0000","last_undegraded":"2024-05-21T03:20:34.275268+0000","last_fullsized":"2024-05-21T03:20:34.275268+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:06:54.359922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":40,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:49.469295+0000","last_change":"2024-05-21T03:20:29.103910+0000","last_active":"2024-05-21T03:20:49.469295+0000","last_peered":"2024-05-21T03:20:49.469295+0000","last_clean":"2024-05-21T03:20:49.469295+0000","last_became_active":"2024-05-21T03:20:29.103583+0000","last_became_peered":"2024-05-21T03:20:29.103583+0000","last_unstale":"2024-05-21T03:20:49.469295+0000","last_undegraded":"2024-05-21T03:20:49.469295+0000","last_fullsized":"2024-05-21T03:20:49.469295+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:57:15.665158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.134133+0000","last_change":"2024-05-21T03:20:32.141242+0000","last_active":"2024-05-21T03:20:34.134133+0000","last_peered":"2024-05-21T03:20:34.134133+0000","last_clean":"2024-05-21T03:20:34.134133+0000","last_became_active":"2024-05-21T03:20:32.140892+0000","last_became_peered":"2024-05-21T03:20:32.140892+0000","last_unstale":"2024-05-21T03:20:34.134133+0000","last_undegraded":"2024-05-21T03:20:34.134133+0000","last_fullsized":"2024-05-21T03:20:34.134133+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:42:13.997708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:48.441864+0000","last_change":"2024-05-21T03:20:29.098973+0000","last_active":"2024-05-21T03:20:48.441864+0000","last_peered":"2024-05-21T03:20:48.441864+0000","last_clean":"2024-05-21T03:20:48.441864+0000","last_became_active":"2024-05-21T03:20:29.098694+0000","last_became_peered":"2024-05-21T03:20:29.098694+0000","last_unstale":"2024-05-21T03:20:48.441864+0000","last_undegraded":"2024-05-21T03:20:48.441864+0000","last_fullsized":"2024-05-21T03:20:48.441864+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:55:17.915820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.276573+0000","last_change":"2024-05-21T03:20:32.147465+0000","last_active":"2024-05-21T03:20:34.276573+0000","last_peered":"2024-05-21T03:20:34.276573+0000","last_clean":"2024-05-21T03:20:34.276573+0000","last_became_active":"2024-05-21T03:20:32.147073+0000","last_became_peered":"2024-05-21T03:20:32.147073+0000","last_unstale":"2024-05-21T03:20:34.276573+0000","last_undegraded":"2024-05-21T03:20:34.276573+0000","last_fullsized":"2024-05-21T03:20:34.276573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:29:05.451370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.274131+0000","last_change":"2024-05-21T03:20:29.103452+0000","last_active":"2024-05-21T03:20:34.274131+0000","last_peered":"2024-05-21T03:20:34.274131+0000","last_clean":"2024-05-21T03:20:34.274131+0000","last_became_active":"2024-05-21T03:20:29.103018+0000","last_became_peered":"2024-05-21T03:20:29.103018+0000","last_unstale":"2024-05-21T03:20:34.274131+0000","last_undegraded":"2024-05-21T03:20:34.274131+0000","last_fullsized":"2024-05-21T03:20:34.274131+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:10:01.569500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T03:21:13.553494+0000","last_change":"2024-05-21T03:20:32.146456+0000","last_active":"2024-05-21T03:21:13.553494+0000","last_peered":"2024-05-21T03:21:13.553494+0000","last_clean":"2024-05-21T03:21:13.553494+0000","last_became_active":"2024-05-21T03:20:32.146105+0000","last_became_peered":"2024-05-21T03:20:32.146105+0000","last_unstale":"2024-05-21T03:21:13.553494+0000","last_undegraded":"2024-05-21T03:21:13.553494+0000","last_fullsized":"2024-05-21T03:21:13.553494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:00:30.550178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'6","reported_seq":51,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:21:10.512972+0000","last_change":"2024-05-21T03:20:29.100262+0000","last_active":"2024-05-21T03:21:10.512972+0000","last_peered":"2024-05-21T03:21:10.512972+0000","last_clean":"2024-05-21T03:21:10.512972+0000","last_became_active":"2024-05-21T03:20:29.099917+0000","last_became_peered":"2024-05-21T03:20:29.099917+0000","last_unstale":"2024-05-21T03:21:10.512972+0000","last_undegraded":"2024-05-21T03:21:10.512972+0000","last_fullsized":"2024-05-21T03:21:10.512972+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:55:48.608341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T03:21:13.553487+0000","last_change":"2024-05-21T03:20:32.142776+0000","last_active":"2024-05-21T03:21:13.553487+0000","last_peered":"2024-05-21T03:21:13.553487+0000","last_clean":"2024-05-21T03:21:13.553487+0000","last_became_active":"2024-05-21T03:20:32.142430+0000","last_became_peered":"2024-05-21T03:20:32.142430+0000","last_unstale":"2024-05-21T03:21:13.553487+0000","last_undegraded":"2024-05-21T03:21:13.553487+0000","last_fullsized":"2024-05-21T03:21:13.553487+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:54:28.682343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.134490+0000","last_change":"2024-05-21T03:20:32.139506+0000","last_active":"2024-05-21T03:20:34.134490+0000","last_peered":"2024-05-21T03:20:34.134490+0000","last_clean":"2024-05-21T03:20:34.134490+0000","last_became_active":"2024-05-21T03:20:32.138136+0000","last_became_peered":"2024-05-21T03:20:32.138136+0000","last_unstale":"2024-05-21T03:20:34.134490+0000","last_undegraded":"2024-05-21T03:20:34.134490+0000","last_fullsized":"2024-05-21T03:20:34.134490+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:31:24.996276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":49,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:21:09.510780+0000","last_change":"2024-05-21T03:20:29.096546+0000","last_active":"2024-05-21T03:21:09.510780+0000","last_peered":"2024-05-21T03:21:09.510780+0000","last_clean":"2024-05-21T03:21:09.510780+0000","last_became_active":"2024-05-21T03:20:29.096184+0000","last_became_peered":"2024-05-21T03:20:29.096184+0000","last_unstale":"2024-05-21T03:21:09.510780+0000","last_undegraded":"2024-05-21T03:21:09.510780+0000","last_fullsized":"2024-05-21T03:21:09.510780+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:09:12.832745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561399+0000","last_change":"2024-05-21T03:20:32.156231+0000","last_active":"2024-05-21T03:21:14.561399+0000","last_peered":"2024-05-21T03:21:14.561399+0000","last_clean":"2024-05-21T03:21:14.561399+0000","last_became_active":"2024-05-21T03:20:32.154035+0000","last_became_peered":"2024-05-21T03:20:32.154035+0000","last_unstale":"2024-05-21T03:21:14.561399+0000","last_undegraded":"2024-05-21T03:21:14.561399+0000","last_fullsized":"2024-05-21T03:21:14.561399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:30:53.820292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:50.481092+0000","last_change":"2024-05-21T03:20:29.100422+0000","last_active":"2024-05-21T03:20:50.481092+0000","last_peered":"2024-05-21T03:20:50.481092+0000","last_clean":"2024-05-21T03:20:50.481092+0000","last_became_active":"2024-05-21T03:20:29.099993+0000","last_became_peered":"2024-05-21T03:20:29.099993+0000","last_unstale":"2024-05-21T03:20:50.481092+0000","last_undegraded":"2024-05-21T03:20:50.481092+0000","last_fullsized":"2024-05-21T03:20:50.481092+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:03:15.456447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":94,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.275026+0000","last_change":"2024-05-21T03:19:05.647808+0000","last_active":"2024-05-21T03:20:34.275026+0000","last_peered":"2024-05-21T03:20:34.275026+0000","last_clean":"2024-05-21T03:20:34.275026+0000","last_became_active":"2024-05-21T03:19:05.641865+0000","last_became_peered":"2024-05-21T03:19:05.641865+0000","last_unstale":"2024-05-21T03:20:34.275026+0000","last_undegraded":"2024-05-21T03:20:34.275026+0000","last_fullsized":"2024-05-21T03:20:34.275026+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:15:42.008493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:15:42.008493+0000","last_clean_scrub_stamp":"2024-05-21T03:15:42.008493+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:26:58.040196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.270394+0000","last_change":"2024-05-21T03:20:32.138182+0000","last_active":"2024-05-21T03:20:34.270394+0000","last_peered":"2024-05-21T03:20:34.270394+0000","last_clean":"2024-05-21T03:20:34.270394+0000","last_became_active":"2024-05-21T03:20:32.137842+0000","last_became_peered":"2024-05-21T03:20:32.137842+0000","last_unstale":"2024-05-21T03:20:34.270394+0000","last_undegraded":"2024-05-21T03:20:34.270394+0000","last_fullsized":"2024-05-21T03:20:34.270394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:43:50.070160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:47.447160+0000","last_change":"2024-05-21T03:20:29.104246+0000","last_active":"2024-05-21T03:20:47.447160+0000","last_peered":"2024-05-21T03:20:47.447160+0000","last_clean":"2024-05-21T03:20:47.447160+0000","last_became_active":"2024-05-21T03:20:29.103910+0000","last_became_peered":"2024-05-21T03:20:29.103910+0000","last_unstale":"2024-05-21T03:20:47.447160+0000","last_undegraded":"2024-05-21T03:20:47.447160+0000","last_fullsized":"2024-05-21T03:20:47.447160+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:39:28.029283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:49.470226+0000","last_change":"2024-05-21T03:20:29.102020+0000","last_active":"2024-05-21T03:20:49.470226+0000","last_peered":"2024-05-21T03:20:49.470226+0000","last_clean":"2024-05-21T03:20:49.470226+0000","last_became_active":"2024-05-21T03:20:29.100429+0000","last_became_peered":"2024-05-21T03:20:29.100429+0000","last_unstale":"2024-05-21T03:20:49.470226+0000","last_undegraded":"2024-05-21T03:20:49.470226+0000","last_fullsized":"2024-05-21T03:20:49.470226+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:09:01.621074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.270792+0000","last_change":"2024-05-21T03:20:32.137130+0000","last_active":"2024-05-21T03:20:34.270792+0000","last_peered":"2024-05-21T03:20:34.270792+0000","last_clean":"2024-05-21T03:20:34.270792+0000","last_became_active":"2024-05-21T03:20:32.136821+0000","last_became_peered":"2024-05-21T03:20:32.136821+0000","last_unstale":"2024-05-21T03:20:34.270792+0000","last_undegraded":"2024-05-21T03:20:34.270792+0000","last_fullsized":"2024-05-21T03:20:34.270792+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:53:41.290370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'13","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:21:08.274520+0000","last_change":"2024-05-21T03:20:29.099869+0000","last_active":"2024-05-21T03:21:08.274520+0000","last_peered":"2024-05-21T03:21:08.274520+0000","last_clean":"2024-05-21T03:21:08.274520+0000","last_became_active":"2024-05-21T03:20:29.099534+0000","last_became_peered":"2024-05-21T03:20:29.099534+0000","last_unstale":"2024-05-21T03:21:08.274520+0000","last_undegraded":"2024-05-21T03:21:08.274520+0000","last_fullsized":"2024-05-21T03:21:08.274520+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:01:11.281544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561631+0000","last_change":"2024-05-21T03:20:32.153921+0000","last_active":"2024-05-21T03:21:14.561631+0000","last_peered":"2024-05-21T03:21:14.561631+0000","last_clean":"2024-05-21T03:21:14.561631+0000","last_became_active":"2024-05-21T03:20:32.153594+0000","last_became_peered":"2024-05-21T03:20:32.153594+0000","last_unstale":"2024-05-21T03:21:14.561631+0000","last_undegraded":"2024-05-21T03:21:14.561631+0000","last_fullsized":"2024-05-21T03:21:14.561631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:27:59.426942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:49.471859+0000","last_change":"2024-05-21T03:20:29.097399+0000","last_active":"2024-05-21T03:20:49.471859+0000","last_peered":"2024-05-21T03:20:49.471859+0000","last_clean":"2024-05-21T03:20:49.471859+0000","last_became_active":"2024-05-21T03:20:29.097010+0000","last_became_peered":"2024-05-21T03:20:29.097010+0000","last_unstale":"2024-05-21T03:20:49.471859+0000","last_undegraded":"2024-05-21T03:20:49.471859+0000","last_fullsized":"2024-05-21T03:20:49.471859+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:54:53.273947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"peering","last_fresh":"2024-05-21T03:20:34.304156+0000","last_change":"2024-05-21T03:21:13.549909+0000","last_active":"2024-05-21T03:21:13.549909+0000","last_peered":"2024-05-21T03:20:34.304156+0000","last_clean":"2024-05-21T03:20:34.304156+0000","last_became_active":"2024-05-21T03:20:32.152251+0000","last_became_peered":"2024-05-21T03:20:32.152251+0000","last_unstale":"2024-05-21T03:20:34.304156+0000","last_undegraded":"2024-05-21T03:20:34.304156+0000","last_fullsized":"2024-05-21T03:20:34.304156+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,4],"acting":[0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,4,8],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:46.433642+0000","last_change":"2024-05-21T03:20:29.103526+0000","last_active":"2024-05-21T03:20:46.433642+0000","last_peered":"2024-05-21T03:20:46.433642+0000","last_clean":"2024-05-21T03:20:46.433642+0000","last_became_active":"2024-05-21T03:20:29.103067+0000","last_became_peered":"2024-05-21T03:20:29.103067+0000","last_unstale":"2024-05-21T03:20:46.433642+0000","last_undegraded":"2024-05-21T03:20:46.433642+0000","last_fullsized":"2024-05-21T03:20:46.433642+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:54:47.566949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.270829+0000","last_change":"2024-05-21T03:20:32.141088+0000","last_active":"2024-05-21T03:20:34.270829+0000","last_peered":"2024-05-21T03:20:34.270829+0000","last_clean":"2024-05-21T03:20:34.270829+0000","last_became_active":"2024-05-21T03:20:32.140744+0000","last_became_peered":"2024-05-21T03:20:32.140744+0000","last_unstale":"2024-05-21T03:20:34.270829+0000","last_undegraded":"2024-05-21T03:20:34.270829+0000","last_fullsized":"2024-05-21T03:20:34.270829+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:57:35.591164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.560757+0000","last_change":"2024-05-21T03:20:29.093440+0000","last_active":"2024-05-21T03:21:14.560757+0000","last_peered":"2024-05-21T03:21:14.560757+0000","last_clean":"2024-05-21T03:21:14.560757+0000","last_became_active":"2024-05-21T03:20:29.092935+0000","last_became_peered":"2024-05-21T03:20:29.092935+0000","last_unstale":"2024-05-21T03:21:14.560757+0000","last_undegraded":"2024-05-21T03:21:14.560757+0000","last_fullsized":"2024-05-21T03:21:14.560757+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:18:00.690465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.276618+0000","last_change":"2024-05-21T03:20:32.143781+0000","last_active":"2024-05-21T03:20:34.276618+0000","last_peered":"2024-05-21T03:20:34.276618+0000","last_clean":"2024-05-21T03:20:34.276618+0000","last_became_active":"2024-05-21T03:20:32.143391+0000","last_became_peered":"2024-05-21T03:20:32.143391+0000","last_unstale":"2024-05-21T03:20:34.276618+0000","last_undegraded":"2024-05-21T03:20:34.276618+0000","last_fullsized":"2024-05-21T03:20:34.276618+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:00:41.506561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:48.455879+0000","last_change":"2024-05-21T03:20:29.101342+0000","last_active":"2024-05-21T03:20:48.455879+0000","last_peered":"2024-05-21T03:20:48.455879+0000","last_clean":"2024-05-21T03:20:48.455879+0000","last_became_active":"2024-05-21T03:20:29.100997+0000","last_became_peered":"2024-05-21T03:20:29.100997+0000","last_unstale":"2024-05-21T03:20:48.455879+0000","last_undegraded":"2024-05-21T03:20:48.455879+0000","last_fullsized":"2024-05-21T03:20:48.455879+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:05:31.801669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561184+0000","last_change":"2024-05-21T03:20:32.150563+0000","last_active":"2024-05-21T03:21:14.561184+0000","last_peered":"2024-05-21T03:21:14.561184+0000","last_clean":"2024-05-21T03:21:14.561184+0000","last_became_active":"2024-05-21T03:20:32.150231+0000","last_became_peered":"2024-05-21T03:20:32.150231+0000","last_unstale":"2024-05-21T03:21:14.561184+0000","last_undegraded":"2024-05-21T03:21:14.561184+0000","last_fullsized":"2024-05-21T03:21:14.561184+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:10:00.251284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:46.431890+0000","last_change":"2024-05-21T03:20:29.101576+0000","last_active":"2024-05-21T03:20:46.431890+0000","last_peered":"2024-05-21T03:20:46.431890+0000","last_clean":"2024-05-21T03:20:46.431890+0000","last_became_active":"2024-05-21T03:20:29.101270+0000","last_became_peered":"2024-05-21T03:20:29.101270+0000","last_unstale":"2024-05-21T03:20:46.431890+0000","last_undegraded":"2024-05-21T03:20:46.431890+0000","last_fullsized":"2024-05-21T03:20:46.431890+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:50:10.579901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.275702+0000","last_change":"2024-05-21T03:20:32.141340+0000","last_active":"2024-05-21T03:20:34.275702+0000","last_peered":"2024-05-21T03:20:34.275702+0000","last_clean":"2024-05-21T03:20:34.275702+0000","last_became_active":"2024-05-21T03:20:32.140792+0000","last_became_peered":"2024-05-21T03:20:32.140792+0000","last_unstale":"2024-05-21T03:20:34.275702+0000","last_undegraded":"2024-05-21T03:20:34.275702+0000","last_fullsized":"2024-05-21T03:20:34.275702+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:35:05.842929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'6","reported_seq":51,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:21:09.512106+0000","last_change":"2024-05-21T03:20:29.100715+0000","last_active":"2024-05-21T03:21:09.512106+0000","last_peered":"2024-05-21T03:21:09.512106+0000","last_clean":"2024-05-21T03:21:09.512106+0000","last_became_active":"2024-05-21T03:20:29.099198+0000","last_became_peered":"2024-05-21T03:20:29.099198+0000","last_unstale":"2024-05-21T03:21:09.512106+0000","last_undegraded":"2024-05-21T03:21:09.512106+0000","last_fullsized":"2024-05-21T03:21:09.512106+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:18:36.637058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.276680+0000","last_change":"2024-05-21T03:20:32.146674+0000","last_active":"2024-05-21T03:20:34.276680+0000","last_peered":"2024-05-21T03:20:34.276680+0000","last_clean":"2024-05-21T03:20:34.276680+0000","last_became_active":"2024-05-21T03:20:32.146150+0000","last_became_peered":"2024-05-21T03:20:32.146150+0000","last_unstale":"2024-05-21T03:20:34.276680+0000","last_undegraded":"2024-05-21T03:20:34.276680+0000","last_fullsized":"2024-05-21T03:20:34.276680+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:39:09.905640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:49.470981+0000","last_change":"2024-05-21T03:20:29.097613+0000","last_active":"2024-05-21T03:20:49.470981+0000","last_peered":"2024-05-21T03:20:49.470981+0000","last_clean":"2024-05-21T03:20:49.470981+0000","last_became_active":"2024-05-21T03:20:29.097331+0000","last_became_peered":"2024-05-21T03:20:29.097331+0000","last_unstale":"2024-05-21T03:20:49.470981+0000","last_undegraded":"2024-05-21T03:20:49.470981+0000","last_fullsized":"2024-05-21T03:20:49.470981+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:56:14.720420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.275506+0000","last_change":"2024-05-21T03:20:32.151569+0000","last_active":"2024-05-21T03:20:34.275506+0000","last_peered":"2024-05-21T03:20:34.275506+0000","last_clean":"2024-05-21T03:20:34.275506+0000","last_became_active":"2024-05-21T03:20:32.151180+0000","last_became_peered":"2024-05-21T03:20:32.151180+0000","last_unstale":"2024-05-21T03:20:34.275506+0000","last_undegraded":"2024-05-21T03:20:34.275506+0000","last_fullsized":"2024-05-21T03:20:34.275506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:36:33.011475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:49.469417+0000","last_change":"2024-05-21T03:20:29.103456+0000","last_active":"2024-05-21T03:20:49.469417+0000","last_peered":"2024-05-21T03:20:49.469417+0000","last_clean":"2024-05-21T03:20:49.469417+0000","last_became_active":"2024-05-21T03:20:29.103008+0000","last_became_peered":"2024-05-21T03:20:29.103008+0000","last_unstale":"2024-05-21T03:20:49.469417+0000","last_undegraded":"2024-05-21T03:20:49.469417+0000","last_fullsized":"2024-05-21T03:20:49.469417+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:02:37.951347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.277153+0000","last_change":"2024-05-21T03:20:32.146986+0000","last_active":"2024-05-21T03:20:34.277153+0000","last_peered":"2024-05-21T03:20:34.277153+0000","last_clean":"2024-05-21T03:20:34.277153+0000","last_became_active":"2024-05-21T03:20:32.146355+0000","last_became_peered":"2024-05-21T03:20:32.146355+0000","last_unstale":"2024-05-21T03:20:34.277153+0000","last_undegraded":"2024-05-21T03:20:34.277153+0000","last_fullsized":"2024-05-21T03:20:34.277153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:36:29.503099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.276808+0000","last_change":"2024-05-21T03:20:32.148699+0000","last_active":"2024-05-21T03:20:34.276808+0000","last_peered":"2024-05-21T03:20:34.276808+0000","last_clean":"2024-05-21T03:20:34.276808+0000","last_became_active":"2024-05-21T03:20:32.148361+0000","last_became_peered":"2024-05-21T03:20:32.148361+0000","last_unstale":"2024-05-21T03:20:34.276808+0000","last_undegraded":"2024-05-21T03:20:34.276808+0000","last_fullsized":"2024-05-21T03:20:34.276808+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:11:46.822382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.273385+0000","last_change":"2024-05-21T03:20:32.137532+0000","last_active":"2024-05-21T03:20:34.273385+0000","last_peered":"2024-05-21T03:20:34.273385+0000","last_clean":"2024-05-21T03:20:34.273385+0000","last_became_active":"2024-05-21T03:20:32.137182+0000","last_became_peered":"2024-05-21T03:20:32.137182+0000","last_unstale":"2024-05-21T03:20:34.273385+0000","last_undegraded":"2024-05-21T03:20:34.273385+0000","last_fullsized":"2024-05-21T03:20:34.273385+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:25:44.207866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.273695+0000","last_change":"2024-05-21T03:20:32.153912+0000","last_active":"2024-05-21T03:20:34.273695+0000","last_peered":"2024-05-21T03:20:34.273695+0000","last_clean":"2024-05-21T03:20:34.273695+0000","last_became_active":"2024-05-21T03:20:32.153541+0000","last_became_peered":"2024-05-21T03:20:32.153541+0000","last_unstale":"2024-05-21T03:20:34.273695+0000","last_undegraded":"2024-05-21T03:20:34.273695+0000","last_fullsized":"2024-05-21T03:20:34.273695+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:02:58.688378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T03:21:13.551303+0000","last_change":"2024-05-21T03:20:32.143689+0000","last_active":"2024-05-21T03:21:13.551303+0000","last_peered":"2024-05-21T03:21:13.551303+0000","last_clean":"2024-05-21T03:21:13.551303+0000","last_became_active":"2024-05-21T03:20:32.143399+0000","last_became_peered":"2024-05-21T03:20:32.143399+0000","last_unstale":"2024-05-21T03:21:13.551303+0000","last_undegraded":"2024-05-21T03:21:13.551303+0000","last_fullsized":"2024-05-21T03:21:13.551303+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:27:55.116599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.270948+0000","last_change":"2024-05-21T03:20:32.141355+0000","last_active":"2024-05-21T03:20:34.270948+0000","last_peered":"2024-05-21T03:20:34.270948+0000","last_clean":"2024-05-21T03:20:34.270948+0000","last_became_active":"2024-05-21T03:20:32.140960+0000","last_became_peered":"2024-05-21T03:20:32.140960+0000","last_unstale":"2024-05-21T03:20:34.270948+0000","last_undegraded":"2024-05-21T03:20:34.270948+0000","last_fullsized":"2024-05-21T03:20:34.270948+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:23:50.287864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.276057+0000","last_change":"2024-05-21T03:20:32.142145+0000","last_active":"2024-05-21T03:20:34.276057+0000","last_peered":"2024-05-21T03:20:34.276057+0000","last_clean":"2024-05-21T03:20:34.276057+0000","last_became_active":"2024-05-21T03:20:32.141802+0000","last_became_peered":"2024-05-21T03:20:32.141802+0000","last_unstale":"2024-05-21T03:20:34.276057+0000","last_undegraded":"2024-05-21T03:20:34.276057+0000","last_fullsized":"2024-05-21T03:20:34.276057+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:26:28.707616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.274572+0000","last_change":"2024-05-21T03:20:32.141606+0000","last_active":"2024-05-21T03:20:34.274572+0000","last_peered":"2024-05-21T03:20:34.274572+0000","last_clean":"2024-05-21T03:20:34.274572+0000","last_became_active":"2024-05-21T03:20:32.141258+0000","last_became_peered":"2024-05-21T03:20:32.141258+0000","last_unstale":"2024-05-21T03:20:34.274572+0000","last_undegraded":"2024-05-21T03:20:34.274572+0000","last_fullsized":"2024-05-21T03:20:34.274572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:22.372723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561186+0000","last_change":"2024-05-21T03:20:32.151621+0000","last_active":"2024-05-21T03:21:14.561186+0000","last_peered":"2024-05-21T03:21:14.561186+0000","last_clean":"2024-05-21T03:21:14.561186+0000","last_became_active":"2024-05-21T03:20:32.151260+0000","last_became_peered":"2024-05-21T03:20:32.151260+0000","last_unstale":"2024-05-21T03:21:14.561186+0000","last_undegraded":"2024-05-21T03:21:14.561186+0000","last_fullsized":"2024-05-21T03:21:14.561186+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:48:59.751179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.134030+0000","last_change":"2024-05-21T03:20:32.153843+0000","last_active":"2024-05-21T03:20:34.134030+0000","last_peered":"2024-05-21T03:20:34.134030+0000","last_clean":"2024-05-21T03:20:34.134030+0000","last_became_active":"2024-05-21T03:20:32.153418+0000","last_became_peered":"2024-05-21T03:20:32.153418+0000","last_unstale":"2024-05-21T03:20:34.134030+0000","last_undegraded":"2024-05-21T03:20:34.134030+0000","last_fullsized":"2024-05-21T03:20:34.134030+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:26:29.663966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.560653+0000","last_change":"2024-05-21T03:20:32.153942+0000","last_active":"2024-05-21T03:21:14.560653+0000","last_peered":"2024-05-21T03:21:14.560653+0000","last_clean":"2024-05-21T03:21:14.560653+0000","last_became_active":"2024-05-21T03:20:32.153566+0000","last_became_peered":"2024-05-21T03:20:32.153566+0000","last_unstale":"2024-05-21T03:21:14.560653+0000","last_undegraded":"2024-05-21T03:21:14.560653+0000","last_fullsized":"2024-05-21T03:21:14.560653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:06.899039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.134388+0000","last_change":"2024-05-21T03:20:32.153243+0000","last_active":"2024-05-21T03:20:34.134388+0000","last_peered":"2024-05-21T03:20:34.134388+0000","last_clean":"2024-05-21T03:20:34.134388+0000","last_became_active":"2024-05-21T03:20:32.152386+0000","last_became_peered":"2024-05-21T03:20:32.152386+0000","last_unstale":"2024-05-21T03:20:34.134388+0000","last_undegraded":"2024-05-21T03:20:34.134388+0000","last_fullsized":"2024-05-21T03:20:34.134388+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:25:28.935183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.270775+0000","last_change":"2024-05-21T03:20:32.142536+0000","last_active":"2024-05-21T03:20:34.270775+0000","last_peered":"2024-05-21T03:20:34.270775+0000","last_clean":"2024-05-21T03:20:34.270775+0000","last_became_active":"2024-05-21T03:20:32.141793+0000","last_became_peered":"2024-05-21T03:20:32.141793+0000","last_unstale":"2024-05-21T03:20:34.270775+0000","last_undegraded":"2024-05-21T03:20:34.270775+0000","last_fullsized":"2024-05-21T03:20:34.270775+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:21:43.547103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.276201+0000","last_change":"2024-05-21T03:20:32.140368+0000","last_active":"2024-05-21T03:20:34.276201+0000","last_peered":"2024-05-21T03:20:34.276201+0000","last_clean":"2024-05-21T03:20:34.276201+0000","last_became_active":"2024-05-21T03:20:32.139747+0000","last_became_peered":"2024-05-21T03:20:32.139747+0000","last_unstale":"2024-05-21T03:20:34.276201+0000","last_undegraded":"2024-05-21T03:20:34.276201+0000","last_fullsized":"2024-05-21T03:20:34.276201+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:05:44.851191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61962,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":268,"num_read_kb":205,"num_write":239,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":185886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645335,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":868352,"data_stored":416574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841564,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1061970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005090,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":416113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201317,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":888832,"data_stored":416491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397548,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32628,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712524,"statfs":{"total":95995035648,"available":95961624576,"internally_reserved":0,"allocated":1486848,"data_stored":1009400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561072,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":876544,"data_stored":416221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":471501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":913408,"data_stored":426319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":467464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149760,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":432814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953531,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":876544,"data_stored":416082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757302,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32640,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712512,"statfs":{"total":95995035648,"available":95961612288,"internally_reserved":0,"allocated":1499136,"data_stored":1008296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":56510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":56730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":56494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T03:21:16.347 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-21T03:21:16.598 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:21:16.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:16 smithi027 ceph-mon[32179]: pgmap v313: 97 pgs: 3 peering, 94 active+clean; 637 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2024-05-21T03:21:16.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:16 smithi027 ceph-mon[32179]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:21:17.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:16 smithi039 ceph-mon[36429]: pgmap v313: 97 pgs: 3 peering, 94 active+clean; 637 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2024-05-21T03:21:17.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:16 smithi039 ceph-mon[36429]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:21:17.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:16 smithi165 ceph-mon[36269]: pgmap v313: 97 pgs: 3 peering, 94 active+clean; 637 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2024-05-21T03:21:17.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:16 smithi165 ceph-mon[36269]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:21:18.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.253+0000 7fe93497c700 1 -- 172.21.15.27:0/944117742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9301026a0 msgr2=0x7fe930108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:18.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.253+0000 7fe93497c700 1 --2- 172.21.15.27:0/944117742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9301026a0 0x7fe930108200 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe9240099c0 tx=0x7fe924009cd0 comp rx=0 tx=0).stop 2024-05-21T03:21:18.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.253+0000 7fe93497c700 1 -- 172.21.15.27:0/944117742 shutdown_connections 2024-05-21T03:21:18.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.253+0000 7fe93497c700 1 --2- 172.21.15.27:0/944117742 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe930108930 0x7fe9301104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:18.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.253+0000 7fe93497c700 1 --2- 172.21.15.27:0/944117742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe930101cf0 0x7fe9301020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:18.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.253+0000 7fe93497c700 1 --2- 172.21.15.27:0/944117742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9301026a0 0x7fe930108200 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:18.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.253+0000 7fe93497c700 1 -- 172.21.15.27:0/944117742 >> 172.21.15.27:0/944117742 conn(0x7fe930078a40 msgr2=0x7fe9300ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:18.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.253+0000 7fe93497c700 1 -- 172.21.15.27:0/944117742 shutdown_connections 2024-05-21T03:21:18.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.253+0000 7fe93497c700 1 -- 172.21.15.27:0/944117742 wait complete. 2024-05-21T03:21:18.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.254+0000 7fe93497c700 1 Processor -- start 2024-05-21T03:21:18.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.254+0000 7fe93497c700 1 -- start start 2024-05-21T03:21:18.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.254+0000 7fe93497c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe930101cf0 0x7fe93019f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:18.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.254+0000 7fe93497c700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9301026a0 0x7fe93019fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:18.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.254+0000 7fe93497c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe930108930 0x7fe930199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:18.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.254+0000 7fe93497c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe930113710 con 0x7fe930101cf0 2024-05-21T03:21:18.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.254+0000 7fe93497c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe930113590 con 0x7fe930108930 2024-05-21T03:21:18.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.254+0000 7fe93497c700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe930113890 con 0x7fe9301026a0 2024-05-21T03:21:18.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.254+0000 7fe92e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe930101cf0 0x7fe93019f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:18.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.254+0000 7fe92e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe930101cf0 0x7fe93019f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44836/0 (socket says 172.21.15.27:44836) 2024-05-21T03:21:18.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.254+0000 7fe92e59c700 1 -- 172.21.15.27:0/263596338 learned_addr learned my addr 172.21.15.27:0/263596338 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:21:18.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.254+0000 7fe92ed9d700 1 --2- 172.21.15.27:0/263596338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe930108930 0x7fe930199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:18.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.254+0000 7fe92dd9b700 1 --2- 172.21.15.27:0/263596338 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9301026a0 0x7fe93019fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:18.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.254+0000 7fe92e59c700 1 -- 172.21.15.27:0/263596338 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9301026a0 msgr2=0x7fe93019fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:18.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.254+0000 7fe92e59c700 1 --2- 172.21.15.27:0/263596338 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9301026a0 0x7fe93019fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:18.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.254+0000 7fe92e59c700 1 -- 172.21.15.27:0/263596338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe930108930 msgr2=0x7fe930199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:18.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.254+0000 7fe92e59c700 1 --2- 172.21.15.27:0/263596338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe930108930 0x7fe930199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:18.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.254+0000 7fe92e59c700 1 -- 172.21.15.27:0/263596338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93019a1f0 con 0x7fe930101cf0 2024-05-21T03:21:18.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.254+0000 7fe92e59c700 1 --2- 172.21.15.27:0/263596338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe930101cf0 0x7fe93019f880 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe92000d980 tx=0x7fe92000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:18.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.255+0000 7fe91f7fe700 1 -- 172.21.15.27:0/263596338 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe920016070 con 0x7fe930101cf0 2024-05-21T03:21:18.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.255+0000 7fe93497c700 1 -- 172.21.15.27:0/263596338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe93019a4e0 con 0x7fe930101cf0 2024-05-21T03:21:18.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.255+0000 7fe91f7fe700 1 -- 172.21.15.27:0/263596338 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe920004830 con 0x7fe930101cf0 2024-05-21T03:21:18.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.255+0000 7fe91f7fe700 1 -- 172.21.15.27:0/263596338 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe920005540 con 0x7fe930101cf0 2024-05-21T03:21:18.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.255+0000 7fe93497c700 1 -- 172.21.15.27:0/263596338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9301a6750 con 0x7fe930101cf0 2024-05-21T03:21:18.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.256+0000 7fe93497c700 1 -- 172.21.15.27:0/263596338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe93006a230 con 0x7fe930101cf0 2024-05-21T03:21:18.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.257+0000 7fe91f7fe700 1 -- 172.21.15.27:0/263596338 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe920004220 con 0x7fe930101cf0 2024-05-21T03:21:18.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.258+0000 7fe91f7fe700 1 --2- 172.21.15.27:0/263596338 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe9180b2910 0x7fe9180b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:18.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.258+0000 7fe92dd9b700 1 --2- 172.21.15.27:0/263596338 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe9180b2910 0x7fe9180b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:18.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.258+0000 7fe91f7fe700 1 -- 172.21.15.27:0/263596338 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fe9200e26e0 con 0x7fe930101cf0 2024-05-21T03:21:18.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.259+0000 7fe92dd9b700 1 --2- 172.21.15.27:0/263596338 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe9180b2910 0x7fe9180b4dd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe924009990 tx=0x7fe924009650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:18.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.262+0000 7fe91f7fe700 1 -- 172.21.15.27:0/263596338 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe92001f020 con 0x7fe930101cf0 2024-05-21T03:21:18.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.473+0000 7fe93497c700 1 -- 172.21.15.27:0/263596338 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe930063980 con 0x7fe9180b2910 2024-05-21T03:21:18.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.480+0000 7fe91f7fe700 1 -- 172.21.15.27:0/263596338 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258648 (secure 0 0 0) 0x7fe930063980 con 0x7fe9180b2910 2024-05-21T03:21:18.484 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:21:18.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.483+0000 7fe93497c700 1 -- 172.21.15.27:0/263596338 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe9180b2910 msgr2=0x7fe9180b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:18.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.483+0000 7fe93497c700 1 --2- 172.21.15.27:0/263596338 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe9180b2910 0x7fe9180b4dd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe924009990 tx=0x7fe924009650 comp rx=0 tx=0).stop 2024-05-21T03:21:18.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.483+0000 7fe93497c700 1 -- 172.21.15.27:0/263596338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe930101cf0 msgr2=0x7fe93019f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:18.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.483+0000 7fe93497c700 1 --2- 172.21.15.27:0/263596338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe930101cf0 0x7fe93019f880 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe92000d980 tx=0x7fe92000dc90 comp rx=0 tx=0).stop 2024-05-21T03:21:18.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.484+0000 7fe93497c700 1 -- 172.21.15.27:0/263596338 shutdown_connections 2024-05-21T03:21:18.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.484+0000 7fe93497c700 1 --2- 172.21.15.27:0/263596338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe930108930 0x7fe930199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:18.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.484+0000 7fe93497c700 1 --2- 172.21.15.27:0/263596338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe930101cf0 0x7fe93019f880 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:18.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.484+0000 7fe93497c700 1 --2- 172.21.15.27:0/263596338 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe9180b2910 0x7fe9180b4dd0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:18.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.484+0000 7fe93497c700 1 --2- 172.21.15.27:0/263596338 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9301026a0 0x7fe93019fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:18.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.484+0000 7fe93497c700 1 -- 172.21.15.27:0/263596338 >> 172.21.15.27:0/263596338 conn(0x7fe930078a40 msgr2=0x7fe930105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:18.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.484+0000 7fe93497c700 1 -- 172.21.15.27:0/263596338 shutdown_connections 2024-05-21T03:21:18.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:18.484+0000 7fe93497c700 1 -- 172.21.15.27:0/263596338 wait complete. 2024-05-21T03:21:18.487 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-05-21T03:21:18.825 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:18 smithi027 ceph-mon[32179]: pgmap v314: 97 pgs: 3 peering, 94 active+clean; 637 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 7 op/s; 209 B/s, 1 keys/s, 1 objects/s recovering 2024-05-21T03:21:19.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:18 smithi039 ceph-mon[36429]: pgmap v314: 97 pgs: 3 peering, 94 active+clean; 637 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 7 op/s; 209 B/s, 1 keys/s, 1 objects/s recovering 2024-05-21T03:21:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:18 smithi165 ceph-mon[36269]: pgmap v314: 97 pgs: 3 peering, 94 active+clean; 637 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 7 op/s; 209 B/s, 1 keys/s, 1 objects/s recovering 2024-05-21T03:21:19.112 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-05-21T03:21:17.344544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652564,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":351,"num_read_kb":276,"num_write":364,"num_write_kb":2279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2953518,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":300,"ondisk_log_size":300,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357700,"kb_used_data":12640,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584124,"statfs":{"total":1151940427776,"available":1151574142976,"internally_reserved":0,"allocated":12943360,"data_stored":7107105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19139,"internal_metadata":353154365},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4257,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":32,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1678,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001984"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.554128+0000","last_change":"2024-05-21T03:20:32.149961+0000","last_active":"2024-05-21T03:21:14.554128+0000","last_peered":"2024-05-21T03:21:14.554128+0000","last_clean":"2024-05-21T03:21:14.554128+0000","last_became_active":"2024-05-21T03:20:32.149552+0000","last_became_peered":"2024-05-21T03:20:32.149552+0000","last_unstale":"2024-05-21T03:21:14.554128+0000","last_undegraded":"2024-05-21T03:21:14.554128+0000","last_fullsized":"2024-05-21T03:21:14.554128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:20:38.543551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T03:21:13.550802+0000","last_change":"2024-05-21T03:20:32.146413+0000","last_active":"2024-05-21T03:21:13.550802+0000","last_peered":"2024-05-21T03:21:13.550802+0000","last_clean":"2024-05-21T03:21:13.550802+0000","last_became_active":"2024-05-21T03:20:32.146006+0000","last_became_peered":"2024-05-21T03:20:32.146006+0000","last_unstale":"2024-05-21T03:21:13.550802+0000","last_undegraded":"2024-05-21T03:21:13.550802+0000","last_fullsized":"2024-05-21T03:21:13.550802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:05:12.888550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T03:21:13.551438+0000","last_change":"2024-05-21T03:20:32.153654+0000","last_active":"2024-05-21T03:21:13.551438+0000","last_peered":"2024-05-21T03:21:13.551438+0000","last_clean":"2024-05-21T03:21:13.551438+0000","last_became_active":"2024-05-21T03:20:32.153296+0000","last_became_peered":"2024-05-21T03:20:32.153296+0000","last_unstale":"2024-05-21T03:21:13.551438+0000","last_undegraded":"2024-05-21T03:21:13.551438+0000","last_fullsized":"2024-05-21T03:21:13.551438+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:16:09.423236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556628+0000","last_change":"2024-05-21T03:20:32.138672+0000","last_active":"2024-05-21T03:21:14.556628+0000","last_peered":"2024-05-21T03:21:14.556628+0000","last_clean":"2024-05-21T03:21:14.556628+0000","last_became_active":"2024-05-21T03:20:32.138163+0000","last_became_peered":"2024-05-21T03:20:32.138163+0000","last_unstale":"2024-05-21T03:21:14.556628+0000","last_undegraded":"2024-05-21T03:21:14.556628+0000","last_fullsized":"2024-05-21T03:21:14.556628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:26:55.012897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556450+0000","last_change":"2024-05-21T03:20:32.153950+0000","last_active":"2024-05-21T03:21:14.556450+0000","last_peered":"2024-05-21T03:21:14.556450+0000","last_clean":"2024-05-21T03:21:14.556450+0000","last_became_active":"2024-05-21T03:20:32.153567+0000","last_became_peered":"2024-05-21T03:20:32.153567+0000","last_unstale":"2024-05-21T03:21:14.556450+0000","last_undegraded":"2024-05-21T03:21:14.556450+0000","last_fullsized":"2024-05-21T03:21:14.556450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:28:34.621193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T03:21:13.553377+0000","last_change":"2024-05-21T03:20:32.140038+0000","last_active":"2024-05-21T03:21:13.553377+0000","last_peered":"2024-05-21T03:21:13.553377+0000","last_clean":"2024-05-21T03:21:13.553377+0000","last_became_active":"2024-05-21T03:20:32.139690+0000","last_became_peered":"2024-05-21T03:20:32.139690+0000","last_unstale":"2024-05-21T03:21:13.553377+0000","last_undegraded":"2024-05-21T03:21:13.553377+0000","last_fullsized":"2024-05-21T03:21:13.553377+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:31:24.270755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556482+0000","last_change":"2024-05-21T03:20:32.143735+0000","last_active":"2024-05-21T03:21:14.556482+0000","last_peered":"2024-05-21T03:21:14.556482+0000","last_clean":"2024-05-21T03:21:14.556482+0000","last_became_active":"2024-05-21T03:20:32.143397+0000","last_became_peered":"2024-05-21T03:20:32.143397+0000","last_unstale":"2024-05-21T03:21:14.556482+0000","last_undegraded":"2024-05-21T03:21:14.556482+0000","last_fullsized":"2024-05-21T03:21:14.556482+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:40:01.951608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.560831+0000","last_change":"2024-05-21T03:20:32.156999+0000","last_active":"2024-05-21T03:21:14.560831+0000","last_peered":"2024-05-21T03:21:14.560831+0000","last_clean":"2024-05-21T03:21:14.560831+0000","last_became_active":"2024-05-21T03:20:32.156677+0000","last_became_peered":"2024-05-21T03:20:32.156677+0000","last_unstale":"2024-05-21T03:21:14.560831+0000","last_undegraded":"2024-05-21T03:21:14.560831+0000","last_fullsized":"2024-05-21T03:21:14.560831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:33:24.225667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558931+0000","last_change":"2024-05-21T03:20:32.145039+0000","last_active":"2024-05-21T03:21:14.558931+0000","last_peered":"2024-05-21T03:21:14.558931+0000","last_clean":"2024-05-21T03:21:14.558931+0000","last_became_active":"2024-05-21T03:20:32.140926+0000","last_became_peered":"2024-05-21T03:20:32.140926+0000","last_unstale":"2024-05-21T03:21:14.558931+0000","last_undegraded":"2024-05-21T03:21:14.558931+0000","last_fullsized":"2024-05-21T03:21:14.558931+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:11:27.049426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T03:21:13.554169+0000","last_change":"2024-05-21T03:20:32.145840+0000","last_active":"2024-05-21T03:21:13.554169+0000","last_peered":"2024-05-21T03:21:13.554169+0000","last_clean":"2024-05-21T03:21:13.554169+0000","last_became_active":"2024-05-21T03:20:32.145291+0000","last_became_peered":"2024-05-21T03:20:32.145291+0000","last_unstale":"2024-05-21T03:21:13.554169+0000","last_undegraded":"2024-05-21T03:21:13.554169+0000","last_fullsized":"2024-05-21T03:21:13.554169+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:22:27.696588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558336+0000","last_change":"2024-05-21T03:20:32.142522+0000","last_active":"2024-05-21T03:21:14.558336+0000","last_peered":"2024-05-21T03:21:14.558336+0000","last_clean":"2024-05-21T03:21:14.558336+0000","last_became_active":"2024-05-21T03:20:32.142165+0000","last_became_peered":"2024-05-21T03:20:32.142165+0000","last_unstale":"2024-05-21T03:21:14.558336+0000","last_undegraded":"2024-05-21T03:21:14.558336+0000","last_fullsized":"2024-05-21T03:21:14.558336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:58:39.085834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556536+0000","last_change":"2024-05-21T03:20:32.142526+0000","last_active":"2024-05-21T03:21:14.556536+0000","last_peered":"2024-05-21T03:21:14.556536+0000","last_clean":"2024-05-21T03:21:14.556536+0000","last_became_active":"2024-05-21T03:20:32.141143+0000","last_became_peered":"2024-05-21T03:20:32.141143+0000","last_unstale":"2024-05-21T03:21:14.556536+0000","last_undegraded":"2024-05-21T03:21:14.556536+0000","last_fullsized":"2024-05-21T03:21:14.556536+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:35:11.336796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552998+0000","last_change":"2024-05-21T03:20:32.135631+0000","last_active":"2024-05-21T03:21:14.552998+0000","last_peered":"2024-05-21T03:21:14.552998+0000","last_clean":"2024-05-21T03:21:14.552998+0000","last_became_active":"2024-05-21T03:20:32.135302+0000","last_became_peered":"2024-05-21T03:20:32.135302+0000","last_unstale":"2024-05-21T03:21:14.552998+0000","last_undegraded":"2024-05-21T03:21:14.552998+0000","last_fullsized":"2024-05-21T03:21:14.552998+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:05:55.000690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T03:21:13.550967+0000","last_change":"2024-05-21T03:20:32.146580+0000","last_active":"2024-05-21T03:21:13.550967+0000","last_peered":"2024-05-21T03:21:13.550967+0000","last_clean":"2024-05-21T03:21:13.550967+0000","last_became_active":"2024-05-21T03:20:32.145803+0000","last_became_peered":"2024-05-21T03:20:32.145803+0000","last_unstale":"2024-05-21T03:21:13.550967+0000","last_undegraded":"2024-05-21T03:21:13.550967+0000","last_fullsized":"2024-05-21T03:21:13.550967+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:34:09.108799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"peering","last_fresh":"2024-05-21T03:20:34.304094+0000","last_change":"2024-05-21T03:21:13.549865+0000","last_active":"2024-05-21T03:21:13.549865+0000","last_peered":"2024-05-21T03:20:34.304094+0000","last_clean":"2024-05-21T03:20:34.304094+0000","last_became_active":"2024-05-21T03:20:32.142338+0000","last_became_peered":"2024-05-21T03:20:32.142338+0000","last_unstale":"2024-05-21T03:20:34.304094+0000","last_undegraded":"2024-05-21T03:20:34.304094+0000","last_fullsized":"2024-05-21T03:20:34.304094+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4,6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.557368+0000","last_change":"2024-05-21T03:20:32.148944+0000","last_active":"2024-05-21T03:21:14.557368+0000","last_peered":"2024-05-21T03:21:14.557368+0000","last_clean":"2024-05-21T03:21:14.557368+0000","last_became_active":"2024-05-21T03:20:32.148502+0000","last_became_peered":"2024-05-21T03:20:32.148502+0000","last_unstale":"2024-05-21T03:21:14.557368+0000","last_undegraded":"2024-05-21T03:21:14.557368+0000","last_fullsized":"2024-05-21T03:21:14.557368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:11:00.178105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.554478+0000","last_change":"2024-05-21T03:20:32.150609+0000","last_active":"2024-05-21T03:21:14.554478+0000","last_peered":"2024-05-21T03:21:14.554478+0000","last_clean":"2024-05-21T03:21:14.554478+0000","last_became_active":"2024-05-21T03:20:32.150314+0000","last_became_peered":"2024-05-21T03:20:32.150314+0000","last_unstale":"2024-05-21T03:21:14.554478+0000","last_undegraded":"2024-05-21T03:21:14.554478+0000","last_fullsized":"2024-05-21T03:21:14.554478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:34:21.869304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.271753+0000","last_change":"2024-05-21T03:20:32.142472+0000","last_active":"2024-05-21T03:20:34.271753+0000","last_peered":"2024-05-21T03:20:34.271753+0000","last_clean":"2024-05-21T03:20:34.271753+0000","last_became_active":"2024-05-21T03:20:32.142161+0000","last_became_peered":"2024-05-21T03:20:32.142161+0000","last_unstale":"2024-05-21T03:20:34.271753+0000","last_undegraded":"2024-05-21T03:20:34.271753+0000","last_fullsized":"2024-05-21T03:20:34.271753+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:26:57.281655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.557572+0000","last_change":"2024-05-21T03:20:32.153987+0000","last_active":"2024-05-21T03:21:14.557572+0000","last_peered":"2024-05-21T03:21:14.557572+0000","last_clean":"2024-05-21T03:21:14.557572+0000","last_became_active":"2024-05-21T03:20:32.153617+0000","last_became_peered":"2024-05-21T03:20:32.153617+0000","last_unstale":"2024-05-21T03:21:14.557572+0000","last_undegraded":"2024-05-21T03:21:14.557572+0000","last_fullsized":"2024-05-21T03:21:14.557572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:57:07.341758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":58,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:21:11.513921+0000","last_change":"2024-05-21T03:20:29.099928+0000","last_active":"2024-05-21T03:21:11.513921+0000","last_peered":"2024-05-21T03:21:11.513921+0000","last_clean":"2024-05-21T03:21:11.513921+0000","last_became_active":"2024-05-21T03:20:29.099592+0000","last_became_peered":"2024-05-21T03:20:29.099592+0000","last_unstale":"2024-05-21T03:21:11.513921+0000","last_undegraded":"2024-05-21T03:21:11.513921+0000","last_fullsized":"2024-05-21T03:21:11.513921+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:35:34.843325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561455+0000","last_change":"2024-05-21T03:20:32.154962+0000","last_active":"2024-05-21T03:21:14.561455+0000","last_peered":"2024-05-21T03:21:14.561455+0000","last_clean":"2024-05-21T03:21:14.561455+0000","last_became_active":"2024-05-21T03:20:32.154620+0000","last_became_peered":"2024-05-21T03:20:32.154620+0000","last_unstale":"2024-05-21T03:21:14.561455+0000","last_undegraded":"2024-05-21T03:21:14.561455+0000","last_fullsized":"2024-05-21T03:21:14.561455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:56:21.553272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561738+0000","last_change":"2024-05-21T03:20:29.097312+0000","last_active":"2024-05-21T03:21:14.561738+0000","last_peered":"2024-05-21T03:21:14.561738+0000","last_clean":"2024-05-21T03:21:14.561738+0000","last_became_active":"2024-05-21T03:20:29.096977+0000","last_became_peered":"2024-05-21T03:20:29.096977+0000","last_unstale":"2024-05-21T03:21:14.561738+0000","last_undegraded":"2024-05-21T03:21:14.561738+0000","last_fullsized":"2024-05-21T03:21:14.561738+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:28:45.467446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T03:21:13.550899+0000","last_change":"2024-05-21T03:20:32.150035+0000","last_active":"2024-05-21T03:21:13.550899+0000","last_peered":"2024-05-21T03:21:13.550899+0000","last_clean":"2024-05-21T03:21:13.550899+0000","last_became_active":"2024-05-21T03:20:32.149688+0000","last_became_peered":"2024-05-21T03:20:32.149688+0000","last_unstale":"2024-05-21T03:21:13.550899+0000","last_undegraded":"2024-05-21T03:21:13.550899+0000","last_fullsized":"2024-05-21T03:21:13.550899+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:27:00.410770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.573309+0000","last_change":"2024-05-21T03:21:14.572832+0000","last_active":"2024-05-21T03:21:14.573309+0000","last_peered":"2024-05-21T03:21:14.573309+0000","last_clean":"2024-05-21T03:21:14.573309+0000","last_became_active":"2024-05-21T03:21:14.559381+0000","last_became_peered":"2024-05-21T03:21:14.559381+0000","last_unstale":"2024-05-21T03:21:14.573309+0000","last_undegraded":"2024-05-21T03:21:14.573309+0000","last_fullsized":"2024-05-21T03:21:14.573309+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:02:53.004031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558214+0000","last_change":"2024-05-21T03:20:32.150301+0000","last_active":"2024-05-21T03:21:14.558214+0000","last_peered":"2024-05-21T03:21:14.558214+0000","last_clean":"2024-05-21T03:21:14.558214+0000","last_became_active":"2024-05-21T03:20:32.148475+0000","last_became_peered":"2024-05-21T03:20:32.148475+0000","last_unstale":"2024-05-21T03:21:14.558214+0000","last_undegraded":"2024-05-21T03:21:14.558214+0000","last_fullsized":"2024-05-21T03:21:14.558214+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:59:30.339660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556899+0000","last_change":"2024-05-21T03:20:29.098861+0000","last_active":"2024-05-21T03:21:14.556899+0000","last_peered":"2024-05-21T03:21:14.556899+0000","last_clean":"2024-05-21T03:21:14.556899+0000","last_became_active":"2024-05-21T03:20:29.097874+0000","last_became_peered":"2024-05-21T03:20:29.097874+0000","last_unstale":"2024-05-21T03:21:14.556899+0000","last_undegraded":"2024-05-21T03:21:14.556899+0000","last_fullsized":"2024-05-21T03:21:14.556899+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:25:36.227339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561801+0000","last_change":"2024-05-21T03:20:32.142820+0000","last_active":"2024-05-21T03:21:14.561801+0000","last_peered":"2024-05-21T03:21:14.561801+0000","last_clean":"2024-05-21T03:21:14.561801+0000","last_became_active":"2024-05-21T03:20:32.142396+0000","last_became_peered":"2024-05-21T03:20:32.142396+0000","last_unstale":"2024-05-21T03:21:14.561801+0000","last_undegraded":"2024-05-21T03:21:14.561801+0000","last_fullsized":"2024-05-21T03:21:14.561801+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:30:14.452389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":60,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:15.521050+0000","last_change":"2024-05-21T03:20:29.103153+0000","last_active":"2024-05-21T03:21:15.521050+0000","last_peered":"2024-05-21T03:21:15.521050+0000","last_clean":"2024-05-21T03:21:15.521050+0000","last_became_active":"2024-05-21T03:20:29.102819+0000","last_became_peered":"2024-05-21T03:20:29.102819+0000","last_unstale":"2024-05-21T03:21:15.521050+0000","last_undegraded":"2024-05-21T03:21:15.521050+0000","last_fullsized":"2024-05-21T03:21:15.521050+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:47:49.843926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556528+0000","last_change":"2024-05-21T03:20:32.142263+0000","last_active":"2024-05-21T03:21:14.556528+0000","last_peered":"2024-05-21T03:21:14.556528+0000","last_clean":"2024-05-21T03:21:14.556528+0000","last_became_active":"2024-05-21T03:20:32.141820+0000","last_became_peered":"2024-05-21T03:20:32.141820+0000","last_unstale":"2024-05-21T03:21:14.556528+0000","last_undegraded":"2024-05-21T03:21:14.556528+0000","last_fullsized":"2024-05-21T03:21:14.556528+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:45:20.021237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T03:21:13.553201+0000","last_change":"2024-05-21T03:20:29.100088+0000","last_active":"2024-05-21T03:21:13.553201+0000","last_peered":"2024-05-21T03:21:13.553201+0000","last_clean":"2024-05-21T03:21:13.553201+0000","last_became_active":"2024-05-21T03:20:29.097343+0000","last_became_peered":"2024-05-21T03:20:29.097343+0000","last_unstale":"2024-05-21T03:21:13.553201+0000","last_undegraded":"2024-05-21T03:21:13.553201+0000","last_fullsized":"2024-05-21T03:21:13.553201+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:00:35.682393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.134153+0000","last_change":"2024-05-21T03:20:32.153907+0000","last_active":"2024-05-21T03:20:34.134153+0000","last_peered":"2024-05-21T03:20:34.134153+0000","last_clean":"2024-05-21T03:20:34.134153+0000","last_became_active":"2024-05-21T03:20:32.153507+0000","last_became_peered":"2024-05-21T03:20:32.153507+0000","last_unstale":"2024-05-21T03:20:34.134153+0000","last_undegraded":"2024-05-21T03:20:34.134153+0000","last_fullsized":"2024-05-21T03:20:34.134153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:09:15.503685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552729+0000","last_change":"2024-05-21T03:20:29.097945+0000","last_active":"2024-05-21T03:21:14.552729+0000","last_peered":"2024-05-21T03:21:14.552729+0000","last_clean":"2024-05-21T03:21:14.552729+0000","last_became_active":"2024-05-21T03:20:29.097624+0000","last_became_peered":"2024-05-21T03:20:29.097624+0000","last_unstale":"2024-05-21T03:21:14.552729+0000","last_undegraded":"2024-05-21T03:21:14.552729+0000","last_fullsized":"2024-05-21T03:21:14.552729+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:01:01.694893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.560719+0000","last_change":"2024-05-21T03:20:32.148955+0000","last_active":"2024-05-21T03:21:14.560719+0000","last_peered":"2024-05-21T03:21:14.560719+0000","last_clean":"2024-05-21T03:21:14.560719+0000","last_became_active":"2024-05-21T03:20:32.148631+0000","last_became_peered":"2024-05-21T03:20:32.148631+0000","last_unstale":"2024-05-21T03:21:14.560719+0000","last_undegraded":"2024-05-21T03:21:14.560719+0000","last_fullsized":"2024-05-21T03:21:14.560719+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:36:25.820580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561173+0000","last_change":"2024-05-21T03:20:29.100902+0000","last_active":"2024-05-21T03:21:14.561173+0000","last_peered":"2024-05-21T03:21:14.561173+0000","last_clean":"2024-05-21T03:21:14.561173+0000","last_became_active":"2024-05-21T03:20:29.100570+0000","last_became_peered":"2024-05-21T03:20:29.100570+0000","last_unstale":"2024-05-21T03:21:14.561173+0000","last_undegraded":"2024-05-21T03:21:14.561173+0000","last_fullsized":"2024-05-21T03:21:14.561173+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:58:36.651338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.273110+0000","last_change":"2024-05-21T03:20:32.150724+0000","last_active":"2024-05-21T03:20:34.273110+0000","last_peered":"2024-05-21T03:20:34.273110+0000","last_clean":"2024-05-21T03:20:34.273110+0000","last_became_active":"2024-05-21T03:20:32.150371+0000","last_became_peered":"2024-05-21T03:20:32.150371+0000","last_unstale":"2024-05-21T03:20:34.273110+0000","last_undegraded":"2024-05-21T03:20:34.273110+0000","last_fullsized":"2024-05-21T03:20:34.273110+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:44:32.321192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":76,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T03:21:13.553954+0000","last_change":"2024-05-21T03:20:29.095045+0000","last_active":"2024-05-21T03:21:13.553954+0000","last_peered":"2024-05-21T03:21:13.553954+0000","last_clean":"2024-05-21T03:21:13.553954+0000","last_became_active":"2024-05-21T03:20:29.094552+0000","last_became_peered":"2024-05-21T03:20:29.094552+0000","last_unstale":"2024-05-21T03:21:13.553954+0000","last_undegraded":"2024-05-21T03:21:13.553954+0000","last_fullsized":"2024-05-21T03:21:13.553954+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:36:15.155125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558897+0000","last_change":"2024-05-21T03:20:32.149813+0000","last_active":"2024-05-21T03:21:14.558897+0000","last_peered":"2024-05-21T03:21:14.558897+0000","last_clean":"2024-05-21T03:21:14.558897+0000","last_became_active":"2024-05-21T03:20:32.148422+0000","last_became_peered":"2024-05-21T03:20:32.148422+0000","last_unstale":"2024-05-21T03:21:14.558897+0000","last_undegraded":"2024-05-21T03:21:14.558897+0000","last_fullsized":"2024-05-21T03:21:14.558897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:55:35.115463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558265+0000","last_change":"2024-05-21T03:20:29.101463+0000","last_active":"2024-05-21T03:21:14.558265+0000","last_peered":"2024-05-21T03:21:14.558265+0000","last_clean":"2024-05-21T03:21:14.558265+0000","last_became_active":"2024-05-21T03:20:29.101127+0000","last_became_peered":"2024-05-21T03:20:29.101127+0000","last_unstale":"2024-05-21T03:21:14.558265+0000","last_undegraded":"2024-05-21T03:21:14.558265+0000","last_fullsized":"2024-05-21T03:21:14.558265+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:58:17.408315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"peering","last_fresh":"2024-05-21T03:20:34.304165+0000","last_change":"2024-05-21T03:21:13.549888+0000","last_active":"2024-05-21T03:21:13.549888+0000","last_peered":"2024-05-21T03:20:34.304165+0000","last_clean":"2024-05-21T03:20:34.304165+0000","last_became_active":"2024-05-21T03:20:32.144900+0000","last_became_peered":"2024-05-21T03:20:32.144900+0000","last_unstale":"2024-05-21T03:20:34.304165+0000","last_undegraded":"2024-05-21T03:20:34.304165+0000","last_fullsized":"2024-05-21T03:20:34.304165+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5,8],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.562216+0000","last_change":"2024-05-21T03:20:29.098725+0000","last_active":"2024-05-21T03:21:14.562216+0000","last_peered":"2024-05-21T03:21:14.562216+0000","last_clean":"2024-05-21T03:21:14.562216+0000","last_became_active":"2024-05-21T03:20:29.098471+0000","last_became_peered":"2024-05-21T03:20:29.098471+0000","last_unstale":"2024-05-21T03:21:14.562216+0000","last_undegraded":"2024-05-21T03:21:14.562216+0000","last_fullsized":"2024-05-21T03:21:14.562216+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:17:11.059825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.560975+0000","last_change":"2024-05-21T03:21:14.560975+0000","last_active":"2024-05-21T03:21:14.560975+0000","last_peered":"2024-05-21T03:21:14.560975+0000","last_clean":"2024-05-21T03:21:14.560975+0000","last_became_active":"2024-05-21T03:21:14.560528+0000","last_became_peered":"2024-05-21T03:21:14.560528+0000","last_unstale":"2024-05-21T03:21:14.560975+0000","last_undegraded":"2024-05-21T03:21:14.560975+0000","last_fullsized":"2024-05-21T03:21:14.560975+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:51:47.469291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'28","reported_seq":108,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T03:21:13.553898+0000","last_change":"2024-05-21T03:20:29.099968+0000","last_active":"2024-05-21T03:21:13.553898+0000","last_peered":"2024-05-21T03:21:13.553898+0000","last_clean":"2024-05-21T03:21:13.553898+0000","last_became_active":"2024-05-21T03:20:29.099181+0000","last_became_peered":"2024-05-21T03:20:29.099181+0000","last_unstale":"2024-05-21T03:21:13.553898+0000","last_undegraded":"2024-05-21T03:21:13.553898+0000","last_fullsized":"2024-05-21T03:21:13.553898+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:35:33.934412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54367,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":56,"num_read_kb":80,"num_write":30,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556604+0000","last_change":"2024-05-21T03:20:32.144024+0000","last_active":"2024-05-21T03:21:14.556604+0000","last_peered":"2024-05-21T03:21:14.556604+0000","last_clean":"2024-05-21T03:21:14.556604+0000","last_became_active":"2024-05-21T03:20:32.143668+0000","last_became_peered":"2024-05-21T03:20:32.143668+0000","last_unstale":"2024-05-21T03:21:14.556604+0000","last_undegraded":"2024-05-21T03:21:14.556604+0000","last_fullsized":"2024-05-21T03:21:14.556604+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:11:31.100802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556933+0000","last_change":"2024-05-21T03:20:29.101770+0000","last_active":"2024-05-21T03:21:14.556933+0000","last_peered":"2024-05-21T03:21:14.556933+0000","last_clean":"2024-05-21T03:21:14.556933+0000","last_became_active":"2024-05-21T03:20:29.100407+0000","last_became_peered":"2024-05-21T03:20:29.100407+0000","last_unstale":"2024-05-21T03:21:14.556933+0000","last_undegraded":"2024-05-21T03:21:14.556933+0000","last_fullsized":"2024-05-21T03:21:14.556933+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:01:33.991246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552656+0000","last_change":"2024-05-21T03:20:32.147479+0000","last_active":"2024-05-21T03:21:14.552656+0000","last_peered":"2024-05-21T03:21:14.552656+0000","last_clean":"2024-05-21T03:21:14.552656+0000","last_became_active":"2024-05-21T03:20:32.147160+0000","last_became_peered":"2024-05-21T03:20:32.147160+0000","last_unstale":"2024-05-21T03:21:14.552656+0000","last_undegraded":"2024-05-21T03:21:14.552656+0000","last_fullsized":"2024-05-21T03:21:14.552656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:51:59.343380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.562216+0000","last_change":"2024-05-21T03:20:29.101875+0000","last_active":"2024-05-21T03:21:14.562216+0000","last_peered":"2024-05-21T03:21:14.562216+0000","last_clean":"2024-05-21T03:21:14.562216+0000","last_became_active":"2024-05-21T03:20:29.101468+0000","last_became_peered":"2024-05-21T03:20:29.101468+0000","last_unstale":"2024-05-21T03:21:14.562216+0000","last_undegraded":"2024-05-21T03:21:14.562216+0000","last_fullsized":"2024-05-21T03:21:14.562216+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:10:19.577836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.557393+0000","last_change":"2024-05-21T03:20:32.139002+0000","last_active":"2024-05-21T03:21:14.557393+0000","last_peered":"2024-05-21T03:21:14.557393+0000","last_clean":"2024-05-21T03:21:14.557393+0000","last_became_active":"2024-05-21T03:20:32.138619+0000","last_became_peered":"2024-05-21T03:20:32.138619+0000","last_unstale":"2024-05-21T03:21:14.557393+0000","last_undegraded":"2024-05-21T03:21:14.557393+0000","last_fullsized":"2024-05-21T03:21:14.557393+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:21:36.136283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":61,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:15.520348+0000","last_change":"2024-05-21T03:20:29.101532+0000","last_active":"2024-05-21T03:21:15.520348+0000","last_peered":"2024-05-21T03:21:15.520348+0000","last_clean":"2024-05-21T03:21:15.520348+0000","last_became_active":"2024-05-21T03:20:29.101206+0000","last_became_peered":"2024-05-21T03:20:29.101206+0000","last_unstale":"2024-05-21T03:21:15.520348+0000","last_undegraded":"2024-05-21T03:21:15.520348+0000","last_fullsized":"2024-05-21T03:21:15.520348+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:47:38.555995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":31,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556597+0000","last_change":"2024-05-21T03:20:32.146866+0000","last_active":"2024-05-21T03:21:14.556597+0000","last_peered":"2024-05-21T03:21:14.556597+0000","last_clean":"2024-05-21T03:21:14.556597+0000","last_became_active":"2024-05-21T03:20:32.146463+0000","last_became_peered":"2024-05-21T03:20:32.146463+0000","last_unstale":"2024-05-21T03:21:14.556597+0000","last_undegraded":"2024-05-21T03:21:14.556597+0000","last_fullsized":"2024-05-21T03:21:14.556597+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:06:54.359922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556379+0000","last_change":"2024-05-21T03:20:29.103910+0000","last_active":"2024-05-21T03:21:14.556379+0000","last_peered":"2024-05-21T03:21:14.556379+0000","last_clean":"2024-05-21T03:21:14.556379+0000","last_became_active":"2024-05-21T03:20:29.103583+0000","last_became_peered":"2024-05-21T03:20:29.103583+0000","last_unstale":"2024-05-21T03:21:14.556379+0000","last_undegraded":"2024-05-21T03:21:14.556379+0000","last_fullsized":"2024-05-21T03:21:14.556379+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:57:15.665158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561420+0000","last_change":"2024-05-21T03:20:32.141242+0000","last_active":"2024-05-21T03:21:14.561420+0000","last_peered":"2024-05-21T03:21:14.561420+0000","last_clean":"2024-05-21T03:21:14.561420+0000","last_became_active":"2024-05-21T03:20:32.140892+0000","last_became_peered":"2024-05-21T03:20:32.140892+0000","last_unstale":"2024-05-21T03:21:14.561420+0000","last_undegraded":"2024-05-21T03:21:14.561420+0000","last_fullsized":"2024-05-21T03:21:14.561420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:42:13.997708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.557026+0000","last_change":"2024-05-21T03:20:29.098973+0000","last_active":"2024-05-21T03:21:14.557026+0000","last_peered":"2024-05-21T03:21:14.557026+0000","last_clean":"2024-05-21T03:21:14.557026+0000","last_became_active":"2024-05-21T03:20:29.098694+0000","last_became_peered":"2024-05-21T03:20:29.098694+0000","last_unstale":"2024-05-21T03:21:14.557026+0000","last_undegraded":"2024-05-21T03:21:14.557026+0000","last_fullsized":"2024-05-21T03:21:14.557026+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:55:17.915820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.553963+0000","last_change":"2024-05-21T03:20:32.147465+0000","last_active":"2024-05-21T03:21:14.553963+0000","last_peered":"2024-05-21T03:21:14.553963+0000","last_clean":"2024-05-21T03:21:14.553963+0000","last_became_active":"2024-05-21T03:20:32.147073+0000","last_became_peered":"2024-05-21T03:20:32.147073+0000","last_unstale":"2024-05-21T03:21:14.553963+0000","last_undegraded":"2024-05-21T03:21:14.553963+0000","last_fullsized":"2024-05-21T03:21:14.553963+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:29:05.451370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:20:34.274131+0000","last_change":"2024-05-21T03:20:29.103452+0000","last_active":"2024-05-21T03:20:34.274131+0000","last_peered":"2024-05-21T03:20:34.274131+0000","last_clean":"2024-05-21T03:20:34.274131+0000","last_became_active":"2024-05-21T03:20:29.103018+0000","last_became_peered":"2024-05-21T03:20:29.103018+0000","last_unstale":"2024-05-21T03:20:34.274131+0000","last_undegraded":"2024-05-21T03:20:34.274131+0000","last_fullsized":"2024-05-21T03:20:34.274131+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:10:01.569500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T03:21:13.553494+0000","last_change":"2024-05-21T03:20:32.146456+0000","last_active":"2024-05-21T03:21:13.553494+0000","last_peered":"2024-05-21T03:21:13.553494+0000","last_clean":"2024-05-21T03:21:13.553494+0000","last_became_active":"2024-05-21T03:20:32.146105+0000","last_became_peered":"2024-05-21T03:20:32.146105+0000","last_unstale":"2024-05-21T03:21:13.553494+0000","last_undegraded":"2024-05-21T03:21:13.553494+0000","last_fullsized":"2024-05-21T03:21:13.553494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:00:30.550178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'6","reported_seq":60,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:15.519771+0000","last_change":"2024-05-21T03:20:29.100262+0000","last_active":"2024-05-21T03:21:15.519771+0000","last_peered":"2024-05-21T03:21:15.519771+0000","last_clean":"2024-05-21T03:21:15.519771+0000","last_became_active":"2024-05-21T03:20:29.099917+0000","last_became_peered":"2024-05-21T03:20:29.099917+0000","last_unstale":"2024-05-21T03:21:15.519771+0000","last_undegraded":"2024-05-21T03:21:15.519771+0000","last_fullsized":"2024-05-21T03:21:15.519771+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:55:48.608341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T03:21:13.553487+0000","last_change":"2024-05-21T03:20:32.142776+0000","last_active":"2024-05-21T03:21:13.553487+0000","last_peered":"2024-05-21T03:21:13.553487+0000","last_clean":"2024-05-21T03:21:13.553487+0000","last_became_active":"2024-05-21T03:20:32.142430+0000","last_became_peered":"2024-05-21T03:20:32.142430+0000","last_unstale":"2024-05-21T03:21:13.553487+0000","last_undegraded":"2024-05-21T03:21:13.553487+0000","last_fullsized":"2024-05-21T03:21:13.553487+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:54:28.682343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561357+0000","last_change":"2024-05-21T03:20:32.139506+0000","last_active":"2024-05-21T03:21:14.561357+0000","last_peered":"2024-05-21T03:21:14.561357+0000","last_clean":"2024-05-21T03:21:14.561357+0000","last_became_active":"2024-05-21T03:20:32.138136+0000","last_became_peered":"2024-05-21T03:20:32.138136+0000","last_unstale":"2024-05-21T03:21:14.561357+0000","last_undegraded":"2024-05-21T03:21:14.561357+0000","last_fullsized":"2024-05-21T03:21:14.561357+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:31:24.996276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":58,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552693+0000","last_change":"2024-05-21T03:20:29.096546+0000","last_active":"2024-05-21T03:21:14.552693+0000","last_peered":"2024-05-21T03:21:14.552693+0000","last_clean":"2024-05-21T03:21:14.552693+0000","last_became_active":"2024-05-21T03:20:29.096184+0000","last_became_peered":"2024-05-21T03:20:29.096184+0000","last_unstale":"2024-05-21T03:21:14.552693+0000","last_undegraded":"2024-05-21T03:21:14.552693+0000","last_fullsized":"2024-05-21T03:21:14.552693+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:09:12.832745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561399+0000","last_change":"2024-05-21T03:20:32.156231+0000","last_active":"2024-05-21T03:21:14.561399+0000","last_peered":"2024-05-21T03:21:14.561399+0000","last_clean":"2024-05-21T03:21:14.561399+0000","last_became_active":"2024-05-21T03:20:32.154035+0000","last_became_peered":"2024-05-21T03:20:32.154035+0000","last_unstale":"2024-05-21T03:21:14.561399+0000","last_undegraded":"2024-05-21T03:21:14.561399+0000","last_fullsized":"2024-05-21T03:21:14.561399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:30:53.820292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556130+0000","last_change":"2024-05-21T03:20:29.100422+0000","last_active":"2024-05-21T03:21:14.556130+0000","last_peered":"2024-05-21T03:21:14.556130+0000","last_clean":"2024-05-21T03:21:14.556130+0000","last_became_active":"2024-05-21T03:20:29.099993+0000","last_became_peered":"2024-05-21T03:20:29.099993+0000","last_unstale":"2024-05-21T03:21:14.556130+0000","last_undegraded":"2024-05-21T03:21:14.556130+0000","last_fullsized":"2024-05-21T03:21:14.556130+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:03:15.456447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":98,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556799+0000","last_change":"2024-05-21T03:19:05.647808+0000","last_active":"2024-05-21T03:21:14.556799+0000","last_peered":"2024-05-21T03:21:14.556799+0000","last_clean":"2024-05-21T03:21:14.556799+0000","last_became_active":"2024-05-21T03:19:05.641865+0000","last_became_peered":"2024-05-21T03:19:05.641865+0000","last_unstale":"2024-05-21T03:21:14.556799+0000","last_undegraded":"2024-05-21T03:21:14.556799+0000","last_fullsized":"2024-05-21T03:21:14.556799+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:15:42.008493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:15:42.008493+0000","last_clean_scrub_stamp":"2024-05-21T03:15:42.008493+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:26:58.040196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552599+0000","last_change":"2024-05-21T03:20:32.138182+0000","last_active":"2024-05-21T03:21:14.552599+0000","last_peered":"2024-05-21T03:21:14.552599+0000","last_clean":"2024-05-21T03:21:14.552599+0000","last_became_active":"2024-05-21T03:20:32.137842+0000","last_became_peered":"2024-05-21T03:20:32.137842+0000","last_unstale":"2024-05-21T03:21:14.552599+0000","last_undegraded":"2024-05-21T03:21:14.552599+0000","last_fullsized":"2024-05-21T03:21:14.552599+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:43:50.070160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556989+0000","last_change":"2024-05-21T03:20:29.104246+0000","last_active":"2024-05-21T03:21:14.556989+0000","last_peered":"2024-05-21T03:21:14.556989+0000","last_clean":"2024-05-21T03:21:14.556989+0000","last_became_active":"2024-05-21T03:20:29.103910+0000","last_became_peered":"2024-05-21T03:20:29.103910+0000","last_unstale":"2024-05-21T03:21:14.556989+0000","last_undegraded":"2024-05-21T03:21:14.556989+0000","last_fullsized":"2024-05-21T03:21:14.556989+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:39:28.029283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556715+0000","last_change":"2024-05-21T03:20:29.102020+0000","last_active":"2024-05-21T03:21:14.556715+0000","last_peered":"2024-05-21T03:21:14.556715+0000","last_clean":"2024-05-21T03:21:14.556715+0000","last_became_active":"2024-05-21T03:20:29.100429+0000","last_became_peered":"2024-05-21T03:20:29.100429+0000","last_unstale":"2024-05-21T03:21:14.556715+0000","last_undegraded":"2024-05-21T03:21:14.556715+0000","last_fullsized":"2024-05-21T03:21:14.556715+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:09:01.621074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552834+0000","last_change":"2024-05-21T03:20:32.137130+0000","last_active":"2024-05-21T03:21:14.552834+0000","last_peered":"2024-05-21T03:21:14.552834+0000","last_clean":"2024-05-21T03:21:14.552834+0000","last_became_active":"2024-05-21T03:20:32.136821+0000","last_became_peered":"2024-05-21T03:20:32.136821+0000","last_unstale":"2024-05-21T03:21:14.552834+0000","last_undegraded":"2024-05-21T03:21:14.552834+0000","last_fullsized":"2024-05-21T03:21:14.552834+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:53:41.290370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'13","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T03:21:08.274520+0000","last_change":"2024-05-21T03:20:29.099869+0000","last_active":"2024-05-21T03:21:08.274520+0000","last_peered":"2024-05-21T03:21:08.274520+0000","last_clean":"2024-05-21T03:21:08.274520+0000","last_became_active":"2024-05-21T03:20:29.099534+0000","last_became_peered":"2024-05-21T03:20:29.099534+0000","last_unstale":"2024-05-21T03:21:08.274520+0000","last_undegraded":"2024-05-21T03:21:08.274520+0000","last_fullsized":"2024-05-21T03:21:08.274520+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:01:11.281544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561631+0000","last_change":"2024-05-21T03:20:32.153921+0000","last_active":"2024-05-21T03:21:14.561631+0000","last_peered":"2024-05-21T03:21:14.561631+0000","last_clean":"2024-05-21T03:21:14.561631+0000","last_became_active":"2024-05-21T03:20:32.153594+0000","last_became_peered":"2024-05-21T03:20:32.153594+0000","last_unstale":"2024-05-21T03:21:14.561631+0000","last_undegraded":"2024-05-21T03:21:14.561631+0000","last_fullsized":"2024-05-21T03:21:14.561631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:27:59.426942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556915+0000","last_change":"2024-05-21T03:20:29.097399+0000","last_active":"2024-05-21T03:21:14.556915+0000","last_peered":"2024-05-21T03:21:14.556915+0000","last_clean":"2024-05-21T03:21:14.556915+0000","last_became_active":"2024-05-21T03:20:29.097010+0000","last_became_peered":"2024-05-21T03:20:29.097010+0000","last_unstale":"2024-05-21T03:21:14.556915+0000","last_undegraded":"2024-05-21T03:21:14.556915+0000","last_fullsized":"2024-05-21T03:21:14.556915+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:54:53.273947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"peering","last_fresh":"2024-05-21T03:20:34.304156+0000","last_change":"2024-05-21T03:21:13.549909+0000","last_active":"2024-05-21T03:21:13.549909+0000","last_peered":"2024-05-21T03:20:34.304156+0000","last_clean":"2024-05-21T03:20:34.304156+0000","last_became_active":"2024-05-21T03:20:32.152251+0000","last_became_peered":"2024-05-21T03:20:32.152251+0000","last_unstale":"2024-05-21T03:20:34.304156+0000","last_undegraded":"2024-05-21T03:20:34.304156+0000","last_fullsized":"2024-05-21T03:20:34.304156+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,4],"acting":[0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,4,8],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.557892+0000","last_change":"2024-05-21T03:20:29.103526+0000","last_active":"2024-05-21T03:21:14.557892+0000","last_peered":"2024-05-21T03:21:14.557892+0000","last_clean":"2024-05-21T03:21:14.557892+0000","last_became_active":"2024-05-21T03:20:29.103067+0000","last_became_peered":"2024-05-21T03:20:29.103067+0000","last_unstale":"2024-05-21T03:21:14.557892+0000","last_undegraded":"2024-05-21T03:21:14.557892+0000","last_fullsized":"2024-05-21T03:21:14.557892+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:54:47.566949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556409+0000","last_change":"2024-05-21T03:20:32.141088+0000","last_active":"2024-05-21T03:21:14.556409+0000","last_peered":"2024-05-21T03:21:14.556409+0000","last_clean":"2024-05-21T03:21:14.556409+0000","last_became_active":"2024-05-21T03:20:32.140744+0000","last_became_peered":"2024-05-21T03:20:32.140744+0000","last_unstale":"2024-05-21T03:21:14.556409+0000","last_undegraded":"2024-05-21T03:21:14.556409+0000","last_fullsized":"2024-05-21T03:21:14.556409+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:57:35.591164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.560757+0000","last_change":"2024-05-21T03:20:29.093440+0000","last_active":"2024-05-21T03:21:14.560757+0000","last_peered":"2024-05-21T03:21:14.560757+0000","last_clean":"2024-05-21T03:21:14.560757+0000","last_became_active":"2024-05-21T03:20:29.092935+0000","last_became_peered":"2024-05-21T03:20:29.092935+0000","last_unstale":"2024-05-21T03:21:14.560757+0000","last_undegraded":"2024-05-21T03:21:14.560757+0000","last_fullsized":"2024-05-21T03:21:14.560757+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:18:00.690465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.553665+0000","last_change":"2024-05-21T03:20:32.143781+0000","last_active":"2024-05-21T03:21:14.553665+0000","last_peered":"2024-05-21T03:21:14.553665+0000","last_clean":"2024-05-21T03:21:14.553665+0000","last_became_active":"2024-05-21T03:20:32.143391+0000","last_became_peered":"2024-05-21T03:20:32.143391+0000","last_unstale":"2024-05-21T03:21:14.553665+0000","last_undegraded":"2024-05-21T03:21:14.553665+0000","last_fullsized":"2024-05-21T03:21:14.553665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:00:41.506561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556256+0000","last_change":"2024-05-21T03:20:29.101342+0000","last_active":"2024-05-21T03:21:14.556256+0000","last_peered":"2024-05-21T03:21:14.556256+0000","last_clean":"2024-05-21T03:21:14.556256+0000","last_became_active":"2024-05-21T03:20:29.100997+0000","last_became_peered":"2024-05-21T03:20:29.100997+0000","last_unstale":"2024-05-21T03:21:14.556256+0000","last_undegraded":"2024-05-21T03:21:14.556256+0000","last_fullsized":"2024-05-21T03:21:14.556256+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:05:31.801669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561184+0000","last_change":"2024-05-21T03:20:32.150563+0000","last_active":"2024-05-21T03:21:14.561184+0000","last_peered":"2024-05-21T03:21:14.561184+0000","last_clean":"2024-05-21T03:21:14.561184+0000","last_became_active":"2024-05-21T03:20:32.150231+0000","last_became_peered":"2024-05-21T03:20:32.150231+0000","last_unstale":"2024-05-21T03:21:14.561184+0000","last_undegraded":"2024-05-21T03:21:14.561184+0000","last_fullsized":"2024-05-21T03:21:14.561184+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:10:00.251284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556609+0000","last_change":"2024-05-21T03:20:29.101576+0000","last_active":"2024-05-21T03:21:14.556609+0000","last_peered":"2024-05-21T03:21:14.556609+0000","last_clean":"2024-05-21T03:21:14.556609+0000","last_became_active":"2024-05-21T03:20:29.101270+0000","last_became_peered":"2024-05-21T03:20:29.101270+0000","last_unstale":"2024-05-21T03:21:14.556609+0000","last_undegraded":"2024-05-21T03:21:14.556609+0000","last_fullsized":"2024-05-21T03:21:14.556609+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:50:10.579901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.557991+0000","last_change":"2024-05-21T03:20:32.141340+0000","last_active":"2024-05-21T03:21:14.557991+0000","last_peered":"2024-05-21T03:21:14.557991+0000","last_clean":"2024-05-21T03:21:14.557991+0000","last_became_active":"2024-05-21T03:20:32.140792+0000","last_became_peered":"2024-05-21T03:20:32.140792+0000","last_unstale":"2024-05-21T03:21:14.557991+0000","last_undegraded":"2024-05-21T03:21:14.557991+0000","last_fullsized":"2024-05-21T03:21:14.557991+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:35:05.842929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'6","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:16.567139+0000","last_change":"2024-05-21T03:21:14.566886+0000","last_active":"2024-05-21T03:21:16.567139+0000","last_peered":"2024-05-21T03:21:16.567139+0000","last_clean":"2024-05-21T03:21:16.567139+0000","last_became_active":"2024-05-21T03:21:14.559816+0000","last_became_peered":"2024-05-21T03:21:14.559816+0000","last_unstale":"2024-05-21T03:21:16.567139+0000","last_undegraded":"2024-05-21T03:21:16.567139+0000","last_fullsized":"2024-05-21T03:21:16.567139+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:34:43.109424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1561,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558407+0000","last_change":"2024-05-21T03:20:32.146674+0000","last_active":"2024-05-21T03:21:14.558407+0000","last_peered":"2024-05-21T03:21:14.558407+0000","last_clean":"2024-05-21T03:21:14.558407+0000","last_became_active":"2024-05-21T03:20:32.146150+0000","last_became_peered":"2024-05-21T03:20:32.146150+0000","last_unstale":"2024-05-21T03:21:14.558407+0000","last_undegraded":"2024-05-21T03:21:14.558407+0000","last_fullsized":"2024-05-21T03:21:14.558407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:39:09.905640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561667+0000","last_change":"2024-05-21T03:20:29.097613+0000","last_active":"2024-05-21T03:21:14.561667+0000","last_peered":"2024-05-21T03:21:14.561667+0000","last_clean":"2024-05-21T03:21:14.561667+0000","last_became_active":"2024-05-21T03:20:29.097331+0000","last_became_peered":"2024-05-21T03:20:29.097331+0000","last_unstale":"2024-05-21T03:21:14.561667+0000","last_undegraded":"2024-05-21T03:21:14.561667+0000","last_fullsized":"2024-05-21T03:21:14.561667+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:56:14.720420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.566406+0000","last_change":"2024-05-21T03:21:14.566406+0000","last_active":"2024-05-21T03:21:14.566406+0000","last_peered":"2024-05-21T03:21:14.566406+0000","last_clean":"2024-05-21T03:21:14.566406+0000","last_became_active":"2024-05-21T03:21:14.563023+0000","last_became_peered":"2024-05-21T03:21:14.563023+0000","last_unstale":"2024-05-21T03:21:14.566406+0000","last_undegraded":"2024-05-21T03:21:14.566406+0000","last_fullsized":"2024-05-21T03:21:14.566406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:12:53.125136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552818+0000","last_change":"2024-05-21T03:20:29.103456+0000","last_active":"2024-05-21T03:21:14.552818+0000","last_peered":"2024-05-21T03:21:14.552818+0000","last_clean":"2024-05-21T03:21:14.552818+0000","last_became_active":"2024-05-21T03:20:29.103008+0000","last_became_peered":"2024-05-21T03:20:29.103008+0000","last_unstale":"2024-05-21T03:21:14.552818+0000","last_undegraded":"2024-05-21T03:21:14.552818+0000","last_fullsized":"2024-05-21T03:21:14.552818+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:02:37.951347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558574+0000","last_change":"2024-05-21T03:20:32.146986+0000","last_active":"2024-05-21T03:21:14.558574+0000","last_peered":"2024-05-21T03:21:14.558574+0000","last_clean":"2024-05-21T03:21:14.558574+0000","last_became_active":"2024-05-21T03:20:32.146355+0000","last_became_peered":"2024-05-21T03:20:32.146355+0000","last_unstale":"2024-05-21T03:21:14.558574+0000","last_undegraded":"2024-05-21T03:21:14.558574+0000","last_fullsized":"2024-05-21T03:21:14.558574+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:36:29.503099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.559658+0000","last_change":"2024-05-21T03:21:14.559658+0000","last_active":"2024-05-21T03:21:14.559658+0000","last_peered":"2024-05-21T03:21:14.559658+0000","last_clean":"2024-05-21T03:21:14.559658+0000","last_became_active":"2024-05-21T03:21:14.559194+0000","last_became_peered":"2024-05-21T03:21:14.559194+0000","last_unstale":"2024-05-21T03:21:14.559658+0000","last_undegraded":"2024-05-21T03:21:14.559658+0000","last_fullsized":"2024-05-21T03:21:14.559658+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:56:31.709545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556799+0000","last_change":"2024-05-21T03:20:32.137532+0000","last_active":"2024-05-21T03:21:14.556799+0000","last_peered":"2024-05-21T03:21:14.556799+0000","last_clean":"2024-05-21T03:21:14.556799+0000","last_became_active":"2024-05-21T03:20:32.137182+0000","last_became_peered":"2024-05-21T03:20:32.137182+0000","last_unstale":"2024-05-21T03:21:14.556799+0000","last_undegraded":"2024-05-21T03:21:14.556799+0000","last_fullsized":"2024-05-21T03:21:14.556799+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:25:44.207866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556286+0000","last_change":"2024-05-21T03:20:32.153912+0000","last_active":"2024-05-21T03:21:14.556286+0000","last_peered":"2024-05-21T03:21:14.556286+0000","last_clean":"2024-05-21T03:21:14.556286+0000","last_became_active":"2024-05-21T03:20:32.153541+0000","last_became_peered":"2024-05-21T03:20:32.153541+0000","last_unstale":"2024-05-21T03:21:14.556286+0000","last_undegraded":"2024-05-21T03:21:14.556286+0000","last_fullsized":"2024-05-21T03:21:14.556286+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:02:58.688378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T03:21:13.551303+0000","last_change":"2024-05-21T03:20:32.143689+0000","last_active":"2024-05-21T03:21:13.551303+0000","last_peered":"2024-05-21T03:21:13.551303+0000","last_clean":"2024-05-21T03:21:13.551303+0000","last_became_active":"2024-05-21T03:20:32.143399+0000","last_became_peered":"2024-05-21T03:20:32.143399+0000","last_unstale":"2024-05-21T03:21:13.551303+0000","last_undegraded":"2024-05-21T03:21:13.551303+0000","last_fullsized":"2024-05-21T03:21:13.551303+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:27:55.116599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552907+0000","last_change":"2024-05-21T03:20:32.141355+0000","last_active":"2024-05-21T03:21:14.552907+0000","last_peered":"2024-05-21T03:21:14.552907+0000","last_clean":"2024-05-21T03:21:14.552907+0000","last_became_active":"2024-05-21T03:20:32.140960+0000","last_became_peered":"2024-05-21T03:20:32.140960+0000","last_unstale":"2024-05-21T03:21:14.552907+0000","last_undegraded":"2024-05-21T03:21:14.552907+0000","last_fullsized":"2024-05-21T03:21:14.552907+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:23:50.287864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.553971+0000","last_change":"2024-05-21T03:20:32.142145+0000","last_active":"2024-05-21T03:21:14.553971+0000","last_peered":"2024-05-21T03:21:14.553971+0000","last_clean":"2024-05-21T03:21:14.553971+0000","last_became_active":"2024-05-21T03:20:32.141802+0000","last_became_peered":"2024-05-21T03:20:32.141802+0000","last_unstale":"2024-05-21T03:21:14.553971+0000","last_undegraded":"2024-05-21T03:21:14.553971+0000","last_fullsized":"2024-05-21T03:21:14.553971+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:26:28.707616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556601+0000","last_change":"2024-05-21T03:20:32.141606+0000","last_active":"2024-05-21T03:21:14.556601+0000","last_peered":"2024-05-21T03:21:14.556601+0000","last_clean":"2024-05-21T03:21:14.556601+0000","last_became_active":"2024-05-21T03:20:32.141258+0000","last_became_peered":"2024-05-21T03:20:32.141258+0000","last_unstale":"2024-05-21T03:21:14.556601+0000","last_undegraded":"2024-05-21T03:21:14.556601+0000","last_fullsized":"2024-05-21T03:21:14.556601+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:22.372723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561186+0000","last_change":"2024-05-21T03:20:32.151621+0000","last_active":"2024-05-21T03:21:14.561186+0000","last_peered":"2024-05-21T03:21:14.561186+0000","last_clean":"2024-05-21T03:21:14.561186+0000","last_became_active":"2024-05-21T03:20:32.151260+0000","last_became_peered":"2024-05-21T03:20:32.151260+0000","last_unstale":"2024-05-21T03:21:14.561186+0000","last_undegraded":"2024-05-21T03:21:14.561186+0000","last_fullsized":"2024-05-21T03:21:14.561186+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:48:59.751179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561107+0000","last_change":"2024-05-21T03:20:32.153843+0000","last_active":"2024-05-21T03:21:14.561107+0000","last_peered":"2024-05-21T03:21:14.561107+0000","last_clean":"2024-05-21T03:21:14.561107+0000","last_became_active":"2024-05-21T03:20:32.153418+0000","last_became_peered":"2024-05-21T03:20:32.153418+0000","last_unstale":"2024-05-21T03:21:14.561107+0000","last_undegraded":"2024-05-21T03:21:14.561107+0000","last_fullsized":"2024-05-21T03:21:14.561107+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:26:29.663966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.560653+0000","last_change":"2024-05-21T03:20:32.153942+0000","last_active":"2024-05-21T03:21:14.560653+0000","last_peered":"2024-05-21T03:21:14.560653+0000","last_clean":"2024-05-21T03:21:14.560653+0000","last_became_active":"2024-05-21T03:20:32.153566+0000","last_became_peered":"2024-05-21T03:20:32.153566+0000","last_unstale":"2024-05-21T03:21:14.560653+0000","last_undegraded":"2024-05-21T03:21:14.560653+0000","last_fullsized":"2024-05-21T03:21:14.560653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:06.899039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558337+0000","last_change":"2024-05-21T03:20:32.153243+0000","last_active":"2024-05-21T03:21:14.558337+0000","last_peered":"2024-05-21T03:21:14.558337+0000","last_clean":"2024-05-21T03:21:14.558337+0000","last_became_active":"2024-05-21T03:20:32.152386+0000","last_became_peered":"2024-05-21T03:20:32.152386+0000","last_unstale":"2024-05-21T03:21:14.558337+0000","last_undegraded":"2024-05-21T03:21:14.558337+0000","last_fullsized":"2024-05-21T03:21:14.558337+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:25:28.935183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552892+0000","last_change":"2024-05-21T03:20:32.142536+0000","last_active":"2024-05-21T03:21:14.552892+0000","last_peered":"2024-05-21T03:21:14.552892+0000","last_clean":"2024-05-21T03:21:14.552892+0000","last_became_active":"2024-05-21T03:20:32.141793+0000","last_became_peered":"2024-05-21T03:20:32.141793+0000","last_unstale":"2024-05-21T03:21:14.552892+0000","last_undegraded":"2024-05-21T03:21:14.552892+0000","last_fullsized":"2024-05-21T03:21:14.552892+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:21:43.547103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.553745+0000","last_change":"2024-05-21T03:20:32.140368+0000","last_active":"2024-05-21T03:21:14.553745+0000","last_peered":"2024-05-21T03:21:14.553745+0000","last_clean":"2024-05-21T03:21:14.553745+0000","last_became_active":"2024-05-21T03:20:32.139747+0000","last_became_peered":"2024-05-21T03:20:32.139747+0000","last_unstale":"2024-05-21T03:21:14.553745+0000","last_undegraded":"2024-05-21T03:21:14.553745+0000","last_fullsized":"2024-05-21T03:21:14.553745+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:05:44.851191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61962,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":270,"num_read_kb":207,"num_write":239,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1561,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561152,"data_stored":196676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645336,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":901120,"data_stored":433850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841564,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1061970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005090,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":416113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201318,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":433767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397549,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32668,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712484,"statfs":{"total":95995035648,"available":95961583616,"internally_reserved":0,"allocated":1519616,"data_stored":1026676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561073,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":433497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":494172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":913408,"data_stored":426319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":487436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149760,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":432814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953532,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":434919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757303,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32680,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712472,"statfs":{"total":95995035648,"available":95961571328,"internally_reserved":0,"allocated":1531904,"data_stored":1025572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":56494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T03:21:19.118 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-21T03:21:19.368 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:21:19.762 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:19 smithi027 ceph-mon[32179]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:21:20.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:19 smithi039 ceph-mon[36429]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:21:20.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:19 smithi165 ceph-mon[36269]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:21:20.642 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:20 smithi027 ceph-mon[32179]: pgmap v315: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.6 KiB/s wr, 9 op/s; 209 B/s, 1 keys/s, 1 objects/s recovering 2024-05-21T03:21:20.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.874+0000 7fbb4c4b8700 1 -- 172.21.15.27:0/879742627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb441039d0 msgr2=0x7fbb4410f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:20.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.874+0000 7fbb4c4b8700 1 --2- 172.21.15.27:0/879742627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb441039d0 0x7fbb4410f3e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00b0d0 tx=0x7fbb3c00b3e0 comp rx=0 tx=0).stop 2024-05-21T03:21:20.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.874+0000 7fbb4c4b8700 1 -- 172.21.15.27:0/879742627 shutdown_connections 2024-05-21T03:21:20.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.874+0000 7fbb4c4b8700 1 --2- 172.21.15.27:0/879742627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb441039d0 0x7fbb4410f3e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:20.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.874+0000 7fbb4c4b8700 1 --2- 172.21.15.27:0/879742627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb44108730 0x7fbb44108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:20.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.874+0000 7fbb4c4b8700 1 --2- 172.21.15.27:0/879742627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb44103010 0x7fbb44103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:20.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.874+0000 7fbb4c4b8700 1 -- 172.21.15.27:0/879742627 >> 172.21.15.27:0/879742627 conn(0x7fbb44077780 msgr2=0x7fbb44077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:20.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.874+0000 7fbb4c4b8700 1 -- 172.21.15.27:0/879742627 shutdown_connections 2024-05-21T03:21:20.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.874+0000 7fbb4c4b8700 1 -- 172.21.15.27:0/879742627 wait complete. 2024-05-21T03:21:20.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.875+0000 7fbb4c4b8700 1 Processor -- start 2024-05-21T03:21:20.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.875+0000 7fbb4c4b8700 1 -- start start 2024-05-21T03:21:20.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.875+0000 7fbb4c4b8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb44103010 0x7fbb44104190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:20.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.875+0000 7fbb4c4b8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb441039d0 0x7fbb441046d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:20.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.875+0000 7fbb4c4b8700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb44108730 0x7fbb44104c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:20.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.875+0000 7fbb4c4b8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbb441127b0 con 0x7fbb44103010 2024-05-21T03:21:20.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.875+0000 7fbb4c4b8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbb44112630 con 0x7fbb441039d0 2024-05-21T03:21:20.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.875+0000 7fbb4c4b8700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbb44112930 con 0x7fbb44108730 2024-05-21T03:21:20.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.875+0000 7fbb4a254700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb44103010 0x7fbb44104190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:20.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.875+0000 7fbb4a254700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb44103010 0x7fbb44104190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44846/0 (socket says 172.21.15.27:44846) 2024-05-21T03:21:20.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.875+0000 7fbb4a254700 1 -- 172.21.15.27:0/740910492 learned_addr learned my addr 172.21.15.27:0/740910492 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:21:20.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.875+0000 7fbb4aa55700 1 --2- 172.21.15.27:0/740910492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb44108730 0x7fbb44104c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:20.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.875+0000 7fbb49a53700 1 --2- 172.21.15.27:0/740910492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb441039d0 0x7fbb441046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:20.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.875+0000 7fbb4a254700 1 -- 172.21.15.27:0/740910492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb44108730 msgr2=0x7fbb44104c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:20.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.875+0000 7fbb4a254700 1 --2- 172.21.15.27:0/740910492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb44108730 0x7fbb44104c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:20.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.875+0000 7fbb4a254700 1 -- 172.21.15.27:0/740910492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb441039d0 msgr2=0x7fbb441046d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:20.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.875+0000 7fbb4a254700 1 --2- 172.21.15.27:0/740910492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb441039d0 0x7fbb441046d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:20.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.875+0000 7fbb4a254700 1 -- 172.21.15.27:0/740910492 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb441a8150 con 0x7fbb44103010 2024-05-21T03:21:20.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.875+0000 7fbb49a53700 1 --2- 172.21.15.27:0/740910492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb441039d0 0x7fbb441046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:21:20.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.876+0000 7fbb4a254700 1 --2- 172.21.15.27:0/740910492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb44103010 0x7fbb44104190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbb4000e9a0 tx=0x7fbb4000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:20.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.876+0000 7fbb377fe700 1 -- 172.21.15.27:0/740910492 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4000cbd0 con 0x7fbb44103010 2024-05-21T03:21:20.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.876+0000 7fbb4c4b8700 1 -- 172.21.15.27:0/740910492 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb441a8440 con 0x7fbb44103010 2024-05-21T03:21:20.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.876+0000 7fbb377fe700 1 -- 172.21.15.27:0/740910492 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb4000cd70 con 0x7fbb44103010 2024-05-21T03:21:20.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.876+0000 7fbb377fe700 1 -- 172.21.15.27:0/740910492 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb40018680 con 0x7fbb44103010 2024-05-21T03:21:20.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.876+0000 7fbb4c4b8700 1 -- 172.21.15.27:0/740910492 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb441a89d0 con 0x7fbb44103010 2024-05-21T03:21:20.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.877+0000 7fbb4c4b8700 1 -- 172.21.15.27:0/740910492 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb4406a230 con 0x7fbb44103010 2024-05-21T03:21:20.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.878+0000 7fbb377fe700 1 -- 172.21.15.27:0/740910492 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fbb40010970 con 0x7fbb44103010 2024-05-21T03:21:20.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.879+0000 7fbb377fe700 1 --2- 172.21.15.27:0/740910492 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fbb300b2910 0x7fbb300b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:20.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.879+0000 7fbb49a53700 1 --2- 172.21.15.27:0/740910492 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fbb300b2910 0x7fbb300b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:20.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.879+0000 7fbb377fe700 1 -- 172.21.15.27:0/740910492 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fbb400e2b90 con 0x7fbb44103010 2024-05-21T03:21:20.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.880+0000 7fbb49a53700 1 --2- 172.21.15.27:0/740910492 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fbb300b2910 0x7fbb300b4dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbb38005fd0 tx=0x7fbb38005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:20.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:20.883+0000 7fbb377fe700 1 -- 172.21.15.27:0/740910492 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbb400aa730 con 0x7fbb44103010 2024-05-21T03:21:21.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:20 smithi039 ceph-mon[36429]: pgmap v315: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.6 KiB/s wr, 9 op/s; 209 B/s, 1 keys/s, 1 objects/s recovering 2024-05-21T03:21:21.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:20 smithi165 ceph-mon[36269]: pgmap v315: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.6 KiB/s wr, 9 op/s; 209 B/s, 1 keys/s, 1 objects/s recovering 2024-05-21T03:21:21.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:21.097+0000 7fbb4c4b8700 1 -- 172.21.15.27:0/740910492 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbb44063980 con 0x7fbb300b2910 2024-05-21T03:21:21.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:21.104+0000 7fbb377fe700 1 -- 172.21.15.27:0/740910492 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258769 (secure 0 0 0) 0x7fbb44063980 con 0x7fbb300b2910 2024-05-21T03:21:21.108 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:21:21.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:21.108+0000 7fbb4c4b8700 1 -- 172.21.15.27:0/740910492 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fbb300b2910 msgr2=0x7fbb300b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:21.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:21.108+0000 7fbb4c4b8700 1 --2- 172.21.15.27:0/740910492 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fbb300b2910 0x7fbb300b4dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbb38005fd0 tx=0x7fbb38005e20 comp rx=0 tx=0).stop 2024-05-21T03:21:21.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:21.108+0000 7fbb4c4b8700 1 -- 172.21.15.27:0/740910492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb44103010 msgr2=0x7fbb44104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:21.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:21.108+0000 7fbb4c4b8700 1 --2- 172.21.15.27:0/740910492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb44103010 0x7fbb44104190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbb4000e9a0 tx=0x7fbb4000ed60 comp rx=0 tx=0).stop 2024-05-21T03:21:21.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:21.108+0000 7fbb4c4b8700 1 -- 172.21.15.27:0/740910492 shutdown_connections 2024-05-21T03:21:21.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:21.108+0000 7fbb4c4b8700 1 --2- 172.21.15.27:0/740910492 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fbb300b2910 0x7fbb300b4dd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:21.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:21.108+0000 7fbb4c4b8700 1 --2- 172.21.15.27:0/740910492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb44108730 0x7fbb44104c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:21.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:21.108+0000 7fbb4c4b8700 1 --2- 172.21.15.27:0/740910492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb44103010 0x7fbb44104190 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:21.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:21.108+0000 7fbb4c4b8700 1 --2- 172.21.15.27:0/740910492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb441039d0 0x7fbb441046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:21.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:21.108+0000 7fbb4c4b8700 1 -- 172.21.15.27:0/740910492 >> 172.21.15.27:0/740910492 conn(0x7fbb44077780 msgr2=0x7fbb44102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:21.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:21.109+0000 7fbb4c4b8700 1 -- 172.21.15.27:0/740910492 shutdown_connections 2024-05-21T03:21:21.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:21.109+0000 7fbb4c4b8700 1 -- 172.21.15.27:0/740910492 wait complete. 2024-05-21T03:21:21.111 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-05-21T03:21:21.809 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2024-05-21T03:21:19.345054+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657959,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":376,"num_read_kb":301,"num_write":369,"num_write_kb":2287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2953518,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":294,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357804,"kb_used_data":12728,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584020,"statfs":{"total":1151940427776,"available":1151574036480,"internally_reserved":0,"allocated":13033472,"data_stored":7155686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19139,"internal_metadata":353154365},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":68,"num_read_kb":52,"num_write":10,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1678,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002036"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.554128+0000","last_change":"2024-05-21T03:20:32.149961+0000","last_active":"2024-05-21T03:21:14.554128+0000","last_peered":"2024-05-21T03:21:14.554128+0000","last_clean":"2024-05-21T03:21:14.554128+0000","last_became_active":"2024-05-21T03:20:32.149552+0000","last_became_peered":"2024-05-21T03:20:32.149552+0000","last_unstale":"2024-05-21T03:21:14.554128+0000","last_undegraded":"2024-05-21T03:21:14.554128+0000","last_fullsized":"2024-05-21T03:21:14.554128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:20:38.543551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556541+0000","last_change":"2024-05-21T03:20:32.146413+0000","last_active":"2024-05-21T03:21:14.556541+0000","last_peered":"2024-05-21T03:21:14.556541+0000","last_clean":"2024-05-21T03:21:14.556541+0000","last_became_active":"2024-05-21T03:20:32.146006+0000","last_became_peered":"2024-05-21T03:20:32.146006+0000","last_unstale":"2024-05-21T03:21:14.556541+0000","last_undegraded":"2024-05-21T03:21:14.556541+0000","last_fullsized":"2024-05-21T03:21:14.556541+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:05:12.888550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556573+0000","last_change":"2024-05-21T03:20:32.153654+0000","last_active":"2024-05-21T03:21:14.556573+0000","last_peered":"2024-05-21T03:21:14.556573+0000","last_clean":"2024-05-21T03:21:14.556573+0000","last_became_active":"2024-05-21T03:20:32.153296+0000","last_became_peered":"2024-05-21T03:20:32.153296+0000","last_unstale":"2024-05-21T03:21:14.556573+0000","last_undegraded":"2024-05-21T03:21:14.556573+0000","last_fullsized":"2024-05-21T03:21:14.556573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:16:09.423236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556628+0000","last_change":"2024-05-21T03:20:32.138672+0000","last_active":"2024-05-21T03:21:14.556628+0000","last_peered":"2024-05-21T03:21:14.556628+0000","last_clean":"2024-05-21T03:21:14.556628+0000","last_became_active":"2024-05-21T03:20:32.138163+0000","last_became_peered":"2024-05-21T03:20:32.138163+0000","last_unstale":"2024-05-21T03:21:14.556628+0000","last_undegraded":"2024-05-21T03:21:14.556628+0000","last_fullsized":"2024-05-21T03:21:14.556628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:26:55.012897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556450+0000","last_change":"2024-05-21T03:20:32.153950+0000","last_active":"2024-05-21T03:21:14.556450+0000","last_peered":"2024-05-21T03:21:14.556450+0000","last_clean":"2024-05-21T03:21:14.556450+0000","last_became_active":"2024-05-21T03:20:32.153567+0000","last_became_peered":"2024-05-21T03:20:32.153567+0000","last_unstale":"2024-05-21T03:21:14.556450+0000","last_undegraded":"2024-05-21T03:21:14.556450+0000","last_fullsized":"2024-05-21T03:21:14.556450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:28:34.621193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.554034+0000","last_change":"2024-05-21T03:20:32.140038+0000","last_active":"2024-05-21T03:21:14.554034+0000","last_peered":"2024-05-21T03:21:14.554034+0000","last_clean":"2024-05-21T03:21:14.554034+0000","last_became_active":"2024-05-21T03:20:32.139690+0000","last_became_peered":"2024-05-21T03:20:32.139690+0000","last_unstale":"2024-05-21T03:21:14.554034+0000","last_undegraded":"2024-05-21T03:21:14.554034+0000","last_fullsized":"2024-05-21T03:21:14.554034+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:31:24.270755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556482+0000","last_change":"2024-05-21T03:20:32.143735+0000","last_active":"2024-05-21T03:21:14.556482+0000","last_peered":"2024-05-21T03:21:14.556482+0000","last_clean":"2024-05-21T03:21:14.556482+0000","last_became_active":"2024-05-21T03:20:32.143397+0000","last_became_peered":"2024-05-21T03:20:32.143397+0000","last_unstale":"2024-05-21T03:21:14.556482+0000","last_undegraded":"2024-05-21T03:21:14.556482+0000","last_fullsized":"2024-05-21T03:21:14.556482+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:40:01.951608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.560831+0000","last_change":"2024-05-21T03:20:32.156999+0000","last_active":"2024-05-21T03:21:14.560831+0000","last_peered":"2024-05-21T03:21:14.560831+0000","last_clean":"2024-05-21T03:21:14.560831+0000","last_became_active":"2024-05-21T03:20:32.156677+0000","last_became_peered":"2024-05-21T03:20:32.156677+0000","last_unstale":"2024-05-21T03:21:14.560831+0000","last_undegraded":"2024-05-21T03:21:14.560831+0000","last_fullsized":"2024-05-21T03:21:14.560831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:33:24.225667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558931+0000","last_change":"2024-05-21T03:20:32.145039+0000","last_active":"2024-05-21T03:21:14.558931+0000","last_peered":"2024-05-21T03:21:14.558931+0000","last_clean":"2024-05-21T03:21:14.558931+0000","last_became_active":"2024-05-21T03:20:32.140926+0000","last_became_peered":"2024-05-21T03:20:32.140926+0000","last_unstale":"2024-05-21T03:21:14.558931+0000","last_undegraded":"2024-05-21T03:21:14.558931+0000","last_fullsized":"2024-05-21T03:21:14.558931+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:11:27.049426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.554577+0000","last_change":"2024-05-21T03:20:32.145840+0000","last_active":"2024-05-21T03:21:14.554577+0000","last_peered":"2024-05-21T03:21:14.554577+0000","last_clean":"2024-05-21T03:21:14.554577+0000","last_became_active":"2024-05-21T03:20:32.145291+0000","last_became_peered":"2024-05-21T03:20:32.145291+0000","last_unstale":"2024-05-21T03:21:14.554577+0000","last_undegraded":"2024-05-21T03:21:14.554577+0000","last_fullsized":"2024-05-21T03:21:14.554577+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:22:27.696588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558336+0000","last_change":"2024-05-21T03:20:32.142522+0000","last_active":"2024-05-21T03:21:14.558336+0000","last_peered":"2024-05-21T03:21:14.558336+0000","last_clean":"2024-05-21T03:21:14.558336+0000","last_became_active":"2024-05-21T03:20:32.142165+0000","last_became_peered":"2024-05-21T03:20:32.142165+0000","last_unstale":"2024-05-21T03:21:14.558336+0000","last_undegraded":"2024-05-21T03:21:14.558336+0000","last_fullsized":"2024-05-21T03:21:14.558336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:58:39.085834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556536+0000","last_change":"2024-05-21T03:20:32.142526+0000","last_active":"2024-05-21T03:21:14.556536+0000","last_peered":"2024-05-21T03:21:14.556536+0000","last_clean":"2024-05-21T03:21:14.556536+0000","last_became_active":"2024-05-21T03:20:32.141143+0000","last_became_peered":"2024-05-21T03:20:32.141143+0000","last_unstale":"2024-05-21T03:21:14.556536+0000","last_undegraded":"2024-05-21T03:21:14.556536+0000","last_fullsized":"2024-05-21T03:21:14.556536+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:35:11.336796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552998+0000","last_change":"2024-05-21T03:20:32.135631+0000","last_active":"2024-05-21T03:21:14.552998+0000","last_peered":"2024-05-21T03:21:14.552998+0000","last_clean":"2024-05-21T03:21:14.552998+0000","last_became_active":"2024-05-21T03:20:32.135302+0000","last_became_peered":"2024-05-21T03:20:32.135302+0000","last_unstale":"2024-05-21T03:21:14.552998+0000","last_undegraded":"2024-05-21T03:21:14.552998+0000","last_fullsized":"2024-05-21T03:21:14.552998+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:05:55.000690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.555811+0000","last_change":"2024-05-21T03:20:32.146580+0000","last_active":"2024-05-21T03:21:14.555811+0000","last_peered":"2024-05-21T03:21:14.555811+0000","last_clean":"2024-05-21T03:21:14.555811+0000","last_became_active":"2024-05-21T03:20:32.145803+0000","last_became_peered":"2024-05-21T03:20:32.145803+0000","last_unstale":"2024-05-21T03:21:14.555811+0000","last_undegraded":"2024-05-21T03:21:14.555811+0000","last_fullsized":"2024-05-21T03:21:14.555811+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:34:09.108799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.559498+0000","last_change":"2024-05-21T03:21:14.559498+0000","last_active":"2024-05-21T03:21:14.559498+0000","last_peered":"2024-05-21T03:21:14.559498+0000","last_clean":"2024-05-21T03:21:14.559498+0000","last_became_active":"2024-05-21T03:21:14.559039+0000","last_became_peered":"2024-05-21T03:21:14.559039+0000","last_unstale":"2024-05-21T03:21:14.559498+0000","last_undegraded":"2024-05-21T03:21:14.559498+0000","last_fullsized":"2024-05-21T03:21:14.559498+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:04:36.676857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.557368+0000","last_change":"2024-05-21T03:20:32.148944+0000","last_active":"2024-05-21T03:21:14.557368+0000","last_peered":"2024-05-21T03:21:14.557368+0000","last_clean":"2024-05-21T03:21:14.557368+0000","last_became_active":"2024-05-21T03:20:32.148502+0000","last_became_peered":"2024-05-21T03:20:32.148502+0000","last_unstale":"2024-05-21T03:21:14.557368+0000","last_undegraded":"2024-05-21T03:21:14.557368+0000","last_fullsized":"2024-05-21T03:21:14.557368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:11:00.178105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.554478+0000","last_change":"2024-05-21T03:20:32.150609+0000","last_active":"2024-05-21T03:21:14.554478+0000","last_peered":"2024-05-21T03:21:14.554478+0000","last_clean":"2024-05-21T03:21:14.554478+0000","last_became_active":"2024-05-21T03:20:32.150314+0000","last_became_peered":"2024-05-21T03:20:32.150314+0000","last_unstale":"2024-05-21T03:21:14.554478+0000","last_undegraded":"2024-05-21T03:21:14.554478+0000","last_fullsized":"2024-05-21T03:21:14.554478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:34:21.869304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556008+0000","last_change":"2024-05-21T03:20:32.142472+0000","last_active":"2024-05-21T03:21:14.556008+0000","last_peered":"2024-05-21T03:21:14.556008+0000","last_clean":"2024-05-21T03:21:14.556008+0000","last_became_active":"2024-05-21T03:20:32.142161+0000","last_became_peered":"2024-05-21T03:20:32.142161+0000","last_unstale":"2024-05-21T03:21:14.556008+0000","last_undegraded":"2024-05-21T03:21:14.556008+0000","last_fullsized":"2024-05-21T03:21:14.556008+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:26:57.281655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.557572+0000","last_change":"2024-05-21T03:20:32.153987+0000","last_active":"2024-05-21T03:21:14.557572+0000","last_peered":"2024-05-21T03:21:14.557572+0000","last_clean":"2024-05-21T03:21:14.557572+0000","last_became_active":"2024-05-21T03:20:32.153617+0000","last_became_peered":"2024-05-21T03:20:32.153617+0000","last_unstale":"2024-05-21T03:21:14.557572+0000","last_undegraded":"2024-05-21T03:21:14.557572+0000","last_fullsized":"2024-05-21T03:21:14.557572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:57:07.341758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:16.520602+0000","last_change":"2024-05-21T03:20:29.099928+0000","last_active":"2024-05-21T03:21:16.520602+0000","last_peered":"2024-05-21T03:21:16.520602+0000","last_clean":"2024-05-21T03:21:16.520602+0000","last_became_active":"2024-05-21T03:20:29.099592+0000","last_became_peered":"2024-05-21T03:20:29.099592+0000","last_unstale":"2024-05-21T03:21:16.520602+0000","last_undegraded":"2024-05-21T03:21:16.520602+0000","last_fullsized":"2024-05-21T03:21:16.520602+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:35:34.843325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561455+0000","last_change":"2024-05-21T03:20:32.154962+0000","last_active":"2024-05-21T03:21:14.561455+0000","last_peered":"2024-05-21T03:21:14.561455+0000","last_clean":"2024-05-21T03:21:14.561455+0000","last_became_active":"2024-05-21T03:20:32.154620+0000","last_became_peered":"2024-05-21T03:20:32.154620+0000","last_unstale":"2024-05-21T03:21:14.561455+0000","last_undegraded":"2024-05-21T03:21:14.561455+0000","last_fullsized":"2024-05-21T03:21:14.561455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:56:21.553272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561738+0000","last_change":"2024-05-21T03:20:29.097312+0000","last_active":"2024-05-21T03:21:14.561738+0000","last_peered":"2024-05-21T03:21:14.561738+0000","last_clean":"2024-05-21T03:21:14.561738+0000","last_became_active":"2024-05-21T03:20:29.096977+0000","last_became_peered":"2024-05-21T03:20:29.096977+0000","last_unstale":"2024-05-21T03:21:14.561738+0000","last_undegraded":"2024-05-21T03:21:14.561738+0000","last_fullsized":"2024-05-21T03:21:14.561738+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:28:45.467446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556973+0000","last_change":"2024-05-21T03:20:32.150035+0000","last_active":"2024-05-21T03:21:14.556973+0000","last_peered":"2024-05-21T03:21:14.556973+0000","last_clean":"2024-05-21T03:21:14.556973+0000","last_became_active":"2024-05-21T03:20:32.149688+0000","last_became_peered":"2024-05-21T03:20:32.149688+0000","last_unstale":"2024-05-21T03:21:14.556973+0000","last_undegraded":"2024-05-21T03:21:14.556973+0000","last_fullsized":"2024-05-21T03:21:14.556973+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:27:00.410770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.573309+0000","last_change":"2024-05-21T03:21:14.572832+0000","last_active":"2024-05-21T03:21:14.573309+0000","last_peered":"2024-05-21T03:21:14.573309+0000","last_clean":"2024-05-21T03:21:14.573309+0000","last_became_active":"2024-05-21T03:21:14.559381+0000","last_became_peered":"2024-05-21T03:21:14.559381+0000","last_unstale":"2024-05-21T03:21:14.573309+0000","last_undegraded":"2024-05-21T03:21:14.573309+0000","last_fullsized":"2024-05-21T03:21:14.573309+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:02:53.004031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558214+0000","last_change":"2024-05-21T03:20:32.150301+0000","last_active":"2024-05-21T03:21:14.558214+0000","last_peered":"2024-05-21T03:21:14.558214+0000","last_clean":"2024-05-21T03:21:14.558214+0000","last_became_active":"2024-05-21T03:20:32.148475+0000","last_became_peered":"2024-05-21T03:20:32.148475+0000","last_unstale":"2024-05-21T03:21:14.558214+0000","last_undegraded":"2024-05-21T03:21:14.558214+0000","last_fullsized":"2024-05-21T03:21:14.558214+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:59:30.339660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556899+0000","last_change":"2024-05-21T03:20:29.098861+0000","last_active":"2024-05-21T03:21:14.556899+0000","last_peered":"2024-05-21T03:21:14.556899+0000","last_clean":"2024-05-21T03:21:14.556899+0000","last_became_active":"2024-05-21T03:20:29.097874+0000","last_became_peered":"2024-05-21T03:20:29.097874+0000","last_unstale":"2024-05-21T03:21:14.556899+0000","last_undegraded":"2024-05-21T03:21:14.556899+0000","last_fullsized":"2024-05-21T03:21:14.556899+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:25:36.227339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561801+0000","last_change":"2024-05-21T03:20:32.142820+0000","last_active":"2024-05-21T03:21:14.561801+0000","last_peered":"2024-05-21T03:21:14.561801+0000","last_clean":"2024-05-21T03:21:14.561801+0000","last_became_active":"2024-05-21T03:20:32.142396+0000","last_became_peered":"2024-05-21T03:20:32.142396+0000","last_unstale":"2024-05-21T03:21:14.561801+0000","last_undegraded":"2024-05-21T03:21:14.561801+0000","last_fullsized":"2024-05-21T03:21:14.561801+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:30:14.452389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":60,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:15.521050+0000","last_change":"2024-05-21T03:20:29.103153+0000","last_active":"2024-05-21T03:21:15.521050+0000","last_peered":"2024-05-21T03:21:15.521050+0000","last_clean":"2024-05-21T03:21:15.521050+0000","last_became_active":"2024-05-21T03:20:29.102819+0000","last_became_peered":"2024-05-21T03:20:29.102819+0000","last_unstale":"2024-05-21T03:21:15.521050+0000","last_undegraded":"2024-05-21T03:21:15.521050+0000","last_fullsized":"2024-05-21T03:21:15.521050+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:47:49.843926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556528+0000","last_change":"2024-05-21T03:20:32.142263+0000","last_active":"2024-05-21T03:21:14.556528+0000","last_peered":"2024-05-21T03:21:14.556528+0000","last_clean":"2024-05-21T03:21:14.556528+0000","last_became_active":"2024-05-21T03:20:32.141820+0000","last_became_peered":"2024-05-21T03:20:32.141820+0000","last_unstale":"2024-05-21T03:21:14.556528+0000","last_undegraded":"2024-05-21T03:21:14.556528+0000","last_fullsized":"2024-05-21T03:21:14.556528+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:45:20.021237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.554011+0000","last_change":"2024-05-21T03:20:29.100088+0000","last_active":"2024-05-21T03:21:14.554011+0000","last_peered":"2024-05-21T03:21:14.554011+0000","last_clean":"2024-05-21T03:21:14.554011+0000","last_became_active":"2024-05-21T03:20:29.097343+0000","last_became_peered":"2024-05-21T03:20:29.097343+0000","last_unstale":"2024-05-21T03:21:14.554011+0000","last_undegraded":"2024-05-21T03:21:14.554011+0000","last_fullsized":"2024-05-21T03:21:14.554011+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:00:35.682393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.562687+0000","last_change":"2024-05-21T03:21:14.562281+0000","last_active":"2024-05-21T03:21:14.562687+0000","last_peered":"2024-05-21T03:21:14.562687+0000","last_clean":"2024-05-21T03:21:14.562687+0000","last_became_active":"2024-05-21T03:21:14.561857+0000","last_became_peered":"2024-05-21T03:21:14.561857+0000","last_unstale":"2024-05-21T03:21:14.562687+0000","last_undegraded":"2024-05-21T03:21:14.562687+0000","last_fullsized":"2024-05-21T03:21:14.562687+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:37:35.283529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552729+0000","last_change":"2024-05-21T03:20:29.097945+0000","last_active":"2024-05-21T03:21:14.552729+0000","last_peered":"2024-05-21T03:21:14.552729+0000","last_clean":"2024-05-21T03:21:14.552729+0000","last_became_active":"2024-05-21T03:20:29.097624+0000","last_became_peered":"2024-05-21T03:20:29.097624+0000","last_unstale":"2024-05-21T03:21:14.552729+0000","last_undegraded":"2024-05-21T03:21:14.552729+0000","last_fullsized":"2024-05-21T03:21:14.552729+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:01:01.694893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.560719+0000","last_change":"2024-05-21T03:20:32.148955+0000","last_active":"2024-05-21T03:21:14.560719+0000","last_peered":"2024-05-21T03:21:14.560719+0000","last_clean":"2024-05-21T03:21:14.560719+0000","last_became_active":"2024-05-21T03:20:32.148631+0000","last_became_peered":"2024-05-21T03:20:32.148631+0000","last_unstale":"2024-05-21T03:21:14.560719+0000","last_undegraded":"2024-05-21T03:21:14.560719+0000","last_fullsized":"2024-05-21T03:21:14.560719+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:36:25.820580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561173+0000","last_change":"2024-05-21T03:20:29.100902+0000","last_active":"2024-05-21T03:21:14.561173+0000","last_peered":"2024-05-21T03:21:14.561173+0000","last_clean":"2024-05-21T03:21:14.561173+0000","last_became_active":"2024-05-21T03:20:29.100570+0000","last_became_peered":"2024-05-21T03:20:29.100570+0000","last_unstale":"2024-05-21T03:21:14.561173+0000","last_undegraded":"2024-05-21T03:21:14.561173+0000","last_fullsized":"2024-05-21T03:21:14.561173+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:58:36.651338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.555588+0000","last_change":"2024-05-21T03:20:32.150724+0000","last_active":"2024-05-21T03:21:14.555588+0000","last_peered":"2024-05-21T03:21:14.555588+0000","last_clean":"2024-05-21T03:21:14.555588+0000","last_became_active":"2024-05-21T03:20:32.150371+0000","last_became_peered":"2024-05-21T03:20:32.150371+0000","last_unstale":"2024-05-21T03:21:14.555588+0000","last_undegraded":"2024-05-21T03:21:14.555588+0000","last_fullsized":"2024-05-21T03:21:14.555588+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:44:32.321192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":87,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:17.567785+0000","last_change":"2024-05-21T03:20:29.095045+0000","last_active":"2024-05-21T03:21:17.567785+0000","last_peered":"2024-05-21T03:21:17.567785+0000","last_clean":"2024-05-21T03:21:17.567785+0000","last_became_active":"2024-05-21T03:20:29.094552+0000","last_became_peered":"2024-05-21T03:20:29.094552+0000","last_unstale":"2024-05-21T03:21:17.567785+0000","last_undegraded":"2024-05-21T03:21:17.567785+0000","last_fullsized":"2024-05-21T03:21:17.567785+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:36:15.155125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":57,"num_read_kb":57,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558897+0000","last_change":"2024-05-21T03:20:32.149813+0000","last_active":"2024-05-21T03:21:14.558897+0000","last_peered":"2024-05-21T03:21:14.558897+0000","last_clean":"2024-05-21T03:21:14.558897+0000","last_became_active":"2024-05-21T03:20:32.148422+0000","last_became_peered":"2024-05-21T03:20:32.148422+0000","last_unstale":"2024-05-21T03:21:14.558897+0000","last_undegraded":"2024-05-21T03:21:14.558897+0000","last_fullsized":"2024-05-21T03:21:14.558897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:55:35.115463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558265+0000","last_change":"2024-05-21T03:20:29.101463+0000","last_active":"2024-05-21T03:21:14.558265+0000","last_peered":"2024-05-21T03:21:14.558265+0000","last_clean":"2024-05-21T03:21:14.558265+0000","last_became_active":"2024-05-21T03:20:29.101127+0000","last_became_peered":"2024-05-21T03:20:29.101127+0000","last_unstale":"2024-05-21T03:21:14.558265+0000","last_undegraded":"2024-05-21T03:21:14.558265+0000","last_fullsized":"2024-05-21T03:21:14.558265+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:58:17.408315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.563434+0000","last_change":"2024-05-21T03:21:14.563434+0000","last_active":"2024-05-21T03:21:14.563434+0000","last_peered":"2024-05-21T03:21:14.563434+0000","last_clean":"2024-05-21T03:21:14.563434+0000","last_became_active":"2024-05-21T03:21:14.563081+0000","last_became_peered":"2024-05-21T03:21:14.563081+0000","last_unstale":"2024-05-21T03:21:14.563434+0000","last_undegraded":"2024-05-21T03:21:14.563434+0000","last_fullsized":"2024-05-21T03:21:14.563434+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:28:55.497338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.562216+0000","last_change":"2024-05-21T03:20:29.098725+0000","last_active":"2024-05-21T03:21:14.562216+0000","last_peered":"2024-05-21T03:21:14.562216+0000","last_clean":"2024-05-21T03:21:14.562216+0000","last_became_active":"2024-05-21T03:20:29.098471+0000","last_became_peered":"2024-05-21T03:20:29.098471+0000","last_unstale":"2024-05-21T03:21:14.562216+0000","last_undegraded":"2024-05-21T03:21:14.562216+0000","last_fullsized":"2024-05-21T03:21:14.562216+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:17:11.059825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.560975+0000","last_change":"2024-05-21T03:21:14.560975+0000","last_active":"2024-05-21T03:21:14.560975+0000","last_peered":"2024-05-21T03:21:14.560975+0000","last_clean":"2024-05-21T03:21:14.560975+0000","last_became_active":"2024-05-21T03:21:14.560528+0000","last_became_peered":"2024-05-21T03:21:14.560528+0000","last_unstale":"2024-05-21T03:21:14.560975+0000","last_undegraded":"2024-05-21T03:21:14.560975+0000","last_fullsized":"2024-05-21T03:21:14.560975+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:51:47.469291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'31","reported_seq":124,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:18.524649+0000","last_change":"2024-05-21T03:20:29.099968+0000","last_active":"2024-05-21T03:21:18.524649+0000","last_peered":"2024-05-21T03:21:18.524649+0000","last_clean":"2024-05-21T03:21:18.524649+0000","last_became_active":"2024-05-21T03:20:29.099181+0000","last_became_peered":"2024-05-21T03:20:29.099181+0000","last_unstale":"2024-05-21T03:21:18.524649+0000","last_undegraded":"2024-05-21T03:21:18.524649+0000","last_fullsized":"2024-05-21T03:21:18.524649+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:35:33.934412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59762,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":91,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556604+0000","last_change":"2024-05-21T03:20:32.144024+0000","last_active":"2024-05-21T03:21:14.556604+0000","last_peered":"2024-05-21T03:21:14.556604+0000","last_clean":"2024-05-21T03:21:14.556604+0000","last_became_active":"2024-05-21T03:20:32.143668+0000","last_became_peered":"2024-05-21T03:20:32.143668+0000","last_unstale":"2024-05-21T03:21:14.556604+0000","last_undegraded":"2024-05-21T03:21:14.556604+0000","last_fullsized":"2024-05-21T03:21:14.556604+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:11:31.100802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556933+0000","last_change":"2024-05-21T03:20:29.101770+0000","last_active":"2024-05-21T03:21:14.556933+0000","last_peered":"2024-05-21T03:21:14.556933+0000","last_clean":"2024-05-21T03:21:14.556933+0000","last_became_active":"2024-05-21T03:20:29.100407+0000","last_became_peered":"2024-05-21T03:20:29.100407+0000","last_unstale":"2024-05-21T03:21:14.556933+0000","last_undegraded":"2024-05-21T03:21:14.556933+0000","last_fullsized":"2024-05-21T03:21:14.556933+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:01:33.991246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552656+0000","last_change":"2024-05-21T03:20:32.147479+0000","last_active":"2024-05-21T03:21:14.552656+0000","last_peered":"2024-05-21T03:21:14.552656+0000","last_clean":"2024-05-21T03:21:14.552656+0000","last_became_active":"2024-05-21T03:20:32.147160+0000","last_became_peered":"2024-05-21T03:20:32.147160+0000","last_unstale":"2024-05-21T03:21:14.552656+0000","last_undegraded":"2024-05-21T03:21:14.552656+0000","last_fullsized":"2024-05-21T03:21:14.552656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:51:59.343380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.562216+0000","last_change":"2024-05-21T03:20:29.101875+0000","last_active":"2024-05-21T03:21:14.562216+0000","last_peered":"2024-05-21T03:21:14.562216+0000","last_clean":"2024-05-21T03:21:14.562216+0000","last_became_active":"2024-05-21T03:20:29.101468+0000","last_became_peered":"2024-05-21T03:20:29.101468+0000","last_unstale":"2024-05-21T03:21:14.562216+0000","last_undegraded":"2024-05-21T03:21:14.562216+0000","last_fullsized":"2024-05-21T03:21:14.562216+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:10:19.577836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.557393+0000","last_change":"2024-05-21T03:20:32.139002+0000","last_active":"2024-05-21T03:21:14.557393+0000","last_peered":"2024-05-21T03:21:14.557393+0000","last_clean":"2024-05-21T03:21:14.557393+0000","last_became_active":"2024-05-21T03:20:32.138619+0000","last_became_peered":"2024-05-21T03:20:32.138619+0000","last_unstale":"2024-05-21T03:21:14.557393+0000","last_undegraded":"2024-05-21T03:21:14.557393+0000","last_fullsized":"2024-05-21T03:21:14.557393+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:21:36.136283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":61,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:15.520348+0000","last_change":"2024-05-21T03:20:29.101532+0000","last_active":"2024-05-21T03:21:15.520348+0000","last_peered":"2024-05-21T03:21:15.520348+0000","last_clean":"2024-05-21T03:21:15.520348+0000","last_became_active":"2024-05-21T03:20:29.101206+0000","last_became_peered":"2024-05-21T03:20:29.101206+0000","last_unstale":"2024-05-21T03:21:15.520348+0000","last_undegraded":"2024-05-21T03:21:15.520348+0000","last_fullsized":"2024-05-21T03:21:15.520348+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:47:38.555995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":31,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556597+0000","last_change":"2024-05-21T03:20:32.146866+0000","last_active":"2024-05-21T03:21:14.556597+0000","last_peered":"2024-05-21T03:21:14.556597+0000","last_clean":"2024-05-21T03:21:14.556597+0000","last_became_active":"2024-05-21T03:20:32.146463+0000","last_became_peered":"2024-05-21T03:20:32.146463+0000","last_unstale":"2024-05-21T03:21:14.556597+0000","last_undegraded":"2024-05-21T03:21:14.556597+0000","last_fullsized":"2024-05-21T03:21:14.556597+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:06:54.359922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556379+0000","last_change":"2024-05-21T03:20:29.103910+0000","last_active":"2024-05-21T03:21:14.556379+0000","last_peered":"2024-05-21T03:21:14.556379+0000","last_clean":"2024-05-21T03:21:14.556379+0000","last_became_active":"2024-05-21T03:20:29.103583+0000","last_became_peered":"2024-05-21T03:20:29.103583+0000","last_unstale":"2024-05-21T03:21:14.556379+0000","last_undegraded":"2024-05-21T03:21:14.556379+0000","last_fullsized":"2024-05-21T03:21:14.556379+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:57:15.665158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561420+0000","last_change":"2024-05-21T03:20:32.141242+0000","last_active":"2024-05-21T03:21:14.561420+0000","last_peered":"2024-05-21T03:21:14.561420+0000","last_clean":"2024-05-21T03:21:14.561420+0000","last_became_active":"2024-05-21T03:20:32.140892+0000","last_became_peered":"2024-05-21T03:20:32.140892+0000","last_unstale":"2024-05-21T03:21:14.561420+0000","last_undegraded":"2024-05-21T03:21:14.561420+0000","last_fullsized":"2024-05-21T03:21:14.561420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:42:13.997708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.557026+0000","last_change":"2024-05-21T03:20:29.098973+0000","last_active":"2024-05-21T03:21:14.557026+0000","last_peered":"2024-05-21T03:21:14.557026+0000","last_clean":"2024-05-21T03:21:14.557026+0000","last_became_active":"2024-05-21T03:20:29.098694+0000","last_became_peered":"2024-05-21T03:20:29.098694+0000","last_unstale":"2024-05-21T03:21:14.557026+0000","last_undegraded":"2024-05-21T03:21:14.557026+0000","last_fullsized":"2024-05-21T03:21:14.557026+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:55:17.915820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.553963+0000","last_change":"2024-05-21T03:20:32.147465+0000","last_active":"2024-05-21T03:21:14.553963+0000","last_peered":"2024-05-21T03:21:14.553963+0000","last_clean":"2024-05-21T03:21:14.553963+0000","last_became_active":"2024-05-21T03:20:32.147073+0000","last_became_peered":"2024-05-21T03:20:32.147073+0000","last_unstale":"2024-05-21T03:21:14.553963+0000","last_undegraded":"2024-05-21T03:21:14.553963+0000","last_fullsized":"2024-05-21T03:21:14.553963+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:29:05.451370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.560568+0000","last_change":"2024-05-21T03:21:14.560568+0000","last_active":"2024-05-21T03:21:14.560568+0000","last_peered":"2024-05-21T03:21:14.560568+0000","last_clean":"2024-05-21T03:21:14.560568+0000","last_became_active":"2024-05-21T03:21:14.560093+0000","last_became_peered":"2024-05-21T03:21:14.560093+0000","last_unstale":"2024-05-21T03:21:14.560568+0000","last_undegraded":"2024-05-21T03:21:14.560568+0000","last_fullsized":"2024-05-21T03:21:14.560568+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:55:39.839942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.554395+0000","last_change":"2024-05-21T03:20:32.146456+0000","last_active":"2024-05-21T03:21:14.554395+0000","last_peered":"2024-05-21T03:21:14.554395+0000","last_clean":"2024-05-21T03:21:14.554395+0000","last_became_active":"2024-05-21T03:20:32.146105+0000","last_became_peered":"2024-05-21T03:20:32.146105+0000","last_unstale":"2024-05-21T03:21:14.554395+0000","last_undegraded":"2024-05-21T03:21:14.554395+0000","last_fullsized":"2024-05-21T03:21:14.554395+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:00:30.550178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'6","reported_seq":60,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:15.519771+0000","last_change":"2024-05-21T03:20:29.100262+0000","last_active":"2024-05-21T03:21:15.519771+0000","last_peered":"2024-05-21T03:21:15.519771+0000","last_clean":"2024-05-21T03:21:15.519771+0000","last_became_active":"2024-05-21T03:20:29.099917+0000","last_became_peered":"2024-05-21T03:20:29.099917+0000","last_unstale":"2024-05-21T03:21:15.519771+0000","last_undegraded":"2024-05-21T03:21:15.519771+0000","last_fullsized":"2024-05-21T03:21:15.519771+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:55:48.608341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.554083+0000","last_change":"2024-05-21T03:20:32.142776+0000","last_active":"2024-05-21T03:21:14.554083+0000","last_peered":"2024-05-21T03:21:14.554083+0000","last_clean":"2024-05-21T03:21:14.554083+0000","last_became_active":"2024-05-21T03:20:32.142430+0000","last_became_peered":"2024-05-21T03:20:32.142430+0000","last_unstale":"2024-05-21T03:21:14.554083+0000","last_undegraded":"2024-05-21T03:21:14.554083+0000","last_fullsized":"2024-05-21T03:21:14.554083+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:54:28.682343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561357+0000","last_change":"2024-05-21T03:20:32.139506+0000","last_active":"2024-05-21T03:21:14.561357+0000","last_peered":"2024-05-21T03:21:14.561357+0000","last_clean":"2024-05-21T03:21:14.561357+0000","last_became_active":"2024-05-21T03:20:32.138136+0000","last_became_peered":"2024-05-21T03:20:32.138136+0000","last_unstale":"2024-05-21T03:21:14.561357+0000","last_undegraded":"2024-05-21T03:21:14.561357+0000","last_fullsized":"2024-05-21T03:21:14.561357+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:31:24.996276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":58,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552693+0000","last_change":"2024-05-21T03:20:29.096546+0000","last_active":"2024-05-21T03:21:14.552693+0000","last_peered":"2024-05-21T03:21:14.552693+0000","last_clean":"2024-05-21T03:21:14.552693+0000","last_became_active":"2024-05-21T03:20:29.096184+0000","last_became_peered":"2024-05-21T03:20:29.096184+0000","last_unstale":"2024-05-21T03:21:14.552693+0000","last_undegraded":"2024-05-21T03:21:14.552693+0000","last_fullsized":"2024-05-21T03:21:14.552693+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:09:12.832745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561399+0000","last_change":"2024-05-21T03:20:32.156231+0000","last_active":"2024-05-21T03:21:14.561399+0000","last_peered":"2024-05-21T03:21:14.561399+0000","last_clean":"2024-05-21T03:21:14.561399+0000","last_became_active":"2024-05-21T03:20:32.154035+0000","last_became_peered":"2024-05-21T03:20:32.154035+0000","last_unstale":"2024-05-21T03:21:14.561399+0000","last_undegraded":"2024-05-21T03:21:14.561399+0000","last_fullsized":"2024-05-21T03:21:14.561399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:30:53.820292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556130+0000","last_change":"2024-05-21T03:20:29.100422+0000","last_active":"2024-05-21T03:21:14.556130+0000","last_peered":"2024-05-21T03:21:14.556130+0000","last_clean":"2024-05-21T03:21:14.556130+0000","last_became_active":"2024-05-21T03:20:29.099993+0000","last_became_peered":"2024-05-21T03:20:29.099993+0000","last_unstale":"2024-05-21T03:21:14.556130+0000","last_undegraded":"2024-05-21T03:21:14.556130+0000","last_fullsized":"2024-05-21T03:21:14.556130+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:03:15.456447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":98,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556799+0000","last_change":"2024-05-21T03:19:05.647808+0000","last_active":"2024-05-21T03:21:14.556799+0000","last_peered":"2024-05-21T03:21:14.556799+0000","last_clean":"2024-05-21T03:21:14.556799+0000","last_became_active":"2024-05-21T03:19:05.641865+0000","last_became_peered":"2024-05-21T03:19:05.641865+0000","last_unstale":"2024-05-21T03:21:14.556799+0000","last_undegraded":"2024-05-21T03:21:14.556799+0000","last_fullsized":"2024-05-21T03:21:14.556799+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:15:42.008493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:15:42.008493+0000","last_clean_scrub_stamp":"2024-05-21T03:15:42.008493+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:26:58.040196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552599+0000","last_change":"2024-05-21T03:20:32.138182+0000","last_active":"2024-05-21T03:21:14.552599+0000","last_peered":"2024-05-21T03:21:14.552599+0000","last_clean":"2024-05-21T03:21:14.552599+0000","last_became_active":"2024-05-21T03:20:32.137842+0000","last_became_peered":"2024-05-21T03:20:32.137842+0000","last_unstale":"2024-05-21T03:21:14.552599+0000","last_undegraded":"2024-05-21T03:21:14.552599+0000","last_fullsized":"2024-05-21T03:21:14.552599+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:43:50.070160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556989+0000","last_change":"2024-05-21T03:20:29.104246+0000","last_active":"2024-05-21T03:21:14.556989+0000","last_peered":"2024-05-21T03:21:14.556989+0000","last_clean":"2024-05-21T03:21:14.556989+0000","last_became_active":"2024-05-21T03:20:29.103910+0000","last_became_peered":"2024-05-21T03:20:29.103910+0000","last_unstale":"2024-05-21T03:21:14.556989+0000","last_undegraded":"2024-05-21T03:21:14.556989+0000","last_fullsized":"2024-05-21T03:21:14.556989+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:39:28.029283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556715+0000","last_change":"2024-05-21T03:20:29.102020+0000","last_active":"2024-05-21T03:21:14.556715+0000","last_peered":"2024-05-21T03:21:14.556715+0000","last_clean":"2024-05-21T03:21:14.556715+0000","last_became_active":"2024-05-21T03:20:29.100429+0000","last_became_peered":"2024-05-21T03:20:29.100429+0000","last_unstale":"2024-05-21T03:21:14.556715+0000","last_undegraded":"2024-05-21T03:21:14.556715+0000","last_fullsized":"2024-05-21T03:21:14.556715+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:09:01.621074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552834+0000","last_change":"2024-05-21T03:20:32.137130+0000","last_active":"2024-05-21T03:21:14.552834+0000","last_peered":"2024-05-21T03:21:14.552834+0000","last_clean":"2024-05-21T03:21:14.552834+0000","last_became_active":"2024-05-21T03:20:32.136821+0000","last_became_peered":"2024-05-21T03:20:32.136821+0000","last_unstale":"2024-05-21T03:21:14.552834+0000","last_undegraded":"2024-05-21T03:21:14.552834+0000","last_fullsized":"2024-05-21T03:21:14.552834+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:53:41.290370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'14","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.555888+0000","last_change":"2024-05-21T03:20:29.099869+0000","last_active":"2024-05-21T03:21:14.555888+0000","last_peered":"2024-05-21T03:21:14.555888+0000","last_clean":"2024-05-21T03:21:14.555888+0000","last_became_active":"2024-05-21T03:20:29.099534+0000","last_became_peered":"2024-05-21T03:20:29.099534+0000","last_unstale":"2024-05-21T03:21:14.555888+0000","last_undegraded":"2024-05-21T03:21:14.555888+0000","last_fullsized":"2024-05-21T03:21:14.555888+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:01:11.281544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561631+0000","last_change":"2024-05-21T03:20:32.153921+0000","last_active":"2024-05-21T03:21:14.561631+0000","last_peered":"2024-05-21T03:21:14.561631+0000","last_clean":"2024-05-21T03:21:14.561631+0000","last_became_active":"2024-05-21T03:20:32.153594+0000","last_became_peered":"2024-05-21T03:20:32.153594+0000","last_unstale":"2024-05-21T03:21:14.561631+0000","last_undegraded":"2024-05-21T03:21:14.561631+0000","last_fullsized":"2024-05-21T03:21:14.561631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:27:59.426942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556915+0000","last_change":"2024-05-21T03:20:29.097399+0000","last_active":"2024-05-21T03:21:14.556915+0000","last_peered":"2024-05-21T03:21:14.556915+0000","last_clean":"2024-05-21T03:21:14.556915+0000","last_became_active":"2024-05-21T03:20:29.097010+0000","last_became_peered":"2024-05-21T03:20:29.097010+0000","last_unstale":"2024-05-21T03:21:14.556915+0000","last_undegraded":"2024-05-21T03:21:14.556915+0000","last_fullsized":"2024-05-21T03:21:14.556915+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:54:53.273947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.559609+0000","last_change":"2024-05-21T03:21:14.559609+0000","last_active":"2024-05-21T03:21:14.559609+0000","last_peered":"2024-05-21T03:21:14.559609+0000","last_clean":"2024-05-21T03:21:14.559609+0000","last_became_active":"2024-05-21T03:21:14.559218+0000","last_became_peered":"2024-05-21T03:21:14.559218+0000","last_unstale":"2024-05-21T03:21:14.559609+0000","last_undegraded":"2024-05-21T03:21:14.559609+0000","last_fullsized":"2024-05-21T03:21:14.559609+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:32:11.156037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,4],"acting":[0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.557892+0000","last_change":"2024-05-21T03:20:29.103526+0000","last_active":"2024-05-21T03:21:14.557892+0000","last_peered":"2024-05-21T03:21:14.557892+0000","last_clean":"2024-05-21T03:21:14.557892+0000","last_became_active":"2024-05-21T03:20:29.103067+0000","last_became_peered":"2024-05-21T03:20:29.103067+0000","last_unstale":"2024-05-21T03:21:14.557892+0000","last_undegraded":"2024-05-21T03:21:14.557892+0000","last_fullsized":"2024-05-21T03:21:14.557892+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:54:47.566949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556409+0000","last_change":"2024-05-21T03:20:32.141088+0000","last_active":"2024-05-21T03:21:14.556409+0000","last_peered":"2024-05-21T03:21:14.556409+0000","last_clean":"2024-05-21T03:21:14.556409+0000","last_became_active":"2024-05-21T03:20:32.140744+0000","last_became_peered":"2024-05-21T03:20:32.140744+0000","last_unstale":"2024-05-21T03:21:14.556409+0000","last_undegraded":"2024-05-21T03:21:14.556409+0000","last_fullsized":"2024-05-21T03:21:14.556409+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:57:35.591164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.560757+0000","last_change":"2024-05-21T03:20:29.093440+0000","last_active":"2024-05-21T03:21:14.560757+0000","last_peered":"2024-05-21T03:21:14.560757+0000","last_clean":"2024-05-21T03:21:14.560757+0000","last_became_active":"2024-05-21T03:20:29.092935+0000","last_became_peered":"2024-05-21T03:20:29.092935+0000","last_unstale":"2024-05-21T03:21:14.560757+0000","last_undegraded":"2024-05-21T03:21:14.560757+0000","last_fullsized":"2024-05-21T03:21:14.560757+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:18:00.690465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.553665+0000","last_change":"2024-05-21T03:20:32.143781+0000","last_active":"2024-05-21T03:21:14.553665+0000","last_peered":"2024-05-21T03:21:14.553665+0000","last_clean":"2024-05-21T03:21:14.553665+0000","last_became_active":"2024-05-21T03:20:32.143391+0000","last_became_peered":"2024-05-21T03:20:32.143391+0000","last_unstale":"2024-05-21T03:21:14.553665+0000","last_undegraded":"2024-05-21T03:21:14.553665+0000","last_fullsized":"2024-05-21T03:21:14.553665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:00:41.506561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556256+0000","last_change":"2024-05-21T03:20:29.101342+0000","last_active":"2024-05-21T03:21:14.556256+0000","last_peered":"2024-05-21T03:21:14.556256+0000","last_clean":"2024-05-21T03:21:14.556256+0000","last_became_active":"2024-05-21T03:20:29.100997+0000","last_became_peered":"2024-05-21T03:20:29.100997+0000","last_unstale":"2024-05-21T03:21:14.556256+0000","last_undegraded":"2024-05-21T03:21:14.556256+0000","last_fullsized":"2024-05-21T03:21:14.556256+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:05:31.801669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561184+0000","last_change":"2024-05-21T03:20:32.150563+0000","last_active":"2024-05-21T03:21:14.561184+0000","last_peered":"2024-05-21T03:21:14.561184+0000","last_clean":"2024-05-21T03:21:14.561184+0000","last_became_active":"2024-05-21T03:20:32.150231+0000","last_became_peered":"2024-05-21T03:20:32.150231+0000","last_unstale":"2024-05-21T03:21:14.561184+0000","last_undegraded":"2024-05-21T03:21:14.561184+0000","last_fullsized":"2024-05-21T03:21:14.561184+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:10:00.251284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556609+0000","last_change":"2024-05-21T03:20:29.101576+0000","last_active":"2024-05-21T03:21:14.556609+0000","last_peered":"2024-05-21T03:21:14.556609+0000","last_clean":"2024-05-21T03:21:14.556609+0000","last_became_active":"2024-05-21T03:20:29.101270+0000","last_became_peered":"2024-05-21T03:20:29.101270+0000","last_unstale":"2024-05-21T03:21:14.556609+0000","last_undegraded":"2024-05-21T03:21:14.556609+0000","last_fullsized":"2024-05-21T03:21:14.556609+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:50:10.579901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.557991+0000","last_change":"2024-05-21T03:20:32.141340+0000","last_active":"2024-05-21T03:21:14.557991+0000","last_peered":"2024-05-21T03:21:14.557991+0000","last_clean":"2024-05-21T03:21:14.557991+0000","last_became_active":"2024-05-21T03:20:32.140792+0000","last_became_peered":"2024-05-21T03:20:32.140792+0000","last_unstale":"2024-05-21T03:21:14.557991+0000","last_undegraded":"2024-05-21T03:21:14.557991+0000","last_fullsized":"2024-05-21T03:21:14.557991+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:35:05.842929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'6","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:16.567139+0000","last_change":"2024-05-21T03:21:14.566886+0000","last_active":"2024-05-21T03:21:16.567139+0000","last_peered":"2024-05-21T03:21:16.567139+0000","last_clean":"2024-05-21T03:21:16.567139+0000","last_became_active":"2024-05-21T03:21:14.559816+0000","last_became_peered":"2024-05-21T03:21:14.559816+0000","last_unstale":"2024-05-21T03:21:16.567139+0000","last_undegraded":"2024-05-21T03:21:16.567139+0000","last_fullsized":"2024-05-21T03:21:16.567139+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:34:43.109424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1561,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558407+0000","last_change":"2024-05-21T03:20:32.146674+0000","last_active":"2024-05-21T03:21:14.558407+0000","last_peered":"2024-05-21T03:21:14.558407+0000","last_clean":"2024-05-21T03:21:14.558407+0000","last_became_active":"2024-05-21T03:20:32.146150+0000","last_became_peered":"2024-05-21T03:20:32.146150+0000","last_unstale":"2024-05-21T03:21:14.558407+0000","last_undegraded":"2024-05-21T03:21:14.558407+0000","last_fullsized":"2024-05-21T03:21:14.558407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:39:09.905640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561667+0000","last_change":"2024-05-21T03:20:29.097613+0000","last_active":"2024-05-21T03:21:14.561667+0000","last_peered":"2024-05-21T03:21:14.561667+0000","last_clean":"2024-05-21T03:21:14.561667+0000","last_became_active":"2024-05-21T03:20:29.097331+0000","last_became_peered":"2024-05-21T03:20:29.097331+0000","last_unstale":"2024-05-21T03:21:14.561667+0000","last_undegraded":"2024-05-21T03:21:14.561667+0000","last_fullsized":"2024-05-21T03:21:14.561667+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:56:14.720420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.566406+0000","last_change":"2024-05-21T03:21:14.566406+0000","last_active":"2024-05-21T03:21:14.566406+0000","last_peered":"2024-05-21T03:21:14.566406+0000","last_clean":"2024-05-21T03:21:14.566406+0000","last_became_active":"2024-05-21T03:21:14.563023+0000","last_became_peered":"2024-05-21T03:21:14.563023+0000","last_unstale":"2024-05-21T03:21:14.566406+0000","last_undegraded":"2024-05-21T03:21:14.566406+0000","last_fullsized":"2024-05-21T03:21:14.566406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:12:53.125136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552818+0000","last_change":"2024-05-21T03:20:29.103456+0000","last_active":"2024-05-21T03:21:14.552818+0000","last_peered":"2024-05-21T03:21:14.552818+0000","last_clean":"2024-05-21T03:21:14.552818+0000","last_became_active":"2024-05-21T03:20:29.103008+0000","last_became_peered":"2024-05-21T03:20:29.103008+0000","last_unstale":"2024-05-21T03:21:14.552818+0000","last_undegraded":"2024-05-21T03:21:14.552818+0000","last_fullsized":"2024-05-21T03:21:14.552818+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:02:37.951347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558574+0000","last_change":"2024-05-21T03:20:32.146986+0000","last_active":"2024-05-21T03:21:14.558574+0000","last_peered":"2024-05-21T03:21:14.558574+0000","last_clean":"2024-05-21T03:21:14.558574+0000","last_became_active":"2024-05-21T03:20:32.146355+0000","last_became_peered":"2024-05-21T03:20:32.146355+0000","last_unstale":"2024-05-21T03:21:14.558574+0000","last_undegraded":"2024-05-21T03:21:14.558574+0000","last_fullsized":"2024-05-21T03:21:14.558574+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:36:29.503099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.559658+0000","last_change":"2024-05-21T03:21:14.559658+0000","last_active":"2024-05-21T03:21:14.559658+0000","last_peered":"2024-05-21T03:21:14.559658+0000","last_clean":"2024-05-21T03:21:14.559658+0000","last_became_active":"2024-05-21T03:21:14.559194+0000","last_became_peered":"2024-05-21T03:21:14.559194+0000","last_unstale":"2024-05-21T03:21:14.559658+0000","last_undegraded":"2024-05-21T03:21:14.559658+0000","last_fullsized":"2024-05-21T03:21:14.559658+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:56:31.709545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556799+0000","last_change":"2024-05-21T03:20:32.137532+0000","last_active":"2024-05-21T03:21:14.556799+0000","last_peered":"2024-05-21T03:21:14.556799+0000","last_clean":"2024-05-21T03:21:14.556799+0000","last_became_active":"2024-05-21T03:20:32.137182+0000","last_became_peered":"2024-05-21T03:20:32.137182+0000","last_unstale":"2024-05-21T03:21:14.556799+0000","last_undegraded":"2024-05-21T03:21:14.556799+0000","last_fullsized":"2024-05-21T03:21:14.556799+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:25:44.207866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556286+0000","last_change":"2024-05-21T03:20:32.153912+0000","last_active":"2024-05-21T03:21:14.556286+0000","last_peered":"2024-05-21T03:21:14.556286+0000","last_clean":"2024-05-21T03:21:14.556286+0000","last_became_active":"2024-05-21T03:20:32.153541+0000","last_became_peered":"2024-05-21T03:20:32.153541+0000","last_unstale":"2024-05-21T03:21:14.556286+0000","last_undegraded":"2024-05-21T03:21:14.556286+0000","last_fullsized":"2024-05-21T03:21:14.556286+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:02:58.688378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556461+0000","last_change":"2024-05-21T03:20:32.143689+0000","last_active":"2024-05-21T03:21:14.556461+0000","last_peered":"2024-05-21T03:21:14.556461+0000","last_clean":"2024-05-21T03:21:14.556461+0000","last_became_active":"2024-05-21T03:20:32.143399+0000","last_became_peered":"2024-05-21T03:20:32.143399+0000","last_unstale":"2024-05-21T03:21:14.556461+0000","last_undegraded":"2024-05-21T03:21:14.556461+0000","last_fullsized":"2024-05-21T03:21:14.556461+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:27:55.116599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552907+0000","last_change":"2024-05-21T03:20:32.141355+0000","last_active":"2024-05-21T03:21:14.552907+0000","last_peered":"2024-05-21T03:21:14.552907+0000","last_clean":"2024-05-21T03:21:14.552907+0000","last_became_active":"2024-05-21T03:20:32.140960+0000","last_became_peered":"2024-05-21T03:20:32.140960+0000","last_unstale":"2024-05-21T03:21:14.552907+0000","last_undegraded":"2024-05-21T03:21:14.552907+0000","last_fullsized":"2024-05-21T03:21:14.552907+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:23:50.287864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.553971+0000","last_change":"2024-05-21T03:20:32.142145+0000","last_active":"2024-05-21T03:21:14.553971+0000","last_peered":"2024-05-21T03:21:14.553971+0000","last_clean":"2024-05-21T03:21:14.553971+0000","last_became_active":"2024-05-21T03:20:32.141802+0000","last_became_peered":"2024-05-21T03:20:32.141802+0000","last_unstale":"2024-05-21T03:21:14.553971+0000","last_undegraded":"2024-05-21T03:21:14.553971+0000","last_fullsized":"2024-05-21T03:21:14.553971+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:26:28.707616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556601+0000","last_change":"2024-05-21T03:20:32.141606+0000","last_active":"2024-05-21T03:21:14.556601+0000","last_peered":"2024-05-21T03:21:14.556601+0000","last_clean":"2024-05-21T03:21:14.556601+0000","last_became_active":"2024-05-21T03:20:32.141258+0000","last_became_peered":"2024-05-21T03:20:32.141258+0000","last_unstale":"2024-05-21T03:21:14.556601+0000","last_undegraded":"2024-05-21T03:21:14.556601+0000","last_fullsized":"2024-05-21T03:21:14.556601+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:22.372723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561186+0000","last_change":"2024-05-21T03:20:32.151621+0000","last_active":"2024-05-21T03:21:14.561186+0000","last_peered":"2024-05-21T03:21:14.561186+0000","last_clean":"2024-05-21T03:21:14.561186+0000","last_became_active":"2024-05-21T03:20:32.151260+0000","last_became_peered":"2024-05-21T03:20:32.151260+0000","last_unstale":"2024-05-21T03:21:14.561186+0000","last_undegraded":"2024-05-21T03:21:14.561186+0000","last_fullsized":"2024-05-21T03:21:14.561186+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:48:59.751179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561107+0000","last_change":"2024-05-21T03:20:32.153843+0000","last_active":"2024-05-21T03:21:14.561107+0000","last_peered":"2024-05-21T03:21:14.561107+0000","last_clean":"2024-05-21T03:21:14.561107+0000","last_became_active":"2024-05-21T03:20:32.153418+0000","last_became_peered":"2024-05-21T03:20:32.153418+0000","last_unstale":"2024-05-21T03:21:14.561107+0000","last_undegraded":"2024-05-21T03:21:14.561107+0000","last_fullsized":"2024-05-21T03:21:14.561107+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:26:29.663966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.560653+0000","last_change":"2024-05-21T03:20:32.153942+0000","last_active":"2024-05-21T03:21:14.560653+0000","last_peered":"2024-05-21T03:21:14.560653+0000","last_clean":"2024-05-21T03:21:14.560653+0000","last_became_active":"2024-05-21T03:20:32.153566+0000","last_became_peered":"2024-05-21T03:20:32.153566+0000","last_unstale":"2024-05-21T03:21:14.560653+0000","last_undegraded":"2024-05-21T03:21:14.560653+0000","last_fullsized":"2024-05-21T03:21:14.560653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:06.899039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558337+0000","last_change":"2024-05-21T03:20:32.153243+0000","last_active":"2024-05-21T03:21:14.558337+0000","last_peered":"2024-05-21T03:21:14.558337+0000","last_clean":"2024-05-21T03:21:14.558337+0000","last_became_active":"2024-05-21T03:20:32.152386+0000","last_became_peered":"2024-05-21T03:20:32.152386+0000","last_unstale":"2024-05-21T03:21:14.558337+0000","last_undegraded":"2024-05-21T03:21:14.558337+0000","last_fullsized":"2024-05-21T03:21:14.558337+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:25:28.935183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552892+0000","last_change":"2024-05-21T03:20:32.142536+0000","last_active":"2024-05-21T03:21:14.552892+0000","last_peered":"2024-05-21T03:21:14.552892+0000","last_clean":"2024-05-21T03:21:14.552892+0000","last_became_active":"2024-05-21T03:20:32.141793+0000","last_became_peered":"2024-05-21T03:20:32.141793+0000","last_unstale":"2024-05-21T03:21:14.552892+0000","last_undegraded":"2024-05-21T03:21:14.552892+0000","last_fullsized":"2024-05-21T03:21:14.552892+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:21:43.547103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.553745+0000","last_change":"2024-05-21T03:20:32.140368+0000","last_active":"2024-05-21T03:21:14.553745+0000","last_peered":"2024-05-21T03:21:14.553745+0000","last_clean":"2024-05-21T03:21:14.553745+0000","last_became_active":"2024-05-21T03:20:32.139747+0000","last_became_peered":"2024-05-21T03:20:32.139747+0000","last_unstale":"2024-05-21T03:21:14.553745+0000","last_undegraded":"2024-05-21T03:21:14.553745+0000","last_fullsized":"2024-05-21T03:21:14.553745+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:05:44.851191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67357,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":295,"num_read_kb":232,"num_write":244,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1561,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":202071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645336,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":901120,"data_stored":433850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841565,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1601536,"data_stored":1084641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005091,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":433506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201318,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":433767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397549,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32668,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712484,"statfs":{"total":95995035648,"available":95961583616,"internally_reserved":0,"allocated":1519616,"data_stored":1026676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561073,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":433497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":494172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":434836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":487436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149760,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":432814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953532,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":434919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757303,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32680,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712472,"statfs":{"total":95995035648,"available":95961571328,"internally_reserved":0,"allocated":1531904,"data_stored":1025572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T03:21:21.814 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph status --format=json 2024-05-21T03:21:22.099 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:21:22.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:22 smithi027 ceph-mon[32179]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:21:22.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:22 smithi027 ceph-mon[32179]: pgmap v316: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.6 KiB/s wr, 7 op/s; 209 B/s, 1 keys/s, 1 objects/s recovering 2024-05-21T03:21:23.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:22 smithi039 ceph-mon[36429]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:21:23.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:22 smithi039 ceph-mon[36429]: pgmap v316: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.6 KiB/s wr, 7 op/s; 209 B/s, 1 keys/s, 1 objects/s recovering 2024-05-21T03:21:23.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:22 smithi165 ceph-mon[36269]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:21:23.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:22 smithi165 ceph-mon[36269]: pgmap v316: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.6 KiB/s wr, 7 op/s; 209 B/s, 1 keys/s, 1 objects/s recovering 2024-05-21T03:21:23.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.706+0000 7fea9fdc1700 1 -- 172.21.15.27:0/133440663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea981095d0 msgr2=0x7fea981099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:23.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.706+0000 7fea9fdc1700 1 --2- 172.21.15.27:0/133440663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea981095d0 0x7fea981099b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fea94009990 tx=0x7fea94009ca0 comp rx=0 tx=0).stop 2024-05-21T03:21:23.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.706+0000 7fea9fdc1700 1 -- 172.21.15.27:0/133440663 shutdown_connections 2024-05-21T03:21:23.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.706+0000 7fea9fdc1700 1 --2- 172.21.15.27:0/133440663 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea98103f80 0x7fea98110410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:23.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.706+0000 7fea9fdc1700 1 --2- 172.21.15.27:0/133440663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea981095d0 0x7fea981099b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:23.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.706+0000 7fea9fdc1700 1 --2- 172.21.15.27:0/133440663 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea981035c0 0x7fea98103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:23.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.706+0000 7fea9fdc1700 1 -- 172.21.15.27:0/133440663 >> 172.21.15.27:0/133440663 conn(0x7fea9807a610 msgr2=0x7fea9807aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:23.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.706+0000 7fea9fdc1700 1 -- 172.21.15.27:0/133440663 shutdown_connections 2024-05-21T03:21:23.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.707+0000 7fea9fdc1700 1 -- 172.21.15.27:0/133440663 wait complete. 2024-05-21T03:21:23.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.707+0000 7fea9fdc1700 1 Processor -- start 2024-05-21T03:21:23.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.707+0000 7fea9fdc1700 1 -- start start 2024-05-21T03:21:23.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.707+0000 7fea9fdc1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea981035c0 0x7fea98075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:23.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.708+0000 7fea9fdc1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea98103f80 0x7fea98075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:23.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.708+0000 7fea9fdc1700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea981095d0 0x7fea980762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:23.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.708+0000 7fea9fdc1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fea98113720 con 0x7fea98103f80 2024-05-21T03:21:23.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.708+0000 7fea9fdc1700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fea981135a0 con 0x7fea981035c0 2024-05-21T03:21:23.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.708+0000 7fea9fdc1700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fea981138a0 con 0x7fea981095d0 2024-05-21T03:21:23.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.708+0000 7fea9d35c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea98103f80 0x7fea98075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:23.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.708+0000 7fea9d35c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea98103f80 0x7fea98075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39554/0 (socket says 172.21.15.27:39554) 2024-05-21T03:21:23.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.708+0000 7fea9d35c700 1 -- 172.21.15.27:0/3977549946 learned_addr learned my addr 172.21.15.27:0/3977549946 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:21:23.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.708+0000 7fea9db5d700 1 --2- 172.21.15.27:0/3977549946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea981035c0 0x7fea98075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:23.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.708+0000 7fea9e35e700 1 --2- 172.21.15.27:0/3977549946 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea981095d0 0x7fea980762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:23.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.708+0000 7fea9d35c700 1 -- 172.21.15.27:0/3977549946 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea981095d0 msgr2=0x7fea980762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:23.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.708+0000 7fea9d35c700 1 --2- 172.21.15.27:0/3977549946 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea981095d0 0x7fea980762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:23.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.708+0000 7fea9d35c700 1 -- 172.21.15.27:0/3977549946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea981035c0 msgr2=0x7fea98075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:23.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.708+0000 7fea9d35c700 1 --2- 172.21.15.27:0/3977549946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea981035c0 0x7fea98075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:23.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.708+0000 7fea9d35c700 1 -- 172.21.15.27:0/3977549946 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea981a80a0 con 0x7fea98103f80 2024-05-21T03:21:23.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.708+0000 7fea9db5d700 1 --2- 172.21.15.27:0/3977549946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea981035c0 0x7fea98075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:21:23.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.708+0000 7fea9d35c700 1 --2- 172.21.15.27:0/3977549946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea98103f80 0x7fea98075d80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fea8800d980 tx=0x7fea8800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:23.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.708+0000 7fea8effd700 1 -- 172.21.15.27:0/3977549946 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea88016070 con 0x7fea98103f80 2024-05-21T03:21:23.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.709+0000 7fea9fdc1700 1 -- 172.21.15.27:0/3977549946 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea981a8390 con 0x7fea98103f80 2024-05-21T03:21:23.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.709+0000 7fea8effd700 1 -- 172.21.15.27:0/3977549946 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea88004830 con 0x7fea98103f80 2024-05-21T03:21:23.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.709+0000 7fea8effd700 1 -- 172.21.15.27:0/3977549946 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea88005540 con 0x7fea98103f80 2024-05-21T03:21:23.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.709+0000 7fea9fdc1700 1 -- 172.21.15.27:0/3977549946 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea981a8920 con 0x7fea98103f80 2024-05-21T03:21:23.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.710+0000 7fea9fdc1700 1 -- 172.21.15.27:0/3977549946 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea98104cb0 con 0x7fea98103f80 2024-05-21T03:21:23.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.711+0000 7fea8effd700 1 -- 172.21.15.27:0/3977549946 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fea88004220 con 0x7fea98103f80 2024-05-21T03:21:23.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.712+0000 7fea8effd700 1 --2- 172.21.15.27:0/3977549946 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fea840b27e0 0x7fea840b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:23.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.712+0000 7fea9db5d700 1 --2- 172.21.15.27:0/3977549946 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fea840b27e0 0x7fea840b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:23.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.712+0000 7fea8effd700 1 -- 172.21.15.27:0/3977549946 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fea880e1ef0 con 0x7fea98103f80 2024-05-21T03:21:23.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.712+0000 7fea9db5d700 1 --2- 172.21.15.27:0/3977549946 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fea840b27e0 0x7fea840b4ca0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fea94006010 tx=0x7fea9400b560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:23.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:23.716+0000 7fea8effd700 1 -- 172.21.15.27:0/3977549946 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fea8801f030 con 0x7fea98103f80 2024-05-21T03:21:24.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:24.020+0000 7fea9fdc1700 1 -- 172.21.15.27:0/3977549946 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fea98076d70 con 0x7fea98103f80 2024-05-21T03:21:24.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:24.020+0000 7fea8effd700 1 -- 172.21.15.27:0/3977549946 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+3293 (secure 0 0 0) 0x7fea880a9a90 con 0x7fea98103f80 2024-05-21T03:21:24.020 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:21:24.020 INFO:teuthology.orchestra.run.smithi027.stdout:{"fsid":"c9ca94be-171f-11ef-bc9a-c7b262605968","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":432,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1716261570,"num_in_osds":12,"osd_in_since":1716261549,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":102,"data_bytes":657959,"bytes_used":366387200,"bytes_avail":1151574040576,"bytes_total":1151940427776,"recovering_objects_per_sec":1,"recovering_bytes_per_sec":171,"recovering_keys_per_sec":1,"num_objects_recovered":11,"num_bytes_recovered":1678,"num_keys_recovered":10,"read_bytes_sec":5953,"write_bytes_sec":1357,"read_op_per_sec":7,"write_op_per_sec":0},"fsmap":{"epoch":28,"id":1,"up":5,"in":5,"max":5,"by_rank":[{"filesystem_id":1,"rank":4,"name":"g","status":"up:standby-replay","gid":14613},{"filesystem_id":1,"rank":3,"name":"a","status":"up:standby-replay","gid":14625},{"filesystem_id":1,"rank":2,"name":"k","status":"up:standby-replay","gid":24401},{"filesystem_id":1,"rank":1,"name":"l","status":"up:standby-replay","gid":24418},{"filesystem_id":1,"rank":0,"name":"e","status":"up:standby-replay","gid":24422},{"filesystem_id":1,"rank":4,"name":"f","status":"up:active","gid":24451},{"filesystem_id":1,"rank":3,"name":"h","status":"up:active","gid":24452},{"filesystem_id":1,"rank":2,"name":"c","status":"up:active","gid":24457},{"filesystem_id":1,"rank":1,"name":"i","status":"up:active","gid":24472},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24476}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.27:8443/"}},"servicemap":{"epoch":22,"modified":"2024-05-21T03:20:57.339863+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"g":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"k":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"l":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-21T03:21:24.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:24.022+0000 7fea9fdc1700 1 -- 172.21.15.27:0/3977549946 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fea840b27e0 msgr2=0x7fea840b4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:24.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:24.022+0000 7fea9fdc1700 1 --2- 172.21.15.27:0/3977549946 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fea840b27e0 0x7fea840b4ca0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fea94006010 tx=0x7fea9400b560 comp rx=0 tx=0).stop 2024-05-21T03:21:24.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:24.022+0000 7fea9fdc1700 1 -- 172.21.15.27:0/3977549946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea98103f80 msgr2=0x7fea98075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:24.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:24.022+0000 7fea9fdc1700 1 --2- 172.21.15.27:0/3977549946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea98103f80 0x7fea98075d80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fea8800d980 tx=0x7fea8800dc90 comp rx=0 tx=0).stop 2024-05-21T03:21:24.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:24.023+0000 7fea9fdc1700 1 -- 172.21.15.27:0/3977549946 shutdown_connections 2024-05-21T03:21:24.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:24.023+0000 7fea9fdc1700 1 --2- 172.21.15.27:0/3977549946 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fea840b27e0 0x7fea840b4ca0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:24.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:24.023+0000 7fea9fdc1700 1 --2- 172.21.15.27:0/3977549946 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea981095d0 0x7fea980762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:24.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:24.023+0000 7fea9fdc1700 1 --2- 172.21.15.27:0/3977549946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea981035c0 0x7fea98075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:24.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:24.023+0000 7fea9fdc1700 1 --2- 172.21.15.27:0/3977549946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea98103f80 0x7fea98075d80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:24.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:24.023+0000 7fea9fdc1700 1 -- 172.21.15.27:0/3977549946 >> 172.21.15.27:0/3977549946 conn(0x7fea9807a610 msgr2=0x7fea981021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:24.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:24.023+0000 7fea9fdc1700 1 -- 172.21.15.27:0/3977549946 shutdown_connections 2024-05-21T03:21:24.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:24.023+0000 7fea9fdc1700 1 -- 172.21.15.27:0/3977549946 wait complete. 2024-05-21T03:21:24.623 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-05-21T03:21:24.624 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-21T03:21:24.873 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:21:24.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:24 smithi027 ceph-mon[32179]: pgmap v317: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.3 KiB/s wr, 8 op/s; 171 B/s, 1 keys/s, 1 objects/s recovering 2024-05-21T03:21:24.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:24 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3977549946' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-05-21T03:21:25.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:24 smithi039 ceph-mon[36429]: pgmap v317: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.3 KiB/s wr, 8 op/s; 171 B/s, 1 keys/s, 1 objects/s recovering 2024-05-21T03:21:25.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:24 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/3977549946' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-05-21T03:21:25.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:24 smithi165 ceph-mon[36269]: pgmap v317: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.3 KiB/s wr, 8 op/s; 171 B/s, 1 keys/s, 1 objects/s recovering 2024-05-21T03:21:25.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:24 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/3977549946' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-05-21T03:21:26.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.552+0000 7fb5a9716700 1 -- 172.21.15.27:0/76678995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5a41039b0 msgr2=0x7fb5a410f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:26.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.552+0000 7fb5a9716700 1 --2- 172.21.15.27:0/76678995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5a41039b0 0x7fb5a410f3c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb59c00b0d0 tx=0x7fb59c00b3e0 comp rx=0 tx=0).stop 2024-05-21T03:21:26.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.553+0000 7fb5a9716700 1 -- 172.21.15.27:0/76678995 shutdown_connections 2024-05-21T03:21:26.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.553+0000 7fb5a9716700 1 --2- 172.21.15.27:0/76678995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5a41039b0 0x7fb5a410f3c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:26.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.553+0000 7fb5a9716700 1 --2- 172.21.15.27:0/76678995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5a4108710 0x7fb5a4108af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:26.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.553+0000 7fb5a9716700 1 --2- 172.21.15.27:0/76678995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5a4102ff0 0x7fb5a4103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:26.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.553+0000 7fb5a9716700 1 -- 172.21.15.27:0/76678995 >> 172.21.15.27:0/76678995 conn(0x7fb5a4077760 msgr2=0x7fb5a4077b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:26.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.553+0000 7fb5a9716700 1 -- 172.21.15.27:0/76678995 shutdown_connections 2024-05-21T03:21:26.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.553+0000 7fb5a9716700 1 -- 172.21.15.27:0/76678995 wait complete. 2024-05-21T03:21:26.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.553+0000 7fb5a9716700 1 Processor -- start 2024-05-21T03:21:26.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.553+0000 7fb5a9716700 1 -- start start 2024-05-21T03:21:26.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.554+0000 7fb5a9716700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5a4102ff0 0x7fb5a419f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:26.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.554+0000 7fb5a9716700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5a41039b0 0x7fb5a419fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:26.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.554+0000 7fb5a9716700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5a4108710 0x7fb5a4199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:26.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.554+0000 7fb5a9716700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a41127e0 con 0x7fb5a4102ff0 2024-05-21T03:21:26.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.554+0000 7fb5a9716700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a4112660 con 0x7fb5a4108710 2024-05-21T03:21:26.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.554+0000 7fb5a9716700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a4112960 con 0x7fb5a41039b0 2024-05-21T03:21:26.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.554+0000 7fb5a2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5a4102ff0 0x7fb5a419f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:26.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.554+0000 7fb5a2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5a4102ff0 0x7fb5a419f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39578/0 (socket says 172.21.15.27:39578) 2024-05-21T03:21:26.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.554+0000 7fb5a2ffd700 1 -- 172.21.15.27:0/2076532987 learned_addr learned my addr 172.21.15.27:0/2076532987 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:21:26.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.554+0000 7fb5a27fc700 1 --2- 172.21.15.27:0/2076532987 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5a41039b0 0x7fb5a419fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:26.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.554+0000 7fb5a37fe700 1 --2- 172.21.15.27:0/2076532987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5a4108710 0x7fb5a4199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:26.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.554+0000 7fb5a2ffd700 1 -- 172.21.15.27:0/2076532987 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5a41039b0 msgr2=0x7fb5a419fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:26.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.554+0000 7fb5a2ffd700 1 --2- 172.21.15.27:0/2076532987 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5a41039b0 0x7fb5a419fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:26.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.554+0000 7fb5a2ffd700 1 -- 172.21.15.27:0/2076532987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5a4108710 msgr2=0x7fb5a4199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:26.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.554+0000 7fb5a2ffd700 1 --2- 172.21.15.27:0/2076532987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5a4108710 0x7fb5a4199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:26.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.554+0000 7fb5a2ffd700 1 -- 172.21.15.27:0/2076532987 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a419a220 con 0x7fb5a4102ff0 2024-05-21T03:21:26.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.554+0000 7fb5a37fe700 1 --2- 172.21.15.27:0/2076532987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5a4108710 0x7fb5a4199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:21:26.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.554+0000 7fb5a2ffd700 1 --2- 172.21.15.27:0/2076532987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5a4102ff0 0x7fb5a419f900 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb59400d950 tx=0x7fb59400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:26.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.555+0000 7fb58bfff700 1 -- 172.21.15.27:0/2076532987 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb594016070 con 0x7fb5a4102ff0 2024-05-21T03:21:26.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.555+0000 7fb58bfff700 1 -- 172.21.15.27:0/2076532987 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb594009df0 con 0x7fb5a4102ff0 2024-05-21T03:21:26.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.555+0000 7fb5a9716700 1 -- 172.21.15.27:0/2076532987 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5a419a510 con 0x7fb5a4102ff0 2024-05-21T03:21:26.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.555+0000 7fb58bfff700 1 -- 172.21.15.27:0/2076532987 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59400f3c0 con 0x7fb5a4102ff0 2024-05-21T03:21:26.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.555+0000 7fb5a9716700 1 -- 172.21.15.27:0/2076532987 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5a41a67d0 con 0x7fb5a4102ff0 2024-05-21T03:21:26.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.556+0000 7fb5a9716700 1 -- 172.21.15.27:0/2076532987 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5a406a230 con 0x7fb5a4102ff0 2024-05-21T03:21:26.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.557+0000 7fb58bfff700 1 -- 172.21.15.27:0/2076532987 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb594019070 con 0x7fb5a4102ff0 2024-05-21T03:21:26.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.558+0000 7fb58bfff700 1 --2- 172.21.15.27:0/2076532987 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb58c0b2990 0x7fb58c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:26.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.558+0000 7fb5a27fc700 1 --2- 172.21.15.27:0/2076532987 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb58c0b2990 0x7fb58c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:26.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.558+0000 7fb58bfff700 1 -- 172.21.15.27:0/2076532987 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fb5940e1f30 con 0x7fb5a4102ff0 2024-05-21T03:21:26.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.558+0000 7fb5a27fc700 1 --2- 172.21.15.27:0/2076532987 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb58c0b2990 0x7fb58c0b4e50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4077d40 tx=0x7fb59800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:26.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.561+0000 7fb58bfff700 1 -- 172.21.15.27:0/2076532987 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb5940a9ad0 con 0x7fb5a4102ff0 2024-05-21T03:21:26.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:26 smithi027 ceph-mon[32179]: pgmap v318: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 758 B/s wr, 7 op/s; 155 B/s, 0 keys/s, 1 objects/s recovering 2024-05-21T03:21:26.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.774+0000 7fb5a9716700 1 -- 172.21.15.27:0/2076532987 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb5a41026b0 con 0x7fb58c0b2990 2024-05-21T03:21:26.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.781+0000 7fb58bfff700 1 -- 172.21.15.27:0/2076532987 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258767 (secure 0 0 0) 0x7fb5a41026b0 con 0x7fb58c0b2990 2024-05-21T03:21:26.784 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:21:26.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.785+0000 7fb5a9716700 1 -- 172.21.15.27:0/2076532987 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb58c0b2990 msgr2=0x7fb58c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:26.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.785+0000 7fb5a9716700 1 --2- 172.21.15.27:0/2076532987 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb58c0b2990 0x7fb58c0b4e50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4077d40 tx=0x7fb59800a300 comp rx=0 tx=0).stop 2024-05-21T03:21:26.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.785+0000 7fb5a9716700 1 -- 172.21.15.27:0/2076532987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5a4102ff0 msgr2=0x7fb5a419f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:26.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.785+0000 7fb5a9716700 1 --2- 172.21.15.27:0/2076532987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5a4102ff0 0x7fb5a419f900 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb59400d950 tx=0x7fb59400dc60 comp rx=0 tx=0).stop 2024-05-21T03:21:26.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.785+0000 7fb5a9716700 1 -- 172.21.15.27:0/2076532987 shutdown_connections 2024-05-21T03:21:26.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.785+0000 7fb5a9716700 1 --2- 172.21.15.27:0/2076532987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5a4108710 0x7fb5a4199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:26.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.785+0000 7fb5a9716700 1 --2- 172.21.15.27:0/2076532987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5a4102ff0 0x7fb5a419f900 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:26.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.785+0000 7fb5a9716700 1 --2- 172.21.15.27:0/2076532987 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb58c0b2990 0x7fb58c0b4e50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:26.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.785+0000 7fb5a9716700 1 --2- 172.21.15.27:0/2076532987 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5a41039b0 0x7fb5a419fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:26.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.785+0000 7fb5a9716700 1 -- 172.21.15.27:0/2076532987 >> 172.21.15.27:0/2076532987 conn(0x7fb5a4077760 msgr2=0x7fb5a4101fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:26.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.785+0000 7fb5a9716700 1 -- 172.21.15.27:0/2076532987 shutdown_connections 2024-05-21T03:21:26.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:26.785+0000 7fb5a9716700 1 -- 172.21.15.27:0/2076532987 wait complete. 2024-05-21T03:21:26.787 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-05-21T03:21:27.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:26 smithi039 ceph-mon[36429]: pgmap v318: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 758 B/s wr, 7 op/s; 155 B/s, 0 keys/s, 1 objects/s recovering 2024-05-21T03:21:27.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:26 smithi165 ceph-mon[36269]: pgmap v318: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 758 B/s wr, 7 op/s; 155 B/s, 0 keys/s, 1 objects/s recovering 2024-05-21T03:21:27.564 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-05-21T03:21:25.346371+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657959,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":426,"num_read_kb":326,"num_write":369,"num_write_kb":2287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2953518,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357800,"kb_used_data":12732,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584024,"statfs":{"total":1151940427776,"available":1151574040576,"internally_reserved":0,"allocated":13037568,"data_stored":7161081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19139,"internal_metadata":353154365},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5395,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":52,"num_write":5,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1678,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.794770"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.554128+0000","last_change":"2024-05-21T03:20:32.149961+0000","last_active":"2024-05-21T03:21:14.554128+0000","last_peered":"2024-05-21T03:21:14.554128+0000","last_clean":"2024-05-21T03:21:14.554128+0000","last_became_active":"2024-05-21T03:20:32.149552+0000","last_became_peered":"2024-05-21T03:20:32.149552+0000","last_unstale":"2024-05-21T03:21:14.554128+0000","last_undegraded":"2024-05-21T03:21:14.554128+0000","last_fullsized":"2024-05-21T03:21:14.554128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:20:38.543551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556541+0000","last_change":"2024-05-21T03:20:32.146413+0000","last_active":"2024-05-21T03:21:14.556541+0000","last_peered":"2024-05-21T03:21:14.556541+0000","last_clean":"2024-05-21T03:21:14.556541+0000","last_became_active":"2024-05-21T03:20:32.146006+0000","last_became_peered":"2024-05-21T03:20:32.146006+0000","last_unstale":"2024-05-21T03:21:14.556541+0000","last_undegraded":"2024-05-21T03:21:14.556541+0000","last_fullsized":"2024-05-21T03:21:14.556541+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:05:12.888550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556573+0000","last_change":"2024-05-21T03:20:32.153654+0000","last_active":"2024-05-21T03:21:14.556573+0000","last_peered":"2024-05-21T03:21:14.556573+0000","last_clean":"2024-05-21T03:21:14.556573+0000","last_became_active":"2024-05-21T03:20:32.153296+0000","last_became_peered":"2024-05-21T03:20:32.153296+0000","last_unstale":"2024-05-21T03:21:14.556573+0000","last_undegraded":"2024-05-21T03:21:14.556573+0000","last_fullsized":"2024-05-21T03:21:14.556573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:16:09.423236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556628+0000","last_change":"2024-05-21T03:20:32.138672+0000","last_active":"2024-05-21T03:21:14.556628+0000","last_peered":"2024-05-21T03:21:14.556628+0000","last_clean":"2024-05-21T03:21:14.556628+0000","last_became_active":"2024-05-21T03:20:32.138163+0000","last_became_peered":"2024-05-21T03:20:32.138163+0000","last_unstale":"2024-05-21T03:21:14.556628+0000","last_undegraded":"2024-05-21T03:21:14.556628+0000","last_fullsized":"2024-05-21T03:21:14.556628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:26:55.012897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556450+0000","last_change":"2024-05-21T03:20:32.153950+0000","last_active":"2024-05-21T03:21:14.556450+0000","last_peered":"2024-05-21T03:21:14.556450+0000","last_clean":"2024-05-21T03:21:14.556450+0000","last_became_active":"2024-05-21T03:20:32.153567+0000","last_became_peered":"2024-05-21T03:20:32.153567+0000","last_unstale":"2024-05-21T03:21:14.556450+0000","last_undegraded":"2024-05-21T03:21:14.556450+0000","last_fullsized":"2024-05-21T03:21:14.556450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:28:34.621193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.554034+0000","last_change":"2024-05-21T03:20:32.140038+0000","last_active":"2024-05-21T03:21:14.554034+0000","last_peered":"2024-05-21T03:21:14.554034+0000","last_clean":"2024-05-21T03:21:14.554034+0000","last_became_active":"2024-05-21T03:20:32.139690+0000","last_became_peered":"2024-05-21T03:20:32.139690+0000","last_unstale":"2024-05-21T03:21:14.554034+0000","last_undegraded":"2024-05-21T03:21:14.554034+0000","last_fullsized":"2024-05-21T03:21:14.554034+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:31:24.270755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556482+0000","last_change":"2024-05-21T03:20:32.143735+0000","last_active":"2024-05-21T03:21:14.556482+0000","last_peered":"2024-05-21T03:21:14.556482+0000","last_clean":"2024-05-21T03:21:14.556482+0000","last_became_active":"2024-05-21T03:20:32.143397+0000","last_became_peered":"2024-05-21T03:20:32.143397+0000","last_unstale":"2024-05-21T03:21:14.556482+0000","last_undegraded":"2024-05-21T03:21:14.556482+0000","last_fullsized":"2024-05-21T03:21:14.556482+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:40:01.951608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.560831+0000","last_change":"2024-05-21T03:20:32.156999+0000","last_active":"2024-05-21T03:21:14.560831+0000","last_peered":"2024-05-21T03:21:14.560831+0000","last_clean":"2024-05-21T03:21:14.560831+0000","last_became_active":"2024-05-21T03:20:32.156677+0000","last_became_peered":"2024-05-21T03:20:32.156677+0000","last_unstale":"2024-05-21T03:21:14.560831+0000","last_undegraded":"2024-05-21T03:21:14.560831+0000","last_fullsized":"2024-05-21T03:21:14.560831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:33:24.225667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558931+0000","last_change":"2024-05-21T03:20:32.145039+0000","last_active":"2024-05-21T03:21:14.558931+0000","last_peered":"2024-05-21T03:21:14.558931+0000","last_clean":"2024-05-21T03:21:14.558931+0000","last_became_active":"2024-05-21T03:20:32.140926+0000","last_became_peered":"2024-05-21T03:20:32.140926+0000","last_unstale":"2024-05-21T03:21:14.558931+0000","last_undegraded":"2024-05-21T03:21:14.558931+0000","last_fullsized":"2024-05-21T03:21:14.558931+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:11:27.049426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.554577+0000","last_change":"2024-05-21T03:20:32.145840+0000","last_active":"2024-05-21T03:21:14.554577+0000","last_peered":"2024-05-21T03:21:14.554577+0000","last_clean":"2024-05-21T03:21:14.554577+0000","last_became_active":"2024-05-21T03:20:32.145291+0000","last_became_peered":"2024-05-21T03:20:32.145291+0000","last_unstale":"2024-05-21T03:21:14.554577+0000","last_undegraded":"2024-05-21T03:21:14.554577+0000","last_fullsized":"2024-05-21T03:21:14.554577+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:22:27.696588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558336+0000","last_change":"2024-05-21T03:20:32.142522+0000","last_active":"2024-05-21T03:21:14.558336+0000","last_peered":"2024-05-21T03:21:14.558336+0000","last_clean":"2024-05-21T03:21:14.558336+0000","last_became_active":"2024-05-21T03:20:32.142165+0000","last_became_peered":"2024-05-21T03:20:32.142165+0000","last_unstale":"2024-05-21T03:21:14.558336+0000","last_undegraded":"2024-05-21T03:21:14.558336+0000","last_fullsized":"2024-05-21T03:21:14.558336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:58:39.085834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556536+0000","last_change":"2024-05-21T03:20:32.142526+0000","last_active":"2024-05-21T03:21:14.556536+0000","last_peered":"2024-05-21T03:21:14.556536+0000","last_clean":"2024-05-21T03:21:14.556536+0000","last_became_active":"2024-05-21T03:20:32.141143+0000","last_became_peered":"2024-05-21T03:20:32.141143+0000","last_unstale":"2024-05-21T03:21:14.556536+0000","last_undegraded":"2024-05-21T03:21:14.556536+0000","last_fullsized":"2024-05-21T03:21:14.556536+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:35:11.336796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552998+0000","last_change":"2024-05-21T03:20:32.135631+0000","last_active":"2024-05-21T03:21:14.552998+0000","last_peered":"2024-05-21T03:21:14.552998+0000","last_clean":"2024-05-21T03:21:14.552998+0000","last_became_active":"2024-05-21T03:20:32.135302+0000","last_became_peered":"2024-05-21T03:20:32.135302+0000","last_unstale":"2024-05-21T03:21:14.552998+0000","last_undegraded":"2024-05-21T03:21:14.552998+0000","last_fullsized":"2024-05-21T03:21:14.552998+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:05:55.000690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.555811+0000","last_change":"2024-05-21T03:20:32.146580+0000","last_active":"2024-05-21T03:21:14.555811+0000","last_peered":"2024-05-21T03:21:14.555811+0000","last_clean":"2024-05-21T03:21:14.555811+0000","last_became_active":"2024-05-21T03:20:32.145803+0000","last_became_peered":"2024-05-21T03:20:32.145803+0000","last_unstale":"2024-05-21T03:21:14.555811+0000","last_undegraded":"2024-05-21T03:21:14.555811+0000","last_fullsized":"2024-05-21T03:21:14.555811+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:34:09.108799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.559498+0000","last_change":"2024-05-21T03:21:14.559498+0000","last_active":"2024-05-21T03:21:14.559498+0000","last_peered":"2024-05-21T03:21:14.559498+0000","last_clean":"2024-05-21T03:21:14.559498+0000","last_became_active":"2024-05-21T03:21:14.559039+0000","last_became_peered":"2024-05-21T03:21:14.559039+0000","last_unstale":"2024-05-21T03:21:14.559498+0000","last_undegraded":"2024-05-21T03:21:14.559498+0000","last_fullsized":"2024-05-21T03:21:14.559498+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:04:36.676857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.557368+0000","last_change":"2024-05-21T03:20:32.148944+0000","last_active":"2024-05-21T03:21:14.557368+0000","last_peered":"2024-05-21T03:21:14.557368+0000","last_clean":"2024-05-21T03:21:14.557368+0000","last_became_active":"2024-05-21T03:20:32.148502+0000","last_became_peered":"2024-05-21T03:20:32.148502+0000","last_unstale":"2024-05-21T03:21:14.557368+0000","last_undegraded":"2024-05-21T03:21:14.557368+0000","last_fullsized":"2024-05-21T03:21:14.557368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:11:00.178105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.554478+0000","last_change":"2024-05-21T03:20:32.150609+0000","last_active":"2024-05-21T03:21:14.554478+0000","last_peered":"2024-05-21T03:21:14.554478+0000","last_clean":"2024-05-21T03:21:14.554478+0000","last_became_active":"2024-05-21T03:20:32.150314+0000","last_became_peered":"2024-05-21T03:20:32.150314+0000","last_unstale":"2024-05-21T03:21:14.554478+0000","last_undegraded":"2024-05-21T03:21:14.554478+0000","last_fullsized":"2024-05-21T03:21:14.554478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:34:21.869304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556008+0000","last_change":"2024-05-21T03:20:32.142472+0000","last_active":"2024-05-21T03:21:14.556008+0000","last_peered":"2024-05-21T03:21:14.556008+0000","last_clean":"2024-05-21T03:21:14.556008+0000","last_became_active":"2024-05-21T03:20:32.142161+0000","last_became_peered":"2024-05-21T03:20:32.142161+0000","last_unstale":"2024-05-21T03:21:14.556008+0000","last_undegraded":"2024-05-21T03:21:14.556008+0000","last_fullsized":"2024-05-21T03:21:14.556008+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:26:57.281655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.557572+0000","last_change":"2024-05-21T03:20:32.153987+0000","last_active":"2024-05-21T03:21:14.557572+0000","last_peered":"2024-05-21T03:21:14.557572+0000","last_clean":"2024-05-21T03:21:14.557572+0000","last_became_active":"2024-05-21T03:20:32.153617+0000","last_became_peered":"2024-05-21T03:20:32.153617+0000","last_unstale":"2024-05-21T03:21:14.557572+0000","last_undegraded":"2024-05-21T03:21:14.557572+0000","last_fullsized":"2024-05-21T03:21:14.557572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:57:07.341758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":72,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:21.526625+0000","last_change":"2024-05-21T03:20:29.099928+0000","last_active":"2024-05-21T03:21:21.526625+0000","last_peered":"2024-05-21T03:21:21.526625+0000","last_clean":"2024-05-21T03:21:21.526625+0000","last_became_active":"2024-05-21T03:20:29.099592+0000","last_became_peered":"2024-05-21T03:20:29.099592+0000","last_unstale":"2024-05-21T03:21:21.526625+0000","last_undegraded":"2024-05-21T03:21:21.526625+0000","last_fullsized":"2024-05-21T03:21:21.526625+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:35:34.843325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561455+0000","last_change":"2024-05-21T03:20:32.154962+0000","last_active":"2024-05-21T03:21:14.561455+0000","last_peered":"2024-05-21T03:21:14.561455+0000","last_clean":"2024-05-21T03:21:14.561455+0000","last_became_active":"2024-05-21T03:20:32.154620+0000","last_became_peered":"2024-05-21T03:20:32.154620+0000","last_unstale":"2024-05-21T03:21:14.561455+0000","last_undegraded":"2024-05-21T03:21:14.561455+0000","last_fullsized":"2024-05-21T03:21:14.561455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:56:21.553272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561738+0000","last_change":"2024-05-21T03:20:29.097312+0000","last_active":"2024-05-21T03:21:14.561738+0000","last_peered":"2024-05-21T03:21:14.561738+0000","last_clean":"2024-05-21T03:21:14.561738+0000","last_became_active":"2024-05-21T03:20:29.096977+0000","last_became_peered":"2024-05-21T03:20:29.096977+0000","last_unstale":"2024-05-21T03:21:14.561738+0000","last_undegraded":"2024-05-21T03:21:14.561738+0000","last_fullsized":"2024-05-21T03:21:14.561738+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:28:45.467446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556973+0000","last_change":"2024-05-21T03:20:32.150035+0000","last_active":"2024-05-21T03:21:14.556973+0000","last_peered":"2024-05-21T03:21:14.556973+0000","last_clean":"2024-05-21T03:21:14.556973+0000","last_became_active":"2024-05-21T03:20:32.149688+0000","last_became_peered":"2024-05-21T03:20:32.149688+0000","last_unstale":"2024-05-21T03:21:14.556973+0000","last_undegraded":"2024-05-21T03:21:14.556973+0000","last_fullsized":"2024-05-21T03:21:14.556973+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:27:00.410770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.573309+0000","last_change":"2024-05-21T03:21:14.572832+0000","last_active":"2024-05-21T03:21:14.573309+0000","last_peered":"2024-05-21T03:21:14.573309+0000","last_clean":"2024-05-21T03:21:14.573309+0000","last_became_active":"2024-05-21T03:21:14.559381+0000","last_became_peered":"2024-05-21T03:21:14.559381+0000","last_unstale":"2024-05-21T03:21:14.573309+0000","last_undegraded":"2024-05-21T03:21:14.573309+0000","last_fullsized":"2024-05-21T03:21:14.573309+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:02:53.004031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558214+0000","last_change":"2024-05-21T03:20:32.150301+0000","last_active":"2024-05-21T03:21:14.558214+0000","last_peered":"2024-05-21T03:21:14.558214+0000","last_clean":"2024-05-21T03:21:14.558214+0000","last_became_active":"2024-05-21T03:20:32.148475+0000","last_became_peered":"2024-05-21T03:20:32.148475+0000","last_unstale":"2024-05-21T03:21:14.558214+0000","last_undegraded":"2024-05-21T03:21:14.558214+0000","last_fullsized":"2024-05-21T03:21:14.558214+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:59:30.339660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556899+0000","last_change":"2024-05-21T03:20:29.098861+0000","last_active":"2024-05-21T03:21:14.556899+0000","last_peered":"2024-05-21T03:21:14.556899+0000","last_clean":"2024-05-21T03:21:14.556899+0000","last_became_active":"2024-05-21T03:20:29.097874+0000","last_became_peered":"2024-05-21T03:20:29.097874+0000","last_unstale":"2024-05-21T03:21:14.556899+0000","last_undegraded":"2024-05-21T03:21:14.556899+0000","last_fullsized":"2024-05-21T03:21:14.556899+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:25:36.227339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561801+0000","last_change":"2024-05-21T03:20:32.142820+0000","last_active":"2024-05-21T03:21:14.561801+0000","last_peered":"2024-05-21T03:21:14.561801+0000","last_clean":"2024-05-21T03:21:14.561801+0000","last_became_active":"2024-05-21T03:20:32.142396+0000","last_became_peered":"2024-05-21T03:20:32.142396+0000","last_unstale":"2024-05-21T03:21:14.561801+0000","last_undegraded":"2024-05-21T03:21:14.561801+0000","last_fullsized":"2024-05-21T03:21:14.561801+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:30:14.452389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":65,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:20.528069+0000","last_change":"2024-05-21T03:20:29.103153+0000","last_active":"2024-05-21T03:21:20.528069+0000","last_peered":"2024-05-21T03:21:20.528069+0000","last_clean":"2024-05-21T03:21:20.528069+0000","last_became_active":"2024-05-21T03:20:29.102819+0000","last_became_peered":"2024-05-21T03:20:29.102819+0000","last_unstale":"2024-05-21T03:21:20.528069+0000","last_undegraded":"2024-05-21T03:21:20.528069+0000","last_fullsized":"2024-05-21T03:21:20.528069+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:47:49.843926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556528+0000","last_change":"2024-05-21T03:20:32.142263+0000","last_active":"2024-05-21T03:21:14.556528+0000","last_peered":"2024-05-21T03:21:14.556528+0000","last_clean":"2024-05-21T03:21:14.556528+0000","last_became_active":"2024-05-21T03:20:32.141820+0000","last_became_peered":"2024-05-21T03:20:32.141820+0000","last_unstale":"2024-05-21T03:21:14.556528+0000","last_undegraded":"2024-05-21T03:21:14.556528+0000","last_fullsized":"2024-05-21T03:21:14.556528+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:45:20.021237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.554011+0000","last_change":"2024-05-21T03:20:29.100088+0000","last_active":"2024-05-21T03:21:14.554011+0000","last_peered":"2024-05-21T03:21:14.554011+0000","last_clean":"2024-05-21T03:21:14.554011+0000","last_became_active":"2024-05-21T03:20:29.097343+0000","last_became_peered":"2024-05-21T03:20:29.097343+0000","last_unstale":"2024-05-21T03:21:14.554011+0000","last_undegraded":"2024-05-21T03:21:14.554011+0000","last_fullsized":"2024-05-21T03:21:14.554011+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:00:35.682393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.562687+0000","last_change":"2024-05-21T03:21:14.562281+0000","last_active":"2024-05-21T03:21:14.562687+0000","last_peered":"2024-05-21T03:21:14.562687+0000","last_clean":"2024-05-21T03:21:14.562687+0000","last_became_active":"2024-05-21T03:21:14.561857+0000","last_became_peered":"2024-05-21T03:21:14.561857+0000","last_unstale":"2024-05-21T03:21:14.562687+0000","last_undegraded":"2024-05-21T03:21:14.562687+0000","last_fullsized":"2024-05-21T03:21:14.562687+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:37:35.283529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552729+0000","last_change":"2024-05-21T03:20:29.097945+0000","last_active":"2024-05-21T03:21:14.552729+0000","last_peered":"2024-05-21T03:21:14.552729+0000","last_clean":"2024-05-21T03:21:14.552729+0000","last_became_active":"2024-05-21T03:20:29.097624+0000","last_became_peered":"2024-05-21T03:20:29.097624+0000","last_unstale":"2024-05-21T03:21:14.552729+0000","last_undegraded":"2024-05-21T03:21:14.552729+0000","last_fullsized":"2024-05-21T03:21:14.552729+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:01:01.694893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.560719+0000","last_change":"2024-05-21T03:20:32.148955+0000","last_active":"2024-05-21T03:21:14.560719+0000","last_peered":"2024-05-21T03:21:14.560719+0000","last_clean":"2024-05-21T03:21:14.560719+0000","last_became_active":"2024-05-21T03:20:32.148631+0000","last_became_peered":"2024-05-21T03:20:32.148631+0000","last_unstale":"2024-05-21T03:21:14.560719+0000","last_undegraded":"2024-05-21T03:21:14.560719+0000","last_fullsized":"2024-05-21T03:21:14.560719+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:36:25.820580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561173+0000","last_change":"2024-05-21T03:20:29.100902+0000","last_active":"2024-05-21T03:21:14.561173+0000","last_peered":"2024-05-21T03:21:14.561173+0000","last_clean":"2024-05-21T03:21:14.561173+0000","last_became_active":"2024-05-21T03:20:29.100570+0000","last_became_peered":"2024-05-21T03:20:29.100570+0000","last_unstale":"2024-05-21T03:21:14.561173+0000","last_undegraded":"2024-05-21T03:21:14.561173+0000","last_fullsized":"2024-05-21T03:21:14.561173+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:58:36.651338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.555588+0000","last_change":"2024-05-21T03:20:32.150724+0000","last_active":"2024-05-21T03:21:14.555588+0000","last_peered":"2024-05-21T03:21:14.555588+0000","last_clean":"2024-05-21T03:21:14.555588+0000","last_became_active":"2024-05-21T03:20:32.150371+0000","last_became_peered":"2024-05-21T03:20:32.150371+0000","last_unstale":"2024-05-21T03:21:14.555588+0000","last_undegraded":"2024-05-21T03:21:14.555588+0000","last_fullsized":"2024-05-21T03:21:14.555588+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:44:32.321192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":97,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:22.574340+0000","last_change":"2024-05-21T03:20:29.095045+0000","last_active":"2024-05-21T03:21:22.574340+0000","last_peered":"2024-05-21T03:21:22.574340+0000","last_clean":"2024-05-21T03:21:22.574340+0000","last_became_active":"2024-05-21T03:20:29.094552+0000","last_became_peered":"2024-05-21T03:20:29.094552+0000","last_unstale":"2024-05-21T03:21:22.574340+0000","last_undegraded":"2024-05-21T03:21:22.574340+0000","last_fullsized":"2024-05-21T03:21:22.574340+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:36:15.155125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558897+0000","last_change":"2024-05-21T03:20:32.149813+0000","last_active":"2024-05-21T03:21:14.558897+0000","last_peered":"2024-05-21T03:21:14.558897+0000","last_clean":"2024-05-21T03:21:14.558897+0000","last_became_active":"2024-05-21T03:20:32.148422+0000","last_became_peered":"2024-05-21T03:20:32.148422+0000","last_unstale":"2024-05-21T03:21:14.558897+0000","last_undegraded":"2024-05-21T03:21:14.558897+0000","last_fullsized":"2024-05-21T03:21:14.558897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:55:35.115463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558265+0000","last_change":"2024-05-21T03:20:29.101463+0000","last_active":"2024-05-21T03:21:14.558265+0000","last_peered":"2024-05-21T03:21:14.558265+0000","last_clean":"2024-05-21T03:21:14.558265+0000","last_became_active":"2024-05-21T03:20:29.101127+0000","last_became_peered":"2024-05-21T03:20:29.101127+0000","last_unstale":"2024-05-21T03:21:14.558265+0000","last_undegraded":"2024-05-21T03:21:14.558265+0000","last_fullsized":"2024-05-21T03:21:14.558265+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:58:17.408315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.563434+0000","last_change":"2024-05-21T03:21:14.563434+0000","last_active":"2024-05-21T03:21:14.563434+0000","last_peered":"2024-05-21T03:21:14.563434+0000","last_clean":"2024-05-21T03:21:14.563434+0000","last_became_active":"2024-05-21T03:21:14.563081+0000","last_became_peered":"2024-05-21T03:21:14.563081+0000","last_unstale":"2024-05-21T03:21:14.563434+0000","last_undegraded":"2024-05-21T03:21:14.563434+0000","last_fullsized":"2024-05-21T03:21:14.563434+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:28:55.497338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.562216+0000","last_change":"2024-05-21T03:20:29.098725+0000","last_active":"2024-05-21T03:21:14.562216+0000","last_peered":"2024-05-21T03:21:14.562216+0000","last_clean":"2024-05-21T03:21:14.562216+0000","last_became_active":"2024-05-21T03:20:29.098471+0000","last_became_peered":"2024-05-21T03:20:29.098471+0000","last_unstale":"2024-05-21T03:21:14.562216+0000","last_undegraded":"2024-05-21T03:21:14.562216+0000","last_fullsized":"2024-05-21T03:21:14.562216+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:17:11.059825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.560975+0000","last_change":"2024-05-21T03:21:14.560975+0000","last_active":"2024-05-21T03:21:14.560975+0000","last_peered":"2024-05-21T03:21:14.560975+0000","last_clean":"2024-05-21T03:21:14.560975+0000","last_became_active":"2024-05-21T03:21:14.560528+0000","last_became_peered":"2024-05-21T03:21:14.560528+0000","last_unstale":"2024-05-21T03:21:14.560975+0000","last_undegraded":"2024-05-21T03:21:14.560975+0000","last_fullsized":"2024-05-21T03:21:14.560975+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:51:47.469291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'31","reported_seq":134,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:23.530523+0000","last_change":"2024-05-21T03:20:29.099968+0000","last_active":"2024-05-21T03:21:23.530523+0000","last_peered":"2024-05-21T03:21:23.530523+0000","last_clean":"2024-05-21T03:21:23.530523+0000","last_became_active":"2024-05-21T03:20:29.099181+0000","last_became_peered":"2024-05-21T03:20:29.099181+0000","last_unstale":"2024-05-21T03:21:23.530523+0000","last_undegraded":"2024-05-21T03:21:23.530523+0000","last_fullsized":"2024-05-21T03:21:23.530523+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:35:33.934412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59762,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":77,"num_read_kb":96,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556604+0000","last_change":"2024-05-21T03:20:32.144024+0000","last_active":"2024-05-21T03:21:14.556604+0000","last_peered":"2024-05-21T03:21:14.556604+0000","last_clean":"2024-05-21T03:21:14.556604+0000","last_became_active":"2024-05-21T03:20:32.143668+0000","last_became_peered":"2024-05-21T03:20:32.143668+0000","last_unstale":"2024-05-21T03:21:14.556604+0000","last_undegraded":"2024-05-21T03:21:14.556604+0000","last_fullsized":"2024-05-21T03:21:14.556604+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:11:31.100802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556933+0000","last_change":"2024-05-21T03:20:29.101770+0000","last_active":"2024-05-21T03:21:14.556933+0000","last_peered":"2024-05-21T03:21:14.556933+0000","last_clean":"2024-05-21T03:21:14.556933+0000","last_became_active":"2024-05-21T03:20:29.100407+0000","last_became_peered":"2024-05-21T03:20:29.100407+0000","last_unstale":"2024-05-21T03:21:14.556933+0000","last_undegraded":"2024-05-21T03:21:14.556933+0000","last_fullsized":"2024-05-21T03:21:14.556933+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:01:33.991246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552656+0000","last_change":"2024-05-21T03:20:32.147479+0000","last_active":"2024-05-21T03:21:14.552656+0000","last_peered":"2024-05-21T03:21:14.552656+0000","last_clean":"2024-05-21T03:21:14.552656+0000","last_became_active":"2024-05-21T03:20:32.147160+0000","last_became_peered":"2024-05-21T03:20:32.147160+0000","last_unstale":"2024-05-21T03:21:14.552656+0000","last_undegraded":"2024-05-21T03:21:14.552656+0000","last_fullsized":"2024-05-21T03:21:14.552656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:51:59.343380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.562216+0000","last_change":"2024-05-21T03:20:29.101875+0000","last_active":"2024-05-21T03:21:14.562216+0000","last_peered":"2024-05-21T03:21:14.562216+0000","last_clean":"2024-05-21T03:21:14.562216+0000","last_became_active":"2024-05-21T03:20:29.101468+0000","last_became_peered":"2024-05-21T03:20:29.101468+0000","last_unstale":"2024-05-21T03:21:14.562216+0000","last_undegraded":"2024-05-21T03:21:14.562216+0000","last_fullsized":"2024-05-21T03:21:14.562216+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:10:19.577836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.557393+0000","last_change":"2024-05-21T03:20:32.139002+0000","last_active":"2024-05-21T03:21:14.557393+0000","last_peered":"2024-05-21T03:21:14.557393+0000","last_clean":"2024-05-21T03:21:14.557393+0000","last_became_active":"2024-05-21T03:20:32.138619+0000","last_became_peered":"2024-05-21T03:20:32.138619+0000","last_unstale":"2024-05-21T03:21:14.557393+0000","last_undegraded":"2024-05-21T03:21:14.557393+0000","last_fullsized":"2024-05-21T03:21:14.557393+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:21:36.136283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":66,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:20.526418+0000","last_change":"2024-05-21T03:20:29.101532+0000","last_active":"2024-05-21T03:21:20.526418+0000","last_peered":"2024-05-21T03:21:20.526418+0000","last_clean":"2024-05-21T03:21:20.526418+0000","last_became_active":"2024-05-21T03:20:29.101206+0000","last_became_peered":"2024-05-21T03:20:29.101206+0000","last_unstale":"2024-05-21T03:21:20.526418+0000","last_undegraded":"2024-05-21T03:21:20.526418+0000","last_fullsized":"2024-05-21T03:21:20.526418+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:47:38.555995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556597+0000","last_change":"2024-05-21T03:20:32.146866+0000","last_active":"2024-05-21T03:21:14.556597+0000","last_peered":"2024-05-21T03:21:14.556597+0000","last_clean":"2024-05-21T03:21:14.556597+0000","last_became_active":"2024-05-21T03:20:32.146463+0000","last_became_peered":"2024-05-21T03:20:32.146463+0000","last_unstale":"2024-05-21T03:21:14.556597+0000","last_undegraded":"2024-05-21T03:21:14.556597+0000","last_fullsized":"2024-05-21T03:21:14.556597+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:06:54.359922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556379+0000","last_change":"2024-05-21T03:20:29.103910+0000","last_active":"2024-05-21T03:21:14.556379+0000","last_peered":"2024-05-21T03:21:14.556379+0000","last_clean":"2024-05-21T03:21:14.556379+0000","last_became_active":"2024-05-21T03:20:29.103583+0000","last_became_peered":"2024-05-21T03:20:29.103583+0000","last_unstale":"2024-05-21T03:21:14.556379+0000","last_undegraded":"2024-05-21T03:21:14.556379+0000","last_fullsized":"2024-05-21T03:21:14.556379+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:57:15.665158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561420+0000","last_change":"2024-05-21T03:20:32.141242+0000","last_active":"2024-05-21T03:21:14.561420+0000","last_peered":"2024-05-21T03:21:14.561420+0000","last_clean":"2024-05-21T03:21:14.561420+0000","last_became_active":"2024-05-21T03:20:32.140892+0000","last_became_peered":"2024-05-21T03:20:32.140892+0000","last_unstale":"2024-05-21T03:21:14.561420+0000","last_undegraded":"2024-05-21T03:21:14.561420+0000","last_fullsized":"2024-05-21T03:21:14.561420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:42:13.997708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.557026+0000","last_change":"2024-05-21T03:20:29.098973+0000","last_active":"2024-05-21T03:21:14.557026+0000","last_peered":"2024-05-21T03:21:14.557026+0000","last_clean":"2024-05-21T03:21:14.557026+0000","last_became_active":"2024-05-21T03:20:29.098694+0000","last_became_peered":"2024-05-21T03:20:29.098694+0000","last_unstale":"2024-05-21T03:21:14.557026+0000","last_undegraded":"2024-05-21T03:21:14.557026+0000","last_fullsized":"2024-05-21T03:21:14.557026+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:55:17.915820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.553963+0000","last_change":"2024-05-21T03:20:32.147465+0000","last_active":"2024-05-21T03:21:14.553963+0000","last_peered":"2024-05-21T03:21:14.553963+0000","last_clean":"2024-05-21T03:21:14.553963+0000","last_became_active":"2024-05-21T03:20:32.147073+0000","last_became_peered":"2024-05-21T03:20:32.147073+0000","last_unstale":"2024-05-21T03:21:14.553963+0000","last_undegraded":"2024-05-21T03:21:14.553963+0000","last_fullsized":"2024-05-21T03:21:14.553963+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:29:05.451370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.560568+0000","last_change":"2024-05-21T03:21:14.560568+0000","last_active":"2024-05-21T03:21:14.560568+0000","last_peered":"2024-05-21T03:21:14.560568+0000","last_clean":"2024-05-21T03:21:14.560568+0000","last_became_active":"2024-05-21T03:21:14.560093+0000","last_became_peered":"2024-05-21T03:21:14.560093+0000","last_unstale":"2024-05-21T03:21:14.560568+0000","last_undegraded":"2024-05-21T03:21:14.560568+0000","last_fullsized":"2024-05-21T03:21:14.560568+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:55:39.839942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.554395+0000","last_change":"2024-05-21T03:20:32.146456+0000","last_active":"2024-05-21T03:21:14.554395+0000","last_peered":"2024-05-21T03:21:14.554395+0000","last_clean":"2024-05-21T03:21:14.554395+0000","last_became_active":"2024-05-21T03:20:32.146105+0000","last_became_peered":"2024-05-21T03:20:32.146105+0000","last_unstale":"2024-05-21T03:21:14.554395+0000","last_undegraded":"2024-05-21T03:21:14.554395+0000","last_fullsized":"2024-05-21T03:21:14.554395+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:00:30.550178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'6","reported_seq":65,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:20.525865+0000","last_change":"2024-05-21T03:20:29.100262+0000","last_active":"2024-05-21T03:21:20.525865+0000","last_peered":"2024-05-21T03:21:20.525865+0000","last_clean":"2024-05-21T03:21:20.525865+0000","last_became_active":"2024-05-21T03:20:29.099917+0000","last_became_peered":"2024-05-21T03:20:29.099917+0000","last_unstale":"2024-05-21T03:21:20.525865+0000","last_undegraded":"2024-05-21T03:21:20.525865+0000","last_fullsized":"2024-05-21T03:21:20.525865+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:55:48.608341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.554083+0000","last_change":"2024-05-21T03:20:32.142776+0000","last_active":"2024-05-21T03:21:14.554083+0000","last_peered":"2024-05-21T03:21:14.554083+0000","last_clean":"2024-05-21T03:21:14.554083+0000","last_became_active":"2024-05-21T03:20:32.142430+0000","last_became_peered":"2024-05-21T03:20:32.142430+0000","last_unstale":"2024-05-21T03:21:14.554083+0000","last_undegraded":"2024-05-21T03:21:14.554083+0000","last_fullsized":"2024-05-21T03:21:14.554083+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:54:28.682343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561357+0000","last_change":"2024-05-21T03:20:32.139506+0000","last_active":"2024-05-21T03:21:14.561357+0000","last_peered":"2024-05-21T03:21:14.561357+0000","last_clean":"2024-05-21T03:21:14.561357+0000","last_became_active":"2024-05-21T03:20:32.138136+0000","last_became_peered":"2024-05-21T03:20:32.138136+0000","last_unstale":"2024-05-21T03:21:14.561357+0000","last_undegraded":"2024-05-21T03:21:14.561357+0000","last_fullsized":"2024-05-21T03:21:14.561357+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:31:24.996276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":63,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:19.524901+0000","last_change":"2024-05-21T03:20:29.096546+0000","last_active":"2024-05-21T03:21:19.524901+0000","last_peered":"2024-05-21T03:21:19.524901+0000","last_clean":"2024-05-21T03:21:19.524901+0000","last_became_active":"2024-05-21T03:20:29.096184+0000","last_became_peered":"2024-05-21T03:20:29.096184+0000","last_unstale":"2024-05-21T03:21:19.524901+0000","last_undegraded":"2024-05-21T03:21:19.524901+0000","last_fullsized":"2024-05-21T03:21:19.524901+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:09:12.832745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561399+0000","last_change":"2024-05-21T03:20:32.156231+0000","last_active":"2024-05-21T03:21:14.561399+0000","last_peered":"2024-05-21T03:21:14.561399+0000","last_clean":"2024-05-21T03:21:14.561399+0000","last_became_active":"2024-05-21T03:20:32.154035+0000","last_became_peered":"2024-05-21T03:20:32.154035+0000","last_unstale":"2024-05-21T03:21:14.561399+0000","last_undegraded":"2024-05-21T03:21:14.561399+0000","last_fullsized":"2024-05-21T03:21:14.561399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:30:53.820292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556130+0000","last_change":"2024-05-21T03:20:29.100422+0000","last_active":"2024-05-21T03:21:14.556130+0000","last_peered":"2024-05-21T03:21:14.556130+0000","last_clean":"2024-05-21T03:21:14.556130+0000","last_became_active":"2024-05-21T03:20:29.099993+0000","last_became_peered":"2024-05-21T03:20:29.099993+0000","last_unstale":"2024-05-21T03:21:14.556130+0000","last_undegraded":"2024-05-21T03:21:14.556130+0000","last_fullsized":"2024-05-21T03:21:14.556130+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:03:15.456447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":98,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556799+0000","last_change":"2024-05-21T03:19:05.647808+0000","last_active":"2024-05-21T03:21:14.556799+0000","last_peered":"2024-05-21T03:21:14.556799+0000","last_clean":"2024-05-21T03:21:14.556799+0000","last_became_active":"2024-05-21T03:19:05.641865+0000","last_became_peered":"2024-05-21T03:19:05.641865+0000","last_unstale":"2024-05-21T03:21:14.556799+0000","last_undegraded":"2024-05-21T03:21:14.556799+0000","last_fullsized":"2024-05-21T03:21:14.556799+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:15:42.008493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:15:42.008493+0000","last_clean_scrub_stamp":"2024-05-21T03:15:42.008493+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:26:58.040196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552599+0000","last_change":"2024-05-21T03:20:32.138182+0000","last_active":"2024-05-21T03:21:14.552599+0000","last_peered":"2024-05-21T03:21:14.552599+0000","last_clean":"2024-05-21T03:21:14.552599+0000","last_became_active":"2024-05-21T03:20:32.137842+0000","last_became_peered":"2024-05-21T03:20:32.137842+0000","last_unstale":"2024-05-21T03:21:14.552599+0000","last_undegraded":"2024-05-21T03:21:14.552599+0000","last_fullsized":"2024-05-21T03:21:14.552599+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:43:50.070160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556989+0000","last_change":"2024-05-21T03:20:29.104246+0000","last_active":"2024-05-21T03:21:14.556989+0000","last_peered":"2024-05-21T03:21:14.556989+0000","last_clean":"2024-05-21T03:21:14.556989+0000","last_became_active":"2024-05-21T03:20:29.103910+0000","last_became_peered":"2024-05-21T03:20:29.103910+0000","last_unstale":"2024-05-21T03:21:14.556989+0000","last_undegraded":"2024-05-21T03:21:14.556989+0000","last_fullsized":"2024-05-21T03:21:14.556989+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:39:28.029283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556715+0000","last_change":"2024-05-21T03:20:29.102020+0000","last_active":"2024-05-21T03:21:14.556715+0000","last_peered":"2024-05-21T03:21:14.556715+0000","last_clean":"2024-05-21T03:21:14.556715+0000","last_became_active":"2024-05-21T03:20:29.100429+0000","last_became_peered":"2024-05-21T03:20:29.100429+0000","last_unstale":"2024-05-21T03:21:14.556715+0000","last_undegraded":"2024-05-21T03:21:14.556715+0000","last_fullsized":"2024-05-21T03:21:14.556715+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:09:01.621074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552834+0000","last_change":"2024-05-21T03:20:32.137130+0000","last_active":"2024-05-21T03:21:14.552834+0000","last_peered":"2024-05-21T03:21:14.552834+0000","last_clean":"2024-05-21T03:21:14.552834+0000","last_became_active":"2024-05-21T03:20:32.136821+0000","last_became_peered":"2024-05-21T03:20:32.136821+0000","last_unstale":"2024-05-21T03:21:14.552834+0000","last_undegraded":"2024-05-21T03:21:14.552834+0000","last_fullsized":"2024-05-21T03:21:14.552834+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:53:41.290370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'14","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.555888+0000","last_change":"2024-05-21T03:20:29.099869+0000","last_active":"2024-05-21T03:21:14.555888+0000","last_peered":"2024-05-21T03:21:14.555888+0000","last_clean":"2024-05-21T03:21:14.555888+0000","last_became_active":"2024-05-21T03:20:29.099534+0000","last_became_peered":"2024-05-21T03:20:29.099534+0000","last_unstale":"2024-05-21T03:21:14.555888+0000","last_undegraded":"2024-05-21T03:21:14.555888+0000","last_fullsized":"2024-05-21T03:21:14.555888+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:01:11.281544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561631+0000","last_change":"2024-05-21T03:20:32.153921+0000","last_active":"2024-05-21T03:21:14.561631+0000","last_peered":"2024-05-21T03:21:14.561631+0000","last_clean":"2024-05-21T03:21:14.561631+0000","last_became_active":"2024-05-21T03:20:32.153594+0000","last_became_peered":"2024-05-21T03:20:32.153594+0000","last_unstale":"2024-05-21T03:21:14.561631+0000","last_undegraded":"2024-05-21T03:21:14.561631+0000","last_fullsized":"2024-05-21T03:21:14.561631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:27:59.426942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556915+0000","last_change":"2024-05-21T03:20:29.097399+0000","last_active":"2024-05-21T03:21:14.556915+0000","last_peered":"2024-05-21T03:21:14.556915+0000","last_clean":"2024-05-21T03:21:14.556915+0000","last_became_active":"2024-05-21T03:20:29.097010+0000","last_became_peered":"2024-05-21T03:20:29.097010+0000","last_unstale":"2024-05-21T03:21:14.556915+0000","last_undegraded":"2024-05-21T03:21:14.556915+0000","last_fullsized":"2024-05-21T03:21:14.556915+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:54:53.273947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.559609+0000","last_change":"2024-05-21T03:21:14.559609+0000","last_active":"2024-05-21T03:21:14.559609+0000","last_peered":"2024-05-21T03:21:14.559609+0000","last_clean":"2024-05-21T03:21:14.559609+0000","last_became_active":"2024-05-21T03:21:14.559218+0000","last_became_peered":"2024-05-21T03:21:14.559218+0000","last_unstale":"2024-05-21T03:21:14.559609+0000","last_undegraded":"2024-05-21T03:21:14.559609+0000","last_fullsized":"2024-05-21T03:21:14.559609+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:32:11.156037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,4],"acting":[0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.557892+0000","last_change":"2024-05-21T03:20:29.103526+0000","last_active":"2024-05-21T03:21:14.557892+0000","last_peered":"2024-05-21T03:21:14.557892+0000","last_clean":"2024-05-21T03:21:14.557892+0000","last_became_active":"2024-05-21T03:20:29.103067+0000","last_became_peered":"2024-05-21T03:20:29.103067+0000","last_unstale":"2024-05-21T03:21:14.557892+0000","last_undegraded":"2024-05-21T03:21:14.557892+0000","last_fullsized":"2024-05-21T03:21:14.557892+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:54:47.566949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556409+0000","last_change":"2024-05-21T03:20:32.141088+0000","last_active":"2024-05-21T03:21:14.556409+0000","last_peered":"2024-05-21T03:21:14.556409+0000","last_clean":"2024-05-21T03:21:14.556409+0000","last_became_active":"2024-05-21T03:20:32.140744+0000","last_became_peered":"2024-05-21T03:20:32.140744+0000","last_unstale":"2024-05-21T03:21:14.556409+0000","last_undegraded":"2024-05-21T03:21:14.556409+0000","last_fullsized":"2024-05-21T03:21:14.556409+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:57:35.591164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.560757+0000","last_change":"2024-05-21T03:20:29.093440+0000","last_active":"2024-05-21T03:21:14.560757+0000","last_peered":"2024-05-21T03:21:14.560757+0000","last_clean":"2024-05-21T03:21:14.560757+0000","last_became_active":"2024-05-21T03:20:29.092935+0000","last_became_peered":"2024-05-21T03:20:29.092935+0000","last_unstale":"2024-05-21T03:21:14.560757+0000","last_undegraded":"2024-05-21T03:21:14.560757+0000","last_fullsized":"2024-05-21T03:21:14.560757+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:18:00.690465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.553665+0000","last_change":"2024-05-21T03:20:32.143781+0000","last_active":"2024-05-21T03:21:14.553665+0000","last_peered":"2024-05-21T03:21:14.553665+0000","last_clean":"2024-05-21T03:21:14.553665+0000","last_became_active":"2024-05-21T03:20:32.143391+0000","last_became_peered":"2024-05-21T03:20:32.143391+0000","last_unstale":"2024-05-21T03:21:14.553665+0000","last_undegraded":"2024-05-21T03:21:14.553665+0000","last_fullsized":"2024-05-21T03:21:14.553665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:00:41.506561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556256+0000","last_change":"2024-05-21T03:20:29.101342+0000","last_active":"2024-05-21T03:21:14.556256+0000","last_peered":"2024-05-21T03:21:14.556256+0000","last_clean":"2024-05-21T03:21:14.556256+0000","last_became_active":"2024-05-21T03:20:29.100997+0000","last_became_peered":"2024-05-21T03:20:29.100997+0000","last_unstale":"2024-05-21T03:21:14.556256+0000","last_undegraded":"2024-05-21T03:21:14.556256+0000","last_fullsized":"2024-05-21T03:21:14.556256+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:05:31.801669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561184+0000","last_change":"2024-05-21T03:20:32.150563+0000","last_active":"2024-05-21T03:21:14.561184+0000","last_peered":"2024-05-21T03:21:14.561184+0000","last_clean":"2024-05-21T03:21:14.561184+0000","last_became_active":"2024-05-21T03:20:32.150231+0000","last_became_peered":"2024-05-21T03:20:32.150231+0000","last_unstale":"2024-05-21T03:21:14.561184+0000","last_undegraded":"2024-05-21T03:21:14.561184+0000","last_fullsized":"2024-05-21T03:21:14.561184+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:10:00.251284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556609+0000","last_change":"2024-05-21T03:20:29.101576+0000","last_active":"2024-05-21T03:21:14.556609+0000","last_peered":"2024-05-21T03:21:14.556609+0000","last_clean":"2024-05-21T03:21:14.556609+0000","last_became_active":"2024-05-21T03:20:29.101270+0000","last_became_peered":"2024-05-21T03:20:29.101270+0000","last_unstale":"2024-05-21T03:21:14.556609+0000","last_undegraded":"2024-05-21T03:21:14.556609+0000","last_fullsized":"2024-05-21T03:21:14.556609+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:50:10.579901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.557991+0000","last_change":"2024-05-21T03:20:32.141340+0000","last_active":"2024-05-21T03:21:14.557991+0000","last_peered":"2024-05-21T03:21:14.557991+0000","last_clean":"2024-05-21T03:21:14.557991+0000","last_became_active":"2024-05-21T03:20:32.140792+0000","last_became_peered":"2024-05-21T03:20:32.140792+0000","last_unstale":"2024-05-21T03:21:14.557991+0000","last_undegraded":"2024-05-21T03:21:14.557991+0000","last_fullsized":"2024-05-21T03:21:14.557991+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:35:05.842929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:20.572553+0000","last_change":"2024-05-21T03:21:14.566886+0000","last_active":"2024-05-21T03:21:20.572553+0000","last_peered":"2024-05-21T03:21:20.572553+0000","last_clean":"2024-05-21T03:21:20.572553+0000","last_became_active":"2024-05-21T03:21:14.559816+0000","last_became_peered":"2024-05-21T03:21:14.559816+0000","last_unstale":"2024-05-21T03:21:20.572553+0000","last_undegraded":"2024-05-21T03:21:20.572553+0000","last_fullsized":"2024-05-21T03:21:20.572553+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:34:43.109424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1561,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558407+0000","last_change":"2024-05-21T03:20:32.146674+0000","last_active":"2024-05-21T03:21:14.558407+0000","last_peered":"2024-05-21T03:21:14.558407+0000","last_clean":"2024-05-21T03:21:14.558407+0000","last_became_active":"2024-05-21T03:20:32.146150+0000","last_became_peered":"2024-05-21T03:20:32.146150+0000","last_unstale":"2024-05-21T03:21:14.558407+0000","last_undegraded":"2024-05-21T03:21:14.558407+0000","last_fullsized":"2024-05-21T03:21:14.558407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:39:09.905640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561667+0000","last_change":"2024-05-21T03:20:29.097613+0000","last_active":"2024-05-21T03:21:14.561667+0000","last_peered":"2024-05-21T03:21:14.561667+0000","last_clean":"2024-05-21T03:21:14.561667+0000","last_became_active":"2024-05-21T03:20:29.097331+0000","last_became_peered":"2024-05-21T03:20:29.097331+0000","last_unstale":"2024-05-21T03:21:14.561667+0000","last_undegraded":"2024-05-21T03:21:14.561667+0000","last_fullsized":"2024-05-21T03:21:14.561667+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:56:14.720420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.566406+0000","last_change":"2024-05-21T03:21:14.566406+0000","last_active":"2024-05-21T03:21:14.566406+0000","last_peered":"2024-05-21T03:21:14.566406+0000","last_clean":"2024-05-21T03:21:14.566406+0000","last_became_active":"2024-05-21T03:21:14.563023+0000","last_became_peered":"2024-05-21T03:21:14.563023+0000","last_unstale":"2024-05-21T03:21:14.566406+0000","last_undegraded":"2024-05-21T03:21:14.566406+0000","last_fullsized":"2024-05-21T03:21:14.566406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:12:53.125136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552818+0000","last_change":"2024-05-21T03:20:29.103456+0000","last_active":"2024-05-21T03:21:14.552818+0000","last_peered":"2024-05-21T03:21:14.552818+0000","last_clean":"2024-05-21T03:21:14.552818+0000","last_became_active":"2024-05-21T03:20:29.103008+0000","last_became_peered":"2024-05-21T03:20:29.103008+0000","last_unstale":"2024-05-21T03:21:14.552818+0000","last_undegraded":"2024-05-21T03:21:14.552818+0000","last_fullsized":"2024-05-21T03:21:14.552818+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:02:37.951347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558574+0000","last_change":"2024-05-21T03:20:32.146986+0000","last_active":"2024-05-21T03:21:14.558574+0000","last_peered":"2024-05-21T03:21:14.558574+0000","last_clean":"2024-05-21T03:21:14.558574+0000","last_became_active":"2024-05-21T03:20:32.146355+0000","last_became_peered":"2024-05-21T03:20:32.146355+0000","last_unstale":"2024-05-21T03:21:14.558574+0000","last_undegraded":"2024-05-21T03:21:14.558574+0000","last_fullsized":"2024-05-21T03:21:14.558574+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:36:29.503099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.559658+0000","last_change":"2024-05-21T03:21:14.559658+0000","last_active":"2024-05-21T03:21:14.559658+0000","last_peered":"2024-05-21T03:21:14.559658+0000","last_clean":"2024-05-21T03:21:14.559658+0000","last_became_active":"2024-05-21T03:21:14.559194+0000","last_became_peered":"2024-05-21T03:21:14.559194+0000","last_unstale":"2024-05-21T03:21:14.559658+0000","last_undegraded":"2024-05-21T03:21:14.559658+0000","last_fullsized":"2024-05-21T03:21:14.559658+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:56:31.709545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556799+0000","last_change":"2024-05-21T03:20:32.137532+0000","last_active":"2024-05-21T03:21:14.556799+0000","last_peered":"2024-05-21T03:21:14.556799+0000","last_clean":"2024-05-21T03:21:14.556799+0000","last_became_active":"2024-05-21T03:20:32.137182+0000","last_became_peered":"2024-05-21T03:20:32.137182+0000","last_unstale":"2024-05-21T03:21:14.556799+0000","last_undegraded":"2024-05-21T03:21:14.556799+0000","last_fullsized":"2024-05-21T03:21:14.556799+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:25:44.207866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556286+0000","last_change":"2024-05-21T03:20:32.153912+0000","last_active":"2024-05-21T03:21:14.556286+0000","last_peered":"2024-05-21T03:21:14.556286+0000","last_clean":"2024-05-21T03:21:14.556286+0000","last_became_active":"2024-05-21T03:20:32.153541+0000","last_became_peered":"2024-05-21T03:20:32.153541+0000","last_unstale":"2024-05-21T03:21:14.556286+0000","last_undegraded":"2024-05-21T03:21:14.556286+0000","last_fullsized":"2024-05-21T03:21:14.556286+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:02:58.688378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556461+0000","last_change":"2024-05-21T03:20:32.143689+0000","last_active":"2024-05-21T03:21:14.556461+0000","last_peered":"2024-05-21T03:21:14.556461+0000","last_clean":"2024-05-21T03:21:14.556461+0000","last_became_active":"2024-05-21T03:20:32.143399+0000","last_became_peered":"2024-05-21T03:20:32.143399+0000","last_unstale":"2024-05-21T03:21:14.556461+0000","last_undegraded":"2024-05-21T03:21:14.556461+0000","last_fullsized":"2024-05-21T03:21:14.556461+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:27:55.116599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552907+0000","last_change":"2024-05-21T03:20:32.141355+0000","last_active":"2024-05-21T03:21:14.552907+0000","last_peered":"2024-05-21T03:21:14.552907+0000","last_clean":"2024-05-21T03:21:14.552907+0000","last_became_active":"2024-05-21T03:20:32.140960+0000","last_became_peered":"2024-05-21T03:20:32.140960+0000","last_unstale":"2024-05-21T03:21:14.552907+0000","last_undegraded":"2024-05-21T03:21:14.552907+0000","last_fullsized":"2024-05-21T03:21:14.552907+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:23:50.287864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.553971+0000","last_change":"2024-05-21T03:20:32.142145+0000","last_active":"2024-05-21T03:21:14.553971+0000","last_peered":"2024-05-21T03:21:14.553971+0000","last_clean":"2024-05-21T03:21:14.553971+0000","last_became_active":"2024-05-21T03:20:32.141802+0000","last_became_peered":"2024-05-21T03:20:32.141802+0000","last_unstale":"2024-05-21T03:21:14.553971+0000","last_undegraded":"2024-05-21T03:21:14.553971+0000","last_fullsized":"2024-05-21T03:21:14.553971+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:26:28.707616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556601+0000","last_change":"2024-05-21T03:20:32.141606+0000","last_active":"2024-05-21T03:21:14.556601+0000","last_peered":"2024-05-21T03:21:14.556601+0000","last_clean":"2024-05-21T03:21:14.556601+0000","last_became_active":"2024-05-21T03:20:32.141258+0000","last_became_peered":"2024-05-21T03:20:32.141258+0000","last_unstale":"2024-05-21T03:21:14.556601+0000","last_undegraded":"2024-05-21T03:21:14.556601+0000","last_fullsized":"2024-05-21T03:21:14.556601+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:22.372723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561186+0000","last_change":"2024-05-21T03:20:32.151621+0000","last_active":"2024-05-21T03:21:14.561186+0000","last_peered":"2024-05-21T03:21:14.561186+0000","last_clean":"2024-05-21T03:21:14.561186+0000","last_became_active":"2024-05-21T03:20:32.151260+0000","last_became_peered":"2024-05-21T03:20:32.151260+0000","last_unstale":"2024-05-21T03:21:14.561186+0000","last_undegraded":"2024-05-21T03:21:14.561186+0000","last_fullsized":"2024-05-21T03:21:14.561186+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:48:59.751179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561107+0000","last_change":"2024-05-21T03:20:32.153843+0000","last_active":"2024-05-21T03:21:14.561107+0000","last_peered":"2024-05-21T03:21:14.561107+0000","last_clean":"2024-05-21T03:21:14.561107+0000","last_became_active":"2024-05-21T03:20:32.153418+0000","last_became_peered":"2024-05-21T03:20:32.153418+0000","last_unstale":"2024-05-21T03:21:14.561107+0000","last_undegraded":"2024-05-21T03:21:14.561107+0000","last_fullsized":"2024-05-21T03:21:14.561107+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:26:29.663966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.560653+0000","last_change":"2024-05-21T03:20:32.153942+0000","last_active":"2024-05-21T03:21:14.560653+0000","last_peered":"2024-05-21T03:21:14.560653+0000","last_clean":"2024-05-21T03:21:14.560653+0000","last_became_active":"2024-05-21T03:20:32.153566+0000","last_became_peered":"2024-05-21T03:20:32.153566+0000","last_unstale":"2024-05-21T03:21:14.560653+0000","last_undegraded":"2024-05-21T03:21:14.560653+0000","last_fullsized":"2024-05-21T03:21:14.560653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:06.899039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558337+0000","last_change":"2024-05-21T03:20:32.153243+0000","last_active":"2024-05-21T03:21:14.558337+0000","last_peered":"2024-05-21T03:21:14.558337+0000","last_clean":"2024-05-21T03:21:14.558337+0000","last_became_active":"2024-05-21T03:20:32.152386+0000","last_became_peered":"2024-05-21T03:20:32.152386+0000","last_unstale":"2024-05-21T03:21:14.558337+0000","last_undegraded":"2024-05-21T03:21:14.558337+0000","last_fullsized":"2024-05-21T03:21:14.558337+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:25:28.935183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552892+0000","last_change":"2024-05-21T03:20:32.142536+0000","last_active":"2024-05-21T03:21:14.552892+0000","last_peered":"2024-05-21T03:21:14.552892+0000","last_clean":"2024-05-21T03:21:14.552892+0000","last_became_active":"2024-05-21T03:20:32.141793+0000","last_became_peered":"2024-05-21T03:20:32.141793+0000","last_unstale":"2024-05-21T03:21:14.552892+0000","last_undegraded":"2024-05-21T03:21:14.552892+0000","last_fullsized":"2024-05-21T03:21:14.552892+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:21:43.547103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.553745+0000","last_change":"2024-05-21T03:20:32.140368+0000","last_active":"2024-05-21T03:21:14.553745+0000","last_peered":"2024-05-21T03:21:14.553745+0000","last_clean":"2024-05-21T03:21:14.553745+0000","last_became_active":"2024-05-21T03:20:32.139747+0000","last_became_peered":"2024-05-21T03:20:32.139747+0000","last_unstale":"2024-05-21T03:21:14.553745+0000","last_undegraded":"2024-05-21T03:21:14.553745+0000","last_fullsized":"2024-05-21T03:21:14.553745+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:05:44.851191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67357,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":345,"num_read_kb":257,"num_write":244,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1561,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":202071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645337,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":901120,"data_stored":433850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841566,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1601536,"data_stored":1084641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005092,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":433506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201319,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":433767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397550,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32668,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712484,"statfs":{"total":95995035648,"available":95961583616,"internally_reserved":0,"allocated":1519616,"data_stored":1026676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561074,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":433497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":494172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":434836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":492831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149762,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":432814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953533,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":434919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757304,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32680,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712472,"statfs":{"total":95995035648,"available":95961571328,"internally_reserved":0,"allocated":1531904,"data_stored":1025572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T03:21:27.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:27 smithi027 ceph-mon[32179]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:21:28.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:27 smithi039 ceph-mon[36429]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:21:28.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:27 smithi165 ceph-mon[36269]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:21:28.920 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:28 smithi039 ceph-mon[36429]: pgmap v319: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s; 139 B/s, 0 keys/s, 0 objects/s recovering 2024-05-21T03:21:28.922 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:28 smithi165 ceph-mon[36269]: pgmap v319: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s; 139 B/s, 0 keys/s, 0 objects/s recovering 2024-05-21T03:21:28.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:28 smithi027 ceph-mon[32179]: pgmap v319: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s; 139 B/s, 0 keys/s, 0 objects/s recovering 2024-05-21T03:21:29.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:29 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:21:29.945 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:29 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:21:29.956 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:29 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:21:30.569 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-21T03:21:30.838 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:21:30.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:30 smithi027 ceph-mon[32179]: pgmap v320: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-21T03:21:31.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:30 smithi039 ceph-mon[36429]: pgmap v320: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-21T03:21:31.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:30 smithi165 ceph-mon[36269]: pgmap v320: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-21T03:21:31.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:31 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:21:31.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:31 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:21:31.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:31 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:21:31.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:31 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:21:31.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:31 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:21:31.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:31 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:21:31.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:31 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:21:31.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:31 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:21:31.985 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:31 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:21:32.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:31 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:21:32.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:31 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:21:32.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:31 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:21:32.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:31 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:21:32.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:31 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:21:32.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:31 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:21:32.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:31 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:21:32.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:31 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:21:32.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:31 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:21:32.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:31 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:21:32.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:31 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:21:32.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:31 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:21:32.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:31 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:21:32.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:31 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:21:32.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:31 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:21:32.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:31 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:21:32.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:31 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:21:32.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:31 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:21:32.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.415+0000 7f70212f0700 1 -- 172.21.15.27:0/4212677927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f701c06bbe0 msgr2=0x7f701c1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:32.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.415+0000 7f70212f0700 1 --2- 172.21.15.27:0/4212677927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f701c06bbe0 0x7f701c1103d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7014009990 tx=0x7f7014009ca0 comp rx=0 tx=0).stop 2024-05-21T03:21:32.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.415+0000 7f70212f0700 1 -- 172.21.15.27:0/4212677927 shutdown_connections 2024-05-21T03:21:32.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.415+0000 7f70212f0700 1 --2- 172.21.15.27:0/4212677927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f701c06bbe0 0x7f701c1103d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:32.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.415+0000 7f70212f0700 1 --2- 172.21.15.27:0/4212677927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f701c1095b0 0x7f701c109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:32.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.415+0000 7f70212f0700 1 --2- 172.21.15.27:0/4212677927 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f701c06b220 0x7f701c06b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:32.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.415+0000 7f70212f0700 1 -- 172.21.15.27:0/4212677927 >> 172.21.15.27:0/4212677927 conn(0x7f701c0793f0 msgr2=0x7f701c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:32.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.415+0000 7f70212f0700 1 -- 172.21.15.27:0/4212677927 shutdown_connections 2024-05-21T03:21:32.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.415+0000 7f70212f0700 1 -- 172.21.15.27:0/4212677927 wait complete. 2024-05-21T03:21:32.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.416+0000 7f70212f0700 1 Processor -- start 2024-05-21T03:21:32.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.416+0000 7f70212f0700 1 -- start start 2024-05-21T03:21:32.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.416+0000 7f70212f0700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f701c06b220 0x7f701c19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:32.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.416+0000 7f70212f0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f701c06bbe0 0x7f701c19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:32.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.416+0000 7f70212f0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f701c1095b0 0x7f701c1999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:32.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.416+0000 7f70212f0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f701c1136f0 con 0x7f701c1095b0 2024-05-21T03:21:32.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.416+0000 7f70212f0700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f701c113570 con 0x7f701c06bbe0 2024-05-21T03:21:32.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.416+0000 7f70212f0700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f701c113870 con 0x7f701c06b220 2024-05-21T03:21:32.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.417+0000 7f701b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f701c1095b0 0x7f701c1999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:32.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.417+0000 7f701b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f701c1095b0 0x7f701c1999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39600/0 (socket says 172.21.15.27:39600) 2024-05-21T03:21:32.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.417+0000 7f701b7fe700 1 -- 172.21.15.27:0/2148900640 learned_addr learned my addr 172.21.15.27:0/2148900640 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:21:32.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.417+0000 7f701affd700 1 --2- 172.21.15.27:0/2148900640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f701c06b220 0x7f701c19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:32.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.417+0000 7f701a7fc700 1 --2- 172.21.15.27:0/2148900640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f701c06bbe0 0x7f701c19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:32.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.417+0000 7f701b7fe700 1 -- 172.21.15.27:0/2148900640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f701c06b220 msgr2=0x7f701c19f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:32.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.417+0000 7f701b7fe700 1 --2- 172.21.15.27:0/2148900640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f701c06b220 0x7f701c19f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:32.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.417+0000 7f701b7fe700 1 -- 172.21.15.27:0/2148900640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f701c06bbe0 msgr2=0x7f701c19fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:32.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.417+0000 7f701b7fe700 1 --2- 172.21.15.27:0/2148900640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f701c06bbe0 0x7f701c19fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:32.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.417+0000 7f701b7fe700 1 -- 172.21.15.27:0/2148900640 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701c19a150 con 0x7f701c1095b0 2024-05-21T03:21:32.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.417+0000 7f701b7fe700 1 --2- 172.21.15.27:0/2148900640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f701c1095b0 0x7f701c1999f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f70140047b0 tx=0x7f7014005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:32.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.417+0000 7f7013fff700 1 -- 172.21.15.27:0/2148900640 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7014026070 con 0x7f701c1095b0 2024-05-21T03:21:32.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.417+0000 7f7013fff700 1 -- 172.21.15.27:0/2148900640 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7014004c90 con 0x7f701c1095b0 2024-05-21T03:21:32.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.417+0000 7f70212f0700 1 -- 172.21.15.27:0/2148900640 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f701c19a3e0 con 0x7f701c1095b0 2024-05-21T03:21:32.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.417+0000 7f7013fff700 1 -- 172.21.15.27:0/2148900640 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7014003cd0 con 0x7f701c1095b0 2024-05-21T03:21:32.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.417+0000 7f70212f0700 1 -- 172.21.15.27:0/2148900640 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f701c1a66b0 con 0x7f701c1095b0 2024-05-21T03:21:32.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.418+0000 7f70212f0700 1 -- 172.21.15.27:0/2148900640 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f701c06a230 con 0x7f701c1095b0 2024-05-21T03:21:32.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.420+0000 7f7013fff700 1 -- 172.21.15.27:0/2148900640 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7014029070 con 0x7f701c1095b0 2024-05-21T03:21:32.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.420+0000 7f7013fff700 1 --2- 172.21.15.27:0/2148900640 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f70080b2910 0x7f70080b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:32.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.420+0000 7f7013fff700 1 -- 172.21.15.27:0/2148900640 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f70140e3000 con 0x7f701c1095b0 2024-05-21T03:21:32.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.421+0000 7f701affd700 1 --2- 172.21.15.27:0/2148900640 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f70080b2910 0x7f70080b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:32.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.421+0000 7f701affd700 1 --2- 172.21.15.27:0/2148900640 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f70080b2910 0x7f70080b4dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f700c009550 tx=0x7f700c0091f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:32.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.424+0000 7f7013fff700 1 -- 172.21.15.27:0/2148900640 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7014030020 con 0x7f701c1095b0 2024-05-21T03:21:32.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.634+0000 7f70212f0700 1 -- 172.21.15.27:0/2148900640 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f701c19aed0 con 0x7f70080b2910 2024-05-21T03:21:32.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.642+0000 7f7013fff700 1 -- 172.21.15.27:0/2148900640 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258762 (secure 0 0 0) 0x7f701c19aed0 con 0x7f70080b2910 2024-05-21T03:21:32.645 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:21:32.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.646+0000 7f70212f0700 1 -- 172.21.15.27:0/2148900640 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f70080b2910 msgr2=0x7f70080b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:32.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.646+0000 7f70212f0700 1 --2- 172.21.15.27:0/2148900640 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f70080b2910 0x7f70080b4dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f700c009550 tx=0x7f700c0091f0 comp rx=0 tx=0).stop 2024-05-21T03:21:32.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.646+0000 7f70212f0700 1 -- 172.21.15.27:0/2148900640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f701c1095b0 msgr2=0x7f701c1999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:32.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.646+0000 7f70212f0700 1 --2- 172.21.15.27:0/2148900640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f701c1095b0 0x7f701c1999f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f70140047b0 tx=0x7f7014005e80 comp rx=0 tx=0).stop 2024-05-21T03:21:32.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.646+0000 7f70212f0700 1 -- 172.21.15.27:0/2148900640 shutdown_connections 2024-05-21T03:21:32.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.646+0000 7f70212f0700 1 --2- 172.21.15.27:0/2148900640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f701c1095b0 0x7f701c1999f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:32.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.646+0000 7f70212f0700 1 --2- 172.21.15.27:0/2148900640 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f70080b2910 0x7f70080b4dd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:32.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.646+0000 7f70212f0700 1 --2- 172.21.15.27:0/2148900640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f701c06b220 0x7f701c19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:32.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.646+0000 7f70212f0700 1 --2- 172.21.15.27:0/2148900640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f701c06bbe0 0x7f701c19fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:32.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.646+0000 7f70212f0700 1 -- 172.21.15.27:0/2148900640 >> 172.21.15.27:0/2148900640 conn(0x7f701c0793f0 msgr2=0x7f701c101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:32.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.646+0000 7f70212f0700 1 -- 172.21.15.27:0/2148900640 shutdown_connections 2024-05-21T03:21:32.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:32.646+0000 7f70212f0700 1 -- 172.21.15.27:0/2148900640 wait complete. 2024-05-21T03:21:32.648 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-05-21T03:21:33.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:32 smithi027 ceph-mon[32179]: pgmap v321: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:21:33.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:32 smithi039 ceph-mon[36429]: pgmap v321: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:21:33.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:32 smithi165 ceph-mon[36269]: pgmap v321: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:21:33.375 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-05-21T03:21:31.347805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657959,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":481,"num_read_kb":356,"num_write":369,"num_write_kb":2287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2953518,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357800,"kb_used_data":12732,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584024,"statfs":{"total":1151940427776,"available":1151574040576,"internally_reserved":0,"allocated":13037568,"data_stored":7161081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19139,"internal_metadata":353154365},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002751"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.554128+0000","last_change":"2024-05-21T03:20:32.149961+0000","last_active":"2024-05-21T03:21:14.554128+0000","last_peered":"2024-05-21T03:21:14.554128+0000","last_clean":"2024-05-21T03:21:14.554128+0000","last_became_active":"2024-05-21T03:20:32.149552+0000","last_became_peered":"2024-05-21T03:20:32.149552+0000","last_unstale":"2024-05-21T03:21:14.554128+0000","last_undegraded":"2024-05-21T03:21:14.554128+0000","last_fullsized":"2024-05-21T03:21:14.554128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:20:38.543551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556541+0000","last_change":"2024-05-21T03:20:32.146413+0000","last_active":"2024-05-21T03:21:14.556541+0000","last_peered":"2024-05-21T03:21:14.556541+0000","last_clean":"2024-05-21T03:21:14.556541+0000","last_became_active":"2024-05-21T03:20:32.146006+0000","last_became_peered":"2024-05-21T03:20:32.146006+0000","last_unstale":"2024-05-21T03:21:14.556541+0000","last_undegraded":"2024-05-21T03:21:14.556541+0000","last_fullsized":"2024-05-21T03:21:14.556541+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:05:12.888550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556573+0000","last_change":"2024-05-21T03:20:32.153654+0000","last_active":"2024-05-21T03:21:14.556573+0000","last_peered":"2024-05-21T03:21:14.556573+0000","last_clean":"2024-05-21T03:21:14.556573+0000","last_became_active":"2024-05-21T03:20:32.153296+0000","last_became_peered":"2024-05-21T03:20:32.153296+0000","last_unstale":"2024-05-21T03:21:14.556573+0000","last_undegraded":"2024-05-21T03:21:14.556573+0000","last_fullsized":"2024-05-21T03:21:14.556573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:16:09.423236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556628+0000","last_change":"2024-05-21T03:20:32.138672+0000","last_active":"2024-05-21T03:21:14.556628+0000","last_peered":"2024-05-21T03:21:14.556628+0000","last_clean":"2024-05-21T03:21:14.556628+0000","last_became_active":"2024-05-21T03:20:32.138163+0000","last_became_peered":"2024-05-21T03:20:32.138163+0000","last_unstale":"2024-05-21T03:21:14.556628+0000","last_undegraded":"2024-05-21T03:21:14.556628+0000","last_fullsized":"2024-05-21T03:21:14.556628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:26:55.012897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556450+0000","last_change":"2024-05-21T03:20:32.153950+0000","last_active":"2024-05-21T03:21:14.556450+0000","last_peered":"2024-05-21T03:21:14.556450+0000","last_clean":"2024-05-21T03:21:14.556450+0000","last_became_active":"2024-05-21T03:20:32.153567+0000","last_became_peered":"2024-05-21T03:20:32.153567+0000","last_unstale":"2024-05-21T03:21:14.556450+0000","last_undegraded":"2024-05-21T03:21:14.556450+0000","last_fullsized":"2024-05-21T03:21:14.556450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:28:34.621193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.554034+0000","last_change":"2024-05-21T03:20:32.140038+0000","last_active":"2024-05-21T03:21:14.554034+0000","last_peered":"2024-05-21T03:21:14.554034+0000","last_clean":"2024-05-21T03:21:14.554034+0000","last_became_active":"2024-05-21T03:20:32.139690+0000","last_became_peered":"2024-05-21T03:20:32.139690+0000","last_unstale":"2024-05-21T03:21:14.554034+0000","last_undegraded":"2024-05-21T03:21:14.554034+0000","last_fullsized":"2024-05-21T03:21:14.554034+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:31:24.270755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556482+0000","last_change":"2024-05-21T03:20:32.143735+0000","last_active":"2024-05-21T03:21:14.556482+0000","last_peered":"2024-05-21T03:21:14.556482+0000","last_clean":"2024-05-21T03:21:14.556482+0000","last_became_active":"2024-05-21T03:20:32.143397+0000","last_became_peered":"2024-05-21T03:20:32.143397+0000","last_unstale":"2024-05-21T03:21:14.556482+0000","last_undegraded":"2024-05-21T03:21:14.556482+0000","last_fullsized":"2024-05-21T03:21:14.556482+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:40:01.951608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.560831+0000","last_change":"2024-05-21T03:20:32.156999+0000","last_active":"2024-05-21T03:21:14.560831+0000","last_peered":"2024-05-21T03:21:14.560831+0000","last_clean":"2024-05-21T03:21:14.560831+0000","last_became_active":"2024-05-21T03:20:32.156677+0000","last_became_peered":"2024-05-21T03:20:32.156677+0000","last_unstale":"2024-05-21T03:21:14.560831+0000","last_undegraded":"2024-05-21T03:21:14.560831+0000","last_fullsized":"2024-05-21T03:21:14.560831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:33:24.225667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558931+0000","last_change":"2024-05-21T03:20:32.145039+0000","last_active":"2024-05-21T03:21:14.558931+0000","last_peered":"2024-05-21T03:21:14.558931+0000","last_clean":"2024-05-21T03:21:14.558931+0000","last_became_active":"2024-05-21T03:20:32.140926+0000","last_became_peered":"2024-05-21T03:20:32.140926+0000","last_unstale":"2024-05-21T03:21:14.558931+0000","last_undegraded":"2024-05-21T03:21:14.558931+0000","last_fullsized":"2024-05-21T03:21:14.558931+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:11:27.049426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.554577+0000","last_change":"2024-05-21T03:20:32.145840+0000","last_active":"2024-05-21T03:21:14.554577+0000","last_peered":"2024-05-21T03:21:14.554577+0000","last_clean":"2024-05-21T03:21:14.554577+0000","last_became_active":"2024-05-21T03:20:32.145291+0000","last_became_peered":"2024-05-21T03:20:32.145291+0000","last_unstale":"2024-05-21T03:21:14.554577+0000","last_undegraded":"2024-05-21T03:21:14.554577+0000","last_fullsized":"2024-05-21T03:21:14.554577+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:22:27.696588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558336+0000","last_change":"2024-05-21T03:20:32.142522+0000","last_active":"2024-05-21T03:21:14.558336+0000","last_peered":"2024-05-21T03:21:14.558336+0000","last_clean":"2024-05-21T03:21:14.558336+0000","last_became_active":"2024-05-21T03:20:32.142165+0000","last_became_peered":"2024-05-21T03:20:32.142165+0000","last_unstale":"2024-05-21T03:21:14.558336+0000","last_undegraded":"2024-05-21T03:21:14.558336+0000","last_fullsized":"2024-05-21T03:21:14.558336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:58:39.085834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556536+0000","last_change":"2024-05-21T03:20:32.142526+0000","last_active":"2024-05-21T03:21:14.556536+0000","last_peered":"2024-05-21T03:21:14.556536+0000","last_clean":"2024-05-21T03:21:14.556536+0000","last_became_active":"2024-05-21T03:20:32.141143+0000","last_became_peered":"2024-05-21T03:20:32.141143+0000","last_unstale":"2024-05-21T03:21:14.556536+0000","last_undegraded":"2024-05-21T03:21:14.556536+0000","last_fullsized":"2024-05-21T03:21:14.556536+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:35:11.336796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552998+0000","last_change":"2024-05-21T03:20:32.135631+0000","last_active":"2024-05-21T03:21:14.552998+0000","last_peered":"2024-05-21T03:21:14.552998+0000","last_clean":"2024-05-21T03:21:14.552998+0000","last_became_active":"2024-05-21T03:20:32.135302+0000","last_became_peered":"2024-05-21T03:20:32.135302+0000","last_unstale":"2024-05-21T03:21:14.552998+0000","last_undegraded":"2024-05-21T03:21:14.552998+0000","last_fullsized":"2024-05-21T03:21:14.552998+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:05:55.000690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.555811+0000","last_change":"2024-05-21T03:20:32.146580+0000","last_active":"2024-05-21T03:21:14.555811+0000","last_peered":"2024-05-21T03:21:14.555811+0000","last_clean":"2024-05-21T03:21:14.555811+0000","last_became_active":"2024-05-21T03:20:32.145803+0000","last_became_peered":"2024-05-21T03:20:32.145803+0000","last_unstale":"2024-05-21T03:21:14.555811+0000","last_undegraded":"2024-05-21T03:21:14.555811+0000","last_fullsized":"2024-05-21T03:21:14.555811+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:34:09.108799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.559498+0000","last_change":"2024-05-21T03:21:14.559498+0000","last_active":"2024-05-21T03:21:14.559498+0000","last_peered":"2024-05-21T03:21:14.559498+0000","last_clean":"2024-05-21T03:21:14.559498+0000","last_became_active":"2024-05-21T03:21:14.559039+0000","last_became_peered":"2024-05-21T03:21:14.559039+0000","last_unstale":"2024-05-21T03:21:14.559498+0000","last_undegraded":"2024-05-21T03:21:14.559498+0000","last_fullsized":"2024-05-21T03:21:14.559498+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:04:36.676857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.557368+0000","last_change":"2024-05-21T03:20:32.148944+0000","last_active":"2024-05-21T03:21:14.557368+0000","last_peered":"2024-05-21T03:21:14.557368+0000","last_clean":"2024-05-21T03:21:14.557368+0000","last_became_active":"2024-05-21T03:20:32.148502+0000","last_became_peered":"2024-05-21T03:20:32.148502+0000","last_unstale":"2024-05-21T03:21:14.557368+0000","last_undegraded":"2024-05-21T03:21:14.557368+0000","last_fullsized":"2024-05-21T03:21:14.557368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:11:00.178105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.554478+0000","last_change":"2024-05-21T03:20:32.150609+0000","last_active":"2024-05-21T03:21:14.554478+0000","last_peered":"2024-05-21T03:21:14.554478+0000","last_clean":"2024-05-21T03:21:14.554478+0000","last_became_active":"2024-05-21T03:20:32.150314+0000","last_became_peered":"2024-05-21T03:20:32.150314+0000","last_unstale":"2024-05-21T03:21:14.554478+0000","last_undegraded":"2024-05-21T03:21:14.554478+0000","last_fullsized":"2024-05-21T03:21:14.554478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:34:21.869304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556008+0000","last_change":"2024-05-21T03:20:32.142472+0000","last_active":"2024-05-21T03:21:14.556008+0000","last_peered":"2024-05-21T03:21:14.556008+0000","last_clean":"2024-05-21T03:21:14.556008+0000","last_became_active":"2024-05-21T03:20:32.142161+0000","last_became_peered":"2024-05-21T03:20:32.142161+0000","last_unstale":"2024-05-21T03:21:14.556008+0000","last_undegraded":"2024-05-21T03:21:14.556008+0000","last_fullsized":"2024-05-21T03:21:14.556008+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:26:57.281655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.557572+0000","last_change":"2024-05-21T03:20:32.153987+0000","last_active":"2024-05-21T03:21:14.557572+0000","last_peered":"2024-05-21T03:21:14.557572+0000","last_clean":"2024-05-21T03:21:14.557572+0000","last_became_active":"2024-05-21T03:20:32.153617+0000","last_became_peered":"2024-05-21T03:20:32.153617+0000","last_unstale":"2024-05-21T03:21:14.557572+0000","last_undegraded":"2024-05-21T03:21:14.557572+0000","last_fullsized":"2024-05-21T03:21:14.557572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:57:07.341758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":77,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:26.533962+0000","last_change":"2024-05-21T03:20:29.099928+0000","last_active":"2024-05-21T03:21:26.533962+0000","last_peered":"2024-05-21T03:21:26.533962+0000","last_clean":"2024-05-21T03:21:26.533962+0000","last_became_active":"2024-05-21T03:20:29.099592+0000","last_became_peered":"2024-05-21T03:20:29.099592+0000","last_unstale":"2024-05-21T03:21:26.533962+0000","last_undegraded":"2024-05-21T03:21:26.533962+0000","last_fullsized":"2024-05-21T03:21:26.533962+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:35:34.843325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561455+0000","last_change":"2024-05-21T03:20:32.154962+0000","last_active":"2024-05-21T03:21:14.561455+0000","last_peered":"2024-05-21T03:21:14.561455+0000","last_clean":"2024-05-21T03:21:14.561455+0000","last_became_active":"2024-05-21T03:20:32.154620+0000","last_became_peered":"2024-05-21T03:20:32.154620+0000","last_unstale":"2024-05-21T03:21:14.561455+0000","last_undegraded":"2024-05-21T03:21:14.561455+0000","last_fullsized":"2024-05-21T03:21:14.561455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:56:21.553272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561738+0000","last_change":"2024-05-21T03:20:29.097312+0000","last_active":"2024-05-21T03:21:14.561738+0000","last_peered":"2024-05-21T03:21:14.561738+0000","last_clean":"2024-05-21T03:21:14.561738+0000","last_became_active":"2024-05-21T03:20:29.096977+0000","last_became_peered":"2024-05-21T03:20:29.096977+0000","last_unstale":"2024-05-21T03:21:14.561738+0000","last_undegraded":"2024-05-21T03:21:14.561738+0000","last_fullsized":"2024-05-21T03:21:14.561738+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:28:45.467446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556973+0000","last_change":"2024-05-21T03:20:32.150035+0000","last_active":"2024-05-21T03:21:14.556973+0000","last_peered":"2024-05-21T03:21:14.556973+0000","last_clean":"2024-05-21T03:21:14.556973+0000","last_became_active":"2024-05-21T03:20:32.149688+0000","last_became_peered":"2024-05-21T03:20:32.149688+0000","last_unstale":"2024-05-21T03:21:14.556973+0000","last_undegraded":"2024-05-21T03:21:14.556973+0000","last_fullsized":"2024-05-21T03:21:14.556973+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:27:00.410770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.573309+0000","last_change":"2024-05-21T03:21:14.572832+0000","last_active":"2024-05-21T03:21:14.573309+0000","last_peered":"2024-05-21T03:21:14.573309+0000","last_clean":"2024-05-21T03:21:14.573309+0000","last_became_active":"2024-05-21T03:21:14.559381+0000","last_became_peered":"2024-05-21T03:21:14.559381+0000","last_unstale":"2024-05-21T03:21:14.573309+0000","last_undegraded":"2024-05-21T03:21:14.573309+0000","last_fullsized":"2024-05-21T03:21:14.573309+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:02:53.004031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558214+0000","last_change":"2024-05-21T03:20:32.150301+0000","last_active":"2024-05-21T03:21:14.558214+0000","last_peered":"2024-05-21T03:21:14.558214+0000","last_clean":"2024-05-21T03:21:14.558214+0000","last_became_active":"2024-05-21T03:20:32.148475+0000","last_became_peered":"2024-05-21T03:20:32.148475+0000","last_unstale":"2024-05-21T03:21:14.558214+0000","last_undegraded":"2024-05-21T03:21:14.558214+0000","last_fullsized":"2024-05-21T03:21:14.558214+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:59:30.339660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556899+0000","last_change":"2024-05-21T03:20:29.098861+0000","last_active":"2024-05-21T03:21:14.556899+0000","last_peered":"2024-05-21T03:21:14.556899+0000","last_clean":"2024-05-21T03:21:14.556899+0000","last_became_active":"2024-05-21T03:20:29.097874+0000","last_became_peered":"2024-05-21T03:20:29.097874+0000","last_unstale":"2024-05-21T03:21:14.556899+0000","last_undegraded":"2024-05-21T03:21:14.556899+0000","last_fullsized":"2024-05-21T03:21:14.556899+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:25:36.227339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561801+0000","last_change":"2024-05-21T03:20:32.142820+0000","last_active":"2024-05-21T03:21:14.561801+0000","last_peered":"2024-05-21T03:21:14.561801+0000","last_clean":"2024-05-21T03:21:14.561801+0000","last_became_active":"2024-05-21T03:20:32.142396+0000","last_became_peered":"2024-05-21T03:20:32.142396+0000","last_unstale":"2024-05-21T03:21:14.561801+0000","last_undegraded":"2024-05-21T03:21:14.561801+0000","last_fullsized":"2024-05-21T03:21:14.561801+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:30:14.452389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:25.534110+0000","last_change":"2024-05-21T03:20:29.103153+0000","last_active":"2024-05-21T03:21:25.534110+0000","last_peered":"2024-05-21T03:21:25.534110+0000","last_clean":"2024-05-21T03:21:25.534110+0000","last_became_active":"2024-05-21T03:20:29.102819+0000","last_became_peered":"2024-05-21T03:20:29.102819+0000","last_unstale":"2024-05-21T03:21:25.534110+0000","last_undegraded":"2024-05-21T03:21:25.534110+0000","last_fullsized":"2024-05-21T03:21:25.534110+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:47:49.843926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556528+0000","last_change":"2024-05-21T03:20:32.142263+0000","last_active":"2024-05-21T03:21:14.556528+0000","last_peered":"2024-05-21T03:21:14.556528+0000","last_clean":"2024-05-21T03:21:14.556528+0000","last_became_active":"2024-05-21T03:20:32.141820+0000","last_became_peered":"2024-05-21T03:20:32.141820+0000","last_unstale":"2024-05-21T03:21:14.556528+0000","last_undegraded":"2024-05-21T03:21:14.556528+0000","last_fullsized":"2024-05-21T03:21:14.556528+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:45:20.021237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.554011+0000","last_change":"2024-05-21T03:20:29.100088+0000","last_active":"2024-05-21T03:21:14.554011+0000","last_peered":"2024-05-21T03:21:14.554011+0000","last_clean":"2024-05-21T03:21:14.554011+0000","last_became_active":"2024-05-21T03:20:29.097343+0000","last_became_peered":"2024-05-21T03:20:29.097343+0000","last_unstale":"2024-05-21T03:21:14.554011+0000","last_undegraded":"2024-05-21T03:21:14.554011+0000","last_fullsized":"2024-05-21T03:21:14.554011+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:00:35.682393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.562687+0000","last_change":"2024-05-21T03:21:14.562281+0000","last_active":"2024-05-21T03:21:14.562687+0000","last_peered":"2024-05-21T03:21:14.562687+0000","last_clean":"2024-05-21T03:21:14.562687+0000","last_became_active":"2024-05-21T03:21:14.561857+0000","last_became_peered":"2024-05-21T03:21:14.561857+0000","last_unstale":"2024-05-21T03:21:14.562687+0000","last_undegraded":"2024-05-21T03:21:14.562687+0000","last_fullsized":"2024-05-21T03:21:14.562687+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:37:35.283529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552729+0000","last_change":"2024-05-21T03:20:29.097945+0000","last_active":"2024-05-21T03:21:14.552729+0000","last_peered":"2024-05-21T03:21:14.552729+0000","last_clean":"2024-05-21T03:21:14.552729+0000","last_became_active":"2024-05-21T03:20:29.097624+0000","last_became_peered":"2024-05-21T03:20:29.097624+0000","last_unstale":"2024-05-21T03:21:14.552729+0000","last_undegraded":"2024-05-21T03:21:14.552729+0000","last_fullsized":"2024-05-21T03:21:14.552729+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:01:01.694893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.560719+0000","last_change":"2024-05-21T03:20:32.148955+0000","last_active":"2024-05-21T03:21:14.560719+0000","last_peered":"2024-05-21T03:21:14.560719+0000","last_clean":"2024-05-21T03:21:14.560719+0000","last_became_active":"2024-05-21T03:20:32.148631+0000","last_became_peered":"2024-05-21T03:20:32.148631+0000","last_unstale":"2024-05-21T03:21:14.560719+0000","last_undegraded":"2024-05-21T03:21:14.560719+0000","last_fullsized":"2024-05-21T03:21:14.560719+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:36:25.820580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561173+0000","last_change":"2024-05-21T03:20:29.100902+0000","last_active":"2024-05-21T03:21:14.561173+0000","last_peered":"2024-05-21T03:21:14.561173+0000","last_clean":"2024-05-21T03:21:14.561173+0000","last_became_active":"2024-05-21T03:20:29.100570+0000","last_became_peered":"2024-05-21T03:20:29.100570+0000","last_unstale":"2024-05-21T03:21:14.561173+0000","last_undegraded":"2024-05-21T03:21:14.561173+0000","last_fullsized":"2024-05-21T03:21:14.561173+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:58:36.651338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.555588+0000","last_change":"2024-05-21T03:20:32.150724+0000","last_active":"2024-05-21T03:21:14.555588+0000","last_peered":"2024-05-21T03:21:14.555588+0000","last_clean":"2024-05-21T03:21:14.555588+0000","last_became_active":"2024-05-21T03:20:32.150371+0000","last_became_peered":"2024-05-21T03:20:32.150371+0000","last_unstale":"2024-05-21T03:21:14.555588+0000","last_undegraded":"2024-05-21T03:21:14.555588+0000","last_fullsized":"2024-05-21T03:21:14.555588+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:44:32.321192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":107,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:27.580293+0000","last_change":"2024-05-21T03:20:29.095045+0000","last_active":"2024-05-21T03:21:27.580293+0000","last_peered":"2024-05-21T03:21:27.580293+0000","last_clean":"2024-05-21T03:21:27.580293+0000","last_became_active":"2024-05-21T03:20:29.094552+0000","last_became_peered":"2024-05-21T03:20:29.094552+0000","last_unstale":"2024-05-21T03:21:27.580293+0000","last_undegraded":"2024-05-21T03:21:27.580293+0000","last_fullsized":"2024-05-21T03:21:27.580293+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:36:15.155125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":77,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558897+0000","last_change":"2024-05-21T03:20:32.149813+0000","last_active":"2024-05-21T03:21:14.558897+0000","last_peered":"2024-05-21T03:21:14.558897+0000","last_clean":"2024-05-21T03:21:14.558897+0000","last_became_active":"2024-05-21T03:20:32.148422+0000","last_became_peered":"2024-05-21T03:20:32.148422+0000","last_unstale":"2024-05-21T03:21:14.558897+0000","last_undegraded":"2024-05-21T03:21:14.558897+0000","last_fullsized":"2024-05-21T03:21:14.558897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:55:35.115463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558265+0000","last_change":"2024-05-21T03:20:29.101463+0000","last_active":"2024-05-21T03:21:14.558265+0000","last_peered":"2024-05-21T03:21:14.558265+0000","last_clean":"2024-05-21T03:21:14.558265+0000","last_became_active":"2024-05-21T03:20:29.101127+0000","last_became_peered":"2024-05-21T03:20:29.101127+0000","last_unstale":"2024-05-21T03:21:14.558265+0000","last_undegraded":"2024-05-21T03:21:14.558265+0000","last_fullsized":"2024-05-21T03:21:14.558265+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:58:17.408315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.563434+0000","last_change":"2024-05-21T03:21:14.563434+0000","last_active":"2024-05-21T03:21:14.563434+0000","last_peered":"2024-05-21T03:21:14.563434+0000","last_clean":"2024-05-21T03:21:14.563434+0000","last_became_active":"2024-05-21T03:21:14.563081+0000","last_became_peered":"2024-05-21T03:21:14.563081+0000","last_unstale":"2024-05-21T03:21:14.563434+0000","last_undegraded":"2024-05-21T03:21:14.563434+0000","last_fullsized":"2024-05-21T03:21:14.563434+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:28:55.497338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.562216+0000","last_change":"2024-05-21T03:20:29.098725+0000","last_active":"2024-05-21T03:21:14.562216+0000","last_peered":"2024-05-21T03:21:14.562216+0000","last_clean":"2024-05-21T03:21:14.562216+0000","last_became_active":"2024-05-21T03:20:29.098471+0000","last_became_peered":"2024-05-21T03:20:29.098471+0000","last_unstale":"2024-05-21T03:21:14.562216+0000","last_undegraded":"2024-05-21T03:21:14.562216+0000","last_fullsized":"2024-05-21T03:21:14.562216+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:17:11.059825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.560975+0000","last_change":"2024-05-21T03:21:14.560975+0000","last_active":"2024-05-21T03:21:14.560975+0000","last_peered":"2024-05-21T03:21:14.560975+0000","last_clean":"2024-05-21T03:21:14.560975+0000","last_became_active":"2024-05-21T03:21:14.560528+0000","last_became_peered":"2024-05-21T03:21:14.560528+0000","last_unstale":"2024-05-21T03:21:14.560975+0000","last_undegraded":"2024-05-21T03:21:14.560975+0000","last_fullsized":"2024-05-21T03:21:14.560975+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:51:47.469291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'31","reported_seq":144,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:28.537243+0000","last_change":"2024-05-21T03:20:29.099968+0000","last_active":"2024-05-21T03:21:28.537243+0000","last_peered":"2024-05-21T03:21:28.537243+0000","last_clean":"2024-05-21T03:21:28.537243+0000","last_became_active":"2024-05-21T03:20:29.099181+0000","last_became_peered":"2024-05-21T03:20:29.099181+0000","last_unstale":"2024-05-21T03:21:28.537243+0000","last_undegraded":"2024-05-21T03:21:28.537243+0000","last_fullsized":"2024-05-21T03:21:28.537243+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:35:33.934412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59762,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":87,"num_read_kb":101,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556604+0000","last_change":"2024-05-21T03:20:32.144024+0000","last_active":"2024-05-21T03:21:14.556604+0000","last_peered":"2024-05-21T03:21:14.556604+0000","last_clean":"2024-05-21T03:21:14.556604+0000","last_became_active":"2024-05-21T03:20:32.143668+0000","last_became_peered":"2024-05-21T03:20:32.143668+0000","last_unstale":"2024-05-21T03:21:14.556604+0000","last_undegraded":"2024-05-21T03:21:14.556604+0000","last_fullsized":"2024-05-21T03:21:14.556604+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:11:31.100802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556933+0000","last_change":"2024-05-21T03:20:29.101770+0000","last_active":"2024-05-21T03:21:14.556933+0000","last_peered":"2024-05-21T03:21:14.556933+0000","last_clean":"2024-05-21T03:21:14.556933+0000","last_became_active":"2024-05-21T03:20:29.100407+0000","last_became_peered":"2024-05-21T03:20:29.100407+0000","last_unstale":"2024-05-21T03:21:14.556933+0000","last_undegraded":"2024-05-21T03:21:14.556933+0000","last_fullsized":"2024-05-21T03:21:14.556933+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:01:33.991246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552656+0000","last_change":"2024-05-21T03:20:32.147479+0000","last_active":"2024-05-21T03:21:14.552656+0000","last_peered":"2024-05-21T03:21:14.552656+0000","last_clean":"2024-05-21T03:21:14.552656+0000","last_became_active":"2024-05-21T03:20:32.147160+0000","last_became_peered":"2024-05-21T03:20:32.147160+0000","last_unstale":"2024-05-21T03:21:14.552656+0000","last_undegraded":"2024-05-21T03:21:14.552656+0000","last_fullsized":"2024-05-21T03:21:14.552656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:51:59.343380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.562216+0000","last_change":"2024-05-21T03:20:29.101875+0000","last_active":"2024-05-21T03:21:14.562216+0000","last_peered":"2024-05-21T03:21:14.562216+0000","last_clean":"2024-05-21T03:21:14.562216+0000","last_became_active":"2024-05-21T03:20:29.101468+0000","last_became_peered":"2024-05-21T03:20:29.101468+0000","last_unstale":"2024-05-21T03:21:14.562216+0000","last_undegraded":"2024-05-21T03:21:14.562216+0000","last_fullsized":"2024-05-21T03:21:14.562216+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:10:19.577836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.557393+0000","last_change":"2024-05-21T03:20:32.139002+0000","last_active":"2024-05-21T03:21:14.557393+0000","last_peered":"2024-05-21T03:21:14.557393+0000","last_clean":"2024-05-21T03:21:14.557393+0000","last_became_active":"2024-05-21T03:20:32.138619+0000","last_became_peered":"2024-05-21T03:20:32.138619+0000","last_unstale":"2024-05-21T03:21:14.557393+0000","last_undegraded":"2024-05-21T03:21:14.557393+0000","last_fullsized":"2024-05-21T03:21:14.557393+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:21:36.136283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":71,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:25.533306+0000","last_change":"2024-05-21T03:20:29.101532+0000","last_active":"2024-05-21T03:21:25.533306+0000","last_peered":"2024-05-21T03:21:25.533306+0000","last_clean":"2024-05-21T03:21:25.533306+0000","last_became_active":"2024-05-21T03:20:29.101206+0000","last_became_peered":"2024-05-21T03:20:29.101206+0000","last_unstale":"2024-05-21T03:21:25.533306+0000","last_undegraded":"2024-05-21T03:21:25.533306+0000","last_fullsized":"2024-05-21T03:21:25.533306+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:47:38.555995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":41,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556597+0000","last_change":"2024-05-21T03:20:32.146866+0000","last_active":"2024-05-21T03:21:14.556597+0000","last_peered":"2024-05-21T03:21:14.556597+0000","last_clean":"2024-05-21T03:21:14.556597+0000","last_became_active":"2024-05-21T03:20:32.146463+0000","last_became_peered":"2024-05-21T03:20:32.146463+0000","last_unstale":"2024-05-21T03:21:14.556597+0000","last_undegraded":"2024-05-21T03:21:14.556597+0000","last_fullsized":"2024-05-21T03:21:14.556597+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:06:54.359922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556379+0000","last_change":"2024-05-21T03:20:29.103910+0000","last_active":"2024-05-21T03:21:14.556379+0000","last_peered":"2024-05-21T03:21:14.556379+0000","last_clean":"2024-05-21T03:21:14.556379+0000","last_became_active":"2024-05-21T03:20:29.103583+0000","last_became_peered":"2024-05-21T03:20:29.103583+0000","last_unstale":"2024-05-21T03:21:14.556379+0000","last_undegraded":"2024-05-21T03:21:14.556379+0000","last_fullsized":"2024-05-21T03:21:14.556379+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:57:15.665158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561420+0000","last_change":"2024-05-21T03:20:32.141242+0000","last_active":"2024-05-21T03:21:14.561420+0000","last_peered":"2024-05-21T03:21:14.561420+0000","last_clean":"2024-05-21T03:21:14.561420+0000","last_became_active":"2024-05-21T03:20:32.140892+0000","last_became_peered":"2024-05-21T03:20:32.140892+0000","last_unstale":"2024-05-21T03:21:14.561420+0000","last_undegraded":"2024-05-21T03:21:14.561420+0000","last_fullsized":"2024-05-21T03:21:14.561420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:42:13.997708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.557026+0000","last_change":"2024-05-21T03:20:29.098973+0000","last_active":"2024-05-21T03:21:14.557026+0000","last_peered":"2024-05-21T03:21:14.557026+0000","last_clean":"2024-05-21T03:21:14.557026+0000","last_became_active":"2024-05-21T03:20:29.098694+0000","last_became_peered":"2024-05-21T03:20:29.098694+0000","last_unstale":"2024-05-21T03:21:14.557026+0000","last_undegraded":"2024-05-21T03:21:14.557026+0000","last_fullsized":"2024-05-21T03:21:14.557026+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:55:17.915820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.553963+0000","last_change":"2024-05-21T03:20:32.147465+0000","last_active":"2024-05-21T03:21:14.553963+0000","last_peered":"2024-05-21T03:21:14.553963+0000","last_clean":"2024-05-21T03:21:14.553963+0000","last_became_active":"2024-05-21T03:20:32.147073+0000","last_became_peered":"2024-05-21T03:20:32.147073+0000","last_unstale":"2024-05-21T03:21:14.553963+0000","last_undegraded":"2024-05-21T03:21:14.553963+0000","last_fullsized":"2024-05-21T03:21:14.553963+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:29:05.451370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.560568+0000","last_change":"2024-05-21T03:21:14.560568+0000","last_active":"2024-05-21T03:21:14.560568+0000","last_peered":"2024-05-21T03:21:14.560568+0000","last_clean":"2024-05-21T03:21:14.560568+0000","last_became_active":"2024-05-21T03:21:14.560093+0000","last_became_peered":"2024-05-21T03:21:14.560093+0000","last_unstale":"2024-05-21T03:21:14.560568+0000","last_undegraded":"2024-05-21T03:21:14.560568+0000","last_fullsized":"2024-05-21T03:21:14.560568+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:55:39.839942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.554395+0000","last_change":"2024-05-21T03:20:32.146456+0000","last_active":"2024-05-21T03:21:14.554395+0000","last_peered":"2024-05-21T03:21:14.554395+0000","last_clean":"2024-05-21T03:21:14.554395+0000","last_became_active":"2024-05-21T03:20:32.146105+0000","last_became_peered":"2024-05-21T03:20:32.146105+0000","last_unstale":"2024-05-21T03:21:14.554395+0000","last_undegraded":"2024-05-21T03:21:14.554395+0000","last_fullsized":"2024-05-21T03:21:14.554395+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:00:30.550178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'6","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:25.533172+0000","last_change":"2024-05-21T03:20:29.100262+0000","last_active":"2024-05-21T03:21:25.533172+0000","last_peered":"2024-05-21T03:21:25.533172+0000","last_clean":"2024-05-21T03:21:25.533172+0000","last_became_active":"2024-05-21T03:20:29.099917+0000","last_became_peered":"2024-05-21T03:20:29.099917+0000","last_unstale":"2024-05-21T03:21:25.533172+0000","last_undegraded":"2024-05-21T03:21:25.533172+0000","last_fullsized":"2024-05-21T03:21:25.533172+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:55:48.608341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.554083+0000","last_change":"2024-05-21T03:20:32.142776+0000","last_active":"2024-05-21T03:21:14.554083+0000","last_peered":"2024-05-21T03:21:14.554083+0000","last_clean":"2024-05-21T03:21:14.554083+0000","last_became_active":"2024-05-21T03:20:32.142430+0000","last_became_peered":"2024-05-21T03:20:32.142430+0000","last_unstale":"2024-05-21T03:21:14.554083+0000","last_undegraded":"2024-05-21T03:21:14.554083+0000","last_fullsized":"2024-05-21T03:21:14.554083+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:54:28.682343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561357+0000","last_change":"2024-05-21T03:20:32.139506+0000","last_active":"2024-05-21T03:21:14.561357+0000","last_peered":"2024-05-21T03:21:14.561357+0000","last_clean":"2024-05-21T03:21:14.561357+0000","last_became_active":"2024-05-21T03:20:32.138136+0000","last_became_peered":"2024-05-21T03:20:32.138136+0000","last_unstale":"2024-05-21T03:21:14.561357+0000","last_undegraded":"2024-05-21T03:21:14.561357+0000","last_fullsized":"2024-05-21T03:21:14.561357+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:31:24.996276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":73,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:29.538253+0000","last_change":"2024-05-21T03:20:29.096546+0000","last_active":"2024-05-21T03:21:29.538253+0000","last_peered":"2024-05-21T03:21:29.538253+0000","last_clean":"2024-05-21T03:21:29.538253+0000","last_became_active":"2024-05-21T03:20:29.096184+0000","last_became_peered":"2024-05-21T03:20:29.096184+0000","last_unstale":"2024-05-21T03:21:29.538253+0000","last_undegraded":"2024-05-21T03:21:29.538253+0000","last_fullsized":"2024-05-21T03:21:29.538253+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:09:12.832745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561399+0000","last_change":"2024-05-21T03:20:32.156231+0000","last_active":"2024-05-21T03:21:14.561399+0000","last_peered":"2024-05-21T03:21:14.561399+0000","last_clean":"2024-05-21T03:21:14.561399+0000","last_became_active":"2024-05-21T03:20:32.154035+0000","last_became_peered":"2024-05-21T03:20:32.154035+0000","last_unstale":"2024-05-21T03:21:14.561399+0000","last_undegraded":"2024-05-21T03:21:14.561399+0000","last_fullsized":"2024-05-21T03:21:14.561399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:30:53.820292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556130+0000","last_change":"2024-05-21T03:20:29.100422+0000","last_active":"2024-05-21T03:21:14.556130+0000","last_peered":"2024-05-21T03:21:14.556130+0000","last_clean":"2024-05-21T03:21:14.556130+0000","last_became_active":"2024-05-21T03:20:29.099993+0000","last_became_peered":"2024-05-21T03:20:29.099993+0000","last_unstale":"2024-05-21T03:21:14.556130+0000","last_undegraded":"2024-05-21T03:21:14.556130+0000","last_fullsized":"2024-05-21T03:21:14.556130+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:03:15.456447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":98,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556799+0000","last_change":"2024-05-21T03:19:05.647808+0000","last_active":"2024-05-21T03:21:14.556799+0000","last_peered":"2024-05-21T03:21:14.556799+0000","last_clean":"2024-05-21T03:21:14.556799+0000","last_became_active":"2024-05-21T03:19:05.641865+0000","last_became_peered":"2024-05-21T03:19:05.641865+0000","last_unstale":"2024-05-21T03:21:14.556799+0000","last_undegraded":"2024-05-21T03:21:14.556799+0000","last_fullsized":"2024-05-21T03:21:14.556799+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:15:42.008493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:15:42.008493+0000","last_clean_scrub_stamp":"2024-05-21T03:15:42.008493+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:26:58.040196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552599+0000","last_change":"2024-05-21T03:20:32.138182+0000","last_active":"2024-05-21T03:21:14.552599+0000","last_peered":"2024-05-21T03:21:14.552599+0000","last_clean":"2024-05-21T03:21:14.552599+0000","last_became_active":"2024-05-21T03:20:32.137842+0000","last_became_peered":"2024-05-21T03:20:32.137842+0000","last_unstale":"2024-05-21T03:21:14.552599+0000","last_undegraded":"2024-05-21T03:21:14.552599+0000","last_fullsized":"2024-05-21T03:21:14.552599+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:43:50.070160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556989+0000","last_change":"2024-05-21T03:20:29.104246+0000","last_active":"2024-05-21T03:21:14.556989+0000","last_peered":"2024-05-21T03:21:14.556989+0000","last_clean":"2024-05-21T03:21:14.556989+0000","last_became_active":"2024-05-21T03:20:29.103910+0000","last_became_peered":"2024-05-21T03:20:29.103910+0000","last_unstale":"2024-05-21T03:21:14.556989+0000","last_undegraded":"2024-05-21T03:21:14.556989+0000","last_fullsized":"2024-05-21T03:21:14.556989+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:39:28.029283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556715+0000","last_change":"2024-05-21T03:20:29.102020+0000","last_active":"2024-05-21T03:21:14.556715+0000","last_peered":"2024-05-21T03:21:14.556715+0000","last_clean":"2024-05-21T03:21:14.556715+0000","last_became_active":"2024-05-21T03:20:29.100429+0000","last_became_peered":"2024-05-21T03:20:29.100429+0000","last_unstale":"2024-05-21T03:21:14.556715+0000","last_undegraded":"2024-05-21T03:21:14.556715+0000","last_fullsized":"2024-05-21T03:21:14.556715+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:09:01.621074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552834+0000","last_change":"2024-05-21T03:20:32.137130+0000","last_active":"2024-05-21T03:21:14.552834+0000","last_peered":"2024-05-21T03:21:14.552834+0000","last_clean":"2024-05-21T03:21:14.552834+0000","last_became_active":"2024-05-21T03:20:32.136821+0000","last_became_peered":"2024-05-21T03:20:32.136821+0000","last_unstale":"2024-05-21T03:21:14.552834+0000","last_undegraded":"2024-05-21T03:21:14.552834+0000","last_fullsized":"2024-05-21T03:21:14.552834+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:53:41.290370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'14","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.555888+0000","last_change":"2024-05-21T03:20:29.099869+0000","last_active":"2024-05-21T03:21:14.555888+0000","last_peered":"2024-05-21T03:21:14.555888+0000","last_clean":"2024-05-21T03:21:14.555888+0000","last_became_active":"2024-05-21T03:20:29.099534+0000","last_became_peered":"2024-05-21T03:20:29.099534+0000","last_unstale":"2024-05-21T03:21:14.555888+0000","last_undegraded":"2024-05-21T03:21:14.555888+0000","last_fullsized":"2024-05-21T03:21:14.555888+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:01:11.281544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561631+0000","last_change":"2024-05-21T03:20:32.153921+0000","last_active":"2024-05-21T03:21:14.561631+0000","last_peered":"2024-05-21T03:21:14.561631+0000","last_clean":"2024-05-21T03:21:14.561631+0000","last_became_active":"2024-05-21T03:20:32.153594+0000","last_became_peered":"2024-05-21T03:20:32.153594+0000","last_unstale":"2024-05-21T03:21:14.561631+0000","last_undegraded":"2024-05-21T03:21:14.561631+0000","last_fullsized":"2024-05-21T03:21:14.561631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:27:59.426942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556915+0000","last_change":"2024-05-21T03:20:29.097399+0000","last_active":"2024-05-21T03:21:14.556915+0000","last_peered":"2024-05-21T03:21:14.556915+0000","last_clean":"2024-05-21T03:21:14.556915+0000","last_became_active":"2024-05-21T03:20:29.097010+0000","last_became_peered":"2024-05-21T03:20:29.097010+0000","last_unstale":"2024-05-21T03:21:14.556915+0000","last_undegraded":"2024-05-21T03:21:14.556915+0000","last_fullsized":"2024-05-21T03:21:14.556915+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:54:53.273947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.559609+0000","last_change":"2024-05-21T03:21:14.559609+0000","last_active":"2024-05-21T03:21:14.559609+0000","last_peered":"2024-05-21T03:21:14.559609+0000","last_clean":"2024-05-21T03:21:14.559609+0000","last_became_active":"2024-05-21T03:21:14.559218+0000","last_became_peered":"2024-05-21T03:21:14.559218+0000","last_unstale":"2024-05-21T03:21:14.559609+0000","last_undegraded":"2024-05-21T03:21:14.559609+0000","last_fullsized":"2024-05-21T03:21:14.559609+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:32:11.156037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,4],"acting":[0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.557892+0000","last_change":"2024-05-21T03:20:29.103526+0000","last_active":"2024-05-21T03:21:14.557892+0000","last_peered":"2024-05-21T03:21:14.557892+0000","last_clean":"2024-05-21T03:21:14.557892+0000","last_became_active":"2024-05-21T03:20:29.103067+0000","last_became_peered":"2024-05-21T03:20:29.103067+0000","last_unstale":"2024-05-21T03:21:14.557892+0000","last_undegraded":"2024-05-21T03:21:14.557892+0000","last_fullsized":"2024-05-21T03:21:14.557892+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:54:47.566949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556409+0000","last_change":"2024-05-21T03:20:32.141088+0000","last_active":"2024-05-21T03:21:14.556409+0000","last_peered":"2024-05-21T03:21:14.556409+0000","last_clean":"2024-05-21T03:21:14.556409+0000","last_became_active":"2024-05-21T03:20:32.140744+0000","last_became_peered":"2024-05-21T03:20:32.140744+0000","last_unstale":"2024-05-21T03:21:14.556409+0000","last_undegraded":"2024-05-21T03:21:14.556409+0000","last_fullsized":"2024-05-21T03:21:14.556409+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:57:35.591164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.560757+0000","last_change":"2024-05-21T03:20:29.093440+0000","last_active":"2024-05-21T03:21:14.560757+0000","last_peered":"2024-05-21T03:21:14.560757+0000","last_clean":"2024-05-21T03:21:14.560757+0000","last_became_active":"2024-05-21T03:20:29.092935+0000","last_became_peered":"2024-05-21T03:20:29.092935+0000","last_unstale":"2024-05-21T03:21:14.560757+0000","last_undegraded":"2024-05-21T03:21:14.560757+0000","last_fullsized":"2024-05-21T03:21:14.560757+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:18:00.690465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.553665+0000","last_change":"2024-05-21T03:20:32.143781+0000","last_active":"2024-05-21T03:21:14.553665+0000","last_peered":"2024-05-21T03:21:14.553665+0000","last_clean":"2024-05-21T03:21:14.553665+0000","last_became_active":"2024-05-21T03:20:32.143391+0000","last_became_peered":"2024-05-21T03:20:32.143391+0000","last_unstale":"2024-05-21T03:21:14.553665+0000","last_undegraded":"2024-05-21T03:21:14.553665+0000","last_fullsized":"2024-05-21T03:21:14.553665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:00:41.506561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556256+0000","last_change":"2024-05-21T03:20:29.101342+0000","last_active":"2024-05-21T03:21:14.556256+0000","last_peered":"2024-05-21T03:21:14.556256+0000","last_clean":"2024-05-21T03:21:14.556256+0000","last_became_active":"2024-05-21T03:20:29.100997+0000","last_became_peered":"2024-05-21T03:20:29.100997+0000","last_unstale":"2024-05-21T03:21:14.556256+0000","last_undegraded":"2024-05-21T03:21:14.556256+0000","last_fullsized":"2024-05-21T03:21:14.556256+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:05:31.801669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561184+0000","last_change":"2024-05-21T03:20:32.150563+0000","last_active":"2024-05-21T03:21:14.561184+0000","last_peered":"2024-05-21T03:21:14.561184+0000","last_clean":"2024-05-21T03:21:14.561184+0000","last_became_active":"2024-05-21T03:20:32.150231+0000","last_became_peered":"2024-05-21T03:20:32.150231+0000","last_unstale":"2024-05-21T03:21:14.561184+0000","last_undegraded":"2024-05-21T03:21:14.561184+0000","last_fullsized":"2024-05-21T03:21:14.561184+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:10:00.251284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556609+0000","last_change":"2024-05-21T03:20:29.101576+0000","last_active":"2024-05-21T03:21:14.556609+0000","last_peered":"2024-05-21T03:21:14.556609+0000","last_clean":"2024-05-21T03:21:14.556609+0000","last_became_active":"2024-05-21T03:20:29.101270+0000","last_became_peered":"2024-05-21T03:20:29.101270+0000","last_unstale":"2024-05-21T03:21:14.556609+0000","last_undegraded":"2024-05-21T03:21:14.556609+0000","last_fullsized":"2024-05-21T03:21:14.556609+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:50:10.579901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.557991+0000","last_change":"2024-05-21T03:20:32.141340+0000","last_active":"2024-05-21T03:21:14.557991+0000","last_peered":"2024-05-21T03:21:14.557991+0000","last_clean":"2024-05-21T03:21:14.557991+0000","last_became_active":"2024-05-21T03:20:32.140792+0000","last_became_peered":"2024-05-21T03:20:32.140792+0000","last_unstale":"2024-05-21T03:21:14.557991+0000","last_undegraded":"2024-05-21T03:21:14.557991+0000","last_fullsized":"2024-05-21T03:21:14.557991+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:35:05.842929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'6","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:25.577905+0000","last_change":"2024-05-21T03:21:14.566886+0000","last_active":"2024-05-21T03:21:25.577905+0000","last_peered":"2024-05-21T03:21:25.577905+0000","last_clean":"2024-05-21T03:21:25.577905+0000","last_became_active":"2024-05-21T03:21:14.559816+0000","last_became_peered":"2024-05-21T03:21:14.559816+0000","last_unstale":"2024-05-21T03:21:25.577905+0000","last_undegraded":"2024-05-21T03:21:25.577905+0000","last_fullsized":"2024-05-21T03:21:25.577905+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:34:43.109424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1561,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558407+0000","last_change":"2024-05-21T03:20:32.146674+0000","last_active":"2024-05-21T03:21:14.558407+0000","last_peered":"2024-05-21T03:21:14.558407+0000","last_clean":"2024-05-21T03:21:14.558407+0000","last_became_active":"2024-05-21T03:20:32.146150+0000","last_became_peered":"2024-05-21T03:20:32.146150+0000","last_unstale":"2024-05-21T03:21:14.558407+0000","last_undegraded":"2024-05-21T03:21:14.558407+0000","last_fullsized":"2024-05-21T03:21:14.558407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:39:09.905640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561667+0000","last_change":"2024-05-21T03:20:29.097613+0000","last_active":"2024-05-21T03:21:14.561667+0000","last_peered":"2024-05-21T03:21:14.561667+0000","last_clean":"2024-05-21T03:21:14.561667+0000","last_became_active":"2024-05-21T03:20:29.097331+0000","last_became_peered":"2024-05-21T03:20:29.097331+0000","last_unstale":"2024-05-21T03:21:14.561667+0000","last_undegraded":"2024-05-21T03:21:14.561667+0000","last_fullsized":"2024-05-21T03:21:14.561667+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:56:14.720420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.566406+0000","last_change":"2024-05-21T03:21:14.566406+0000","last_active":"2024-05-21T03:21:14.566406+0000","last_peered":"2024-05-21T03:21:14.566406+0000","last_clean":"2024-05-21T03:21:14.566406+0000","last_became_active":"2024-05-21T03:21:14.563023+0000","last_became_peered":"2024-05-21T03:21:14.563023+0000","last_unstale":"2024-05-21T03:21:14.566406+0000","last_undegraded":"2024-05-21T03:21:14.566406+0000","last_fullsized":"2024-05-21T03:21:14.566406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:12:53.125136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552818+0000","last_change":"2024-05-21T03:20:29.103456+0000","last_active":"2024-05-21T03:21:14.552818+0000","last_peered":"2024-05-21T03:21:14.552818+0000","last_clean":"2024-05-21T03:21:14.552818+0000","last_became_active":"2024-05-21T03:20:29.103008+0000","last_became_peered":"2024-05-21T03:20:29.103008+0000","last_unstale":"2024-05-21T03:21:14.552818+0000","last_undegraded":"2024-05-21T03:21:14.552818+0000","last_fullsized":"2024-05-21T03:21:14.552818+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:02:37.951347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558574+0000","last_change":"2024-05-21T03:20:32.146986+0000","last_active":"2024-05-21T03:21:14.558574+0000","last_peered":"2024-05-21T03:21:14.558574+0000","last_clean":"2024-05-21T03:21:14.558574+0000","last_became_active":"2024-05-21T03:20:32.146355+0000","last_became_peered":"2024-05-21T03:20:32.146355+0000","last_unstale":"2024-05-21T03:21:14.558574+0000","last_undegraded":"2024-05-21T03:21:14.558574+0000","last_fullsized":"2024-05-21T03:21:14.558574+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:36:29.503099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.559658+0000","last_change":"2024-05-21T03:21:14.559658+0000","last_active":"2024-05-21T03:21:14.559658+0000","last_peered":"2024-05-21T03:21:14.559658+0000","last_clean":"2024-05-21T03:21:14.559658+0000","last_became_active":"2024-05-21T03:21:14.559194+0000","last_became_peered":"2024-05-21T03:21:14.559194+0000","last_unstale":"2024-05-21T03:21:14.559658+0000","last_undegraded":"2024-05-21T03:21:14.559658+0000","last_fullsized":"2024-05-21T03:21:14.559658+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:56:31.709545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556799+0000","last_change":"2024-05-21T03:20:32.137532+0000","last_active":"2024-05-21T03:21:14.556799+0000","last_peered":"2024-05-21T03:21:14.556799+0000","last_clean":"2024-05-21T03:21:14.556799+0000","last_became_active":"2024-05-21T03:20:32.137182+0000","last_became_peered":"2024-05-21T03:20:32.137182+0000","last_unstale":"2024-05-21T03:21:14.556799+0000","last_undegraded":"2024-05-21T03:21:14.556799+0000","last_fullsized":"2024-05-21T03:21:14.556799+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:25:44.207866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556286+0000","last_change":"2024-05-21T03:20:32.153912+0000","last_active":"2024-05-21T03:21:14.556286+0000","last_peered":"2024-05-21T03:21:14.556286+0000","last_clean":"2024-05-21T03:21:14.556286+0000","last_became_active":"2024-05-21T03:20:32.153541+0000","last_became_peered":"2024-05-21T03:20:32.153541+0000","last_unstale":"2024-05-21T03:21:14.556286+0000","last_undegraded":"2024-05-21T03:21:14.556286+0000","last_fullsized":"2024-05-21T03:21:14.556286+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:02:58.688378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556461+0000","last_change":"2024-05-21T03:20:32.143689+0000","last_active":"2024-05-21T03:21:14.556461+0000","last_peered":"2024-05-21T03:21:14.556461+0000","last_clean":"2024-05-21T03:21:14.556461+0000","last_became_active":"2024-05-21T03:20:32.143399+0000","last_became_peered":"2024-05-21T03:20:32.143399+0000","last_unstale":"2024-05-21T03:21:14.556461+0000","last_undegraded":"2024-05-21T03:21:14.556461+0000","last_fullsized":"2024-05-21T03:21:14.556461+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:27:55.116599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552907+0000","last_change":"2024-05-21T03:20:32.141355+0000","last_active":"2024-05-21T03:21:14.552907+0000","last_peered":"2024-05-21T03:21:14.552907+0000","last_clean":"2024-05-21T03:21:14.552907+0000","last_became_active":"2024-05-21T03:20:32.140960+0000","last_became_peered":"2024-05-21T03:20:32.140960+0000","last_unstale":"2024-05-21T03:21:14.552907+0000","last_undegraded":"2024-05-21T03:21:14.552907+0000","last_fullsized":"2024-05-21T03:21:14.552907+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:23:50.287864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.553971+0000","last_change":"2024-05-21T03:20:32.142145+0000","last_active":"2024-05-21T03:21:14.553971+0000","last_peered":"2024-05-21T03:21:14.553971+0000","last_clean":"2024-05-21T03:21:14.553971+0000","last_became_active":"2024-05-21T03:20:32.141802+0000","last_became_peered":"2024-05-21T03:20:32.141802+0000","last_unstale":"2024-05-21T03:21:14.553971+0000","last_undegraded":"2024-05-21T03:21:14.553971+0000","last_fullsized":"2024-05-21T03:21:14.553971+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:26:28.707616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556601+0000","last_change":"2024-05-21T03:20:32.141606+0000","last_active":"2024-05-21T03:21:14.556601+0000","last_peered":"2024-05-21T03:21:14.556601+0000","last_clean":"2024-05-21T03:21:14.556601+0000","last_became_active":"2024-05-21T03:20:32.141258+0000","last_became_peered":"2024-05-21T03:20:32.141258+0000","last_unstale":"2024-05-21T03:21:14.556601+0000","last_undegraded":"2024-05-21T03:21:14.556601+0000","last_fullsized":"2024-05-21T03:21:14.556601+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:22.372723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561186+0000","last_change":"2024-05-21T03:20:32.151621+0000","last_active":"2024-05-21T03:21:14.561186+0000","last_peered":"2024-05-21T03:21:14.561186+0000","last_clean":"2024-05-21T03:21:14.561186+0000","last_became_active":"2024-05-21T03:20:32.151260+0000","last_became_peered":"2024-05-21T03:20:32.151260+0000","last_unstale":"2024-05-21T03:21:14.561186+0000","last_undegraded":"2024-05-21T03:21:14.561186+0000","last_fullsized":"2024-05-21T03:21:14.561186+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:48:59.751179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561107+0000","last_change":"2024-05-21T03:20:32.153843+0000","last_active":"2024-05-21T03:21:14.561107+0000","last_peered":"2024-05-21T03:21:14.561107+0000","last_clean":"2024-05-21T03:21:14.561107+0000","last_became_active":"2024-05-21T03:20:32.153418+0000","last_became_peered":"2024-05-21T03:20:32.153418+0000","last_unstale":"2024-05-21T03:21:14.561107+0000","last_undegraded":"2024-05-21T03:21:14.561107+0000","last_fullsized":"2024-05-21T03:21:14.561107+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:26:29.663966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.560653+0000","last_change":"2024-05-21T03:20:32.153942+0000","last_active":"2024-05-21T03:21:14.560653+0000","last_peered":"2024-05-21T03:21:14.560653+0000","last_clean":"2024-05-21T03:21:14.560653+0000","last_became_active":"2024-05-21T03:20:32.153566+0000","last_became_peered":"2024-05-21T03:20:32.153566+0000","last_unstale":"2024-05-21T03:21:14.560653+0000","last_undegraded":"2024-05-21T03:21:14.560653+0000","last_fullsized":"2024-05-21T03:21:14.560653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:06.899039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558337+0000","last_change":"2024-05-21T03:20:32.153243+0000","last_active":"2024-05-21T03:21:14.558337+0000","last_peered":"2024-05-21T03:21:14.558337+0000","last_clean":"2024-05-21T03:21:14.558337+0000","last_became_active":"2024-05-21T03:20:32.152386+0000","last_became_peered":"2024-05-21T03:20:32.152386+0000","last_unstale":"2024-05-21T03:21:14.558337+0000","last_undegraded":"2024-05-21T03:21:14.558337+0000","last_fullsized":"2024-05-21T03:21:14.558337+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:25:28.935183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552892+0000","last_change":"2024-05-21T03:20:32.142536+0000","last_active":"2024-05-21T03:21:14.552892+0000","last_peered":"2024-05-21T03:21:14.552892+0000","last_clean":"2024-05-21T03:21:14.552892+0000","last_became_active":"2024-05-21T03:20:32.141793+0000","last_became_peered":"2024-05-21T03:20:32.141793+0000","last_unstale":"2024-05-21T03:21:14.552892+0000","last_undegraded":"2024-05-21T03:21:14.552892+0000","last_fullsized":"2024-05-21T03:21:14.552892+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:21:43.547103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.553745+0000","last_change":"2024-05-21T03:20:32.140368+0000","last_active":"2024-05-21T03:21:14.553745+0000","last_peered":"2024-05-21T03:21:14.553745+0000","last_clean":"2024-05-21T03:21:14.553745+0000","last_became_active":"2024-05-21T03:20:32.139747+0000","last_became_peered":"2024-05-21T03:20:32.139747+0000","last_unstale":"2024-05-21T03:21:14.553745+0000","last_undegraded":"2024-05-21T03:21:14.553745+0000","last_fullsized":"2024-05-21T03:21:14.553745+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:05:44.851191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67357,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":400,"num_read_kb":287,"num_write":244,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1561,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":202071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645338,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":901120,"data_stored":433850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841567,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1601536,"data_stored":1084641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005093,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":433506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201321,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":433767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397551,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32668,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712484,"statfs":{"total":95995035648,"available":95961583616,"internally_reserved":0,"allocated":1519616,"data_stored":1026676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561076,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":433497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":494172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":434836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":492831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149763,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":432814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953534,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":434919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757305,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32680,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712472,"statfs":{"total":95995035648,"available":95961571328,"internally_reserved":0,"allocated":1531904,"data_stored":1025572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T03:21:33.378 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-21T03:21:33.379 INFO:tasks.cephadm:Setting up client nodes... 2024-05-21T03:21:33.379 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-21T03:21:33.638 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:21:34.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:33 smithi027 ceph-mon[32179]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:21:34.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:33 smithi039 ceph-mon[36429]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:21:34.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:33 smithi165 ceph-mon[36269]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:21:35.032 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:34 smithi027 ceph-mon[32179]: pgmap v322: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:21:35.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:34 smithi039 ceph-mon[36429]: pgmap v322: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:21:35.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:34 smithi165 ceph-mon[36269]: pgmap v322: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:21:35.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.283+0000 7f90b27fc700 1 -- 172.21.15.27:0/1130617722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90ac105190 msgr2=0x7f90ac10ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:35.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.283+0000 7f90b27fc700 1 --2- 172.21.15.27:0/1130617722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90ac105190 0x7f90ac10ba40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f90a400b0d0 tx=0x7f90a400b3e0 comp rx=0 tx=0).stop 2024-05-21T03:21:35.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.283+0000 7f90b27fc700 1 -- 172.21.15.27:0/1130617722 shutdown_connections 2024-05-21T03:21:35.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.283+0000 7f90b27fc700 1 --2- 172.21.15.27:0/1130617722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90ac105190 0x7f90ac10ba40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:35.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.283+0000 7f90b27fc700 1 --2- 172.21.15.27:0/1130617722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90ac1035f0 0x7f90ac103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:35.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.283+0000 7f90b27fc700 1 --2- 172.21.15.27:0/1130617722 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90ac1047f0 0x7f90ac104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:35.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.283+0000 7f90b27fc700 1 -- 172.21.15.27:0/1130617722 >> 172.21.15.27:0/1130617722 conn(0x7f90ac0feb90 msgr2=0x7f90ac100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:35.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.284+0000 7f90b27fc700 1 -- 172.21.15.27:0/1130617722 shutdown_connections 2024-05-21T03:21:35.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.284+0000 7f90b27fc700 1 -- 172.21.15.27:0/1130617722 wait complete. 2024-05-21T03:21:35.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.284+0000 7f90b27fc700 1 Processor -- start 2024-05-21T03:21:35.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.284+0000 7f90b27fc700 1 -- start start 2024-05-21T03:21:35.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.285+0000 7f90b27fc700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90ac1035f0 0x7f90ac1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:35.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.285+0000 7f90b27fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90ac1047f0 0x7f90ac199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:35.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.285+0000 7f90b27fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90ac105190 0x7f90ac1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:35.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.285+0000 7f90b27fc700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac10f080 con 0x7f90ac1047f0 2024-05-21T03:21:35.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.285+0000 7f90b27fc700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac10ef00 con 0x7f90ac105190 2024-05-21T03:21:35.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.285+0000 7f90b27fc700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac10f200 con 0x7f90ac1035f0 2024-05-21T03:21:35.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.285+0000 7f90a3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90ac1047f0 0x7f90ac199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:35.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.285+0000 7f90a3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90ac1047f0 0x7f90ac199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45958/0 (socket says 172.21.15.27:45958) 2024-05-21T03:21:35.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.285+0000 7f90a3fff700 1 -- 172.21.15.27:0/3291555556 learned_addr learned my addr 172.21.15.27:0/3291555556 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:21:35.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.285+0000 7f90b0d99700 1 --2- 172.21.15.27:0/3291555556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90ac105190 0x7f90ac1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:35.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.285+0000 7f90abfff700 1 --2- 172.21.15.27:0/3291555556 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90ac1035f0 0x7f90ac1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:35.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.285+0000 7f90a3fff700 1 -- 172.21.15.27:0/3291555556 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90ac1035f0 msgr2=0x7f90ac1995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:35.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.285+0000 7f90a3fff700 1 --2- 172.21.15.27:0/3291555556 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90ac1035f0 0x7f90ac1995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:35.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.285+0000 7f90a3fff700 1 -- 172.21.15.27:0/3291555556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90ac105190 msgr2=0x7f90ac1a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:35.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.285+0000 7f90a3fff700 1 --2- 172.21.15.27:0/3291555556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90ac105190 0x7f90ac1a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:35.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.285+0000 7f90a3fff700 1 -- 172.21.15.27:0/3291555556 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90ac1a1200 con 0x7f90ac1047f0 2024-05-21T03:21:35.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.285+0000 7f90a3fff700 1 --2- 172.21.15.27:0/3291555556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90ac1047f0 0x7f90ac199ae0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f909400d9b0 tx=0x7f909400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:35.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.285+0000 7f90a9ffb700 1 -- 172.21.15.27:0/3291555556 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9094017070 con 0x7f90ac1047f0 2024-05-21T03:21:35.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.286+0000 7f90a9ffb700 1 -- 172.21.15.27:0/3291555556 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9094005020 con 0x7f90ac1047f0 2024-05-21T03:21:35.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.286+0000 7f90a9ffb700 1 -- 172.21.15.27:0/3291555556 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9094009a20 con 0x7f90ac1047f0 2024-05-21T03:21:35.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.286+0000 7f90b27fc700 1 -- 172.21.15.27:0/3291555556 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90ac1a14f0 con 0x7f90ac1047f0 2024-05-21T03:21:35.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.286+0000 7f90b27fc700 1 -- 172.21.15.27:0/3291555556 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90ac1a19e0 con 0x7f90ac1047f0 2024-05-21T03:21:35.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.287+0000 7f90b27fc700 1 -- 172.21.15.27:0/3291555556 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90ac051a80 con 0x7f90ac1047f0 2024-05-21T03:21:35.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.288+0000 7f90a9ffb700 1 -- 172.21.15.27:0/3291555556 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f90940051d0 con 0x7f90ac1047f0 2024-05-21T03:21:35.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.289+0000 7f90a9ffb700 1 --2- 172.21.15.27:0/3291555556 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f90980b29b0 0x7f90980b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:35.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.289+0000 7f90abfff700 1 --2- 172.21.15.27:0/3291555556 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f90980b29b0 0x7f90980b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:35.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.289+0000 7f90a9ffb700 1 -- 172.21.15.27:0/3291555556 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f90940e2200 con 0x7f90ac1047f0 2024-05-21T03:21:35.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.289+0000 7f90abfff700 1 --2- 172.21.15.27:0/3291555556 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f90980b29b0 0x7f90980b4e70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f90ac104650 tx=0x7f909c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:35.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.293+0000 7f90a9ffb700 1 -- 172.21.15.27:0/3291555556 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9094020020 con 0x7f90ac1047f0 2024-05-21T03:21:35.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.600+0000 7f90b27fc700 1 -- 172.21.15.27:0/3291555556 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f90ac02e560 con 0x7f90ac1047f0 2024-05-21T03:21:35.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.602+0000 7f90a9ffb700 1 -- 172.21.15.27:0/3291555556 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f90940a9da0 con 0x7f90ac1047f0 2024-05-21T03:21:35.601 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2024-05-21T03:21:35.601 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQA/E0xmQ9fRIxAA1XKM2w+V1CSed56LctvCfg== 2024-05-21T03:21:35.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.605+0000 7f90b27fc700 1 -- 172.21.15.27:0/3291555556 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f90980b29b0 msgr2=0x7f90980b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:35.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.605+0000 7f90b27fc700 1 --2- 172.21.15.27:0/3291555556 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f90980b29b0 0x7f90980b4e70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f90ac104650 tx=0x7f909c00b410 comp rx=0 tx=0).stop 2024-05-21T03:21:35.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.605+0000 7f90b27fc700 1 -- 172.21.15.27:0/3291555556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90ac1047f0 msgr2=0x7f90ac199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:35.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.605+0000 7f90b27fc700 1 --2- 172.21.15.27:0/3291555556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90ac1047f0 0x7f90ac199ae0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f909400d9b0 tx=0x7f909400dcc0 comp rx=0 tx=0).stop 2024-05-21T03:21:35.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.606+0000 7f90b27fc700 1 -- 172.21.15.27:0/3291555556 shutdown_connections 2024-05-21T03:21:35.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.606+0000 7f90b27fc700 1 --2- 172.21.15.27:0/3291555556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90ac105190 0x7f90ac1a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:35.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.606+0000 7f90b27fc700 1 --2- 172.21.15.27:0/3291555556 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f90980b29b0 0x7f90980b4e70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:35.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.606+0000 7f90b27fc700 1 --2- 172.21.15.27:0/3291555556 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90ac1035f0 0x7f90ac1995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:35.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.606+0000 7f90b27fc700 1 --2- 172.21.15.27:0/3291555556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90ac1047f0 0x7f90ac199ae0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:35.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.606+0000 7f90b27fc700 1 -- 172.21.15.27:0/3291555556 >> 172.21.15.27:0/3291555556 conn(0x7f90ac0feb90 msgr2=0x7f90ac107a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:35.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.606+0000 7f90b27fc700 1 -- 172.21.15.27:0/3291555556 shutdown_connections 2024-05-21T03:21:35.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:35.606+0000 7f90b27fc700 1 -- 172.21.15.27:0/3291555556 wait complete. 2024-05-21T03:21:36.212 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:35 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3291555556' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-21T03:21:36.212 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:35 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3291555556' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-21T03:21:36.246 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-05-21T03:21:36.246 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-21T03:21:36.246 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-21T03:21:36.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:35 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/3291555556' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-21T03:21:36.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:35 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/3291555556' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-21T03:21:36.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:35 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/3291555556' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-21T03:21:36.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:35 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/3291555556' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-21T03:21:36.290 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-21T03:21:36.290 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-21T03:21:36.290 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph mgr dump --format=json 2024-05-21T03:21:36.581 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:21:37.032 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:36 smithi027 ceph-mon[32179]: pgmap v323: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:21:37.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:36 smithi039 ceph-mon[36429]: pgmap v323: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:21:37.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:36 smithi165 ceph-mon[36269]: pgmap v323: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:21:38.040 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:37 smithi027 ceph-mon[32179]: pgmap v324: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:21:38.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.148+0000 7f6c50dbd700 1 -- 172.21.15.27:0/1646365634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c4c06bd10 msgr2=0x7f6c4c06c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:38.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.148+0000 7f6c50dbd700 1 --2- 172.21.15.27:0/1646365634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c4c06bd10 0x7f6c4c06c190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0099c0 tx=0x7f6c3c009cd0 comp rx=0 tx=0).stop 2024-05-21T03:21:38.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.149+0000 7f6c50dbd700 1 -- 172.21.15.27:0/1646365634 shutdown_connections 2024-05-21T03:21:38.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.149+0000 7f6c50dbd700 1 --2- 172.21.15.27:0/1646365634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c4c06c6d0 0x7f6c4c110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:38.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.149+0000 7f6c50dbd700 1 --2- 172.21.15.27:0/1646365634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c4c109630 0x7f6c4c109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:38.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.149+0000 7f6c50dbd700 1 --2- 172.21.15.27:0/1646365634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c4c06bd10 0x7f6c4c06c190 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:38.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.149+0000 7f6c50dbd700 1 -- 172.21.15.27:0/1646365634 >> 172.21.15.27:0/1646365634 conn(0x7f6c4c0794e0 msgr2=0x7f6c4c0798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:38.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.149+0000 7f6c50dbd700 1 -- 172.21.15.27:0/1646365634 shutdown_connections 2024-05-21T03:21:38.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.149+0000 7f6c50dbd700 1 -- 172.21.15.27:0/1646365634 wait complete. 2024-05-21T03:21:38.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.149+0000 7f6c50dbd700 1 Processor -- start 2024-05-21T03:21:38.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.149+0000 7f6c50dbd700 1 -- start start 2024-05-21T03:21:38.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.150+0000 7f6c50dbd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c4c06bd10 0x7f6c4c1024b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:38.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.150+0000 7f6c50dbd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c4c06c6d0 0x7f6c4c1029f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:38.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.150+0000 7f6c50dbd700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c4c109630 0x7f6c4c102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:38.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.150+0000 7f6c50dbd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c113790 con 0x7f6c4c06bd10 2024-05-21T03:21:38.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.150+0000 7f6c50dbd700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c113610 con 0x7f6c4c06c6d0 2024-05-21T03:21:38.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.150+0000 7f6c50dbd700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c113910 con 0x7f6c4c109630 2024-05-21T03:21:38.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.150+0000 7f6c4a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c4c06bd10 0x7f6c4c1024b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:38.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.150+0000 7f6c4a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c4c06bd10 0x7f6c4c1024b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45982/0 (socket says 172.21.15.27:45982) 2024-05-21T03:21:38.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.150+0000 7f6c4a59c700 1 -- 172.21.15.27:0/2233732810 learned_addr learned my addr 172.21.15.27:0/2233732810 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:21:38.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.150+0000 7f6c4ad9d700 1 --2- 172.21.15.27:0/2233732810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c4c109630 0x7f6c4c102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:38.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.150+0000 7f6c49d9b700 1 --2- 172.21.15.27:0/2233732810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c4c06c6d0 0x7f6c4c1029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:38.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.150+0000 7f6c4a59c700 1 -- 172.21.15.27:0/2233732810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c4c109630 msgr2=0x7f6c4c102f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:38.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.150+0000 7f6c4a59c700 1 --2- 172.21.15.27:0/2233732810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c4c109630 0x7f6c4c102f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:38.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.150+0000 7f6c4a59c700 1 -- 172.21.15.27:0/2233732810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c4c06c6d0 msgr2=0x7f6c4c1029f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:38.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.150+0000 7f6c4a59c700 1 --2- 172.21.15.27:0/2233732810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c4c06c6d0 0x7f6c4c1029f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:38.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.150+0000 7f6c4a59c700 1 -- 172.21.15.27:0/2233732810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4c1a8030 con 0x7f6c4c06bd10 2024-05-21T03:21:38.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.150+0000 7f6c49d9b700 1 --2- 172.21.15.27:0/2233732810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c4c06c6d0 0x7f6c4c1029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:21:38.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.150+0000 7f6c4a59c700 1 --2- 172.21.15.27:0/2233732810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c4c06bd10 0x7f6c4c1024b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400c9b0 tx=0x7f6c3400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:38.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.151+0000 7f6c437fe700 1 -- 172.21.15.27:0/2233732810 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34007b00 con 0x7f6c4c06bd10 2024-05-21T03:21:38.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.151+0000 7f6c50dbd700 1 -- 172.21.15.27:0/2233732810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c4c1a8320 con 0x7f6c4c06bd10 2024-05-21T03:21:38.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.151+0000 7f6c437fe700 1 -- 172.21.15.27:0/2233732810 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c34007ca0 con 0x7f6c4c06bd10 2024-05-21T03:21:38.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.151+0000 7f6c437fe700 1 -- 172.21.15.27:0/2233732810 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34018660 con 0x7f6c4c06bd10 2024-05-21T03:21:38.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.151+0000 7f6c50dbd700 1 -- 172.21.15.27:0/2233732810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c4c1a8830 con 0x7f6c4c06bd10 2024-05-21T03:21:38.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.152+0000 7f6c50dbd700 1 -- 172.21.15.27:0/2233732810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c4c104ca0 con 0x7f6c4c06bd10 2024-05-21T03:21:38.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.153+0000 7f6c437fe700 1 -- 172.21.15.27:0/2233732810 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6c3400f450 con 0x7f6c4c06bd10 2024-05-21T03:21:38.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.154+0000 7f6c437fe700 1 --2- 172.21.15.27:0/2233732810 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f6c380b2910 0x7f6c380b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:38.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.154+0000 7f6c49d9b700 1 --2- 172.21.15.27:0/2233732810 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f6c380b2910 0x7f6c380b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:38.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.154+0000 7f6c437fe700 1 -- 172.21.15.27:0/2233732810 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f6c340e25d0 con 0x7f6c4c06bd10 2024-05-21T03:21:38.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.154+0000 7f6c49d9b700 1 --2- 172.21.15.27:0/2233732810 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f6c380b2910 0x7f6c380b4dd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c103f40 tx=0x7f6c3c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:38.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.157+0000 7f6c437fe700 1 -- 172.21.15.27:0/2233732810 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6c34014030 con 0x7f6c4c06bd10 2024-05-21T03:21:38.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:37 smithi039 ceph-mon[36429]: pgmap v324: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:21:38.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:37 smithi165 ceph-mon[36269]: pgmap v324: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:21:38.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.446+0000 7f6c50dbd700 1 -- 172.21.15.27:0/2233732810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f6c4c1039e0 con 0x7f6c4c06bd10 2024-05-21T03:21:38.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.452+0000 7f6c437fe700 1 -- 172.21.15.27:0/2233732810 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288646 (secure 0 0 0) 0x7f6c340aa200 con 0x7f6c4c06bd10 2024-05-21T03:21:38.453 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:21:38.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.455+0000 7f6c50dbd700 1 -- 172.21.15.27:0/2233732810 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f6c380b2910 msgr2=0x7f6c380b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:38.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.455+0000 7f6c50dbd700 1 --2- 172.21.15.27:0/2233732810 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f6c380b2910 0x7f6c380b4dd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c103f40 tx=0x7f6c3c01a040 comp rx=0 tx=0).stop 2024-05-21T03:21:38.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.455+0000 7f6c50dbd700 1 -- 172.21.15.27:0/2233732810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c4c06bd10 msgr2=0x7f6c4c1024b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:38.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.455+0000 7f6c50dbd700 1 --2- 172.21.15.27:0/2233732810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c4c06bd10 0x7f6c4c1024b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400c9b0 tx=0x7f6c3400ccc0 comp rx=0 tx=0).stop 2024-05-21T03:21:38.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.455+0000 7f6c50dbd700 1 -- 172.21.15.27:0/2233732810 shutdown_connections 2024-05-21T03:21:38.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.455+0000 7f6c50dbd700 1 --2- 172.21.15.27:0/2233732810 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f6c380b2910 0x7f6c380b4dd0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:38.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.455+0000 7f6c50dbd700 1 --2- 172.21.15.27:0/2233732810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c4c109630 0x7f6c4c102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:38.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.455+0000 7f6c50dbd700 1 --2- 172.21.15.27:0/2233732810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c4c06bd10 0x7f6c4c1024b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:38.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.455+0000 7f6c50dbd700 1 --2- 172.21.15.27:0/2233732810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c4c06c6d0 0x7f6c4c1029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:38.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.455+0000 7f6c50dbd700 1 -- 172.21.15.27:0/2233732810 >> 172.21.15.27:0/2233732810 conn(0x7f6c4c0794e0 msgr2=0x7f6c4c100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:38.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.455+0000 7f6c50dbd700 1 -- 172.21.15.27:0/2233732810 shutdown_connections 2024-05-21T03:21:38.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:38.455+0000 7f6c50dbd700 1 -- 172.21.15.27:0/2233732810 wait complete. 2024-05-21T03:21:39.043 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:38 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2233732810' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-21T03:21:39.073 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3683408243},{"type":"v1","addr":"172.21.15.27:6801","nonce":3683408243}]},"active_addr":"172.21.15.27:6801/3683408243","active_change":"2024-05-21T03:13:13.249383+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24121,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.27:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":4189166571}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":4135246381}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":744303480}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":3254504266}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":3074381090}]}]} 2024-05-21T03:21:39.078 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-21T03:21:39.079 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-21T03:21:39.079 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-21T03:21:39.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:38 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/2233732810' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-21T03:21:39.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:38 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/2233732810' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-21T03:21:39.365 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:21:40.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:39 smithi027 ceph-mon[32179]: pgmap v325: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:21:40.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:39 smithi039 ceph-mon[36429]: pgmap v325: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:21:40.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:39 smithi165 ceph-mon[36269]: pgmap v325: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:21:41.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.006+0000 7f8a1c0d8700 1 -- 172.21.15.27:0/709763620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a14108520 msgr2=0x7f8a14108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:41.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.006+0000 7f8a1c0d8700 1 --2- 172.21.15.27:0/709763620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a14108520 0x7f8a14108900 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8a1000b0a0 tx=0x7f8a1000b3b0 comp rx=0 tx=0).stop 2024-05-21T03:21:41.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.006+0000 7f8a1c0d8700 1 -- 172.21.15.27:0/709763620 shutdown_connections 2024-05-21T03:21:41.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.006+0000 7f8a1c0d8700 1 --2- 172.21.15.27:0/709763620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a14102e00 0x7f8a1410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:41.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.006+0000 7f8a1c0d8700 1 --2- 172.21.15.27:0/709763620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a14108520 0x7f8a14108900 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:41.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.006+0000 7f8a1c0d8700 1 --2- 172.21.15.27:0/709763620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a14102440 0x7f8a141028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:41.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.006+0000 7f8a1c0d8700 1 -- 172.21.15.27:0/709763620 >> 172.21.15.27:0/709763620 conn(0x7f8a140793f0 msgr2=0x7f8a14079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:41.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.006+0000 7f8a1c0d8700 1 -- 172.21.15.27:0/709763620 shutdown_connections 2024-05-21T03:21:41.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.006+0000 7f8a1c0d8700 1 -- 172.21.15.27:0/709763620 wait complete. 2024-05-21T03:21:41.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.007+0000 7f8a1c0d8700 1 Processor -- start 2024-05-21T03:21:41.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.007+0000 7f8a1c0d8700 1 -- start start 2024-05-21T03:21:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.007+0000 7f8a1c0d8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a14102440 0x7f8a1419f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.007+0000 7f8a1c0d8700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a14102e00 0x7f8a1419fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.007+0000 7f8a1c0d8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a14108520 0x7f8a14199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.008+0000 7f8a1c0d8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8a14112650 con 0x7f8a14102440 2024-05-21T03:21:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.008+0000 7f8a1c0d8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8a141124d0 con 0x7f8a14108520 2024-05-21T03:21:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.008+0000 7f8a1c0d8700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8a141127d0 con 0x7f8a14102e00 2024-05-21T03:21:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.008+0000 7f8a19e74700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a14102440 0x7f8a1419f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.008+0000 7f8a19e74700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a14102440 0x7f8a1419f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46002/0 (socket says 172.21.15.27:46002) 2024-05-21T03:21:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.008+0000 7f8a19e74700 1 -- 172.21.15.27:0/2563963663 learned_addr learned my addr 172.21.15.27:0/2563963663 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:21:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.008+0000 7f8a19673700 1 --2- 172.21.15.27:0/2563963663 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a14102e00 0x7f8a1419fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.008+0000 7f8a1a675700 1 --2- 172.21.15.27:0/2563963663 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a14108520 0x7f8a14199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.008+0000 7f8a19e74700 1 -- 172.21.15.27:0/2563963663 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a14102e00 msgr2=0x7f8a1419fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:41.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.008+0000 7f8a19e74700 1 --2- 172.21.15.27:0/2563963663 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a14102e00 0x7f8a1419fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:41.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.008+0000 7f8a19e74700 1 -- 172.21.15.27:0/2563963663 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a14108520 msgr2=0x7f8a14199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:41.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.008+0000 7f8a19e74700 1 --2- 172.21.15.27:0/2563963663 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a14108520 0x7f8a14199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:41.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.008+0000 7f8a19e74700 1 -- 172.21.15.27:0/2563963663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1419a1e0 con 0x7f8a14102440 2024-05-21T03:21:41.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.008+0000 7f8a1a675700 1 --2- 172.21.15.27:0/2563963663 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a14108520 0x7f8a14199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:21:41.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.008+0000 7f8a19e74700 1 --2- 172.21.15.27:0/2563963663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a14102440 0x7f8a1419f8c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8a10006070 tx=0x7f8a10004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:41.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.008+0000 7f8a0affd700 1 -- 172.21.15.27:0/2563963663 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a10021070 con 0x7f8a14102440 2024-05-21T03:21:41.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.008+0000 7f8a0affd700 1 -- 172.21.15.27:0/2563963663 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a10007df0 con 0x7f8a14102440 2024-05-21T03:21:41.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.008+0000 7f8a1c0d8700 1 -- 172.21.15.27:0/2563963663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a1419a470 con 0x7f8a14102440 2024-05-21T03:21:41.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.008+0000 7f8a0affd700 1 -- 172.21.15.27:0/2563963663 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1001d450 con 0x7f8a14102440 2024-05-21T03:21:41.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.009+0000 7f8a1c0d8700 1 -- 172.21.15.27:0/2563963663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a141a6790 con 0x7f8a14102440 2024-05-21T03:21:41.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.010+0000 7f8a1c0d8700 1 -- 172.21.15.27:0/2563963663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89f8005330 con 0x7f8a14102440 2024-05-21T03:21:41.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.011+0000 7f8a0affd700 1 -- 172.21.15.27:0/2563963663 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f8a10004020 con 0x7f8a14102440 2024-05-21T03:21:41.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.012+0000 7f8a0affd700 1 --2- 172.21.15.27:0/2563963663 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f8a000b2990 0x7f8a000b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:41.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.012+0000 7f8a19673700 1 --2- 172.21.15.27:0/2563963663 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f8a000b2990 0x7f8a000b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:41.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.012+0000 7f8a0affd700 1 -- 172.21.15.27:0/2563963663 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f8a1000f020 con 0x7f8a14102440 2024-05-21T03:21:41.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.012+0000 7f8a19673700 1 --2- 172.21.15.27:0/2563963663 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f8a000b2990 0x7f8a000b4e50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8a04006fd0 tx=0x7f8a04008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:41.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.016+0000 7f8a0affd700 1 -- 172.21.15.27:0/2563963663 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8a1002a020 con 0x7f8a14102440 2024-05-21T03:21:41.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.239+0000 7f8a1c0d8700 1 -- 172.21.15.27:0/2563963663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f89f80058f0 con 0x7f8a14102440 2024-05-21T03:21:41.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.241+0000 7f8a0affd700 1 -- 172.21.15.27:0/2563963663 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22683 (secure 0 0 0) 0x7f8a100ab580 con 0x7f8a14102440 2024-05-21T03:21:41.241 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:21:41.241 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":82,"fsid":"c9ca94be-171f-11ef-bc9a-c7b262605968","created":"2024-05-21T03:12:18.217236+0000","modified":"2024-05-21T03:21:14.540989+0000","last_up_change":"2024-05-21T03:19:30.382150+0000","last_in_change":"2024-05-21T03:19:09.058249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T03:15:41.340194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T03:20:27.507346+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T03:20:30.830169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c745c09f-9a76-4d05-af75-c7554f6d5e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6803","nonce":1399996449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6805","nonce":1399996449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6809","nonce":1399996449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6807","nonce":1399996449}]},"public_addr":"172.21.15.27:6803/1399996449","cluster_addr":"172.21.15.27:6805/1399996449","heartbeat_back_addr":"172.21.15.27:6809/1399996449","heartbeat_front_addr":"172.21.15.27:6807/1399996449","state":["exists","up"]},{"osd":1,"uuid":"8b73aa00-4364-438f-9722-53caa4e9bec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6801","nonce":1776318847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6803","nonce":1776318847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6807","nonce":1776318847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6805","nonce":1776318847}]},"public_addr":"172.21.15.39:6801/1776318847","cluster_addr":"172.21.15.39:6803/1776318847","heartbeat_back_addr":"172.21.15.39:6807/1776318847","heartbeat_front_addr":"172.21.15.39:6805/1776318847","state":["exists","up"]},{"osd":2,"uuid":"0822043e-7d3c-4e98-9dc0-94b98b4d562a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6801","nonce":703322625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6803","nonce":703322625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6807","nonce":703322625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6805","nonce":703322625}]},"public_addr":"172.21.15.165:6801/703322625","cluster_addr":"172.21.15.165:6803/703322625","heartbeat_back_addr":"172.21.15.165:6807/703322625","heartbeat_front_addr":"172.21.15.165:6805/703322625","state":["exists","up"]},{"osd":3,"uuid":"c563b1b2-efcb-4f8c-8121-c1938f83131e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6811","nonce":1933403135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6813","nonce":1933403135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6817","nonce":1933403135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6815","nonce":1933403135}]},"public_addr":"172.21.15.27:6811/1933403135","cluster_addr":"172.21.15.27:6813/1933403135","heartbeat_back_addr":"172.21.15.27:6817/1933403135","heartbeat_front_addr":"172.21.15.27:6815/1933403135","state":["exists","up"]},{"osd":4,"uuid":"0c793b87-b045-4b4a-89e7-87ec36deeb0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6809","nonce":155579331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6811","nonce":155579331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6815","nonce":155579331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6813","nonce":155579331}]},"public_addr":"172.21.15.39:6809/155579331","cluster_addr":"172.21.15.39:6811/155579331","heartbeat_back_addr":"172.21.15.39:6815/155579331","heartbeat_front_addr":"172.21.15.39:6813/155579331","state":["exists","up"]},{"osd":5,"uuid":"367eeb5a-9d75-4644-bf46-317c03f33d8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6809","nonce":386850244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6811","nonce":386850244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6815","nonce":386850244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6813","nonce":386850244}]},"public_addr":"172.21.15.165:6809/386850244","cluster_addr":"172.21.15.165:6811/386850244","heartbeat_back_addr":"172.21.15.165:6815/386850244","heartbeat_front_addr":"172.21.15.165:6813/386850244","state":["exists","up"]},{"osd":6,"uuid":"e000167f-24fd-4115-83f0-8afbfa8d8991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6819","nonce":3531376679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6821","nonce":3531376679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6825","nonce":3531376679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6823","nonce":3531376679}]},"public_addr":"172.21.15.27:6819/3531376679","cluster_addr":"172.21.15.27:6821/3531376679","heartbeat_back_addr":"172.21.15.27:6825/3531376679","heartbeat_front_addr":"172.21.15.27:6823/3531376679","state":["exists","up"]},{"osd":7,"uuid":"4b8d486b-868e-4f33-98a1-b99686adeaaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6817","nonce":1550035852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6819","nonce":1550035852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6823","nonce":1550035852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6821","nonce":1550035852}]},"public_addr":"172.21.15.39:6817/1550035852","cluster_addr":"172.21.15.39:6819/1550035852","heartbeat_back_addr":"172.21.15.39:6823/1550035852","heartbeat_front_addr":"172.21.15.39:6821/1550035852","state":["exists","up"]},{"osd":8,"uuid":"2259f35c-2edd-4331-9a63-8c0c1acbdc71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6817","nonce":3184704971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6819","nonce":3184704971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6823","nonce":3184704971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6821","nonce":3184704971}]},"public_addr":"172.21.15.165:6817/3184704971","cluster_addr":"172.21.15.165:6819/3184704971","heartbeat_back_addr":"172.21.15.165:6823/3184704971","heartbeat_front_addr":"172.21.15.165:6821/3184704971","state":["exists","up"]},{"osd":9,"uuid":"10307ecc-8270-4be2-b62b-9708e09bb207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6827","nonce":2800048793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6829","nonce":2800048793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6833","nonce":2800048793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6831","nonce":2800048793}]},"public_addr":"172.21.15.27:6827/2800048793","cluster_addr":"172.21.15.27:6829/2800048793","heartbeat_back_addr":"172.21.15.27:6833/2800048793","heartbeat_front_addr":"172.21.15.27:6831/2800048793","state":["exists","up"]},{"osd":10,"uuid":"dfe1b8ca-ff8a-4b7f-ade5-ad4f00cf60cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6825","nonce":363500811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6827","nonce":363500811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6831","nonce":363500811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6829","nonce":363500811}]},"public_addr":"172.21.15.39:6825/363500811","cluster_addr":"172.21.15.39:6827/363500811","heartbeat_back_addr":"172.21.15.39:6831/363500811","heartbeat_front_addr":"172.21.15.39:6829/363500811","state":["exists","up"]},{"osd":11,"uuid":"f9531145-5d09-4dbd-a050-3159ee09db24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6825","nonce":2274332243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6827","nonce":2274332243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6831","nonce":2274332243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6829","nonce":2274332243}]},"public_addr":"172.21.15.165:6825/2274332243","cluster_addr":"172.21.15.165:6827/2274332243","heartbeat_back_addr":"172.21.15.165:6831/2274332243","heartbeat_front_addr":"172.21.15.165:6829/2274332243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:14:47.077860+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:15:13.272879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:15:38.046651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:03.376483+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:29.179280+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:53.391261+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:17:18.472829+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:17:44.349925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:18:08.652193+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:18:35.282669+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:19:01.637151+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:19:27.461566+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6800/1185518061":"2024-05-22T03:13:13.249220+0000","172.21.15.27:6801/1185518061":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/4194431212":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/1890032101":"2024-05-22T03:13:13.249220+0000","172.21.15.27:6800/175229478":"2024-05-22T03:12:45.739195+0000","172.21.15.27:0/999325706":"2024-05-22T03:12:45.739195+0000","172.21.15.27:6801/175229478":"2024-05-22T03:12:45.739195+0000","172.21.15.27:0/4184128736":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/3000197506":"2024-05-22T03:12:45.739195+0000","172.21.15.27:0/972677319":"2024-05-22T03:12:45.739195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T03:21:41.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.243+0000 7f8a1c0d8700 1 -- 172.21.15.27:0/2563963663 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f8a000b2990 msgr2=0x7f8a000b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:41.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.243+0000 7f8a1c0d8700 1 --2- 172.21.15.27:0/2563963663 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f8a000b2990 0x7f8a000b4e50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8a04006fd0 tx=0x7f8a04008040 comp rx=0 tx=0).stop 2024-05-21T03:21:41.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.243+0000 7f8a1c0d8700 1 -- 172.21.15.27:0/2563963663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a14102440 msgr2=0x7f8a1419f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:41.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.243+0000 7f8a1c0d8700 1 --2- 172.21.15.27:0/2563963663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a14102440 0x7f8a1419f8c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8a10006070 tx=0x7f8a10004ef0 comp rx=0 tx=0).stop 2024-05-21T03:21:41.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.243+0000 7f8a1c0d8700 1 -- 172.21.15.27:0/2563963663 shutdown_connections 2024-05-21T03:21:41.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.243+0000 7f8a1c0d8700 1 --2- 172.21.15.27:0/2563963663 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a14108520 0x7f8a14199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:41.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.243+0000 7f8a1c0d8700 1 --2- 172.21.15.27:0/2563963663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a14102440 0x7f8a1419f8c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:41.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.243+0000 7f8a1c0d8700 1 --2- 172.21.15.27:0/2563963663 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f8a000b2990 0x7f8a000b4e50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:41.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.243+0000 7f8a1c0d8700 1 --2- 172.21.15.27:0/2563963663 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a14102e00 0x7f8a1419fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:41.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.243+0000 7f8a1c0d8700 1 -- 172.21.15.27:0/2563963663 >> 172.21.15.27:0/2563963663 conn(0x7f8a140793f0 msgr2=0x7f8a141067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:41.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.244+0000 7f8a1c0d8700 1 -- 172.21.15.27:0/2563963663 shutdown_connections 2024-05-21T03:21:41.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:41.244+0000 7f8a1c0d8700 1 -- 172.21.15.27:0/2563963663 wait complete. 2024-05-21T03:21:41.566 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:41 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2563963663' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:21:41.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:41 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/2563963663' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:21:41.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:41 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/2563963663' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:21:41.843 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-21T03:21:41.843 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-21T03:21:42.094 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:21:42.512 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:42 smithi027 ceph-mon[32179]: pgmap v326: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:21:42.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:42 smithi039 ceph-mon[36429]: pgmap v326: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:21:42.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:42 smithi165 ceph-mon[36269]: pgmap v326: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:21:43.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.709+0000 7fd87a162700 1 -- 172.21.15.27:0/1923779612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87406c470 msgr2=0x7fd8741103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:43.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.709+0000 7fd87a162700 1 --2- 172.21.15.27:0/1923779612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87406c470 0x7fd8741103c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd86c009990 tx=0x7fd86c009ca0 comp rx=0 tx=0).stop 2024-05-21T03:21:43.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.709+0000 7fd87a162700 1 -- 172.21.15.27:0/1923779612 shutdown_connections 2024-05-21T03:21:43.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.709+0000 7fd87a162700 1 --2- 172.21.15.27:0/1923779612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87406c470 0x7fd8741103c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:43.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.709+0000 7fd87a162700 1 --2- 172.21.15.27:0/1923779612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8741095a0 0x7fd874109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:43.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.709+0000 7fd87a162700 1 --2- 172.21.15.27:0/1923779612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd87406bab0 0x7fd87406bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:43.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.709+0000 7fd87a162700 1 -- 172.21.15.27:0/1923779612 >> 172.21.15.27:0/1923779612 conn(0x7fd8740793b0 msgr2=0x7fd8740797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:43.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.709+0000 7fd87a162700 1 -- 172.21.15.27:0/1923779612 shutdown_connections 2024-05-21T03:21:43.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.709+0000 7fd87a162700 1 -- 172.21.15.27:0/1923779612 wait complete. 2024-05-21T03:21:43.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.710+0000 7fd87a162700 1 Processor -- start 2024-05-21T03:21:43.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.710+0000 7fd87a162700 1 -- start start 2024-05-21T03:21:43.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.710+0000 7fd87a162700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd87406bab0 0x7fd87419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:43.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.710+0000 7fd87a162700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87406c470 0x7fd87419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:43.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.710+0000 7fd87a162700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8741095a0 0x7fd8741999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:43.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.710+0000 7fd87a162700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd8741136d0 con 0x7fd87406c470 2024-05-21T03:21:43.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.710+0000 7fd87a162700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd874113550 con 0x7fd8741095a0 2024-05-21T03:21:43.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.710+0000 7fd87a162700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd874113850 con 0x7fd87406bab0 2024-05-21T03:21:43.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.711+0000 7fd872ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87406c470 0x7fd87419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:43.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.711+0000 7fd872ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87406c470 0x7fd87419fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34212/0 (socket says 172.21.15.27:34212) 2024-05-21T03:21:43.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.711+0000 7fd872ffd700 1 -- 172.21.15.27:0/479782593 learned_addr learned my addr 172.21.15.27:0/479782593 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:21:43.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.711+0000 7fd873fff700 1 --2- 172.21.15.27:0/479782593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8741095a0 0x7fd8741999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:43.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.711+0000 7fd8737fe700 1 --2- 172.21.15.27:0/479782593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd87406bab0 0x7fd87419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:43.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.711+0000 7fd872ffd700 1 -- 172.21.15.27:0/479782593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd87406bab0 msgr2=0x7fd87419f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:43.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.711+0000 7fd872ffd700 1 --2- 172.21.15.27:0/479782593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd87406bab0 0x7fd87419f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:43.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.711+0000 7fd872ffd700 1 -- 172.21.15.27:0/479782593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8741095a0 msgr2=0x7fd8741999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:43.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.711+0000 7fd872ffd700 1 --2- 172.21.15.27:0/479782593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8741095a0 0x7fd8741999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:43.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.711+0000 7fd872ffd700 1 -- 172.21.15.27:0/479782593 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd87419a130 con 0x7fd87406c470 2024-05-21T03:21:43.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.711+0000 7fd872ffd700 1 --2- 172.21.15.27:0/479782593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87406c470 0x7fd87419fd50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd86800d980 tx=0x7fd86800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:43.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.711+0000 7fd870ff9700 1 -- 172.21.15.27:0/479782593 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868017070 con 0x7fd87406c470 2024-05-21T03:21:43.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.711+0000 7fd870ff9700 1 -- 172.21.15.27:0/479782593 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd868005020 con 0x7fd87406c470 2024-05-21T03:21:43.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.711+0000 7fd87a162700 1 -- 172.21.15.27:0/479782593 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd87419a420 con 0x7fd87406c470 2024-05-21T03:21:43.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.711+0000 7fd870ff9700 1 -- 172.21.15.27:0/479782593 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868009b50 con 0x7fd87406c470 2024-05-21T03:21:43.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.711+0000 7fd87a162700 1 -- 172.21.15.27:0/479782593 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8741a66e0 con 0x7fd87406c470 2024-05-21T03:21:43.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.713+0000 7fd87a162700 1 -- 172.21.15.27:0/479782593 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd87406a230 con 0x7fd87406c470 2024-05-21T03:21:43.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.714+0000 7fd870ff9700 1 -- 172.21.15.27:0/479782593 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fd8680051d0 con 0x7fd87406c470 2024-05-21T03:21:43.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.715+0000 7fd870ff9700 1 --2- 172.21.15.27:0/479782593 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd85c0b2910 0x7fd85c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:43.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.715+0000 7fd870ff9700 1 -- 172.21.15.27:0/479782593 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fd8680e22f0 con 0x7fd87406c470 2024-05-21T03:21:43.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.715+0000 7fd8737fe700 1 --2- 172.21.15.27:0/479782593 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd85c0b2910 0x7fd85c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:43.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.715+0000 7fd8737fe700 1 --2- 172.21.15.27:0/479782593 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd85c0b2910 0x7fd85c0b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd87419ab10 tx=0x7fd864009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:43.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.719+0000 7fd870ff9700 1 -- 172.21.15.27:0/479782593 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd868020020 con 0x7fd87406c470 2024-05-21T03:21:43.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.940+0000 7fd87a162700 1 -- 172.21.15.27:0/479782593 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd87402e560 con 0x7fd87406c470 2024-05-21T03:21:43.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.941+0000 7fd870ff9700 1 -- 172.21.15.27:0/479782593 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22683 (secure 0 0 0) 0x7fd8680a9e90 con 0x7fd87406c470 2024-05-21T03:21:43.942 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:21:43.942 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":82,"fsid":"c9ca94be-171f-11ef-bc9a-c7b262605968","created":"2024-05-21T03:12:18.217236+0000","modified":"2024-05-21T03:21:14.540989+0000","last_up_change":"2024-05-21T03:19:30.382150+0000","last_in_change":"2024-05-21T03:19:09.058249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T03:15:41.340194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T03:20:27.507346+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T03:20:30.830169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c745c09f-9a76-4d05-af75-c7554f6d5e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6803","nonce":1399996449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6805","nonce":1399996449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6809","nonce":1399996449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6807","nonce":1399996449}]},"public_addr":"172.21.15.27:6803/1399996449","cluster_addr":"172.21.15.27:6805/1399996449","heartbeat_back_addr":"172.21.15.27:6809/1399996449","heartbeat_front_addr":"172.21.15.27:6807/1399996449","state":["exists","up"]},{"osd":1,"uuid":"8b73aa00-4364-438f-9722-53caa4e9bec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6801","nonce":1776318847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6803","nonce":1776318847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6807","nonce":1776318847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6805","nonce":1776318847}]},"public_addr":"172.21.15.39:6801/1776318847","cluster_addr":"172.21.15.39:6803/1776318847","heartbeat_back_addr":"172.21.15.39:6807/1776318847","heartbeat_front_addr":"172.21.15.39:6805/1776318847","state":["exists","up"]},{"osd":2,"uuid":"0822043e-7d3c-4e98-9dc0-94b98b4d562a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6801","nonce":703322625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6803","nonce":703322625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6807","nonce":703322625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6805","nonce":703322625}]},"public_addr":"172.21.15.165:6801/703322625","cluster_addr":"172.21.15.165:6803/703322625","heartbeat_back_addr":"172.21.15.165:6807/703322625","heartbeat_front_addr":"172.21.15.165:6805/703322625","state":["exists","up"]},{"osd":3,"uuid":"c563b1b2-efcb-4f8c-8121-c1938f83131e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6811","nonce":1933403135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6813","nonce":1933403135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6817","nonce":1933403135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6815","nonce":1933403135}]},"public_addr":"172.21.15.27:6811/1933403135","cluster_addr":"172.21.15.27:6813/1933403135","heartbeat_back_addr":"172.21.15.27:6817/1933403135","heartbeat_front_addr":"172.21.15.27:6815/1933403135","state":["exists","up"]},{"osd":4,"uuid":"0c793b87-b045-4b4a-89e7-87ec36deeb0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6809","nonce":155579331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6811","nonce":155579331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6815","nonce":155579331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6813","nonce":155579331}]},"public_addr":"172.21.15.39:6809/155579331","cluster_addr":"172.21.15.39:6811/155579331","heartbeat_back_addr":"172.21.15.39:6815/155579331","heartbeat_front_addr":"172.21.15.39:6813/155579331","state":["exists","up"]},{"osd":5,"uuid":"367eeb5a-9d75-4644-bf46-317c03f33d8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6809","nonce":386850244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6811","nonce":386850244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6815","nonce":386850244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6813","nonce":386850244}]},"public_addr":"172.21.15.165:6809/386850244","cluster_addr":"172.21.15.165:6811/386850244","heartbeat_back_addr":"172.21.15.165:6815/386850244","heartbeat_front_addr":"172.21.15.165:6813/386850244","state":["exists","up"]},{"osd":6,"uuid":"e000167f-24fd-4115-83f0-8afbfa8d8991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6819","nonce":3531376679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6821","nonce":3531376679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6825","nonce":3531376679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6823","nonce":3531376679}]},"public_addr":"172.21.15.27:6819/3531376679","cluster_addr":"172.21.15.27:6821/3531376679","heartbeat_back_addr":"172.21.15.27:6825/3531376679","heartbeat_front_addr":"172.21.15.27:6823/3531376679","state":["exists","up"]},{"osd":7,"uuid":"4b8d486b-868e-4f33-98a1-b99686adeaaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6817","nonce":1550035852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6819","nonce":1550035852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6823","nonce":1550035852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6821","nonce":1550035852}]},"public_addr":"172.21.15.39:6817/1550035852","cluster_addr":"172.21.15.39:6819/1550035852","heartbeat_back_addr":"172.21.15.39:6823/1550035852","heartbeat_front_addr":"172.21.15.39:6821/1550035852","state":["exists","up"]},{"osd":8,"uuid":"2259f35c-2edd-4331-9a63-8c0c1acbdc71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6817","nonce":3184704971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6819","nonce":3184704971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6823","nonce":3184704971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6821","nonce":3184704971}]},"public_addr":"172.21.15.165:6817/3184704971","cluster_addr":"172.21.15.165:6819/3184704971","heartbeat_back_addr":"172.21.15.165:6823/3184704971","heartbeat_front_addr":"172.21.15.165:6821/3184704971","state":["exists","up"]},{"osd":9,"uuid":"10307ecc-8270-4be2-b62b-9708e09bb207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6827","nonce":2800048793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6829","nonce":2800048793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6833","nonce":2800048793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6831","nonce":2800048793}]},"public_addr":"172.21.15.27:6827/2800048793","cluster_addr":"172.21.15.27:6829/2800048793","heartbeat_back_addr":"172.21.15.27:6833/2800048793","heartbeat_front_addr":"172.21.15.27:6831/2800048793","state":["exists","up"]},{"osd":10,"uuid":"dfe1b8ca-ff8a-4b7f-ade5-ad4f00cf60cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6825","nonce":363500811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6827","nonce":363500811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6831","nonce":363500811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6829","nonce":363500811}]},"public_addr":"172.21.15.39:6825/363500811","cluster_addr":"172.21.15.39:6827/363500811","heartbeat_back_addr":"172.21.15.39:6831/363500811","heartbeat_front_addr":"172.21.15.39:6829/363500811","state":["exists","up"]},{"osd":11,"uuid":"f9531145-5d09-4dbd-a050-3159ee09db24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6825","nonce":2274332243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6827","nonce":2274332243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6831","nonce":2274332243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6829","nonce":2274332243}]},"public_addr":"172.21.15.165:6825/2274332243","cluster_addr":"172.21.15.165:6827/2274332243","heartbeat_back_addr":"172.21.15.165:6831/2274332243","heartbeat_front_addr":"172.21.15.165:6829/2274332243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:14:47.077860+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:15:13.272879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:15:38.046651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:03.376483+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:29.179280+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:53.391261+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:17:18.472829+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:17:44.349925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:18:08.652193+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:18:35.282669+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:19:01.637151+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:19:27.461566+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6800/1185518061":"2024-05-22T03:13:13.249220+0000","172.21.15.27:6801/1185518061":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/4194431212":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/1890032101":"2024-05-22T03:13:13.249220+0000","172.21.15.27:6800/175229478":"2024-05-22T03:12:45.739195+0000","172.21.15.27:0/999325706":"2024-05-22T03:12:45.739195+0000","172.21.15.27:6801/175229478":"2024-05-22T03:12:45.739195+0000","172.21.15.27:0/4184128736":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/3000197506":"2024-05-22T03:12:45.739195+0000","172.21.15.27:0/972677319":"2024-05-22T03:12:45.739195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T03:21:43.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.944+0000 7fd87a162700 1 -- 172.21.15.27:0/479782593 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd85c0b2910 msgr2=0x7fd85c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:43.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.944+0000 7fd87a162700 1 --2- 172.21.15.27:0/479782593 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd85c0b2910 0x7fd85c0b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd87419ab10 tx=0x7fd864009040 comp rx=0 tx=0).stop 2024-05-21T03:21:43.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.944+0000 7fd87a162700 1 -- 172.21.15.27:0/479782593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87406c470 msgr2=0x7fd87419fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:43.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.944+0000 7fd87a162700 1 --2- 172.21.15.27:0/479782593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87406c470 0x7fd87419fd50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd86800d980 tx=0x7fd86800dc90 comp rx=0 tx=0).stop 2024-05-21T03:21:43.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.944+0000 7fd87a162700 1 -- 172.21.15.27:0/479782593 shutdown_connections 2024-05-21T03:21:43.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.944+0000 7fd87a162700 1 --2- 172.21.15.27:0/479782593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8741095a0 0x7fd8741999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:43.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.944+0000 7fd87a162700 1 --2- 172.21.15.27:0/479782593 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd85c0b2910 0x7fd85c0b4dd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:43.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.944+0000 7fd87a162700 1 --2- 172.21.15.27:0/479782593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd87406bab0 0x7fd87419f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:43.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.944+0000 7fd87a162700 1 --2- 172.21.15.27:0/479782593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87406c470 0x7fd87419fd50 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:43.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.944+0000 7fd87a162700 1 -- 172.21.15.27:0/479782593 >> 172.21.15.27:0/479782593 conn(0x7fd8740793b0 msgr2=0x7fd8741010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:43.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.944+0000 7fd87a162700 1 -- 172.21.15.27:0/479782593 shutdown_connections 2024-05-21T03:21:43.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:43.944+0000 7fd87a162700 1 -- 172.21.15.27:0/479782593 wait complete. 2024-05-21T03:21:44.556 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:44 smithi027 ceph-mon[32179]: pgmap v327: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:21:44.556 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:44 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/479782593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:21:44.586 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-21T03:21:44.587 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-21T03:21:44.587 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-21T03:21:44.588 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-21T03:21:44.588 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-21T03:21:44.589 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-21T03:21:44.589 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-21T03:21:44.589 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-21T03:21:44.590 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-21T03:21:44.590 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-21T03:21:44.591 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-21T03:21:44.591 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-21T03:21:44.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:44 smithi039 ceph-mon[36429]: pgmap v327: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:21:44.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:44 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/479782593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:21:44.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:44 smithi165 ceph-mon[36269]: pgmap v327: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:21:44.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:44 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/479782593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:21:45.299 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:21:45.312 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:21:46.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:46 smithi027 ceph-mon[32179]: pgmap v328: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:21:46.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:46 smithi039 ceph-mon[36429]: pgmap v328: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:21:46.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:46 smithi165 ceph-mon[36269]: pgmap v328: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:21:47.407 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:21:47.409 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:21:47.413 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:21:47.416 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:21:47.416 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:21:47.417 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:21:47.418 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:21:47.427 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:21:47.428 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:21:47.430 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:21:48.610 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:48 smithi027 ceph-mon[32179]: pgmap v329: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:21:48.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:48 smithi039 ceph-mon[36429]: pgmap v329: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:21:48.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:48 smithi165 ceph-mon[36269]: pgmap v329: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:21:50.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:50 smithi027 ceph-mon[32179]: pgmap v330: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:21:50.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:50 smithi039 ceph-mon[36429]: pgmap v330: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:21:50.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:50 smithi165 ceph-mon[36269]: pgmap v330: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:21:51.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.336+0000 7fcc724ea700 1 -- 172.21.15.27:0/4087516990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc6c1084f0 msgr2=0x7fcc6c1088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:51.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.336+0000 7fcc724ea700 1 --2- 172.21.15.27:0/4087516990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc6c1084f0 0x7fcc6c1088d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c009990 tx=0x7fcc5c009ca0 comp rx=0 tx=0).stop 2024-05-21T03:21:51.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.336+0000 7fcc724ea700 1 -- 172.21.15.27:0/4087516990 shutdown_connections 2024-05-21T03:21:51.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.336+0000 7fcc724ea700 1 --2- 172.21.15.27:0/4087516990 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc6c102e20 0x7fcc6c10f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:51.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.336+0000 7fcc724ea700 1 --2- 172.21.15.27:0/4087516990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc6c1084f0 0x7fcc6c1088d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:51.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.336+0000 7fcc724ea700 1 --2- 172.21.15.27:0/4087516990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc6c102460 0x7fcc6c1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:51.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.336+0000 7fcc724ea700 1 -- 172.21.15.27:0/4087516990 >> 172.21.15.27:0/4087516990 conn(0x7fcc6c0794e0 msgr2=0x7fcc6c0798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:51.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.336+0000 7fcc724ea700 1 -- 172.21.15.27:0/4087516990 shutdown_connections 2024-05-21T03:21:51.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.336+0000 7fcc724ea700 1 -- 172.21.15.27:0/4087516990 wait complete. 2024-05-21T03:21:51.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.337+0000 7fcc724ea700 1 Processor -- start 2024-05-21T03:21:51.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.337+0000 7fcc724ea700 1 -- start start 2024-05-21T03:21:51.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.337+0000 7fcc724ea700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc6c102460 0x7fcc6c0ff490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:51.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.338+0000 7fcc724ea700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc6c102e20 0x7fcc6c0ff9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:51.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.338+0000 7fcc724ea700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc6c1084f0 0x7fcc6c0fff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:51.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.338+0000 7fcc724ea700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c112680 con 0x7fcc6c1084f0 2024-05-21T03:21:51.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.338+0000 7fcc724ea700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c112500 con 0x7fcc6c102460 2024-05-21T03:21:51.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.338+0000 7fcc724ea700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c112800 con 0x7fcc6c102e20 2024-05-21T03:21:51.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.338+0000 7fcc70a87700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc6c1084f0 0x7fcc6c0fff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:51.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.338+0000 7fcc70a87700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc6c1084f0 0x7fcc6c0fff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34240/0 (socket says 172.21.15.27:34240) 2024-05-21T03:21:51.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.338+0000 7fcc70a87700 1 -- 172.21.15.27:0/4021254569 learned_addr learned my addr 172.21.15.27:0/4021254569 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:21:51.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.338+0000 7fcc6b7fe700 1 --2- 172.21.15.27:0/4021254569 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc6c102e20 0x7fcc6c0ff9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:51.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.338+0000 7fcc6bfff700 1 --2- 172.21.15.27:0/4021254569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc6c102460 0x7fcc6c0ff490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:51.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.338+0000 7fcc70a87700 1 -- 172.21.15.27:0/4021254569 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc6c102e20 msgr2=0x7fcc6c0ff9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:51.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.338+0000 7fcc70a87700 1 --2- 172.21.15.27:0/4021254569 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc6c102e20 0x7fcc6c0ff9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:51.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.338+0000 7fcc70a87700 1 -- 172.21.15.27:0/4021254569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc6c102460 msgr2=0x7fcc6c0ff490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:51.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.338+0000 7fcc70a87700 1 --2- 172.21.15.27:0/4021254569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc6c102460 0x7fcc6c0ff490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:51.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.338+0000 7fcc70a87700 1 -- 172.21.15.27:0/4021254569 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc6c1a8250 con 0x7fcc6c1084f0 2024-05-21T03:21:51.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.338+0000 7fcc70a87700 1 --2- 172.21.15.27:0/4021254569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc6c1084f0 0x7fcc6c0fff10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcc64007ee0 tx=0x7fcc6400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:51.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.338+0000 7fcc697fa700 1 -- 172.21.15.27:0/4021254569 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc64004bb0 con 0x7fcc6c1084f0 2024-05-21T03:21:51.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.338+0000 7fcc724ea700 1 -- 172.21.15.27:0/4021254569 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc6c1a8540 con 0x7fcc6c1084f0 2024-05-21T03:21:51.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.338+0000 7fcc697fa700 1 -- 172.21.15.27:0/4021254569 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc64004d50 con 0x7fcc6c1084f0 2024-05-21T03:21:51.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.338+0000 7fcc697fa700 1 -- 172.21.15.27:0/4021254569 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc64005920 con 0x7fcc6c1084f0 2024-05-21T03:21:51.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.339+0000 7fcc724ea700 1 -- 172.21.15.27:0/4021254569 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc6c1a8a80 con 0x7fcc6c1084f0 2024-05-21T03:21:51.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.339+0000 7fcc724ea700 1 -- 172.21.15.27:0/4021254569 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcc6c063c50 con 0x7fcc6c1084f0 2024-05-21T03:21:51.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.341+0000 7fcc697fa700 1 -- 172.21.15.27:0/4021254569 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fcc6401a020 con 0x7fcc6c1084f0 2024-05-21T03:21:51.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.342+0000 7fcc697fa700 1 --2- 172.21.15.27:0/4021254569 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fcc540b2910 0x7fcc540b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:51.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.342+0000 7fcc6bfff700 1 --2- 172.21.15.27:0/4021254569 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fcc540b2910 0x7fcc540b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:51.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.342+0000 7fcc697fa700 1 -- 172.21.15.27:0/4021254569 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fcc640e2130 con 0x7fcc6c1084f0 2024-05-21T03:21:51.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.342+0000 7fcc697fa700 1 --2- 172.21.15.27:0/4021254569 >> [v2:172.21.15.165:6816/3184704971,v1:172.21.15.165:6817/3184704971] conn(0x7fcc540bc550 0x7fcc540be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:51.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.342+0000 7fcc697fa700 1 -- 172.21.15.27:0/4021254569 --> [v2:172.21.15.165:6816/3184704971,v1:172.21.15.165:6817/3184704971] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcc540bf0c0 con 0x7fcc540bc550 2024-05-21T03:21:51.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.342+0000 7fcc697fa700 1 -- 172.21.15.27:0/4021254569 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fcc640e2580 con 0x7fcc6c1084f0 2024-05-21T03:21:51.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.342+0000 7fcc6bfff700 1 --2- 172.21.15.27:0/4021254569 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fcc540b2910 0x7fcc540b4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00b5c0 tx=0x7fcc5c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:51.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.342+0000 7fcc6b7fe700 1 --2- 172.21.15.27:0/4021254569 >> [v2:172.21.15.165:6816/3184704971,v1:172.21.15.165:6817/3184704971] conn(0x7fcc540bc550 0x7fcc540be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:51.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.343+0000 7fcc6b7fe700 1 --2- 172.21.15.27:0/4021254569 >> [v2:172.21.15.165:6816/3184704971,v1:172.21.15.165:6817/3184704971] conn(0x7fcc540bc550 0x7fcc540be9f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:51.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.344+0000 7fcc697fa700 1 -- 172.21.15.27:0/4021254569 <== osd.8 v2:172.21.15.165:6816/3184704971 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcc540bf0c0 con 0x7fcc540bc550 2024-05-21T03:21:51.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.361+0000 7fcc724ea700 1 -- 172.21.15.27:0/4021254569 --> [v2:172.21.15.165:6816/3184704971,v1:172.21.15.165:6817/3184704971] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcc6c064050 con 0x7fcc540bc550 2024-05-21T03:21:51.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.361+0000 7fcc697fa700 1 -- 172.21.15.27:0/4021254569 <== osd.8 v2:172.21.15.165:6816/3184704971 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcc6c064050 con 0x7fcc540bc550 2024-05-21T03:21:51.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.362+0000 7fcc724ea700 1 -- 172.21.15.27:0/4021254569 >> [v2:172.21.15.165:6816/3184704971,v1:172.21.15.165:6817/3184704971] conn(0x7fcc540bc550 msgr2=0x7fcc540be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:51.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.362+0000 7fcc724ea700 1 --2- 172.21.15.27:0/4021254569 >> [v2:172.21.15.165:6816/3184704971,v1:172.21.15.165:6817/3184704971] conn(0x7fcc540bc550 0x7fcc540be9f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:51.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.362+0000 7fcc724ea700 1 -- 172.21.15.27:0/4021254569 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fcc540b2910 msgr2=0x7fcc540b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:51.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.362+0000 7fcc724ea700 1 --2- 172.21.15.27:0/4021254569 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fcc540b2910 0x7fcc540b4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00b5c0 tx=0x7fcc5c005960 comp rx=0 tx=0).stop 2024-05-21T03:21:51.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.362+0000 7fcc724ea700 1 -- 172.21.15.27:0/4021254569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc6c1084f0 msgr2=0x7fcc6c0fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:51.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.362+0000 7fcc724ea700 1 --2- 172.21.15.27:0/4021254569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc6c1084f0 0x7fcc6c0fff10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcc64007ee0 tx=0x7fcc6400d3b0 comp rx=0 tx=0).stop 2024-05-21T03:21:51.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.362+0000 7fcc70a87700 1 -- 172.21.15.27:0/4021254569 reap_dead start 2024-05-21T03:21:51.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.362+0000 7fcc724ea700 1 -- 172.21.15.27:0/4021254569 shutdown_connections 2024-05-21T03:21:51.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.362+0000 7fcc724ea700 1 -- 172.21.15.27:0/4021254569 >> 172.21.15.27:0/4021254569 conn(0x7fcc6c0794e0 msgr2=0x7fcc6c06c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:51.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.362+0000 7fcc724ea700 1 -- 172.21.15.27:0/4021254569 shutdown_connections 2024-05-21T03:21:51.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:51.362+0000 7fcc724ea700 1 -- 172.21.15.27:0/4021254569 wait complete. 2024-05-21T03:21:52.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:52 smithi027 ceph-mon[32179]: pgmap v331: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:21:52.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:52 smithi039 ceph-mon[36429]: pgmap v331: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:21:52.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:52 smithi165 ceph-mon[36269]: pgmap v331: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:21:53.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.705+0000 7fa380671700 1 -- 172.21.15.27:0/3282194172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3781026a0 msgr2=0x7fa378108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:53.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.705+0000 7fa380671700 1 --2- 172.21.15.27:0/3282194172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3781026a0 0x7fa378108200 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa3680099c0 tx=0x7fa368009cd0 comp rx=0 tx=0).stop 2024-05-21T03:21:53.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.705+0000 7fa380671700 1 -- 172.21.15.27:0/3282194172 shutdown_connections 2024-05-21T03:21:53.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.705+0000 7fa380671700 1 --2- 172.21.15.27:0/3282194172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa378108930 0x7fa3781104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:53.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.705+0000 7fa380671700 1 --2- 172.21.15.27:0/3282194172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa378101cf0 0x7fa3781020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:53.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.705+0000 7fa380671700 1 --2- 172.21.15.27:0/3282194172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3781026a0 0x7fa378108200 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:53.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.705+0000 7fa380671700 1 -- 172.21.15.27:0/3282194172 >> 172.21.15.27:0/3282194172 conn(0x7fa378078a40 msgr2=0x7fa3780ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:53.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.705+0000 7fa380671700 1 -- 172.21.15.27:0/3282194172 shutdown_connections 2024-05-21T03:21:53.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.705+0000 7fa380671700 1 -- 172.21.15.27:0/3282194172 wait complete. 2024-05-21T03:21:53.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.706+0000 7fa380671700 1 Processor -- start 2024-05-21T03:21:53.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.706+0000 7fa380671700 1 -- start start 2024-05-21T03:21:53.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.706+0000 7fa380671700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa378101cf0 0x7fa378102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:53.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.706+0000 7fa380671700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3781026a0 0x7fa3781031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:53.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.706+0000 7fa380671700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa378108930 0x7fa3781036f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:53.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.706+0000 7fa380671700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa378113760 con 0x7fa378101cf0 2024-05-21T03:21:53.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.706+0000 7fa380671700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa3781135e0 con 0x7fa378108930 2024-05-21T03:21:53.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.706+0000 7fa380671700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa3781138e0 con 0x7fa3781026a0 2024-05-21T03:21:53.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.706+0000 7fa37e40d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa378101cf0 0x7fa378102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:53.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.706+0000 7fa37e40d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa378101cf0 0x7fa378102c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56816/0 (socket says 172.21.15.27:56816) 2024-05-21T03:21:53.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.706+0000 7fa37e40d700 1 -- 172.21.15.27:0/1184300838 learned_addr learned my addr 172.21.15.27:0/1184300838 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:21:53.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.706+0000 7fa37dc0c700 1 --2- 172.21.15.27:0/1184300838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3781026a0 0x7fa3781031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:53.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.706+0000 7fa37ec0e700 1 --2- 172.21.15.27:0/1184300838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa378108930 0x7fa3781036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:53.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.706+0000 7fa37e40d700 1 -- 172.21.15.27:0/1184300838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3781026a0 msgr2=0x7fa3781031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:53.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.706+0000 7fa37e40d700 1 --2- 172.21.15.27:0/1184300838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3781026a0 0x7fa3781031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:53.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.706+0000 7fa37e40d700 1 -- 172.21.15.27:0/1184300838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa378108930 msgr2=0x7fa3781036f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:53.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.707+0000 7fa37e40d700 1 --2- 172.21.15.27:0/1184300838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa378108930 0x7fa3781036f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:53.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.707+0000 7fa37e40d700 1 -- 172.21.15.27:0/1184300838 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3781a8100 con 0x7fa378101cf0 2024-05-21T03:21:53.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.707+0000 7fa37ec0e700 1 --2- 172.21.15.27:0/1184300838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa378108930 0x7fa3781036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:21:53.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.707+0000 7fa37e40d700 1 --2- 172.21.15.27:0/1184300838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa378101cf0 0x7fa378102c70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa37400c920 tx=0x7fa37400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:53.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.707+0000 7fa36f7fe700 1 -- 172.21.15.27:0/1184300838 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa374007b00 con 0x7fa378101cf0 2024-05-21T03:21:53.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.707+0000 7fa36f7fe700 1 -- 172.21.15.27:0/1184300838 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa374004ce0 con 0x7fa378101cf0 2024-05-21T03:21:53.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.707+0000 7fa36f7fe700 1 -- 172.21.15.27:0/1184300838 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa374005550 con 0x7fa378101cf0 2024-05-21T03:21:53.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.707+0000 7fa380671700 1 -- 172.21.15.27:0/1184300838 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3781a83f0 con 0x7fa378101cf0 2024-05-21T03:21:53.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.707+0000 7fa380671700 1 -- 172.21.15.27:0/1184300838 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3781a8950 con 0x7fa378101cf0 2024-05-21T03:21:53.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.708+0000 7fa380671700 1 -- 172.21.15.27:0/1184300838 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa378063cd0 con 0x7fa378101cf0 2024-05-21T03:21:53.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.709+0000 7fa36f7fe700 1 -- 172.21.15.27:0/1184300838 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa374007ca0 con 0x7fa378101cf0 2024-05-21T03:21:53.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.710+0000 7fa36f7fe700 1 --2- 172.21.15.27:0/1184300838 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fa3640b2990 0x7fa3640b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:53.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.710+0000 7fa37dc0c700 1 --2- 172.21.15.27:0/1184300838 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fa3640b2990 0x7fa3640b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:53.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.710+0000 7fa36f7fe700 1 -- 172.21.15.27:0/1184300838 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fa3740e2470 con 0x7fa378101cf0 2024-05-21T03:21:53.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.710+0000 7fa36f7fe700 1 --2- 172.21.15.27:0/1184300838 >> [v2:172.21.15.165:6800/703322625,v1:172.21.15.165:6801/703322625] conn(0x7fa3640bc5d0 0x7fa3640bea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:53.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.710+0000 7fa36f7fe700 1 -- 172.21.15.27:0/1184300838 --> [v2:172.21.15.165:6800/703322625,v1:172.21.15.165:6801/703322625] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa374012ec0 con 0x7fa3640bc5d0 2024-05-21T03:21:53.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.710+0000 7fa36f7fe700 1 -- 172.21.15.27:0/1184300838 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fa3740e28c0 con 0x7fa378101cf0 2024-05-21T03:21:53.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.711+0000 7fa37dc0c700 1 --2- 172.21.15.27:0/1184300838 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fa3640b2990 0x7fa3640b4e50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa378104660 tx=0x7fa368005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:53.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.711+0000 7fa37ec0e700 1 --2- 172.21.15.27:0/1184300838 >> [v2:172.21.15.165:6800/703322625,v1:172.21.15.165:6801/703322625] conn(0x7fa3640bc5d0 0x7fa3640bea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:53.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.711+0000 7fa37ec0e700 1 --2- 172.21.15.27:0/1184300838 >> [v2:172.21.15.165:6800/703322625,v1:172.21.15.165:6801/703322625] conn(0x7fa3640bc5d0 0x7fa3640bea70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:53.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.712+0000 7fa36f7fe700 1 -- 172.21.15.27:0/1184300838 <== osd.2 v2:172.21.15.165:6800/703322625 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa374012ec0 con 0x7fa3640bc5d0 2024-05-21T03:21:53.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.729+0000 7fa380671700 1 -- 172.21.15.27:0/1184300838 --> [v2:172.21.15.165:6800/703322625,v1:172.21.15.165:6801/703322625] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa378064150 con 0x7fa3640bc5d0 2024-05-21T03:21:53.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.730+0000 7fa36f7fe700 1 -- 172.21.15.27:0/1184300838 <== osd.2 v2:172.21.15.165:6800/703322625 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa378064150 con 0x7fa3640bc5d0 2024-05-21T03:21:53.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.730+0000 7fa380671700 1 -- 172.21.15.27:0/1184300838 >> [v2:172.21.15.165:6800/703322625,v1:172.21.15.165:6801/703322625] conn(0x7fa3640bc5d0 msgr2=0x7fa3640bea70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:53.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.730+0000 7fa380671700 1 --2- 172.21.15.27:0/1184300838 >> [v2:172.21.15.165:6800/703322625,v1:172.21.15.165:6801/703322625] conn(0x7fa3640bc5d0 0x7fa3640bea70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:53.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.730+0000 7fa380671700 1 -- 172.21.15.27:0/1184300838 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fa3640b2990 msgr2=0x7fa3640b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:53.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.730+0000 7fa380671700 1 --2- 172.21.15.27:0/1184300838 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fa3640b2990 0x7fa3640b4e50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa378104660 tx=0x7fa368005fb0 comp rx=0 tx=0).stop 2024-05-21T03:21:53.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.730+0000 7fa380671700 1 -- 172.21.15.27:0/1184300838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa378101cf0 msgr2=0x7fa378102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:53.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.730+0000 7fa380671700 1 --2- 172.21.15.27:0/1184300838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa378101cf0 0x7fa378102c70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa37400c920 tx=0x7fa37400cc30 comp rx=0 tx=0).stop 2024-05-21T03:21:53.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.730+0000 7fa37ec0e700 1 -- 172.21.15.27:0/1184300838 reap_dead start 2024-05-21T03:21:53.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.730+0000 7fa380671700 1 -- 172.21.15.27:0/1184300838 shutdown_connections 2024-05-21T03:21:53.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.730+0000 7fa380671700 1 -- 172.21.15.27:0/1184300838 >> 172.21.15.27:0/1184300838 conn(0x7fa378078a40 msgr2=0x7fa3781070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:53.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.730+0000 7fa380671700 1 -- 172.21.15.27:0/1184300838 shutdown_connections 2024-05-21T03:21:53.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:53.730+0000 7fa380671700 1 -- 172.21.15.27:0/1184300838 wait complete. 2024-05-21T03:21:54.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:54 smithi027 ceph-mon[32179]: pgmap v332: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:21:54.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:54 smithi039 ceph-mon[36429]: pgmap v332: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:21:54.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:54 smithi165 ceph-mon[36269]: pgmap v332: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:21:55.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.751+0000 7ff808599700 1 -- 172.21.15.27:0/3728646593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff800108510 msgr2=0x7ff8001088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:55.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.751+0000 7ff808599700 1 --2- 172.21.15.27:0/3728646593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff800108510 0x7ff8001088f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc009990 tx=0x7ff7fc009ca0 comp rx=0 tx=0).stop 2024-05-21T03:21:55.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.751+0000 7ff808599700 1 -- 172.21.15.27:0/3728646593 shutdown_connections 2024-05-21T03:21:55.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.751+0000 7ff808599700 1 --2- 172.21.15.27:0/3728646593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff800102df0 0x7ff80010f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:55.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.751+0000 7ff808599700 1 --2- 172.21.15.27:0/3728646593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff800108510 0x7ff8001088f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:55.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.751+0000 7ff808599700 1 --2- 172.21.15.27:0/3728646593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff800102430 0x7ff8001028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:55.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.751+0000 7ff808599700 1 -- 172.21.15.27:0/3728646593 >> 172.21.15.27:0/3728646593 conn(0x7ff8000793b0 msgr2=0x7ff8000797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:55.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.752+0000 7ff808599700 1 -- 172.21.15.27:0/3728646593 shutdown_connections 2024-05-21T03:21:55.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.752+0000 7ff808599700 1 -- 172.21.15.27:0/3728646593 wait complete. 2024-05-21T03:21:55.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.752+0000 7ff808599700 1 Processor -- start 2024-05-21T03:21:55.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.752+0000 7ff808599700 1 -- start start 2024-05-21T03:21:55.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.752+0000 7ff808599700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff800102430 0x7ff800104450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:55.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.752+0000 7ff808599700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff800102df0 0x7ff800104990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:55.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.752+0000 7ff808599700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff800108510 0x7ff800104ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:55.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.752+0000 7ff808599700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff8001125c0 con 0x7ff800102430 2024-05-21T03:21:55.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.752+0000 7ff808599700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff800112440 con 0x7ff800108510 2024-05-21T03:21:55.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.752+0000 7ff808599700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff800112740 con 0x7ff800102df0 2024-05-21T03:21:55.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.753+0000 7ff806335700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff800102430 0x7ff800104450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:55.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.753+0000 7ff806335700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff800102430 0x7ff800104450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56838/0 (socket says 172.21.15.27:56838) 2024-05-21T03:21:55.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.753+0000 7ff806335700 1 -- 172.21.15.27:0/2647324717 learned_addr learned my addr 172.21.15.27:0/2647324717 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:21:55.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.753+0000 7ff806b36700 1 --2- 172.21.15.27:0/2647324717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff800108510 0x7ff800104ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:55.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.753+0000 7ff805b34700 1 --2- 172.21.15.27:0/2647324717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff800102df0 0x7ff800104990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:55.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.753+0000 7ff806335700 1 -- 172.21.15.27:0/2647324717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff800102df0 msgr2=0x7ff800104990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:55.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.753+0000 7ff806335700 1 --2- 172.21.15.27:0/2647324717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff800102df0 0x7ff800104990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:55.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.753+0000 7ff806335700 1 -- 172.21.15.27:0/2647324717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff800108510 msgr2=0x7ff800104ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:55.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.753+0000 7ff806335700 1 --2- 172.21.15.27:0/2647324717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff800108510 0x7ff800104ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:55.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.753+0000 7ff806335700 1 -- 172.21.15.27:0/2647324717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8001a8130 con 0x7ff800102430 2024-05-21T03:21:55.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.753+0000 7ff806335700 1 --2- 172.21.15.27:0/2647324717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff800102430 0x7ff800104450 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc000c00 tx=0x7ff7fc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:55.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.753+0000 7ff7f37fe700 1 -- 172.21.15.27:0/2647324717 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc027070 con 0x7ff800102430 2024-05-21T03:21:55.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.753+0000 7ff7f37fe700 1 -- 172.21.15.27:0/2647324717 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7fc003f20 con 0x7ff800102430 2024-05-21T03:21:55.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.753+0000 7ff808599700 1 -- 172.21.15.27:0/2647324717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8001a83c0 con 0x7ff800102430 2024-05-21T03:21:55.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.753+0000 7ff7f37fe700 1 -- 172.21.15.27:0/2647324717 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc022cf0 con 0x7ff800102430 2024-05-21T03:21:55.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.753+0000 7ff808599700 1 -- 172.21.15.27:0/2647324717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8001a8810 con 0x7ff800102430 2024-05-21T03:21:55.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.754+0000 7ff808599700 1 -- 172.21.15.27:0/2647324717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff8001011c0 con 0x7ff800102430 2024-05-21T03:21:55.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.756+0000 7ff7f37fe700 1 -- 172.21.15.27:0/2647324717 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff7fc02a070 con 0x7ff800102430 2024-05-21T03:21:55.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.757+0000 7ff7f37fe700 1 --2- 172.21.15.27:0/2647324717 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7ff7ec0b2910 0x7ff7ec0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:55.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.757+0000 7ff805b34700 1 --2- 172.21.15.27:0/2647324717 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7ff7ec0b2910 0x7ff7ec0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:55.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.757+0000 7ff7f37fe700 1 -- 172.21.15.27:0/2647324717 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7ff7fc0e37a0 con 0x7ff800102430 2024-05-21T03:21:55.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.757+0000 7ff7f37fe700 1 --2- 172.21.15.27:0/2647324717 >> [v2:172.21.15.39:6808/155579331,v1:172.21.15.39:6809/155579331] conn(0x7ff7ec0bc5b0 0x7ff7ec0bea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:55.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.757+0000 7ff7f37fe700 1 -- 172.21.15.27:0/2647324717 --> [v2:172.21.15.39:6808/155579331,v1:172.21.15.39:6809/155579331] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff7fc023e50 con 0x7ff7ec0bc5b0 2024-05-21T03:21:55.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.757+0000 7ff7f37fe700 1 -- 172.21.15.27:0/2647324717 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7ff7fc0e3bf0 con 0x7ff800102430 2024-05-21T03:21:55.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.757+0000 7ff805b34700 1 --2- 172.21.15.27:0/2647324717 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7ff7ec0b2910 0x7ff7ec0b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4005ea0 tx=0x7ff7f4005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:55.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.757+0000 7ff806b36700 1 --2- 172.21.15.27:0/2647324717 >> [v2:172.21.15.39:6808/155579331,v1:172.21.15.39:6809/155579331] conn(0x7ff7ec0bc5b0 0x7ff7ec0bea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:55.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.758+0000 7ff806b36700 1 --2- 172.21.15.27:0/2647324717 >> [v2:172.21.15.39:6808/155579331,v1:172.21.15.39:6809/155579331] conn(0x7ff7ec0bc5b0 0x7ff7ec0bea50 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:55.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.759+0000 7ff7f37fe700 1 -- 172.21.15.27:0/2647324717 <== osd.4 v2:172.21.15.39:6808/155579331 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff7fc023e50 con 0x7ff7ec0bc5b0 2024-05-21T03:21:55.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.776+0000 7ff808599700 1 -- 172.21.15.27:0/2647324717 --> [v2:172.21.15.39:6808/155579331,v1:172.21.15.39:6809/155579331] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff800103b50 con 0x7ff7ec0bc5b0 2024-05-21T03:21:55.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.776+0000 7ff7f37fe700 1 -- 172.21.15.27:0/2647324717 <== osd.4 v2:172.21.15.39:6808/155579331 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff800103b50 con 0x7ff7ec0bc5b0 2024-05-21T03:21:55.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.777+0000 7ff808599700 1 -- 172.21.15.27:0/2647324717 >> [v2:172.21.15.39:6808/155579331,v1:172.21.15.39:6809/155579331] conn(0x7ff7ec0bc5b0 msgr2=0x7ff7ec0bea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:55.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.777+0000 7ff808599700 1 --2- 172.21.15.27:0/2647324717 >> [v2:172.21.15.39:6808/155579331,v1:172.21.15.39:6809/155579331] conn(0x7ff7ec0bc5b0 0x7ff7ec0bea50 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:55.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.777+0000 7ff808599700 1 -- 172.21.15.27:0/2647324717 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7ff7ec0b2910 msgr2=0x7ff7ec0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:55.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.777+0000 7ff808599700 1 --2- 172.21.15.27:0/2647324717 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7ff7ec0b2910 0x7ff7ec0b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4005ea0 tx=0x7ff7f4005e30 comp rx=0 tx=0).stop 2024-05-21T03:21:55.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.777+0000 7ff808599700 1 -- 172.21.15.27:0/2647324717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff800102430 msgr2=0x7ff800104450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:55.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.777+0000 7ff808599700 1 --2- 172.21.15.27:0/2647324717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff800102430 0x7ff800104450 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc000c00 tx=0x7ff7fc0047b0 comp rx=0 tx=0).stop 2024-05-21T03:21:55.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.777+0000 7ff806b36700 1 -- 172.21.15.27:0/2647324717 reap_dead start 2024-05-21T03:21:55.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.777+0000 7ff808599700 1 -- 172.21.15.27:0/2647324717 shutdown_connections 2024-05-21T03:21:55.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.777+0000 7ff808599700 1 -- 172.21.15.27:0/2647324717 >> 172.21.15.27:0/2647324717 conn(0x7ff8000793b0 msgr2=0x7ff8001066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:55.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.777+0000 7ff808599700 1 -- 172.21.15.27:0/2647324717 shutdown_connections 2024-05-21T03:21:55.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:55.777+0000 7ff808599700 1 -- 172.21.15.27:0/2647324717 wait complete. 2024-05-21T03:21:56.532 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:56 smithi027 ceph-mon[32179]: pgmap v333: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-05-21T03:21:56.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:56 smithi039 ceph-mon[36429]: pgmap v333: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-05-21T03:21:56.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:56 smithi165 ceph-mon[36269]: pgmap v333: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-05-21T03:21:58.528 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:21:58 smithi027 ceph-mon[32179]: pgmap v334: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 10 op/s 2024-05-21T03:21:58.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:21:58 smithi039 ceph-mon[36429]: pgmap v334: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 10 op/s 2024-05-21T03:21:58.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:21:58 smithi165 ceph-mon[36269]: pgmap v334: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 10 op/s 2024-05-21T03:21:59.488 INFO:teuthology.orchestra.run.smithi027.stdout:236223201326 2024-05-21T03:21:59.489 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-21T03:21:59.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.691+0000 7fdc92109700 1 -- 172.21.15.27:0/535953399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc8c1041d0 msgr2=0x7fdc8c1081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:59.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.691+0000 7fdc92109700 1 --2- 172.21.15.27:0/535953399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc8c1041d0 0x7fdc8c1081e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdc800099c0 tx=0x7fdc80009cd0 comp rx=0 tx=0).stop 2024-05-21T03:21:59.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.691+0000 7fdc92109700 1 -- 172.21.15.27:0/535953399 shutdown_connections 2024-05-21T03:21:59.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.691+0000 7fdc92109700 1 --2- 172.21.15.27:0/535953399 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc8c108910 0x7fdc8c110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:59.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.691+0000 7fdc92109700 1 --2- 172.21.15.27:0/535953399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc8c103820 0x7fdc8c103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:59.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.691+0000 7fdc92109700 1 --2- 172.21.15.27:0/535953399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc8c1041d0 0x7fdc8c1081e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:59.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.691+0000 7fdc92109700 1 -- 172.21.15.27:0/535953399 >> 172.21.15.27:0/535953399 conn(0x7fdc8c07a6b0 msgr2=0x7fdc8c07aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:59.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.691+0000 7fdc92109700 1 -- 172.21.15.27:0/535953399 shutdown_connections 2024-05-21T03:21:59.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.691+0000 7fdc92109700 1 -- 172.21.15.27:0/535953399 wait complete. 2024-05-21T03:21:59.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.692+0000 7fdc92109700 1 Processor -- start 2024-05-21T03:21:59.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.693+0000 7fdc92109700 1 -- start start 2024-05-21T03:21:59.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.693+0000 7fdc92109700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc8c103820 0x7fdc8c075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:59.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.693+0000 7fdc92109700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc8c1041d0 0x7fdc8c075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:59.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.693+0000 7fdc92109700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc8c108910 0x7fdc8c0762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:59.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.693+0000 7fdc8b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc8c103820 0x7fdc8c075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:59.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.693+0000 7fdc8b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc8c103820 0x7fdc8c075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56862/0 (socket says 172.21.15.27:56862) 2024-05-21T03:21:59.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.693+0000 7fdc8b7fe700 1 -- 172.21.15.27:0/4246247075 learned_addr learned my addr 172.21.15.27:0/4246247075 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:21:59.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.693+0000 7fdc8affd700 1 --2- 172.21.15.27:0/4246247075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc8c1041d0 0x7fdc8c075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:59.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.693+0000 7fdc92109700 1 -- 172.21.15.27:0/4246247075 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c1136f0 con 0x7fdc8c103820 2024-05-21T03:21:59.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.693+0000 7fdc92109700 1 -- 172.21.15.27:0/4246247075 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c113570 con 0x7fdc8c108910 2024-05-21T03:21:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.693+0000 7fdc92109700 1 -- 172.21.15.27:0/4246247075 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c113870 con 0x7fdc8c1041d0 2024-05-21T03:21:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.693+0000 7fdc8bfff700 1 --2- 172.21.15.27:0/4246247075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc8c108910 0x7fdc8c0762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.693+0000 7fdc8b7fe700 1 -- 172.21.15.27:0/4246247075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc8c1041d0 msgr2=0x7fdc8c075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.693+0000 7fdc8b7fe700 1 --2- 172.21.15.27:0/4246247075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc8c1041d0 0x7fdc8c075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.693+0000 7fdc8b7fe700 1 -- 172.21.15.27:0/4246247075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc8c108910 msgr2=0x7fdc8c0762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.693+0000 7fdc8b7fe700 1 --2- 172.21.15.27:0/4246247075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc8c108910 0x7fdc8c0762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.694+0000 7fdc8b7fe700 1 -- 172.21.15.27:0/4246247075 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc8c1a8150 con 0x7fdc8c103820 2024-05-21T03:21:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.694+0000 7fdc8affd700 1 --2- 172.21.15.27:0/4246247075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc8c1041d0 0x7fdc8c075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:21:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.694+0000 7fdc8b7fe700 1 --2- 172.21.15.27:0/4246247075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc8c103820 0x7fdc8c075840 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c00e9c0 tx=0x7fdc7c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.694+0000 7fdc88ff9700 1 -- 172.21.15.27:0/4246247075 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c00cbd0 con 0x7fdc8c103820 2024-05-21T03:21:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.694+0000 7fdc88ff9700 1 -- 172.21.15.27:0/4246247075 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc7c00ee60 con 0x7fdc8c103820 2024-05-21T03:21:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.694+0000 7fdc92109700 1 -- 172.21.15.27:0/4246247075 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc8c1a8440 con 0x7fdc8c103820 2024-05-21T03:21:59.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.694+0000 7fdc88ff9700 1 -- 172.21.15.27:0/4246247075 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c0055c0 con 0x7fdc8c103820 2024-05-21T03:21:59.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.694+0000 7fdc92109700 1 -- 172.21.15.27:0/4246247075 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc8c1a8900 con 0x7fdc8c103820 2024-05-21T03:21:59.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.695+0000 7fdc92109700 1 -- 172.21.15.27:0/4246247075 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdc8c063c50 con 0x7fdc8c103820 2024-05-21T03:21:59.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.696+0000 7fdc88ff9700 1 -- 172.21.15.27:0/4246247075 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fdc7c004750 con 0x7fdc8c103820 2024-05-21T03:21:59.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.697+0000 7fdc88ff9700 1 --2- 172.21.15.27:0/4246247075 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fdc740b2910 0x7fdc740b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:59.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.697+0000 7fdc8affd700 1 --2- 172.21.15.27:0/4246247075 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fdc740b2910 0x7fdc740b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:59.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.698+0000 7fdc88ff9700 1 -- 172.21.15.27:0/4246247075 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fdc7c0e2a50 con 0x7fdc8c103820 2024-05-21T03:21:59.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.698+0000 7fdc88ff9700 1 --2- 172.21.15.27:0/4246247075 >> [v2:172.21.15.165:6824/2274332243,v1:172.21.15.165:6825/2274332243] conn(0x7fdc740bc550 0x7fdc740be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:21:59.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.698+0000 7fdc88ff9700 1 -- 172.21.15.27:0/4246247075 --> [v2:172.21.15.165:6824/2274332243,v1:172.21.15.165:6825/2274332243] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdc7c00be50 con 0x7fdc740bc550 2024-05-21T03:21:59.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.698+0000 7fdc88ff9700 1 -- 172.21.15.27:0/4246247075 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fdc7c0e2ea0 con 0x7fdc8c103820 2024-05-21T03:21:59.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.698+0000 7fdc8affd700 1 --2- 172.21.15.27:0/4246247075 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fdc740b2910 0x7fdc740b4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c0772d0 tx=0x7fdc80005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:59.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.698+0000 7fdc8bfff700 1 --2- 172.21.15.27:0/4246247075 >> [v2:172.21.15.165:6824/2274332243,v1:172.21.15.165:6825/2274332243] conn(0x7fdc740bc550 0x7fdc740be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:21:59.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.698+0000 7fdc8bfff700 1 --2- 172.21.15.27:0/4246247075 >> [v2:172.21.15.165:6824/2274332243,v1:172.21.15.165:6825/2274332243] conn(0x7fdc740bc550 0x7fdc740be9f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:21:59.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.699+0000 7fdc88ff9700 1 -- 172.21.15.27:0/4246247075 <== osd.11 v2:172.21.15.165:6824/2274332243 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdc7c00be50 con 0x7fdc740bc550 2024-05-21T03:21:59.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.717+0000 7fdc92109700 1 -- 172.21.15.27:0/4246247075 --> [v2:172.21.15.165:6824/2274332243,v1:172.21.15.165:6825/2274332243] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdc8c064050 con 0x7fdc740bc550 2024-05-21T03:21:59.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.718+0000 7fdc88ff9700 1 -- 172.21.15.27:0/4246247075 <== osd.11 v2:172.21.15.165:6824/2274332243 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdc8c064050 con 0x7fdc740bc550 2024-05-21T03:21:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.718+0000 7fdc92109700 1 -- 172.21.15.27:0/4246247075 >> [v2:172.21.15.165:6824/2274332243,v1:172.21.15.165:6825/2274332243] conn(0x7fdc740bc550 msgr2=0x7fdc740be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.718+0000 7fdc92109700 1 --2- 172.21.15.27:0/4246247075 >> [v2:172.21.15.165:6824/2274332243,v1:172.21.15.165:6825/2274332243] conn(0x7fdc740bc550 0x7fdc740be9f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:21:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.718+0000 7fdc92109700 1 -- 172.21.15.27:0/4246247075 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fdc740b2910 msgr2=0x7fdc740b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.718+0000 7fdc92109700 1 --2- 172.21.15.27:0/4246247075 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fdc740b2910 0x7fdc740b4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c0772d0 tx=0x7fdc80005fb0 comp rx=0 tx=0).stop 2024-05-21T03:21:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.718+0000 7fdc92109700 1 -- 172.21.15.27:0/4246247075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc8c103820 msgr2=0x7fdc8c075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:21:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.718+0000 7fdc92109700 1 --2- 172.21.15.27:0/4246247075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc8c103820 0x7fdc8c075840 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c00e9c0 tx=0x7fdc7c00ecd0 comp rx=0 tx=0).stop 2024-05-21T03:21:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.718+0000 7fdc8bfff700 1 -- 172.21.15.27:0/4246247075 reap_dead start 2024-05-21T03:21:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.718+0000 7fdc92109700 1 -- 172.21.15.27:0/4246247075 shutdown_connections 2024-05-21T03:21:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.718+0000 7fdc92109700 1 -- 172.21.15.27:0/4246247075 >> 172.21.15.27:0/4246247075 conn(0x7fdc8c07a6b0 msgr2=0x7fdc8c101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:21:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.719+0000 7fdc92109700 1 -- 172.21.15.27:0/4246247075 shutdown_connections 2024-05-21T03:21:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:21:59.719+0000 7fdc92109700 1 -- 172.21.15.27:0/4246247075 wait complete. 2024-05-21T03:22:00.322 INFO:teuthology.orchestra.run.smithi027.stdout:137438953540 2024-05-21T03:22:00.323 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-21T03:22:00.580 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:00 smithi027 ceph-mon[32179]: pgmap v335: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-21T03:22:00.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:00 smithi039 ceph-mon[36429]: pgmap v335: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-21T03:22:00.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:00 smithi165 ceph-mon[36269]: pgmap v335: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-21T03:22:01.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.923+0000 7f91e5ecd700 1 -- 172.21.15.27:0/2471971878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91e006c150 msgr2=0x7f91e006c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:01.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.923+0000 7f91e5ecd700 1 --2- 172.21.15.27:0/2471971878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91e006c150 0x7f91e006c5b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f91c8009990 tx=0x7f91c8009ca0 comp rx=0 tx=0).stop 2024-05-21T03:22:01.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.923+0000 7f91e5ecd700 1 -- 172.21.15.27:0/2471971878 shutdown_connections 2024-05-21T03:22:01.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.923+0000 7f91e5ecd700 1 --2- 172.21.15.27:0/2471971878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91e010fe90 0x7f91e0112280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:01.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.923+0000 7f91e5ecd700 1 --2- 172.21.15.27:0/2471971878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91e0104450 0x7f91e006bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:01.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.923+0000 7f91e5ecd700 1 --2- 172.21.15.27:0/2471971878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91e006c150 0x7f91e006c5b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:01.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.923+0000 7f91e5ecd700 1 -- 172.21.15.27:0/2471971878 >> 172.21.15.27:0/2471971878 conn(0x7f91e00793f0 msgr2=0x7f91e0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:01.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.923+0000 7f91e5ecd700 1 -- 172.21.15.27:0/2471971878 shutdown_connections 2024-05-21T03:22:01.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.923+0000 7f91e5ecd700 1 -- 172.21.15.27:0/2471971878 wait complete. 2024-05-21T03:22:01.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.924+0000 7f91e5ecd700 1 Processor -- start 2024-05-21T03:22:01.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.924+0000 7f91e5ecd700 1 -- start start 2024-05-21T03:22:01.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.924+0000 7f91e5ecd700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91e006c150 0x7f91e019bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:01.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.924+0000 7f91e5ecd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91e0104450 0x7f91e019c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:01.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.924+0000 7f91e5ecd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91e010fe90 0x7f91e01a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:01.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.924+0000 7f91e5ecd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f91e0115900 con 0x7f91e0104450 2024-05-21T03:22:01.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.924+0000 7f91e5ecd700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f91e0115780 con 0x7f91e010fe90 2024-05-21T03:22:01.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.924+0000 7f91e5ecd700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f91e0115a80 con 0x7f91e006c150 2024-05-21T03:22:01.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.924+0000 7f91deffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91e0104450 0x7f91e019c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:01.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.924+0000 7f91deffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91e0104450 0x7f91e019c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56906/0 (socket says 172.21.15.27:56906) 2024-05-21T03:22:01.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.924+0000 7f91deffd700 1 -- 172.21.15.27:0/2308457813 learned_addr learned my addr 172.21.15.27:0/2308457813 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:01.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.924+0000 7f91dffff700 1 --2- 172.21.15.27:0/2308457813 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91e010fe90 0x7f91e01a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:01.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.925+0000 7f91df7fe700 1 --2- 172.21.15.27:0/2308457813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91e006c150 0x7f91e019bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:01.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.925+0000 7f91deffd700 1 -- 172.21.15.27:0/2308457813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91e006c150 msgr2=0x7f91e019bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:01.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.925+0000 7f91deffd700 1 --2- 172.21.15.27:0/2308457813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91e006c150 0x7f91e019bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.925+0000 7f91deffd700 1 -- 172.21.15.27:0/2308457813 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91e010fe90 msgr2=0x7f91e01a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.925+0000 7f91deffd700 1 --2- 172.21.15.27:0/2308457813 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91e010fe90 0x7f91e01a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.925+0000 7f91deffd700 1 -- 172.21.15.27:0/2308457813 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e01a0da0 con 0x7f91e0104450 2024-05-21T03:22:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.925+0000 7f91deffd700 1 --2- 172.21.15.27:0/2308457813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91e0104450 0x7f91e019c2d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f91c80057c0 tx=0x7f91c8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.925+0000 7f91dcff9700 1 -- 172.21.15.27:0/2308457813 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c8026070 con 0x7f91e0104450 2024-05-21T03:22:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.925+0000 7f91e5ecd700 1 -- 172.21.15.27:0/2308457813 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91e01a1030 con 0x7f91e0104450 2024-05-21T03:22:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.925+0000 7f91dcff9700 1 -- 172.21.15.27:0/2308457813 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91c8021410 con 0x7f91e0104450 2024-05-21T03:22:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.925+0000 7f91dcff9700 1 -- 172.21.15.27:0/2308457813 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c8017650 con 0x7f91e0104450 2024-05-21T03:22:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.925+0000 7f91e5ecd700 1 -- 172.21.15.27:0/2308457813 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91e01a8910 con 0x7f91e0104450 2024-05-21T03:22:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.926+0000 7f91e5ecd700 1 -- 172.21.15.27:0/2308457813 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f91e0102270 con 0x7f91e0104450 2024-05-21T03:22:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.927+0000 7f91dcff9700 1 -- 172.21.15.27:0/2308457813 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f91c8029070 con 0x7f91e0104450 2024-05-21T03:22:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.928+0000 7f91dcff9700 1 --2- 172.21.15.27:0/2308457813 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f91cc0b2910 0x7f91cc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.928+0000 7f91df7fe700 1 --2- 172.21.15.27:0/2308457813 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f91cc0b2910 0x7f91cc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.928+0000 7f91dcff9700 1 -- 172.21.15.27:0/2308457813 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f91c80e3550 con 0x7f91e0104450 2024-05-21T03:22:01.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.929+0000 7f91dcff9700 1 --2- 172.21.15.27:0/2308457813 >> [v2:172.21.15.27:6826/2800048793,v1:172.21.15.27:6827/2800048793] conn(0x7f91cc0bc550 0x7f91cc0be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:01.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.929+0000 7f91dcff9700 1 -- 172.21.15.27:0/2308457813 --> [v2:172.21.15.27:6826/2800048793,v1:172.21.15.27:6827/2800048793] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f91c802be50 con 0x7f91cc0bc550 2024-05-21T03:22:01.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.929+0000 7f91dcff9700 1 -- 172.21.15.27:0/2308457813 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f91c80e39a0 con 0x7f91e0104450 2024-05-21T03:22:01.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.929+0000 7f91df7fe700 1 --2- 172.21.15.27:0/2308457813 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f91cc0b2910 0x7f91cc0b4dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f91d0005ea0 tx=0x7f91d0005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:01.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.929+0000 7f91dffff700 1 --2- 172.21.15.27:0/2308457813 >> [v2:172.21.15.27:6826/2800048793,v1:172.21.15.27:6827/2800048793] conn(0x7f91cc0bc550 0x7f91cc0be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:01.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.929+0000 7f91dffff700 1 --2- 172.21.15.27:0/2308457813 >> [v2:172.21.15.27:6826/2800048793,v1:172.21.15.27:6827/2800048793] conn(0x7f91cc0bc550 0x7f91cc0be9f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:01.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.930+0000 7f91dcff9700 1 -- 172.21.15.27:0/2308457813 <== osd.9 v2:172.21.15.27:6826/2800048793 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f91c802be50 con 0x7f91cc0bc550 2024-05-21T03:22:01.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.947+0000 7f91e5ecd700 1 -- 172.21.15.27:0/2308457813 --> [v2:172.21.15.27:6826/2800048793,v1:172.21.15.27:6827/2800048793] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f91e006b4c0 con 0x7f91cc0bc550 2024-05-21T03:22:01.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.947+0000 7f91dcff9700 1 -- 172.21.15.27:0/2308457813 <== osd.9 v2:172.21.15.27:6826/2800048793 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f91e006b4c0 con 0x7f91cc0bc550 2024-05-21T03:22:01.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.948+0000 7f91e5ecd700 1 -- 172.21.15.27:0/2308457813 >> [v2:172.21.15.27:6826/2800048793,v1:172.21.15.27:6827/2800048793] conn(0x7f91cc0bc550 msgr2=0x7f91cc0be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:01.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.948+0000 7f91e5ecd700 1 --2- 172.21.15.27:0/2308457813 >> [v2:172.21.15.27:6826/2800048793,v1:172.21.15.27:6827/2800048793] conn(0x7f91cc0bc550 0x7f91cc0be9f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:01.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.948+0000 7f91e5ecd700 1 -- 172.21.15.27:0/2308457813 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f91cc0b2910 msgr2=0x7f91cc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:01.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.948+0000 7f91e5ecd700 1 --2- 172.21.15.27:0/2308457813 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f91cc0b2910 0x7f91cc0b4dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f91d0005ea0 tx=0x7f91d0005e30 comp rx=0 tx=0).stop 2024-05-21T03:22:01.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.948+0000 7f91e5ecd700 1 -- 172.21.15.27:0/2308457813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91e0104450 msgr2=0x7f91e019c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:01.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.948+0000 7f91e5ecd700 1 --2- 172.21.15.27:0/2308457813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91e0104450 0x7f91e019c2d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f91c80057c0 tx=0x7f91c8009580 comp rx=0 tx=0).stop 2024-05-21T03:22:01.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.948+0000 7f91dffff700 1 -- 172.21.15.27:0/2308457813 reap_dead start 2024-05-21T03:22:01.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.948+0000 7f91e5ecd700 1 -- 172.21.15.27:0/2308457813 shutdown_connections 2024-05-21T03:22:01.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.948+0000 7f91e5ecd700 1 -- 172.21.15.27:0/2308457813 >> 172.21.15.27:0/2308457813 conn(0x7f91e00793f0 msgr2=0x7f91e0101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:01.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.948+0000 7f91e5ecd700 1 -- 172.21.15.27:0/2308457813 shutdown_connections 2024-05-21T03:22:01.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.948+0000 7f91e5ecd700 1 -- 172.21.15.27:0/2308457813 wait complete. 2024-05-21T03:22:02.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.828+0000 7f5956cd1700 1 -- 172.21.15.27:0/1095426920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f595006bbe0 msgr2=0x7f59501103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:02.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.828+0000 7f5956cd1700 1 --2- 172.21.15.27:0/1095426920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f595006bbe0 0x7f59501103d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f594800b0a0 tx=0x7f594800b3b0 comp rx=0 tx=0).stop 2024-05-21T03:22:02.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.828+0000 7f5956cd1700 1 -- 172.21.15.27:0/1095426920 shutdown_connections 2024-05-21T03:22:02.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.828+0000 7f5956cd1700 1 --2- 172.21.15.27:0/1095426920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f595006bbe0 0x7f59501103d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:02.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.828+0000 7f5956cd1700 1 --2- 172.21.15.27:0/1095426920 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59501095b0 0x7f5950109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:02.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.828+0000 7f5956cd1700 1 --2- 172.21.15.27:0/1095426920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f595006b220 0x7f595006b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:02.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.828+0000 7f5956cd1700 1 -- 172.21.15.27:0/1095426920 >> 172.21.15.27:0/1095426920 conn(0x7f59500793f0 msgr2=0x7f5950079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:02.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.828+0000 7f5956cd1700 1 -- 172.21.15.27:0/1095426920 shutdown_connections 2024-05-21T03:22:02.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.828+0000 7f5956cd1700 1 -- 172.21.15.27:0/1095426920 wait complete. 2024-05-21T03:22:02.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.829+0000 7f5956cd1700 1 Processor -- start 2024-05-21T03:22:02.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.829+0000 7f5956cd1700 1 -- start start 2024-05-21T03:22:02.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.829+0000 7f5956cd1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f595006b220 0x7f5950102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:02.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.829+0000 7f5956cd1700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f595006bbe0 0x7f5950102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:02.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.829+0000 7f5956cd1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59501095b0 0x7f5950102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:02.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.829+0000 7f5956cd1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f59501136e0 con 0x7f59501095b0 2024-05-21T03:22:02.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.829+0000 7f5956cd1700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5950113560 con 0x7f595006b220 2024-05-21T03:22:02.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.829+0000 7f5956cd1700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5950113860 con 0x7f595006bbe0 2024-05-21T03:22:02.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.829+0000 7f595526e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59501095b0 0x7f5950102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:02.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.829+0000 7f594ffff700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f595006bbe0 0x7f5950102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:02.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.829+0000 7f595526e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59501095b0 0x7f5950102f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56876/0 (socket says 172.21.15.27:56876) 2024-05-21T03:22:02.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.829+0000 7f595526e700 1 -- 172.21.15.27:0/3352000619 learned_addr learned my addr 172.21.15.27:0/3352000619 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:02.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.829+0000 7f5954a6d700 1 --2- 172.21.15.27:0/3352000619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f595006b220 0x7f5950102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:02.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.830+0000 7f594ffff700 1 -- 172.21.15.27:0/3352000619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f595006b220 msgr2=0x7f5950102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:02.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.830+0000 7f594ffff700 1 --2- 172.21.15.27:0/3352000619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f595006b220 0x7f5950102510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:02.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.830+0000 7f594ffff700 1 -- 172.21.15.27:0/3352000619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59501095b0 msgr2=0x7f5950102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:02.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.830+0000 7f594ffff700 1 --2- 172.21.15.27:0/3352000619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59501095b0 0x7f5950102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:02.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.830+0000 7f594ffff700 1 -- 172.21.15.27:0/3352000619 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59501a8200 con 0x7f595006bbe0 2024-05-21T03:22:02.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.830+0000 7f5954a6d700 1 --2- 172.21.15.27:0/3352000619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f595006b220 0x7f5950102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:22:02.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.830+0000 7f595526e700 1 --2- 172.21.15.27:0/3352000619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59501095b0 0x7f5950102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:22:02.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.830+0000 7f594ffff700 1 --2- 172.21.15.27:0/3352000619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f595006bbe0 0x7f5950102a50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f593c00d750 tx=0x7f593c00da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:02.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.830+0000 7f594dffb700 1 -- 172.21.15.27:0/3352000619 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c016070 con 0x7f595006bbe0 2024-05-21T03:22:02.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.830+0000 7f5956cd1700 1 -- 172.21.15.27:0/3352000619 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59501a84f0 con 0x7f595006bbe0 2024-05-21T03:22:02.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.830+0000 7f594dffb700 1 -- 172.21.15.27:0/3352000619 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f593c004830 con 0x7f595006bbe0 2024-05-21T03:22:02.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.830+0000 7f594dffb700 1 -- 172.21.15.27:0/3352000619 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c0055d0 con 0x7f595006bbe0 2024-05-21T03:22:02.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.830+0000 7f5956cd1700 1 -- 172.21.15.27:0/3352000619 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59501a8a80 con 0x7f595006bbe0 2024-05-21T03:22:02.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.831+0000 7f5956cd1700 1 -- 172.21.15.27:0/3352000619 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5950063c50 con 0x7f595006bbe0 2024-05-21T03:22:02.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.833+0000 7f594dffb700 1 -- 172.21.15.27:0/3352000619 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f593c004220 con 0x7f595006bbe0 2024-05-21T03:22:02.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.833+0000 7f594dffb700 1 --2- 172.21.15.27:0/3352000619 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f59400b2910 0x7f59400b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:02.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.833+0000 7f594dffb700 1 -- 172.21.15.27:0/3352000619 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f593c0e2790 con 0x7f595006bbe0 2024-05-21T03:22:02.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.833+0000 7f5954a6d700 1 --2- 172.21.15.27:0/3352000619 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f59400b2910 0x7f59400b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:02.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.834+0000 7f594dffb700 1 --2- 172.21.15.27:0/3352000619 >> [v2:172.21.15.39:6816/1550035852,v1:172.21.15.39:6817/1550035852] conn(0x7f59400bc550 0x7f59400be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:02.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.834+0000 7f594dffb700 1 -- 172.21.15.27:0/3352000619 --> [v2:172.21.15.39:6816/1550035852,v1:172.21.15.39:6817/1550035852] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f59400bf0c0 con 0x7f59400bc550 2024-05-21T03:22:02.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.834+0000 7f594dffb700 1 -- 172.21.15.27:0/3352000619 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f593c0e2b40 con 0x7f595006bbe0 2024-05-21T03:22:02.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.834+0000 7f5954a6d700 1 --2- 172.21.15.27:0/3352000619 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f59400b2910 0x7f59400b4dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5944009c90 tx=0x7f5944009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:02.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.834+0000 7f595526e700 1 --2- 172.21.15.27:0/3352000619 >> [v2:172.21.15.39:6816/1550035852,v1:172.21.15.39:6817/1550035852] conn(0x7f59400bc550 0x7f59400be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:02.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.834+0000 7f595526e700 1 --2- 172.21.15.27:0/3352000619 >> [v2:172.21.15.39:6816/1550035852,v1:172.21.15.39:6817/1550035852] conn(0x7f59400bc550 0x7f59400be9f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:02.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.835+0000 7f594dffb700 1 -- 172.21.15.27:0/3352000619 <== osd.7 v2:172.21.15.39:6816/1550035852 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f59400bf0c0 con 0x7f59400bc550 2024-05-21T03:22:02.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.852+0000 7f5956cd1700 1 -- 172.21.15.27:0/3352000619 --> [v2:172.21.15.39:6816/1550035852,v1:172.21.15.39:6817/1550035852] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5950064050 con 0x7f59400bc550 2024-05-21T03:22:02.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.853+0000 7f594dffb700 1 -- 172.21.15.27:0/3352000619 <== osd.7 v2:172.21.15.39:6816/1550035852 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5950064050 con 0x7f59400bc550 2024-05-21T03:22:02.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.853+0000 7f5956cd1700 1 -- 172.21.15.27:0/3352000619 >> [v2:172.21.15.39:6816/1550035852,v1:172.21.15.39:6817/1550035852] conn(0x7f59400bc550 msgr2=0x7f59400be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:02.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.853+0000 7f5956cd1700 1 --2- 172.21.15.27:0/3352000619 >> [v2:172.21.15.39:6816/1550035852,v1:172.21.15.39:6817/1550035852] conn(0x7f59400bc550 0x7f59400be9f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:02.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.854+0000 7f5956cd1700 1 -- 172.21.15.27:0/3352000619 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f59400b2910 msgr2=0x7f59400b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:02.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.854+0000 7f5956cd1700 1 --2- 172.21.15.27:0/3352000619 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f59400b2910 0x7f59400b4dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5944009c90 tx=0x7f5944009260 comp rx=0 tx=0).stop 2024-05-21T03:22:02.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.854+0000 7f5956cd1700 1 -- 172.21.15.27:0/3352000619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f595006bbe0 msgr2=0x7f5950102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:02.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.854+0000 7f5956cd1700 1 --2- 172.21.15.27:0/3352000619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f595006bbe0 0x7f5950102a50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f593c00d750 tx=0x7f593c00da60 comp rx=0 tx=0).stop 2024-05-21T03:22:02.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.854+0000 7f595526e700 1 -- 172.21.15.27:0/3352000619 reap_dead start 2024-05-21T03:22:02.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.854+0000 7f5956cd1700 1 -- 172.21.15.27:0/3352000619 shutdown_connections 2024-05-21T03:22:02.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.854+0000 7f5956cd1700 1 -- 172.21.15.27:0/3352000619 >> 172.21.15.27:0/3352000619 conn(0x7f59500793f0 msgr2=0x7f5950101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:02.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.854+0000 7f5956cd1700 1 -- 172.21.15.27:0/3352000619 shutdown_connections 2024-05-21T03:22:02.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:01.854+0000 7f5956cd1700 1 -- 172.21.15.27:0/3352000619 wait complete. 2024-05-21T03:22:02.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:02 smithi039 ceph-mon[36429]: pgmap v336: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-05-21T03:22:02.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:02 smithi165 ceph-mon[36269]: pgmap v336: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-05-21T03:22:02.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:02 smithi027 ceph-mon[32179]: pgmap v336: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-05-21T03:22:02.970 INFO:teuthology.orchestra.run.smithi027.stdout:210453397559 2024-05-21T03:22:02.970 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-21T03:22:03.935 INFO:teuthology.orchestra.run.smithi027.stdout:309237645345 2024-05-21T03:22:03.935 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-21T03:22:04.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.519+0000 7f2c52c6a700 1 -- 172.21.15.27:0/2204779965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c4c1095a0 msgr2=0x7f2c4c109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:04.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.519+0000 7f2c52c6a700 1 --2- 172.21.15.27:0/2204779965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c4c1095a0 0x7f2c4c109980 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2c40009990 tx=0x7f2c40009ca0 comp rx=0 tx=0).stop 2024-05-21T03:22:04.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.519+0000 7f2c52c6a700 1 -- 172.21.15.27:0/2204779965 shutdown_connections 2024-05-21T03:22:04.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.519+0000 7f2c52c6a700 1 --2- 172.21.15.27:0/2204779965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c4c06c470 0x7f2c4c1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:04.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.519+0000 7f2c52c6a700 1 --2- 172.21.15.27:0/2204779965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c4c1095a0 0x7f2c4c109980 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:04.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.519+0000 7f2c52c6a700 1 --2- 172.21.15.27:0/2204779965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c4c06bab0 0x7f2c4c06bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:04.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.519+0000 7f2c52c6a700 1 -- 172.21.15.27:0/2204779965 >> 172.21.15.27:0/2204779965 conn(0x7f2c4c0793b0 msgr2=0x7f2c4c0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:04.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.519+0000 7f2c52c6a700 1 -- 172.21.15.27:0/2204779965 shutdown_connections 2024-05-21T03:22:04.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.519+0000 7f2c52c6a700 1 -- 172.21.15.27:0/2204779965 wait complete. 2024-05-21T03:22:04.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.520+0000 7f2c52c6a700 1 Processor -- start 2024-05-21T03:22:04.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.520+0000 7f2c52c6a700 1 -- start start 2024-05-21T03:22:04.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.520+0000 7f2c52c6a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c4c06bab0 0x7f2c4c19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:04.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.520+0000 7f2c52c6a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c4c06c470 0x7f2c4c19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:04.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.520+0000 7f2c52c6a700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c4c1095a0 0x7f2c4c199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:04.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.520+0000 7f2c52c6a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4c1136b0 con 0x7f2c4c06c470 2024-05-21T03:22:04.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.520+0000 7f2c52c6a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4c113530 con 0x7f2c4c06bab0 2024-05-21T03:22:04.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.520+0000 7f2c52c6a700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4c113830 con 0x7f2c4c1095a0 2024-05-21T03:22:04.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.520+0000 7f2c4bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c4c06c470 0x7f2c4c19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:04.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.520+0000 7f2c4bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c4c06c470 0x7f2c4c19fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57822/0 (socket says 172.21.15.27:57822) 2024-05-21T03:22:04.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.520+0000 7f2c4bfff700 1 -- 172.21.15.27:0/2768851932 learned_addr learned my addr 172.21.15.27:0/2768851932 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:04.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.520+0000 7f2c51207700 1 --2- 172.21.15.27:0/2768851932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c4c1095a0 0x7f2c4c199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:04.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.520+0000 7f2c50a06700 1 --2- 172.21.15.27:0/2768851932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c4c06bab0 0x7f2c4c19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:04.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.521+0000 7f2c4bfff700 1 -- 172.21.15.27:0/2768851932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c4c1095a0 msgr2=0x7f2c4c199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:04.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.521+0000 7f2c4bfff700 1 --2- 172.21.15.27:0/2768851932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c4c1095a0 0x7f2c4c199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:04.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.521+0000 7f2c4bfff700 1 -- 172.21.15.27:0/2768851932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c4c06bab0 msgr2=0x7f2c4c19f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:04.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.521+0000 7f2c4bfff700 1 --2- 172.21.15.27:0/2768851932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c4c06bab0 0x7f2c4c19f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:04.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.521+0000 7f2c4bfff700 1 -- 172.21.15.27:0/2768851932 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4c19a0f0 con 0x7f2c4c06c470 2024-05-21T03:22:04.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.521+0000 7f2c4bfff700 1 --2- 172.21.15.27:0/2768851932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c4c06c470 0x7f2c4c19fcc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800c9b0 tx=0x7f2c3800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:04.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.521+0000 7f2c49ffb700 1 -- 172.21.15.27:0/2768851932 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38007b00 con 0x7f2c4c06c470 2024-05-21T03:22:04.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.521+0000 7f2c49ffb700 1 -- 172.21.15.27:0/2768851932 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c3800ce50 con 0x7f2c4c06c470 2024-05-21T03:22:04.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.521+0000 7f2c52c6a700 1 -- 172.21.15.27:0/2768851932 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c4c19a3e0 con 0x7f2c4c06c470 2024-05-21T03:22:04.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.521+0000 7f2c49ffb700 1 -- 172.21.15.27:0/2768851932 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38018660 con 0x7f2c4c06c470 2024-05-21T03:22:04.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.521+0000 7f2c52c6a700 1 -- 172.21.15.27:0/2768851932 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c4c1a6650 con 0x7f2c4c06c470 2024-05-21T03:22:04.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.522+0000 7f2c52c6a700 1 -- 172.21.15.27:0/2768851932 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2c4c063cd0 con 0x7f2c4c06c470 2024-05-21T03:22:04.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.523+0000 7f2c49ffb700 1 -- 172.21.15.27:0/2768851932 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f2c3800f450 con 0x7f2c4c06c470 2024-05-21T03:22:04.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.524+0000 7f2c49ffb700 1 --2- 172.21.15.27:0/2768851932 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f2c3c0b2910 0x7f2c3c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:04.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.524+0000 7f2c50a06700 1 --2- 172.21.15.27:0/2768851932 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f2c3c0b2910 0x7f2c3c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:04.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.524+0000 7f2c49ffb700 1 -- 172.21.15.27:0/2768851932 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f2c380e26f0 con 0x7f2c4c06c470 2024-05-21T03:22:04.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.524+0000 7f2c49ffb700 1 --2- 172.21.15.27:0/2768851932 >> [v2:172.21.15.27:6810/1933403135,v1:172.21.15.27:6811/1933403135] conn(0x7f2c3c0bc550 0x7f2c3c0be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:04.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.524+0000 7f2c49ffb700 1 -- 172.21.15.27:0/2768851932 --> [v2:172.21.15.27:6810/1933403135,v1:172.21.15.27:6811/1933403135] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2c38015e70 con 0x7f2c3c0bc550 2024-05-21T03:22:04.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.524+0000 7f2c49ffb700 1 -- 172.21.15.27:0/2768851932 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f2c380e2b40 con 0x7f2c4c06c470 2024-05-21T03:22:04.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.525+0000 7f2c51207700 1 --2- 172.21.15.27:0/2768851932 >> [v2:172.21.15.27:6810/1933403135,v1:172.21.15.27:6811/1933403135] conn(0x7f2c3c0bc550 0x7f2c3c0be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:04.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.525+0000 7f2c50a06700 1 --2- 172.21.15.27:0/2768851932 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f2c3c0b2910 0x7f2c3c0b4dd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000b5c0 tx=0x7f2c40005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:04.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.525+0000 7f2c51207700 1 --2- 172.21.15.27:0/2768851932 >> [v2:172.21.15.27:6810/1933403135,v1:172.21.15.27:6811/1933403135] conn(0x7f2c3c0bc550 0x7f2c3c0be9f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:04.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.526+0000 7f2c49ffb700 1 -- 172.21.15.27:0/2768851932 <== osd.3 v2:172.21.15.27:6810/1933403135 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2c38015e70 con 0x7f2c3c0bc550 2024-05-21T03:22:04.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.542+0000 7f2c52c6a700 1 -- 172.21.15.27:0/2768851932 --> [v2:172.21.15.27:6810/1933403135,v1:172.21.15.27:6811/1933403135] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2c4c064150 con 0x7f2c3c0bc550 2024-05-21T03:22:04.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.543+0000 7f2c49ffb700 1 -- 172.21.15.27:0/2768851932 <== osd.3 v2:172.21.15.27:6810/1933403135 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2c4c064150 con 0x7f2c3c0bc550 2024-05-21T03:22:04.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.543+0000 7f2c52c6a700 1 -- 172.21.15.27:0/2768851932 >> [v2:172.21.15.27:6810/1933403135,v1:172.21.15.27:6811/1933403135] conn(0x7f2c3c0bc550 msgr2=0x7f2c3c0be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:04.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.543+0000 7f2c52c6a700 1 --2- 172.21.15.27:0/2768851932 >> [v2:172.21.15.27:6810/1933403135,v1:172.21.15.27:6811/1933403135] conn(0x7f2c3c0bc550 0x7f2c3c0be9f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:04.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.543+0000 7f2c52c6a700 1 -- 172.21.15.27:0/2768851932 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f2c3c0b2910 msgr2=0x7f2c3c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:04.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.543+0000 7f2c52c6a700 1 --2- 172.21.15.27:0/2768851932 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f2c3c0b2910 0x7f2c3c0b4dd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000b5c0 tx=0x7f2c40005960 comp rx=0 tx=0).stop 2024-05-21T03:22:04.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.543+0000 7f2c52c6a700 1 -- 172.21.15.27:0/2768851932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c4c06c470 msgr2=0x7f2c4c19fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:04.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.543+0000 7f2c52c6a700 1 --2- 172.21.15.27:0/2768851932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c4c06c470 0x7f2c4c19fcc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800c9b0 tx=0x7f2c3800ccc0 comp rx=0 tx=0).stop 2024-05-21T03:22:04.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.543+0000 7f2c51207700 1 -- 172.21.15.27:0/2768851932 reap_dead start 2024-05-21T03:22:04.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.543+0000 7f2c52c6a700 1 -- 172.21.15.27:0/2768851932 shutdown_connections 2024-05-21T03:22:04.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.543+0000 7f2c52c6a700 1 -- 172.21.15.27:0/2768851932 >> 172.21.15.27:0/2768851932 conn(0x7f2c4c0793b0 msgr2=0x7f2c4c101120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:04.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.543+0000 7f2c52c6a700 1 -- 172.21.15.27:0/2768851932 shutdown_connections 2024-05-21T03:22:04.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.543+0000 7f2c52c6a700 1 -- 172.21.15.27:0/2768851932 wait complete. 2024-05-21T03:22:04.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.226+0000 7f47f2451700 1 -- 172.21.15.27:0/2178083072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47ec108520 msgr2=0x7f47ec108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:04.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.226+0000 7f47f2451700 1 --2- 172.21.15.27:0/2178083072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47ec108520 0x7f47ec108900 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f47dc009990 tx=0x7f47dc009ca0 comp rx=0 tx=0).stop 2024-05-21T03:22:04.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.226+0000 7f47f2451700 1 -- 172.21.15.27:0/2178083072 shutdown_connections 2024-05-21T03:22:04.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.226+0000 7f47f2451700 1 --2- 172.21.15.27:0/2178083072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47ec102e00 0x7f47ec10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:04.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.226+0000 7f47f2451700 1 --2- 172.21.15.27:0/2178083072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47ec108520 0x7f47ec108900 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:04.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.226+0000 7f47f2451700 1 --2- 172.21.15.27:0/2178083072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47ec102440 0x7f47ec1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:04.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.226+0000 7f47f2451700 1 -- 172.21.15.27:0/2178083072 >> 172.21.15.27:0/2178083072 conn(0x7f47ec0793f0 msgr2=0x7f47ec079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:04.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.226+0000 7f47f2451700 1 -- 172.21.15.27:0/2178083072 shutdown_connections 2024-05-21T03:22:04.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.226+0000 7f47f2451700 1 -- 172.21.15.27:0/2178083072 wait complete. 2024-05-21T03:22:04.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.226+0000 7f47f2451700 1 Processor -- start 2024-05-21T03:22:04.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.226+0000 7f47f2451700 1 -- start start 2024-05-21T03:22:04.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.227+0000 7f47f2451700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47ec102440 0x7f47ec19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:04.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.227+0000 7f47f2451700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47ec102e00 0x7f47ec19af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:04.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.227+0000 7f47f2451700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47ec108520 0x7f47ec1a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:04.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.227+0000 7f47f2451700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec1125d0 con 0x7f47ec102440 2024-05-21T03:22:04.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.227+0000 7f47f2451700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec112450 con 0x7f47ec108520 2024-05-21T03:22:04.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.227+0000 7f47f2451700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec112750 con 0x7f47ec102e00 2024-05-21T03:22:04.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.227+0000 7f47ebfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47ec102440 0x7f47ec19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:04.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.227+0000 7f47ebfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47ec102440 0x7f47ec19aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57808/0 (socket says 172.21.15.27:57808) 2024-05-21T03:22:04.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.227+0000 7f47ebfff700 1 -- 172.21.15.27:0/159878463 learned_addr learned my addr 172.21.15.27:0/159878463 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:04.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.227+0000 7f47f09ee700 1 --2- 172.21.15.27:0/159878463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47ec108520 0x7f47ec1a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:04.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.227+0000 7f47ebfff700 1 -- 172.21.15.27:0/159878463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47ec102e00 msgr2=0x7f47ec19af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:04.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.227+0000 7f47e3fff700 1 --2- 172.21.15.27:0/159878463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47ec102e00 0x7f47ec19af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:04.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.227+0000 7f47ebfff700 1 --2- 172.21.15.27:0/159878463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47ec102e00 0x7f47ec19af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:04.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.227+0000 7f47ebfff700 1 -- 172.21.15.27:0/159878463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47ec108520 msgr2=0x7f47ec1a5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:04.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.227+0000 7f47ebfff700 1 --2- 172.21.15.27:0/159878463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47ec108520 0x7f47ec1a5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:04.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.227+0000 7f47ebfff700 1 -- 172.21.15.27:0/159878463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47ec1a5ff0 con 0x7f47ec102440 2024-05-21T03:22:04.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.227+0000 7f47f09ee700 1 --2- 172.21.15.27:0/159878463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47ec108520 0x7f47ec1a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:22:04.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.227+0000 7f47ebfff700 1 --2- 172.21.15.27:0/159878463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47ec102440 0x7f47ec19aa30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f47dc000c00 tx=0x7f47dc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:04.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.227+0000 7f47e9ffb700 1 -- 172.21.15.27:0/159878463 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc027070 con 0x7f47ec102440 2024-05-21T03:22:04.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.227+0000 7f47f2451700 1 -- 172.21.15.27:0/159878463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47ec1a61c0 con 0x7f47ec102440 2024-05-21T03:22:04.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.227+0000 7f47e9ffb700 1 -- 172.21.15.27:0/159878463 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47dc003fa0 con 0x7f47ec102440 2024-05-21T03:22:04.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.227+0000 7f47e9ffb700 1 -- 172.21.15.27:0/159878463 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc022db0 con 0x7f47ec102440 2024-05-21T03:22:04.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.228+0000 7f47f2451700 1 -- 172.21.15.27:0/159878463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47ec1a6610 con 0x7f47ec102440 2024-05-21T03:22:04.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.228+0000 7f47f2451700 1 -- 172.21.15.27:0/159878463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f47ec063c50 con 0x7f47ec102440 2024-05-21T03:22:04.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.230+0000 7f47e9ffb700 1 -- 172.21.15.27:0/159878463 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f47dc02a070 con 0x7f47ec102440 2024-05-21T03:22:04.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.230+0000 7f47e9ffb700 1 --2- 172.21.15.27:0/159878463 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f47d80b2910 0x7f47d80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:04.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.231+0000 7f47e3fff700 1 --2- 172.21.15.27:0/159878463 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f47d80b2910 0x7f47d80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:04.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.231+0000 7f47e9ffb700 1 -- 172.21.15.27:0/159878463 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f47dc0e3720 con 0x7f47ec102440 2024-05-21T03:22:04.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.231+0000 7f47e9ffb700 1 --2- 172.21.15.27:0/159878463 >> [v2:172.21.15.27:6818/3531376679,v1:172.21.15.27:6819/3531376679] conn(0x7f47d80bc550 0x7f47d80be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:04.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.231+0000 7f47e9ffb700 1 -- 172.21.15.27:0/159878463 --> [v2:172.21.15.27:6818/3531376679,v1:172.21.15.27:6819/3531376679] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f47dc00eea0 con 0x7f47d80bc550 2024-05-21T03:22:04.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.231+0000 7f47e9ffb700 1 -- 172.21.15.27:0/159878463 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f47dc0e3b70 con 0x7f47ec102440 2024-05-21T03:22:04.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.231+0000 7f47f09ee700 1 --2- 172.21.15.27:0/159878463 >> [v2:172.21.15.27:6818/3531376679,v1:172.21.15.27:6819/3531376679] conn(0x7f47d80bc550 0x7f47d80be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:04.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.231+0000 7f47e3fff700 1 --2- 172.21.15.27:0/159878463 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f47d80b2910 0x7f47d80b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f47d4005950 tx=0x7f47d40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:04.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.231+0000 7f47f09ee700 1 --2- 172.21.15.27:0/159878463 >> [v2:172.21.15.27:6818/3531376679,v1:172.21.15.27:6819/3531376679] conn(0x7f47d80bc550 0x7f47d80be9f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:04.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.232+0000 7f47e9ffb700 1 -- 172.21.15.27:0/159878463 <== osd.6 v2:172.21.15.27:6818/3531376679 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f47dc00eea0 con 0x7f47d80bc550 2024-05-21T03:22:04.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.249+0000 7f47f2451700 1 -- 172.21.15.27:0/159878463 --> [v2:172.21.15.27:6818/3531376679,v1:172.21.15.27:6819/3531376679] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f47ec0640d0 con 0x7f47d80bc550 2024-05-21T03:22:04.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.249+0000 7f47e9ffb700 1 -- 172.21.15.27:0/159878463 <== osd.6 v2:172.21.15.27:6818/3531376679 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f47ec0640d0 con 0x7f47d80bc550 2024-05-21T03:22:04.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.249+0000 7f47f2451700 1 -- 172.21.15.27:0/159878463 >> [v2:172.21.15.27:6818/3531376679,v1:172.21.15.27:6819/3531376679] conn(0x7f47d80bc550 msgr2=0x7f47d80be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:04.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.249+0000 7f47f2451700 1 --2- 172.21.15.27:0/159878463 >> [v2:172.21.15.27:6818/3531376679,v1:172.21.15.27:6819/3531376679] conn(0x7f47d80bc550 0x7f47d80be9f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:04.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.249+0000 7f47f2451700 1 -- 172.21.15.27:0/159878463 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f47d80b2910 msgr2=0x7f47d80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:04.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.249+0000 7f47f2451700 1 --2- 172.21.15.27:0/159878463 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f47d80b2910 0x7f47d80b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f47d4005950 tx=0x7f47d40058e0 comp rx=0 tx=0).stop 2024-05-21T03:22:04.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.249+0000 7f47f2451700 1 -- 172.21.15.27:0/159878463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47ec102440 msgr2=0x7f47ec19aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:04.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.249+0000 7f47f2451700 1 --2- 172.21.15.27:0/159878463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47ec102440 0x7f47ec19aa30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f47dc000c00 tx=0x7f47dc0047b0 comp rx=0 tx=0).stop 2024-05-21T03:22:04.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.249+0000 7f47f09ee700 1 -- 172.21.15.27:0/159878463 reap_dead start 2024-05-21T03:22:04.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.250+0000 7f47f2451700 1 -- 172.21.15.27:0/159878463 shutdown_connections 2024-05-21T03:22:04.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.250+0000 7f47f2451700 1 -- 172.21.15.27:0/159878463 >> 172.21.15.27:0/159878463 conn(0x7f47ec0793f0 msgr2=0x7f47ec101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:04.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.250+0000 7f47f2451700 1 -- 172.21.15.27:0/159878463 shutdown_connections 2024-05-21T03:22:04.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:04.250+0000 7f47f2451700 1 -- 172.21.15.27:0/159878463 wait complete. 2024-05-21T03:22:04.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:04 smithi027 ceph-mon[32179]: pgmap v337: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 10 op/s 2024-05-21T03:22:04.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:04 smithi039 ceph-mon[36429]: pgmap v337: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 10 op/s 2024-05-21T03:22:04.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:04 smithi165 ceph-mon[36269]: pgmap v337: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 10 op/s 2024-05-21T03:22:05.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.070+0000 7fc3636f0700 1 -- 172.21.15.27:0/3146274958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc35c108650 msgr2=0x7fc35c108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:05.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.070+0000 7fc3636f0700 1 --2- 172.21.15.27:0/3146274958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc35c108650 0x7fc35c108a30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc358009990 tx=0x7fc358009ca0 comp rx=0 tx=0).stop 2024-05-21T03:22:05.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.070+0000 7fc3636f0700 1 -- 172.21.15.27:0/3146274958 shutdown_connections 2024-05-21T03:22:05.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.070+0000 7fc3636f0700 1 --2- 172.21.15.27:0/3146274958 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc35c1030e0 0x7fc35c10f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:05.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.070+0000 7fc3636f0700 1 --2- 172.21.15.27:0/3146274958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc35c108650 0x7fc35c108a30 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:05.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.070+0000 7fc3636f0700 1 --2- 172.21.15.27:0/3146274958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc35c102720 0x7fc35c102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:05.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.070+0000 7fc3636f0700 1 -- 172.21.15.27:0/3146274958 >> 172.21.15.27:0/3146274958 conn(0x7fc35c0792d0 msgr2=0x7fc35c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:05.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.071+0000 7fc3636f0700 1 -- 172.21.15.27:0/3146274958 shutdown_connections 2024-05-21T03:22:05.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.071+0000 7fc3636f0700 1 -- 172.21.15.27:0/3146274958 wait complete. 2024-05-21T03:22:05.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.071+0000 7fc3636f0700 1 Processor -- start 2024-05-21T03:22:05.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.071+0000 7fc3636f0700 1 -- start start 2024-05-21T03:22:05.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.071+0000 7fc3636f0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc35c102720 0x7fc35c19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:05.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.071+0000 7fc3636f0700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc35c1030e0 0x7fc35c19af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:05.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.071+0000 7fc3636f0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc35c108650 0x7fc35c1a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:05.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.072+0000 7fc3636f0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c112600 con 0x7fc35c102720 2024-05-21T03:22:05.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.072+0000 7fc3636f0700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c112480 con 0x7fc35c108650 2024-05-21T03:22:05.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.072+0000 7fc3636f0700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c112780 con 0x7fc35c1030e0 2024-05-21T03:22:05.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.072+0000 7fc36148c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc35c102720 0x7fc35c19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:05.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.072+0000 7fc36148c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc35c102720 0x7fc35c19a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57832/0 (socket says 172.21.15.27:57832) 2024-05-21T03:22:05.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.072+0000 7fc36148c700 1 -- 172.21.15.27:0/42251321 learned_addr learned my addr 172.21.15.27:0/42251321 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:05.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.072+0000 7fc361c8d700 1 --2- 172.21.15.27:0/42251321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc35c108650 0x7fc35c1a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:05.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.072+0000 7fc360c8b700 1 --2- 172.21.15.27:0/42251321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc35c1030e0 0x7fc35c19af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:05.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.072+0000 7fc36148c700 1 -- 172.21.15.27:0/42251321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc35c1030e0 msgr2=0x7fc35c19af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:05.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.072+0000 7fc36148c700 1 --2- 172.21.15.27:0/42251321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc35c1030e0 0x7fc35c19af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:05.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.072+0000 7fc36148c700 1 -- 172.21.15.27:0/42251321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc35c108650 msgr2=0x7fc35c1a5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:05.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.072+0000 7fc36148c700 1 --2- 172.21.15.27:0/42251321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc35c108650 0x7fc35c1a5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:05.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.072+0000 7fc36148c700 1 -- 172.21.15.27:0/42251321 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc35c1a5f80 con 0x7fc35c102720 2024-05-21T03:22:05.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.072+0000 7fc36148c700 1 --2- 172.21.15.27:0/42251321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc35c102720 0x7fc35c19a9c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc358000c00 tx=0x7fc3580047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:05.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.072+0000 7fc3527fc700 1 -- 172.21.15.27:0/42251321 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc358027070 con 0x7fc35c102720 2024-05-21T03:22:05.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.072+0000 7fc3527fc700 1 -- 172.21.15.27:0/42251321 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc358003f20 con 0x7fc35c102720 2024-05-21T03:22:05.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.072+0000 7fc3636f0700 1 -- 172.21.15.27:0/42251321 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc35c1a6150 con 0x7fc35c102720 2024-05-21T03:22:05.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.072+0000 7fc3527fc700 1 -- 172.21.15.27:0/42251321 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc358022cf0 con 0x7fc35c102720 2024-05-21T03:22:05.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.072+0000 7fc3636f0700 1 -- 172.21.15.27:0/42251321 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc35c1a65a0 con 0x7fc35c102720 2024-05-21T03:22:05.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.073+0000 7fc3636f0700 1 -- 172.21.15.27:0/42251321 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc35c063cd0 con 0x7fc35c102720 2024-05-21T03:22:05.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.075+0000 7fc3527fc700 1 -- 172.21.15.27:0/42251321 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc35802a070 con 0x7fc35c102720 2024-05-21T03:22:05.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.076+0000 7fc3527fc700 1 --2- 172.21.15.27:0/42251321 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fc3480b2910 0x7fc3480b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:05.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.076+0000 7fc360c8b700 1 --2- 172.21.15.27:0/42251321 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fc3480b2910 0x7fc3480b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:05.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.076+0000 7fc3527fc700 1 -- 172.21.15.27:0/42251321 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fc3580e37b0 con 0x7fc35c102720 2024-05-21T03:22:05.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.076+0000 7fc3527fc700 1 --2- 172.21.15.27:0/42251321 >> [v2:172.21.15.27:6802/1399996449,v1:172.21.15.27:6803/1399996449] conn(0x7fc3480bc550 0x7fc3480be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:05.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.076+0000 7fc3527fc700 1 -- 172.21.15.27:0/42251321 --> [v2:172.21.15.27:6802/1399996449,v1:172.21.15.27:6803/1399996449] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc35801fec0 con 0x7fc3480bc550 2024-05-21T03:22:05.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.076+0000 7fc3527fc700 1 -- 172.21.15.27:0/42251321 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fc3580e3c00 con 0x7fc35c102720 2024-05-21T03:22:05.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.076+0000 7fc361c8d700 1 --2- 172.21.15.27:0/42251321 >> [v2:172.21.15.27:6802/1399996449,v1:172.21.15.27:6803/1399996449] conn(0x7fc3480bc550 0x7fc3480be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:05.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.076+0000 7fc360c8b700 1 --2- 172.21.15.27:0/42251321 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fc3480b2910 0x7fc3480b4dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc34c005fd0 tx=0x7fc34c005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:05.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.076+0000 7fc361c8d700 1 --2- 172.21.15.27:0/42251321 >> [v2:172.21.15.27:6802/1399996449,v1:172.21.15.27:6803/1399996449] conn(0x7fc3480bc550 0x7fc3480be9f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:05.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.077+0000 7fc3527fc700 1 -- 172.21.15.27:0/42251321 <== osd.0 v2:172.21.15.27:6802/1399996449 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc35801fec0 con 0x7fc3480bc550 2024-05-21T03:22:05.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.094+0000 7fc3636f0700 1 -- 172.21.15.27:0/42251321 --> [v2:172.21.15.27:6802/1399996449,v1:172.21.15.27:6803/1399996449] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc35c064150 con 0x7fc3480bc550 2024-05-21T03:22:05.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.095+0000 7fc3527fc700 1 -- 172.21.15.27:0/42251321 <== osd.0 v2:172.21.15.27:6802/1399996449 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc35c064150 con 0x7fc3480bc550 2024-05-21T03:22:05.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.095+0000 7fc3636f0700 1 -- 172.21.15.27:0/42251321 >> [v2:172.21.15.27:6802/1399996449,v1:172.21.15.27:6803/1399996449] conn(0x7fc3480bc550 msgr2=0x7fc3480be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:05.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.095+0000 7fc3636f0700 1 --2- 172.21.15.27:0/42251321 >> [v2:172.21.15.27:6802/1399996449,v1:172.21.15.27:6803/1399996449] conn(0x7fc3480bc550 0x7fc3480be9f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:05.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.095+0000 7fc3636f0700 1 -- 172.21.15.27:0/42251321 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fc3480b2910 msgr2=0x7fc3480b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:05.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.095+0000 7fc3636f0700 1 --2- 172.21.15.27:0/42251321 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fc3480b2910 0x7fc3480b4dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc34c005fd0 tx=0x7fc34c005f60 comp rx=0 tx=0).stop 2024-05-21T03:22:05.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.095+0000 7fc3636f0700 1 -- 172.21.15.27:0/42251321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc35c102720 msgr2=0x7fc35c19a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:05.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.095+0000 7fc3636f0700 1 --2- 172.21.15.27:0/42251321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc35c102720 0x7fc35c19a9c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc358000c00 tx=0x7fc3580047b0 comp rx=0 tx=0).stop 2024-05-21T03:22:05.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.095+0000 7fc361c8d700 1 -- 172.21.15.27:0/42251321 reap_dead start 2024-05-21T03:22:05.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.095+0000 7fc3636f0700 1 -- 172.21.15.27:0/42251321 shutdown_connections 2024-05-21T03:22:05.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.095+0000 7fc3636f0700 1 -- 172.21.15.27:0/42251321 >> 172.21.15.27:0/42251321 conn(0x7fc35c0792d0 msgr2=0x7fc35c101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:05.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.095+0000 7fc3636f0700 1 -- 172.21.15.27:0/42251321 shutdown_connections 2024-05-21T03:22:05.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:05.095+0000 7fc3636f0700 1 -- 172.21.15.27:0/42251321 wait complete. 2024-05-21T03:22:05.575 INFO:teuthology.orchestra.run.smithi027.stdout:261993005100 2024-05-21T03:22:05.576 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-21T03:22:06.727 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:06 smithi027 ceph-mon[32179]: pgmap v338: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s 2024-05-21T03:22:06.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:06 smithi039 ceph-mon[36429]: pgmap v338: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s 2024-05-21T03:22:06.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:06 smithi165 ceph-mon[36269]: pgmap v338: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s 2024-05-21T03:22:07.336 INFO:teuthology.orchestra.run.smithi027.stdout:188978561083 2024-05-21T03:22:07.337 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-21T03:22:07.983 INFO:teuthology.orchestra.run.smithi027.stdout:34359738459 2024-05-21T03:22:07.983 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-21T03:22:08.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.287+0000 7f87caeb6700 1 -- 172.21.15.27:0/310210728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87c4101b30 msgr2=0x7f87c4101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:08.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.287+0000 7f87caeb6700 1 --2- 172.21.15.27:0/310210728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87c4101b30 0x7f87c4101fb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f87b00099c0 tx=0x7f87b0009cd0 comp rx=0 tx=0).stop 2024-05-21T03:22:08.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.287+0000 7f87caeb6700 1 -- 172.21.15.27:0/310210728 shutdown_connections 2024-05-21T03:22:08.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.287+0000 7f87caeb6700 1 --2- 172.21.15.27:0/310210728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87c41024f0 0x7f87c41103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:08.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.287+0000 7f87caeb6700 1 --2- 172.21.15.27:0/310210728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87c4109520 0x7f87c4109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:08.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.287+0000 7f87caeb6700 1 --2- 172.21.15.27:0/310210728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87c4101b30 0x7f87c4101fb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:08.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.287+0000 7f87caeb6700 1 -- 172.21.15.27:0/310210728 >> 172.21.15.27:0/310210728 conn(0x7f87c4078ab0 msgr2=0x7f87c40ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:08.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.287+0000 7f87caeb6700 1 -- 172.21.15.27:0/310210728 shutdown_connections 2024-05-21T03:22:08.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.287+0000 7f87caeb6700 1 -- 172.21.15.27:0/310210728 wait complete. 2024-05-21T03:22:08.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.288+0000 7f87caeb6700 1 Processor -- start 2024-05-21T03:22:08.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.288+0000 7f87caeb6700 1 -- start start 2024-05-21T03:22:08.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.289+0000 7f87caeb6700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87c4101b30 0x7f87c419f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:08.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.289+0000 7f87caeb6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87c41024f0 0x7f87c419fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:08.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.289+0000 7f87caeb6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87c4109520 0x7f87c4199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:08.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.289+0000 7f87caeb6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f87c41137b0 con 0x7f87c41024f0 2024-05-21T03:22:08.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.289+0000 7f87caeb6700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f87c4113630 con 0x7f87c4109520 2024-05-21T03:22:08.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.289+0000 7f87caeb6700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f87c4113930 con 0x7f87c4101b30 2024-05-21T03:22:08.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.289+0000 7f87c3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87c41024f0 0x7f87c419fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:08.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.289+0000 7f87c3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87c41024f0 0x7f87c419fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57846/0 (socket says 172.21.15.27:57846) 2024-05-21T03:22:08.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.289+0000 7f87c3fff700 1 -- 172.21.15.27:0/3221915477 learned_addr learned my addr 172.21.15.27:0/3221915477 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:08.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.289+0000 7f87c8c52700 1 --2- 172.21.15.27:0/3221915477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87c4101b30 0x7f87c419f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:08.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.289+0000 7f87c9453700 1 --2- 172.21.15.27:0/3221915477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87c4109520 0x7f87c4199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:08.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.289+0000 7f87c3fff700 1 -- 172.21.15.27:0/3221915477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87c4101b30 msgr2=0x7f87c419f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:08.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.289+0000 7f87c3fff700 1 --2- 172.21.15.27:0/3221915477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87c4101b30 0x7f87c419f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:08.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.289+0000 7f87c3fff700 1 -- 172.21.15.27:0/3221915477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87c4109520 msgr2=0x7f87c4199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:08.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.289+0000 7f87c3fff700 1 --2- 172.21.15.27:0/3221915477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87c4109520 0x7f87c4199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:08.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.289+0000 7f87c3fff700 1 -- 172.21.15.27:0/3221915477 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c419a1f0 con 0x7f87c41024f0 2024-05-21T03:22:08.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.289+0000 7f87c3fff700 1 --2- 172.21.15.27:0/3221915477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87c41024f0 0x7f87c419fe10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f87b00098e0 tx=0x7f87b0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:08.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.290+0000 7f87c1ffb700 1 -- 172.21.15.27:0/3221915477 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b0026070 con 0x7f87c41024f0 2024-05-21T03:22:08.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.290+0000 7f87c1ffb700 1 -- 172.21.15.27:0/3221915477 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87b0003c70 con 0x7f87c41024f0 2024-05-21T03:22:08.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.290+0000 7f87c1ffb700 1 -- 172.21.15.27:0/3221915477 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b000fb00 con 0x7f87c41024f0 2024-05-21T03:22:08.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.290+0000 7f87caeb6700 1 -- 172.21.15.27:0/3221915477 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87c419a480 con 0x7f87c41024f0 2024-05-21T03:22:08.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.290+0000 7f87caeb6700 1 -- 172.21.15.27:0/3221915477 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87c41a67a0 con 0x7f87c41024f0 2024-05-21T03:22:08.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.291+0000 7f87caeb6700 1 -- 172.21.15.27:0/3221915477 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f87c4063cd0 con 0x7f87c41024f0 2024-05-21T03:22:08.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.293+0000 7f87c1ffb700 1 -- 172.21.15.27:0/3221915477 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f87b0029070 con 0x7f87c41024f0 2024-05-21T03:22:08.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.293+0000 7f87c1ffb700 1 --2- 172.21.15.27:0/3221915477 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f87b40b2910 0x7f87b40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:08.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.294+0000 7f87c8c52700 1 --2- 172.21.15.27:0/3221915477 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f87b40b2910 0x7f87b40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:08.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.294+0000 7f87c1ffb700 1 -- 172.21.15.27:0/3221915477 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f87b00e3400 con 0x7f87c41024f0 2024-05-21T03:22:08.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.294+0000 7f87c1ffb700 1 --2- 172.21.15.27:0/3221915477 >> [v2:172.21.15.39:6800/1776318847,v1:172.21.15.39:6801/1776318847] conn(0x7f87b40bc550 0x7f87b40be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:08.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.294+0000 7f87c1ffb700 1 -- 172.21.15.27:0/3221915477 --> [v2:172.21.15.39:6800/1776318847,v1:172.21.15.39:6801/1776318847] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f87b40bf0c0 con 0x7f87b40bc550 2024-05-21T03:22:08.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.294+0000 7f87c1ffb700 1 -- 172.21.15.27:0/3221915477 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f87b00e3850 con 0x7f87c41024f0 2024-05-21T03:22:08.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.294+0000 7f87c9453700 1 --2- 172.21.15.27:0/3221915477 >> [v2:172.21.15.39:6800/1776318847,v1:172.21.15.39:6801/1776318847] conn(0x7f87b40bc550 0x7f87b40be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:08.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.294+0000 7f87c8c52700 1 --2- 172.21.15.27:0/3221915477 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f87b40b2910 0x7f87b40b4dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f87c419abd0 tx=0x7f87b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:08.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.294+0000 7f87c9453700 1 --2- 172.21.15.27:0/3221915477 >> [v2:172.21.15.39:6800/1776318847,v1:172.21.15.39:6801/1776318847] conn(0x7f87b40bc550 0x7f87b40be9f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:08.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.295+0000 7f87c1ffb700 1 -- 172.21.15.27:0/3221915477 <== osd.1 v2:172.21.15.39:6800/1776318847 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f87b40bf0c0 con 0x7f87b40bc550 2024-05-21T03:22:08.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.313+0000 7f87caeb6700 1 -- 172.21.15.27:0/3221915477 --> [v2:172.21.15.39:6800/1776318847,v1:172.21.15.39:6801/1776318847] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f87c4064150 con 0x7f87b40bc550 2024-05-21T03:22:08.313 INFO:teuthology.orchestra.run.smithi027.stdout:77309411406 2024-05-21T03:22:08.313 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-21T03:22:08.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.313+0000 7f87c1ffb700 1 -- 172.21.15.27:0/3221915477 <== osd.1 v2:172.21.15.39:6800/1776318847 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f87c4064150 con 0x7f87b40bc550 2024-05-21T03:22:08.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.314+0000 7f87caeb6700 1 -- 172.21.15.27:0/3221915477 >> [v2:172.21.15.39:6800/1776318847,v1:172.21.15.39:6801/1776318847] conn(0x7f87b40bc550 msgr2=0x7f87b40be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:08.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.314+0000 7f87caeb6700 1 --2- 172.21.15.27:0/3221915477 >> [v2:172.21.15.39:6800/1776318847,v1:172.21.15.39:6801/1776318847] conn(0x7f87b40bc550 0x7f87b40be9f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:08.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.314+0000 7f87caeb6700 1 -- 172.21.15.27:0/3221915477 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f87b40b2910 msgr2=0x7f87b40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:08.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.314+0000 7f87caeb6700 1 --2- 172.21.15.27:0/3221915477 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f87b40b2910 0x7f87b40b4dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f87c419abd0 tx=0x7f87b8009260 comp rx=0 tx=0).stop 2024-05-21T03:22:08.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.314+0000 7f87caeb6700 1 -- 172.21.15.27:0/3221915477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87c41024f0 msgr2=0x7f87c419fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:08.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.314+0000 7f87caeb6700 1 --2- 172.21.15.27:0/3221915477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87c41024f0 0x7f87c419fe10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f87b00098e0 tx=0x7f87b0004270 comp rx=0 tx=0).stop 2024-05-21T03:22:08.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.314+0000 7f87c9453700 1 -- 172.21.15.27:0/3221915477 reap_dead start 2024-05-21T03:22:08.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.314+0000 7f87caeb6700 1 -- 172.21.15.27:0/3221915477 shutdown_connections 2024-05-21T03:22:08.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.314+0000 7f87caeb6700 1 -- 172.21.15.27:0/3221915477 >> 172.21.15.27:0/3221915477 conn(0x7f87c4078ab0 msgr2=0x7f87c41002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:08.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.314+0000 7f87caeb6700 1 -- 172.21.15.27:0/3221915477 shutdown_connections 2024-05-21T03:22:08.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.314+0000 7f87caeb6700 1 -- 172.21.15.27:0/3221915477 wait complete. 2024-05-21T03:22:08.534 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:08 smithi027 ceph-mon[32179]: pgmap v339: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:22:08.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.511+0000 7f1a0a52d700 1 -- 172.21.15.27:0/3846054242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a04102430 msgr2=0x7f1a041028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:08.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.511+0000 7f1a0a52d700 1 --2- 172.21.15.27:0/3846054242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a04102430 0x7f1a041028b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f19f8009990 tx=0x7f19f8009ca0 comp rx=0 tx=0).stop 2024-05-21T03:22:08.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.511+0000 7f1a0a52d700 1 -- 172.21.15.27:0/3846054242 shutdown_connections 2024-05-21T03:22:08.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.511+0000 7f1a0a52d700 1 --2- 172.21.15.27:0/3846054242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a04102df0 0x7f1a0410f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:08.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.511+0000 7f1a0a52d700 1 --2- 172.21.15.27:0/3846054242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a04108510 0x7f1a041088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:08.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.511+0000 7f1a0a52d700 1 --2- 172.21.15.27:0/3846054242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a04102430 0x7f1a041028b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:08.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.511+0000 7f1a0a52d700 1 -- 172.21.15.27:0/3846054242 >> 172.21.15.27:0/3846054242 conn(0x7f1a040793b0 msgr2=0x7f1a040797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:08.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.511+0000 7f1a0a52d700 1 -- 172.21.15.27:0/3846054242 shutdown_connections 2024-05-21T03:22:08.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.511+0000 7f1a0a52d700 1 -- 172.21.15.27:0/3846054242 wait complete. 2024-05-21T03:22:08.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.512+0000 7f1a0a52d700 1 Processor -- start 2024-05-21T03:22:08.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.512+0000 7f1a0a52d700 1 -- start start 2024-05-21T03:22:08.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.512+0000 7f1a0a52d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a04102430 0x7f1a0419bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:08.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.512+0000 7f1a0a52d700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a04102df0 0x7f1a0419c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:08.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.512+0000 7f1a0a52d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a04108510 0x7f1a041a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:08.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.512+0000 7f1a0a52d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1a041126c0 con 0x7f1a04102430 2024-05-21T03:22:08.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.512+0000 7f1a0a52d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1a04112540 con 0x7f1a04108510 2024-05-21T03:22:08.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.512+0000 7f1a0a52d700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1a04112840 con 0x7f1a04102df0 2024-05-21T03:22:08.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.512+0000 7f1a03fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a04102430 0x7f1a0419bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:08.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.512+0000 7f1a03fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a04102430 0x7f1a0419bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57870/0 (socket says 172.21.15.27:57870) 2024-05-21T03:22:08.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.512+0000 7f1a03fff700 1 -- 172.21.15.27:0/1467940380 learned_addr learned my addr 172.21.15.27:0/1467940380 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:08.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.512+0000 7f1a037fe700 1 --2- 172.21.15.27:0/1467940380 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a04102df0 0x7f1a0419c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:08.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.513+0000 7f1a08aca700 1 --2- 172.21.15.27:0/1467940380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a04108510 0x7f1a041a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:08.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.513+0000 7f1a03fff700 1 -- 172.21.15.27:0/1467940380 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a04102df0 msgr2=0x7f1a0419c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:08.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.513+0000 7f1a03fff700 1 --2- 172.21.15.27:0/1467940380 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a04102df0 0x7f1a0419c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:08.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.513+0000 7f1a03fff700 1 -- 172.21.15.27:0/1467940380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a04108510 msgr2=0x7f1a041a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:08.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.513+0000 7f1a03fff700 1 --2- 172.21.15.27:0/1467940380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a04108510 0x7f1a041a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:08.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.513+0000 7f1a03fff700 1 -- 172.21.15.27:0/1467940380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a041a0f50 con 0x7f1a04102430 2024-05-21T03:22:08.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.513+0000 7f1a03fff700 1 --2- 172.21.15.27:0/1467940380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a04102430 0x7f1a0419bf90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f19f400b530 tx=0x7f19f400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:08.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.513+0000 7f1a017fa700 1 -- 172.21.15.27:0/1467940380 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4015020 con 0x7f1a04102430 2024-05-21T03:22:08.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.513+0000 7f1a017fa700 1 -- 172.21.15.27:0/1467940380 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19f4011ce0 con 0x7f1a04102430 2024-05-21T03:22:08.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.513+0000 7f1a0a52d700 1 -- 172.21.15.27:0/1467940380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a041a1240 con 0x7f1a04102430 2024-05-21T03:22:08.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.513+0000 7f1a017fa700 1 -- 172.21.15.27:0/1467940380 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4010e40 con 0x7f1a04102430 2024-05-21T03:22:08.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.513+0000 7f1a0a52d700 1 -- 172.21.15.27:0/1467940380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a041a8ac0 con 0x7f1a04102430 2024-05-21T03:22:08.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.514+0000 7f1a0a52d700 1 -- 172.21.15.27:0/1467940380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1a0406c590 con 0x7f1a04102430 2024-05-21T03:22:08.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.515+0000 7f1a017fa700 1 -- 172.21.15.27:0/1467940380 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f19f4018020 con 0x7f1a04102430 2024-05-21T03:22:08.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.516+0000 7f1a017fa700 1 --2- 172.21.15.27:0/1467940380 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f19ec0b2910 0x7f19ec0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:08.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.516+0000 7f1a037fe700 1 --2- 172.21.15.27:0/1467940380 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f19ec0b2910 0x7f19ec0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:08.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.516+0000 7f1a017fa700 1 -- 172.21.15.27:0/1467940380 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f19f40e2280 con 0x7f1a04102430 2024-05-21T03:22:08.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.516+0000 7f1a017fa700 1 --2- 172.21.15.27:0/1467940380 >> [v2:172.21.15.165:6808/386850244,v1:172.21.15.165:6809/386850244] conn(0x7f19ec0bc550 0x7f19ec0be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:08.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.516+0000 7f1a017fa700 1 -- 172.21.15.27:0/1467940380 --> [v2:172.21.15.165:6808/386850244,v1:172.21.15.165:6809/386850244] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f19f401be50 con 0x7f19ec0bc550 2024-05-21T03:22:08.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.516+0000 7f1a017fa700 1 -- 172.21.15.27:0/1467940380 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f19f40e26d0 con 0x7f1a04102430 2024-05-21T03:22:08.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.517+0000 7f1a037fe700 1 --2- 172.21.15.27:0/1467940380 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f19ec0b2910 0x7f19ec0b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f19f800b5c0 tx=0x7f19f8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:08.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.517+0000 7f1a08aca700 1 --2- 172.21.15.27:0/1467940380 >> [v2:172.21.15.165:6808/386850244,v1:172.21.15.165:6809/386850244] conn(0x7f19ec0bc550 0x7f19ec0be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:08.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.517+0000 7f1a08aca700 1 --2- 172.21.15.27:0/1467940380 >> [v2:172.21.15.165:6808/386850244,v1:172.21.15.165:6809/386850244] conn(0x7f19ec0bc550 0x7f19ec0be9f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:08.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.518+0000 7f1a017fa700 1 -- 172.21.15.27:0/1467940380 <== osd.5 v2:172.21.15.165:6808/386850244 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f19f401be50 con 0x7f19ec0bc550 2024-05-21T03:22:08.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.535+0000 7f1a0a52d700 1 -- 172.21.15.27:0/1467940380 --> [v2:172.21.15.165:6808/386850244,v1:172.21.15.165:6809/386850244] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1a04103b50 con 0x7f19ec0bc550 2024-05-21T03:22:08.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.535+0000 7f1a017fa700 1 -- 172.21.15.27:0/1467940380 <== osd.5 v2:172.21.15.165:6808/386850244 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1a04103b50 con 0x7f19ec0bc550 2024-05-21T03:22:08.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.536+0000 7f1a0a52d700 1 -- 172.21.15.27:0/1467940380 >> [v2:172.21.15.165:6808/386850244,v1:172.21.15.165:6809/386850244] conn(0x7f19ec0bc550 msgr2=0x7f19ec0be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:08.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.536+0000 7f1a0a52d700 1 --2- 172.21.15.27:0/1467940380 >> [v2:172.21.15.165:6808/386850244,v1:172.21.15.165:6809/386850244] conn(0x7f19ec0bc550 0x7f19ec0be9f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:08.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.536+0000 7f1a0a52d700 1 -- 172.21.15.27:0/1467940380 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f19ec0b2910 msgr2=0x7f19ec0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:08.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.536+0000 7f1a0a52d700 1 --2- 172.21.15.27:0/1467940380 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f19ec0b2910 0x7f19ec0b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f19f800b5c0 tx=0x7f19f8005960 comp rx=0 tx=0).stop 2024-05-21T03:22:08.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.536+0000 7f1a0a52d700 1 -- 172.21.15.27:0/1467940380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a04102430 msgr2=0x7f1a0419bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:08.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.536+0000 7f1a0a52d700 1 --2- 172.21.15.27:0/1467940380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a04102430 0x7f1a0419bf90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f19f400b530 tx=0x7f19f400b840 comp rx=0 tx=0).stop 2024-05-21T03:22:08.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.536+0000 7f1a08aca700 1 -- 172.21.15.27:0/1467940380 reap_dead start 2024-05-21T03:22:08.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.536+0000 7f1a0a52d700 1 -- 172.21.15.27:0/1467940380 shutdown_connections 2024-05-21T03:22:08.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.536+0000 7f1a0a52d700 1 -- 172.21.15.27:0/1467940380 >> 172.21.15.27:0/1467940380 conn(0x7f1a040793b0 msgr2=0x7f1a0406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:08.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.536+0000 7f1a0a52d700 1 -- 172.21.15.27:0/1467940380 shutdown_connections 2024-05-21T03:22:08.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:08.536+0000 7f1a0a52d700 1 -- 172.21.15.27:0/1467940380 wait complete. 2024-05-21T03:22:08.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:08 smithi039 ceph-mon[36429]: pgmap v339: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:22:08.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:08 smithi165 ceph-mon[36269]: pgmap v339: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:22:09.329 INFO:teuthology.orchestra.run.smithi027.stdout:163208757314 2024-05-21T03:22:09.329 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-21T03:22:09.694 INFO:teuthology.orchestra.run.smithi027.stdout:111669149770 2024-05-21T03:22:09.695 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-21T03:22:10.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.133+0000 7fec30872700 1 -- 172.21.15.27:0/4113190308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec2c06b220 msgr2=0x7fec2c06b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:10.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.133+0000 7fec30872700 1 --2- 172.21.15.27:0/4113190308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec2c06b220 0x7fec2c06b6a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fec20009990 tx=0x7fec20009ca0 comp rx=0 tx=0).stop 2024-05-21T03:22:10.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.133+0000 7fec30872700 1 -- 172.21.15.27:0/4113190308 shutdown_connections 2024-05-21T03:22:10.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.133+0000 7fec30872700 1 --2- 172.21.15.27:0/4113190308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec2c06bbe0 0x7fec2c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:10.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.133+0000 7fec30872700 1 --2- 172.21.15.27:0/4113190308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec2c1095b0 0x7fec2c109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:10.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.133+0000 7fec30872700 1 --2- 172.21.15.27:0/4113190308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec2c06b220 0x7fec2c06b6a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:10.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.133+0000 7fec30872700 1 -- 172.21.15.27:0/4113190308 >> 172.21.15.27:0/4113190308 conn(0x7fec2c0793f0 msgr2=0x7fec2c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:10.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.133+0000 7fec30872700 1 -- 172.21.15.27:0/4113190308 shutdown_connections 2024-05-21T03:22:10.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.133+0000 7fec30872700 1 -- 172.21.15.27:0/4113190308 wait complete. 2024-05-21T03:22:10.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.134+0000 7fec30872700 1 Processor -- start 2024-05-21T03:22:10.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.134+0000 7fec30872700 1 -- start start 2024-05-21T03:22:10.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.134+0000 7fec30872700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec2c06b220 0x7fec2c19be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:10.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.134+0000 7fec30872700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec2c06bbe0 0x7fec2c19c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:10.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.134+0000 7fec30872700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec2c1095b0 0x7fec2c1a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:10.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.134+0000 7fec30872700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c113740 con 0x7fec2c06bbe0 2024-05-21T03:22:10.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.134+0000 7fec30872700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c1135c0 con 0x7fec2c1095b0 2024-05-21T03:22:10.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.134+0000 7fec30872700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c1138c0 con 0x7fec2c06b220 2024-05-21T03:22:10.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.134+0000 7fec29d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec2c06bbe0 0x7fec2c19c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:10.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.134+0000 7fec29d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec2c06bbe0 0x7fec2c19c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57876/0 (socket says 172.21.15.27:57876) 2024-05-21T03:22:10.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.134+0000 7fec29d9b700 1 -- 172.21.15.27:0/1415217483 learned_addr learned my addr 172.21.15.27:0/1415217483 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:10.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.135+0000 7fec2a59c700 1 --2- 172.21.15.27:0/1415217483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec2c06b220 0x7fec2c19be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:10.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.135+0000 7fec2ad9d700 1 --2- 172.21.15.27:0/1415217483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec2c1095b0 0x7fec2c1a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:10.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.135+0000 7fec29d9b700 1 -- 172.21.15.27:0/1415217483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec2c06b220 msgr2=0x7fec2c19be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:10.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.135+0000 7fec29d9b700 1 --2- 172.21.15.27:0/1415217483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec2c06b220 0x7fec2c19be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:10.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.135+0000 7fec29d9b700 1 -- 172.21.15.27:0/1415217483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec2c1095b0 msgr2=0x7fec2c1a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:10.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.135+0000 7fec29d9b700 1 --2- 172.21.15.27:0/1415217483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec2c1095b0 0x7fec2c1a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:10.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.135+0000 7fec29d9b700 1 -- 172.21.15.27:0/1415217483 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2c1a0ea0 con 0x7fec2c06bbe0 2024-05-21T03:22:10.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.135+0000 7fec29d9b700 1 --2- 172.21.15.27:0/1415217483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec2c06bbe0 0x7fec2c19c3d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fec20000c00 tx=0x7fec200047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:10.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.135+0000 7fec1b7fe700 1 -- 172.21.15.27:0/1415217483 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec20027070 con 0x7fec2c06bbe0 2024-05-21T03:22:10.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.135+0000 7fec30872700 1 -- 172.21.15.27:0/1415217483 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec2c1a1130 con 0x7fec2c06bbe0 2024-05-21T03:22:10.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.135+0000 7fec1b7fe700 1 -- 172.21.15.27:0/1415217483 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec200041d0 con 0x7fec2c06bbe0 2024-05-21T03:22:10.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.135+0000 7fec1b7fe700 1 -- 172.21.15.27:0/1415217483 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2000f460 con 0x7fec2c06bbe0 2024-05-21T03:22:10.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.135+0000 7fec30872700 1 -- 172.21.15.27:0/1415217483 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec2c1a8a10 con 0x7fec2c06bbe0 2024-05-21T03:22:10.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.136+0000 7fec30872700 1 -- 172.21.15.27:0/1415217483 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fec2c10f160 con 0x7fec2c06bbe0 2024-05-21T03:22:10.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.138+0000 7fec1b7fe700 1 -- 172.21.15.27:0/1415217483 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fec2002a070 con 0x7fec2c06bbe0 2024-05-21T03:22:10.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.138+0000 7fec1b7fe700 1 --2- 172.21.15.27:0/1415217483 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fec140b2990 0x7fec140b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:10.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.138+0000 7fec2a59c700 1 --2- 172.21.15.27:0/1415217483 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fec140b2990 0x7fec140b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:10.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.138+0000 7fec1b7fe700 1 -- 172.21.15.27:0/1415217483 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fec200e3a80 con 0x7fec2c06bbe0 2024-05-21T03:22:10.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.139+0000 7fec1b7fe700 1 --2- 172.21.15.27:0/1415217483 >> [v2:172.21.15.39:6824/363500811,v1:172.21.15.39:6825/363500811] conn(0x7fec140bc5d0 0x7fec140bea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:10.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.139+0000 7fec1b7fe700 1 -- 172.21.15.27:0/1415217483 --> [v2:172.21.15.39:6824/363500811,v1:172.21.15.39:6825/363500811] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fec200043d0 con 0x7fec140bc5d0 2024-05-21T03:22:10.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.139+0000 7fec1b7fe700 1 -- 172.21.15.27:0/1415217483 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fec200e3ed0 con 0x7fec2c06bbe0 2024-05-21T03:22:10.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.139+0000 7fec2a59c700 1 --2- 172.21.15.27:0/1415217483 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fec140b2990 0x7fec140b4e50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fec1c005fd0 tx=0x7fec1c00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:10.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.139+0000 7fec2ad9d700 1 --2- 172.21.15.27:0/1415217483 >> [v2:172.21.15.39:6824/363500811,v1:172.21.15.39:6825/363500811] conn(0x7fec140bc5d0 0x7fec140bea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:10.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.140+0000 7fec2ad9d700 1 --2- 172.21.15.27:0/1415217483 >> [v2:172.21.15.39:6824/363500811,v1:172.21.15.39:6825/363500811] conn(0x7fec140bc5d0 0x7fec140bea70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:10.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.140+0000 7fec1b7fe700 1 -- 172.21.15.27:0/1415217483 <== osd.10 v2:172.21.15.39:6824/363500811 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fec200043d0 con 0x7fec140bc5d0 2024-05-21T03:22:10.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.159+0000 7fec30872700 1 -- 172.21.15.27:0/1415217483 --> [v2:172.21.15.39:6824/363500811,v1:172.21.15.39:6825/363500811] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fec2c104c70 con 0x7fec140bc5d0 2024-05-21T03:22:10.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.160+0000 7fec1b7fe700 1 -- 172.21.15.27:0/1415217483 <== osd.10 v2:172.21.15.39:6824/363500811 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fec2c104c70 con 0x7fec140bc5d0 2024-05-21T03:22:10.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.160+0000 7fec30872700 1 -- 172.21.15.27:0/1415217483 >> [v2:172.21.15.39:6824/363500811,v1:172.21.15.39:6825/363500811] conn(0x7fec140bc5d0 msgr2=0x7fec140bea70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:10.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.160+0000 7fec30872700 1 --2- 172.21.15.27:0/1415217483 >> [v2:172.21.15.39:6824/363500811,v1:172.21.15.39:6825/363500811] conn(0x7fec140bc5d0 0x7fec140bea70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:10.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.161+0000 7fec30872700 1 -- 172.21.15.27:0/1415217483 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fec140b2990 msgr2=0x7fec140b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:10.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.161+0000 7fec30872700 1 --2- 172.21.15.27:0/1415217483 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fec140b2990 0x7fec140b4e50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fec1c005fd0 tx=0x7fec1c00a560 comp rx=0 tx=0).stop 2024-05-21T03:22:10.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.161+0000 7fec30872700 1 -- 172.21.15.27:0/1415217483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec2c06bbe0 msgr2=0x7fec2c19c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:10.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.161+0000 7fec30872700 1 --2- 172.21.15.27:0/1415217483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec2c06bbe0 0x7fec2c19c3d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fec20000c00 tx=0x7fec200047b0 comp rx=0 tx=0).stop 2024-05-21T03:22:10.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.161+0000 7fec2ad9d700 1 -- 172.21.15.27:0/1415217483 reap_dead start 2024-05-21T03:22:10.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.161+0000 7fec30872700 1 -- 172.21.15.27:0/1415217483 shutdown_connections 2024-05-21T03:22:10.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.161+0000 7fec30872700 1 -- 172.21.15.27:0/1415217483 >> 172.21.15.27:0/1415217483 conn(0x7fec2c0793f0 msgr2=0x7fec2c10e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:10.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.161+0000 7fec30872700 1 -- 172.21.15.27:0/1415217483 shutdown_connections 2024-05-21T03:22:10.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:10.161+0000 7fec30872700 1 -- 172.21.15.27:0/1415217483 wait complete. 2024-05-21T03:22:10.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:10 smithi027 ceph-mon[32179]: pgmap v340: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:22:10.752 INFO:teuthology.orchestra.run.smithi027.stdout:55834574934 2024-05-21T03:22:10.752 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-21T03:22:10.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:10 smithi039 ceph-mon[36429]: pgmap v340: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:22:10.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:10 smithi165 ceph-mon[36269]: pgmap v340: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:22:11.402 INFO:teuthology.orchestra.run.smithi027.stdout:283467841576 2024-05-21T03:22:11.402 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-21T03:22:11.495 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:22:11.495 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:22:12.592 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:12 smithi027 ceph-mon[32179]: pgmap v341: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-05-21T03:22:12.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:12 smithi039 ceph-mon[36429]: pgmap v341: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-05-21T03:22:12.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:12 smithi165 ceph-mon[36269]: pgmap v341: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-05-21T03:22:13.374 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:22:13.381 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:22:13.382 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:22:13.392 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:22:13.397 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:22:13.828 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:22:13.833 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:22:13.834 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:22:13.837 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:22:13.973 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:22:14.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.586+0000 7f7a5e025700 1 -- 172.21.15.27:0/3961069128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a581037f0 msgr2=0x7f7a58103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:14.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.586+0000 7f7a5e025700 1 --2- 172.21.15.27:0/3961069128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a581037f0 0x7f7a58103bd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7a480099c0 tx=0x7f7a48009cd0 comp rx=0 tx=0).stop 2024-05-21T03:22:14.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.586+0000 7f7a5e025700 1 -- 172.21.15.27:0/3961069128 shutdown_connections 2024-05-21T03:22:14.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.586+0000 7f7a5e025700 1 --2- 172.21.15.27:0/3961069128 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a581088f0 0x7f7a58110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:14.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.586+0000 7f7a5e025700 1 --2- 172.21.15.27:0/3961069128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a581037f0 0x7f7a58103bd0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:14.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.586+0000 7f7a5e025700 1 --2- 172.21.15.27:0/3961069128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a581041a0 0x7f7a581081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:14.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.586+0000 7f7a5e025700 1 -- 172.21.15.27:0/3961069128 >> 172.21.15.27:0/3961069128 conn(0x7f7a5807a6a0 msgr2=0x7f7a5807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:14.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.587+0000 7f7a5e025700 1 -- 172.21.15.27:0/3961069128 shutdown_connections 2024-05-21T03:22:14.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.587+0000 7f7a5e025700 1 -- 172.21.15.27:0/3961069128 wait complete. 2024-05-21T03:22:14.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.587+0000 7f7a5e025700 1 Processor -- start 2024-05-21T03:22:14.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.587+0000 7f7a5e025700 1 -- start start 2024-05-21T03:22:14.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.588+0000 7f7a5e025700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a581037f0 0x7f7a58075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:14.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.588+0000 7f7a5e025700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a581041a0 0x7f7a58075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:14.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.588+0000 7f7a5e025700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a581088f0 0x7f7a580762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:14.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.588+0000 7f7a5e025700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7a58113790 con 0x7f7a581088f0 2024-05-21T03:22:14.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.588+0000 7f7a5e025700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7a58113610 con 0x7f7a581041a0 2024-05-21T03:22:14.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.588+0000 7f7a5e025700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7a58113910 con 0x7f7a581037f0 2024-05-21T03:22:14.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.588+0000 7f7a57fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a581088f0 0x7f7a580762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:14.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.588+0000 7f7a57fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a581088f0 0x7f7a580762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51694/0 (socket says 172.21.15.27:51694) 2024-05-21T03:22:14.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.588+0000 7f7a57fff700 1 -- 172.21.15.27:0/775829634 learned_addr learned my addr 172.21.15.27:0/775829634 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:14.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.588+0000 7f7a56ffd700 1 --2- 172.21.15.27:0/775829634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a581041a0 0x7f7a58075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:14.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.588+0000 7f7a577fe700 1 --2- 172.21.15.27:0/775829634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a581037f0 0x7f7a58075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:14.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.588+0000 7f7a57fff700 1 -- 172.21.15.27:0/775829634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a581037f0 msgr2=0x7f7a58075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:14.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.588+0000 7f7a57fff700 1 --2- 172.21.15.27:0/775829634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a581037f0 0x7f7a58075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:14.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.588+0000 7f7a57fff700 1 -- 172.21.15.27:0/775829634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a581041a0 msgr2=0x7f7a58075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:14.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.588+0000 7f7a57fff700 1 --2- 172.21.15.27:0/775829634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a581041a0 0x7f7a58075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:14.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.588+0000 7f7a57fff700 1 -- 172.21.15.27:0/775829634 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a581a8190 con 0x7f7a581088f0 2024-05-21T03:22:14.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.588+0000 7f7a57fff700 1 --2- 172.21.15.27:0/775829634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a581088f0 0x7f7a580762c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000b970 tx=0x7f7a5000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:14.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.588+0000 7f7a54ff9700 1 -- 172.21.15.27:0/775829634 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a50019070 con 0x7f7a581088f0 2024-05-21T03:22:14.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.588+0000 7f7a5e025700 1 -- 172.21.15.27:0/775829634 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a581a8480 con 0x7f7a581088f0 2024-05-21T03:22:14.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.588+0000 7f7a54ff9700 1 -- 172.21.15.27:0/775829634 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a5000cde0 con 0x7f7a581088f0 2024-05-21T03:22:14.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.588+0000 7f7a54ff9700 1 -- 172.21.15.27:0/775829634 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a50007370 con 0x7f7a581088f0 2024-05-21T03:22:14.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.589+0000 7f7a5e025700 1 -- 172.21.15.27:0/775829634 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a581a8a10 con 0x7f7a581088f0 2024-05-21T03:22:14.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.589+0000 7f7a5e025700 1 -- 172.21.15.27:0/775829634 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a5810bb30 con 0x7f7a581088f0 2024-05-21T03:22:14.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.591+0000 7f7a54ff9700 1 -- 172.21.15.27:0/775829634 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7a5001c070 con 0x7f7a581088f0 2024-05-21T03:22:14.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.592+0000 7f7a54ff9700 1 --2- 172.21.15.27:0/775829634 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f7a400b2880 0x7f7a400b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:14.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.592+0000 7f7a577fe700 1 --2- 172.21.15.27:0/775829634 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f7a400b2880 0x7f7a400b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:14.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.592+0000 7f7a54ff9700 1 -- 172.21.15.27:0/775829634 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f7a500e1e40 con 0x7f7a581088f0 2024-05-21T03:22:14.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.592+0000 7f7a577fe700 1 --2- 172.21.15.27:0/775829634 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f7a400b2880 0x7f7a400b4d40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7a48003800 tx=0x7f7a4801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:14.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.595+0000 7f7a54ff9700 1 -- 172.21.15.27:0/775829634 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7a50023070 con 0x7f7a581088f0 2024-05-21T03:22:14.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:14 smithi027 ceph-mon[32179]: pgmap v342: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 11 op/s 2024-05-21T03:22:14.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:14 smithi027 ceph-mon[32179]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-21T03:22:14.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:14 smithi039 ceph-mon[36429]: pgmap v342: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 11 op/s 2024-05-21T03:22:14.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:14 smithi039 ceph-mon[36429]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-21T03:22:14.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:14 smithi165 ceph-mon[36269]: pgmap v342: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 11 op/s 2024-05-21T03:22:14.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:14 smithi165 ceph-mon[36269]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-21T03:22:14.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.821+0000 7f7a5e025700 1 -- 172.21.15.27:0/775829634 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f7a58076da0 con 0x7f7a581088f0 2024-05-21T03:22:14.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.821+0000 7f7a54ff9700 1 -- 172.21.15.27:0/775829634 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7a58076da0 con 0x7f7a581088f0 2024-05-21T03:22:14.821 INFO:teuthology.orchestra.run.smithi027.stdout:137438953544 2024-05-21T03:22:14.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.824+0000 7f7a5e025700 1 -- 172.21.15.27:0/775829634 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f7a400b2880 msgr2=0x7f7a400b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:14.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.824+0000 7f7a5e025700 1 --2- 172.21.15.27:0/775829634 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f7a400b2880 0x7f7a400b4d40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7a48003800 tx=0x7f7a4801a040 comp rx=0 tx=0).stop 2024-05-21T03:22:14.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.824+0000 7f7a5e025700 1 -- 172.21.15.27:0/775829634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a581088f0 msgr2=0x7f7a580762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:14.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.824+0000 7f7a5e025700 1 --2- 172.21.15.27:0/775829634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a581088f0 0x7f7a580762c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000b970 tx=0x7f7a5000bc80 comp rx=0 tx=0).stop 2024-05-21T03:22:14.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.824+0000 7f7a5e025700 1 -- 172.21.15.27:0/775829634 shutdown_connections 2024-05-21T03:22:14.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.824+0000 7f7a5e025700 1 --2- 172.21.15.27:0/775829634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a581088f0 0x7f7a580762c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:14.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.824+0000 7f7a5e025700 1 --2- 172.21.15.27:0/775829634 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f7a400b2880 0x7f7a400b4d40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:14.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.824+0000 7f7a5e025700 1 --2- 172.21.15.27:0/775829634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a581037f0 0x7f7a58075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:14.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.824+0000 7f7a5e025700 1 --2- 172.21.15.27:0/775829634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a581041a0 0x7f7a58075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:14.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.824+0000 7f7a5e025700 1 -- 172.21.15.27:0/775829634 >> 172.21.15.27:0/775829634 conn(0x7f7a5807a6a0 msgr2=0x7f7a58101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:14.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.824+0000 7f7a5e025700 1 -- 172.21.15.27:0/775829634 shutdown_connections 2024-05-21T03:22:14.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:14.824+0000 7f7a5e025700 1 -- 172.21.15.27:0/775829634 wait complete. 2024-05-21T03:22:15.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:15 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/775829634' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-21T03:22:15.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:15 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/775829634' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-21T03:22:15.885 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:15 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/775829634' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-21T03:22:15.914 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953540 got 137438953544 for osd.4 2024-05-21T03:22:15.914 DEBUG:teuthology.parallel:result is None 2024-05-21T03:22:16.633 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:16 smithi027 ceph-mon[32179]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-21T03:22:16.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:16 smithi039 ceph-mon[36429]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-21T03:22:16.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:16 smithi165 ceph-mon[36269]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-21T03:22:18.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:18 smithi027 ceph-mon[32179]: pgmap v344: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-21T03:22:18.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:18 smithi039 ceph-mon[36429]: pgmap v344: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-21T03:22:18.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:18 smithi165 ceph-mon[36269]: pgmap v344: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-21T03:22:20.736 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:20 smithi027 ceph-mon[32179]: pgmap v345: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-21T03:22:20.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:20 smithi039 ceph-mon[36429]: pgmap v345: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-21T03:22:20.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:20 smithi165 ceph-mon[36269]: pgmap v345: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-21T03:22:21.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.056+0000 7f8cf7ba2700 1 -- 172.21.15.27:0/107969218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cf01041d0 msgr2=0x7f8cf01081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:21.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.056+0000 7f8cf7ba2700 1 --2- 172.21.15.27:0/107969218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cf01041d0 0x7f8cf01081e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8ce00099c0 tx=0x7f8ce0009cd0 comp rx=0 tx=0).stop 2024-05-21T03:22:21.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.056+0000 7f8cf7ba2700 1 -- 172.21.15.27:0/107969218 shutdown_connections 2024-05-21T03:22:21.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.056+0000 7f8cf7ba2700 1 --2- 172.21.15.27:0/107969218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cf0108910 0x7f8cf0110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:21.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.056+0000 7f8cf7ba2700 1 --2- 172.21.15.27:0/107969218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cf0103820 0x7f8cf0103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:21.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.056+0000 7f8cf7ba2700 1 --2- 172.21.15.27:0/107969218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cf01041d0 0x7f8cf01081e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:21.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.056+0000 7f8cf7ba2700 1 -- 172.21.15.27:0/107969218 >> 172.21.15.27:0/107969218 conn(0x7f8cf007a6b0 msgr2=0x7f8cf007aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:21.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.056+0000 7f8cf7ba2700 1 -- 172.21.15.27:0/107969218 shutdown_connections 2024-05-21T03:22:21.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.056+0000 7f8cf7ba2700 1 -- 172.21.15.27:0/107969218 wait complete. 2024-05-21T03:22:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.056+0000 7f8cf7ba2700 1 Processor -- start 2024-05-21T03:22:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.057+0000 7f8cf7ba2700 1 -- start start 2024-05-21T03:22:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.057+0000 7f8cf7ba2700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cf0103820 0x7f8cf019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.057+0000 7f8cf7ba2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cf01041d0 0x7f8cf019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.057+0000 7f8cf7ba2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cf0108910 0x7f8cf0199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.057+0000 7f8cf7ba2700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf01136d0 con 0x7f8cf01041d0 2024-05-21T03:22:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.057+0000 7f8cf7ba2700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf0113550 con 0x7f8cf0108910 2024-05-21T03:22:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.057+0000 7f8cf7ba2700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf0113850 con 0x7f8cf0103820 2024-05-21T03:22:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.057+0000 7f8cf513d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cf01041d0 0x7f8cf019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.057+0000 7f8cf513d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cf01041d0 0x7f8cf019fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51708/0 (socket says 172.21.15.27:51708) 2024-05-21T03:22:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.057+0000 7f8cf513d700 1 -- 172.21.15.27:0/2047814384 learned_addr learned my addr 172.21.15.27:0/2047814384 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.057+0000 7f8cf593e700 1 --2- 172.21.15.27:0/2047814384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cf0103820 0x7f8cf019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.057+0000 7f8cf613f700 1 --2- 172.21.15.27:0/2047814384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cf0108910 0x7f8cf0199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.057+0000 7f8cf513d700 1 -- 172.21.15.27:0/2047814384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cf0103820 msgr2=0x7f8cf019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.057+0000 7f8cf513d700 1 --2- 172.21.15.27:0/2047814384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cf0103820 0x7f8cf019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.057+0000 7f8cf513d700 1 -- 172.21.15.27:0/2047814384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cf0108910 msgr2=0x7f8cf0199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.057+0000 7f8cf513d700 1 --2- 172.21.15.27:0/2047814384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cf0108910 0x7f8cf0199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.057+0000 7f8cf513d700 1 -- 172.21.15.27:0/2047814384 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf019a030 con 0x7f8cf01041d0 2024-05-21T03:22:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.058+0000 7f8cf513d700 1 --2- 172.21.15.27:0/2047814384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cf01041d0 0x7f8cf019fc10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8ce00098e0 tx=0x7f8ce00057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.058+0000 7f8ce6ffd700 1 -- 172.21.15.27:0/2047814384 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0027070 con 0x7f8cf01041d0 2024-05-21T03:22:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.058+0000 7f8cf7ba2700 1 -- 172.21.15.27:0/2047814384 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf019a2c0 con 0x7f8cf01041d0 2024-05-21T03:22:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.058+0000 7f8ce6ffd700 1 -- 172.21.15.27:0/2047814384 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ce0003f20 con 0x7f8cf01041d0 2024-05-21T03:22:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.058+0000 7f8ce6ffd700 1 -- 172.21.15.27:0/2047814384 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0022d70 con 0x7f8cf01041d0 2024-05-21T03:22:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.058+0000 7f8cf7ba2700 1 -- 172.21.15.27:0/2047814384 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf01a6530 con 0x7f8cf01041d0 2024-05-21T03:22:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.059+0000 7f8cf7ba2700 1 -- 172.21.15.27:0/2047814384 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cf0104ca0 con 0x7f8cf01041d0 2024-05-21T03:22:21.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.060+0000 7f8ce6ffd700 1 -- 172.21.15.27:0/2047814384 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8ce002a070 con 0x7f8cf01041d0 2024-05-21T03:22:21.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.061+0000 7f8ce6ffd700 1 --2- 172.21.15.27:0/2047814384 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f8cdc0b2880 0x7f8cdc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:21.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.061+0000 7f8cf593e700 1 --2- 172.21.15.27:0/2047814384 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f8cdc0b2880 0x7f8cdc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:21.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.061+0000 7f8ce6ffd700 1 -- 172.21.15.27:0/2047814384 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f8ce00e3970 con 0x7f8cf01041d0 2024-05-21T03:22:21.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.062+0000 7f8cf593e700 1 --2- 172.21.15.27:0/2047814384 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f8cdc0b2880 0x7f8cdc0b4d40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8cec00ac50 tx=0x7f8cec00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:21.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.065+0000 7f8ce6ffd700 1 -- 172.21.15.27:0/2047814384 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8ce0013070 con 0x7f8cf01041d0 2024-05-21T03:22:21.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.298+0000 7f8cf7ba2700 1 -- 172.21.15.27:0/2047814384 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f8cf006a230 con 0x7f8cf01041d0 2024-05-21T03:22:21.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.298+0000 7f8ce6ffd700 1 -- 172.21.15.27:0/2047814384 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8cf006a230 con 0x7f8cf01041d0 2024-05-21T03:22:21.299 INFO:teuthology.orchestra.run.smithi027.stdout:236223201331 2024-05-21T03:22:21.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.300+0000 7f8cf7ba2700 1 -- 172.21.15.27:0/2047814384 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f8cdc0b2880 msgr2=0x7f8cdc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:21.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.300+0000 7f8cf7ba2700 1 --2- 172.21.15.27:0/2047814384 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f8cdc0b2880 0x7f8cdc0b4d40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8cec00ac50 tx=0x7f8cec00a380 comp rx=0 tx=0).stop 2024-05-21T03:22:21.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.300+0000 7f8cf7ba2700 1 -- 172.21.15.27:0/2047814384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cf01041d0 msgr2=0x7f8cf019fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:21.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.300+0000 7f8cf7ba2700 1 --2- 172.21.15.27:0/2047814384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cf01041d0 0x7f8cf019fc10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8ce00098e0 tx=0x7f8ce00057c0 comp rx=0 tx=0).stop 2024-05-21T03:22:21.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.300+0000 7f8cf7ba2700 1 -- 172.21.15.27:0/2047814384 shutdown_connections 2024-05-21T03:22:21.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.300+0000 7f8cf7ba2700 1 --2- 172.21.15.27:0/2047814384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cf0108910 0x7f8cf0199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:21.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.300+0000 7f8cf7ba2700 1 --2- 172.21.15.27:0/2047814384 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f8cdc0b2880 0x7f8cdc0b4d40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:21.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.300+0000 7f8cf7ba2700 1 --2- 172.21.15.27:0/2047814384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cf0103820 0x7f8cf019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:21.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.300+0000 7f8cf7ba2700 1 --2- 172.21.15.27:0/2047814384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cf01041d0 0x7f8cf019fc10 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:21.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.300+0000 7f8cf7ba2700 1 -- 172.21.15.27:0/2047814384 >> 172.21.15.27:0/2047814384 conn(0x7f8cf007a6b0 msgr2=0x7f8cf0101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:21.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.300+0000 7f8cf7ba2700 1 -- 172.21.15.27:0/2047814384 shutdown_connections 2024-05-21T03:22:21.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:21.300+0000 7f8cf7ba2700 1 -- 172.21.15.27:0/2047814384 wait complete. 2024-05-21T03:22:21.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:21 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2047814384' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-21T03:22:21.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:21 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/2047814384' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-21T03:22:21.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:21 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/2047814384' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-21T03:22:22.602 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:22 smithi027 ceph-mon[32179]: pgmap v346: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-21T03:22:22.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:22 smithi039 ceph-mon[36429]: pgmap v346: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-21T03:22:22.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:22 smithi165 ceph-mon[36269]: pgmap v346: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-21T03:22:22.899 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201326 got 236223201331 for osd.8 2024-05-21T03:22:22.899 DEBUG:teuthology.parallel:result is None 2024-05-21T03:22:24.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:24 smithi027 ceph-mon[32179]: pgmap v347: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-21T03:22:24.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:24 smithi039 ceph-mon[36429]: pgmap v347: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-21T03:22:24.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:24 smithi165 ceph-mon[36269]: pgmap v347: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-21T03:22:24.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.899+0000 7f6fc6c85700 1 -- 172.21.15.27:0/2991446069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fc006bd10 msgr2=0x7f6fc006c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:24.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.899+0000 7f6fc6c85700 1 --2- 172.21.15.27:0/2991446069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fc006bd10 0x7f6fc006c190 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000b0d0 tx=0x7f6fb000b3e0 comp rx=0 tx=0).stop 2024-05-21T03:22:24.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.899+0000 7f6fc6c85700 1 -- 172.21.15.27:0/2991446069 shutdown_connections 2024-05-21T03:22:24.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.899+0000 7f6fc6c85700 1 --2- 172.21.15.27:0/2991446069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fc006c6d0 0x7f6fc0110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:24.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.899+0000 7f6fc6c85700 1 --2- 172.21.15.27:0/2991446069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fc0109630 0x7f6fc0109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:24.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.899+0000 7f6fc6c85700 1 --2- 172.21.15.27:0/2991446069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fc006bd10 0x7f6fc006c190 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:24.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.899+0000 7f6fc6c85700 1 -- 172.21.15.27:0/2991446069 >> 172.21.15.27:0/2991446069 conn(0x7f6fc00794e0 msgr2=0x7f6fc00798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:24.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.899+0000 7f6fc6c85700 1 -- 172.21.15.27:0/2991446069 shutdown_connections 2024-05-21T03:22:24.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.899+0000 7f6fc6c85700 1 -- 172.21.15.27:0/2991446069 wait complete. 2024-05-21T03:22:24.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.900+0000 7f6fc6c85700 1 Processor -- start 2024-05-21T03:22:24.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.900+0000 7f6fc6c85700 1 -- start start 2024-05-21T03:22:24.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.900+0000 7f6fc6c85700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fc006bd10 0x7f6fc01024b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:24.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.900+0000 7f6fc6c85700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fc006c6d0 0x7f6fc01029f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:24.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.900+0000 7f6fc6c85700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fc0109630 0x7f6fc0102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:24.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.900+0000 7f6fc6c85700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc0113790 con 0x7f6fc006bd10 2024-05-21T03:22:24.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.900+0000 7f6fc6c85700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc0113610 con 0x7f6fc0109630 2024-05-21T03:22:24.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.900+0000 7f6fc6c85700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc0113910 con 0x7f6fc006c6d0 2024-05-21T03:22:24.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.900+0000 7f6fc4a21700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fc006bd10 0x7f6fc01024b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:24.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.900+0000 7f6fc4a21700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fc006bd10 0x7f6fc01024b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36454/0 (socket says 172.21.15.27:36454) 2024-05-21T03:22:24.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.900+0000 7f6fc4a21700 1 -- 172.21.15.27:0/1660840902 learned_addr learned my addr 172.21.15.27:0/1660840902 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:24.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.900+0000 7f6fbffff700 1 --2- 172.21.15.27:0/1660840902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fc006c6d0 0x7f6fc01029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:24.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.900+0000 7f6fc5222700 1 --2- 172.21.15.27:0/1660840902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fc0109630 0x7f6fc0102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:24.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.900+0000 7f6fc4a21700 1 -- 172.21.15.27:0/1660840902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fc006c6d0 msgr2=0x7f6fc01029f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:24.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.900+0000 7f6fc4a21700 1 --2- 172.21.15.27:0/1660840902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fc006c6d0 0x7f6fc01029f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:24.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.900+0000 7f6fc4a21700 1 -- 172.21.15.27:0/1660840902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fc0109630 msgr2=0x7f6fc0102f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:24.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.900+0000 7f6fc4a21700 1 --2- 172.21.15.27:0/1660840902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fc0109630 0x7f6fc0102f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:24.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.900+0000 7f6fc4a21700 1 -- 172.21.15.27:0/1660840902 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc01a8070 con 0x7f6fc006bd10 2024-05-21T03:22:24.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.901+0000 7f6fc4a21700 1 --2- 172.21.15.27:0/1660840902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fc006bd10 0x7f6fc01024b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800d980 tx=0x7f6fa800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:24.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.901+0000 7f6fbdffb700 1 -- 172.21.15.27:0/1660840902 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8017070 con 0x7f6fc006bd10 2024-05-21T03:22:24.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.901+0000 7f6fbdffb700 1 -- 172.21.15.27:0/1660840902 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fa8005020 con 0x7f6fc006bd10 2024-05-21T03:22:24.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.901+0000 7f6fc6c85700 1 -- 172.21.15.27:0/1660840902 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fc01a8360 con 0x7f6fc006bd10 2024-05-21T03:22:24.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.901+0000 7f6fbdffb700 1 -- 172.21.15.27:0/1660840902 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa800fa10 con 0x7f6fc006bd10 2024-05-21T03:22:24.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.901+0000 7f6fc6c85700 1 -- 172.21.15.27:0/1660840902 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fc01a8850 con 0x7f6fc006bd10 2024-05-21T03:22:24.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.902+0000 7f6fc6c85700 1 -- 172.21.15.27:0/1660840902 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fc010bac0 con 0x7f6fc006bd10 2024-05-21T03:22:24.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.903+0000 7f6fbdffb700 1 -- 172.21.15.27:0/1660840902 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6fa8004a30 con 0x7f6fc006bd10 2024-05-21T03:22:24.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.904+0000 7f6fbdffb700 1 --2- 172.21.15.27:0/1660840902 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f6fac0b2880 0x7f6fac0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:24.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.905+0000 7f6fbffff700 1 --2- 172.21.15.27:0/1660840902 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f6fac0b2880 0x7f6fac0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:24.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.905+0000 7f6fbdffb700 1 -- 172.21.15.27:0/1660840902 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f6fa80e24f0 con 0x7f6fc006bd10 2024-05-21T03:22:24.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.905+0000 7f6fbffff700 1 --2- 172.21.15.27:0/1660840902 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f6fac0b2880 0x7f6fac0b4d40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0079a70 tx=0x7f6fb000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:24.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:24.908+0000 7f6fbdffb700 1 -- 172.21.15.27:0/1660840902 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6fa8020020 con 0x7f6fc006bd10 2024-05-21T03:22:25.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.129+0000 7f6fc6c85700 1 -- 172.21.15.27:0/1660840902 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f6fc01039e0 con 0x7f6fc006bd10 2024-05-21T03:22:25.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.130+0000 7f6fbdffb700 1 -- 172.21.15.27:0/1660840902 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6fa80a5020 con 0x7f6fc006bd10 2024-05-21T03:22:25.130 INFO:teuthology.orchestra.run.smithi027.stdout:163208757317 2024-05-21T03:22:25.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.132+0000 7f6fc6c85700 1 -- 172.21.15.27:0/1660840902 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f6fac0b2880 msgr2=0x7f6fac0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:25.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.132+0000 7f6fc6c85700 1 --2- 172.21.15.27:0/1660840902 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f6fac0b2880 0x7f6fac0b4d40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0079a70 tx=0x7f6fb000b040 comp rx=0 tx=0).stop 2024-05-21T03:22:25.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.132+0000 7f6fc6c85700 1 -- 172.21.15.27:0/1660840902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fc006bd10 msgr2=0x7f6fc01024b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:25.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.132+0000 7f6fc6c85700 1 --2- 172.21.15.27:0/1660840902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fc006bd10 0x7f6fc01024b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800d980 tx=0x7f6fa800dc90 comp rx=0 tx=0).stop 2024-05-21T03:22:25.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.132+0000 7f6fc6c85700 1 -- 172.21.15.27:0/1660840902 shutdown_connections 2024-05-21T03:22:25.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.132+0000 7f6fc6c85700 1 --2- 172.21.15.27:0/1660840902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fc0109630 0x7f6fc0102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:25.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.132+0000 7f6fc6c85700 1 --2- 172.21.15.27:0/1660840902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fc006bd10 0x7f6fc01024b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:25.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.132+0000 7f6fc6c85700 1 --2- 172.21.15.27:0/1660840902 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f6fac0b2880 0x7f6fac0b4d40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:25.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.132+0000 7f6fc6c85700 1 --2- 172.21.15.27:0/1660840902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fc006c6d0 0x7f6fc01029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:25.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.132+0000 7f6fc6c85700 1 -- 172.21.15.27:0/1660840902 >> 172.21.15.27:0/1660840902 conn(0x7f6fc00794e0 msgr2=0x7f6fc0100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:25.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.132+0000 7f6fc6c85700 1 -- 172.21.15.27:0/1660840902 shutdown_connections 2024-05-21T03:22:25.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.132+0000 7f6fc6c85700 1 -- 172.21.15.27:0/1660840902 wait complete. 2024-05-21T03:22:25.605 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:25 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/1660840902' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-21T03:22:25.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.711+0000 7f7978b08700 1 -- 172.21.15.27:0/631288095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f797406b220 msgr2=0x7f797406b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:25.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.711+0000 7f7978b08700 1 --2- 172.21.15.27:0/631288095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f797406b220 0x7f797406b6a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7968009990 tx=0x7f7968009ca0 comp rx=0 tx=0).stop 2024-05-21T03:22:25.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.712+0000 7f7978b08700 1 -- 172.21.15.27:0/631288095 shutdown_connections 2024-05-21T03:22:25.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.712+0000 7f7978b08700 1 --2- 172.21.15.27:0/631288095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f797406bbe0 0x7f79741103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:25.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.712+0000 7f7978b08700 1 --2- 172.21.15.27:0/631288095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79741095b0 0x7f7974109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:25.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.712+0000 7f7978b08700 1 --2- 172.21.15.27:0/631288095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f797406b220 0x7f797406b6a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:25.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.712+0000 7f7978b08700 1 -- 172.21.15.27:0/631288095 >> 172.21.15.27:0/631288095 conn(0x7f79740793f0 msgr2=0x7f7974079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:25.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.712+0000 7f7978b08700 1 -- 172.21.15.27:0/631288095 shutdown_connections 2024-05-21T03:22:25.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.712+0000 7f7978b08700 1 -- 172.21.15.27:0/631288095 wait complete. 2024-05-21T03:22:25.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.712+0000 7f7978b08700 1 Processor -- start 2024-05-21T03:22:25.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.713+0000 7f7978b08700 1 -- start start 2024-05-21T03:22:25.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.713+0000 7f7978b08700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f797406b220 0x7f797419f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:25.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.713+0000 7f7978b08700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f797406bbe0 0x7f797419fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:25.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.713+0000 7f7978b08700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79741095b0 0x7f7974199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:25.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.713+0000 7f7978b08700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f79741136e0 con 0x7f797406bbe0 2024-05-21T03:22:25.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.713+0000 7f7978b08700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7974113560 con 0x7f797406b220 2024-05-21T03:22:25.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.713+0000 7f7978b08700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7974113860 con 0x7f79741095b0 2024-05-21T03:22:25.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.713+0000 7f7971d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f797406bbe0 0x7f797419fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.713+0000 7f7971d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f797406bbe0 0x7f797419fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36484/0 (socket says 172.21.15.27:36484) 2024-05-21T03:22:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.713+0000 7f7971d9b700 1 -- 172.21.15.27:0/1856092723 learned_addr learned my addr 172.21.15.27:0/1856092723 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.713+0000 7f797259c700 1 --2- 172.21.15.27:0/1856092723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f797406b220 0x7f797419f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.713+0000 7f7971d9b700 1 -- 172.21.15.27:0/1856092723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79741095b0 msgr2=0x7f7974199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.713+0000 7f7971d9b700 1 --2- 172.21.15.27:0/1856092723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79741095b0 0x7f7974199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.713+0000 7f7971d9b700 1 -- 172.21.15.27:0/1856092723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f797406b220 msgr2=0x7f797419f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.713+0000 7f7971d9b700 1 --2- 172.21.15.27:0/1856092723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f797406b220 0x7f797419f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.713+0000 7f7971d9b700 1 -- 172.21.15.27:0/1856092723 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f797419a180 con 0x7f797406bbe0 2024-05-21T03:22:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.713+0000 7f7971d9b700 1 --2- 172.21.15.27:0/1856092723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f797406bbe0 0x7f797419fda0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7968000c00 tx=0x7f79680047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.714+0000 7f79637fe700 1 -- 172.21.15.27:0/1856092723 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7968027070 con 0x7f797406bbe0 2024-05-21T03:22:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.714+0000 7f79637fe700 1 -- 172.21.15.27:0/1856092723 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7968003fa0 con 0x7f797406bbe0 2024-05-21T03:22:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.714+0000 7f79637fe700 1 -- 172.21.15.27:0/1856092723 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7968022db0 con 0x7f797406bbe0 2024-05-21T03:22:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.714+0000 7f7978b08700 1 -- 172.21.15.27:0/1856092723 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f797419a410 con 0x7f797406bbe0 2024-05-21T03:22:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.714+0000 7f7978b08700 1 -- 172.21.15.27:0/1856092723 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79741a6730 con 0x7f797406bbe0 2024-05-21T03:22:25.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.715+0000 7f7978b08700 1 -- 172.21.15.27:0/1856092723 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f797410ba90 con 0x7f797406bbe0 2024-05-21T03:22:25.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.716+0000 7f79637fe700 1 -- 172.21.15.27:0/1856092723 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f796802a070 con 0x7f797406bbe0 2024-05-21T03:22:25.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.717+0000 7f79637fe700 1 --2- 172.21.15.27:0/1856092723 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f795c0b2900 0x7f795c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:25.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.717+0000 7f797259c700 1 --2- 172.21.15.27:0/1856092723 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f795c0b2900 0x7f795c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:25.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.717+0000 7f79637fe700 1 -- 172.21.15.27:0/1856092723 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f79680e3290 con 0x7f797406bbe0 2024-05-21T03:22:25.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.718+0000 7f797259c700 1 --2- 172.21.15.27:0/1856092723 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f795c0b2900 0x7f795c0b4dc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7964005fd0 tx=0x7f7964005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:25.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.720+0000 7f79637fe700 1 -- 172.21.15.27:0/1856092723 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7968013070 con 0x7f797406bbe0 2024-05-21T03:22:25.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:25 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/1660840902' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-21T03:22:25.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:25 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/1660840902' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-21T03:22:25.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.947+0000 7f7978b08700 1 -- 172.21.15.27:0/1856092723 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f797419af80 con 0x7f797406bbe0 2024-05-21T03:22:25.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.947+0000 7f79637fe700 1 -- 172.21.15.27:0/1856092723 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f797419af80 con 0x7f797406bbe0 2024-05-21T03:22:25.947 INFO:teuthology.orchestra.run.smithi027.stdout:261993005105 2024-05-21T03:22:25.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.949+0000 7f7978b08700 1 -- 172.21.15.27:0/1856092723 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f795c0b2900 msgr2=0x7f795c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:25.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.949+0000 7f7978b08700 1 --2- 172.21.15.27:0/1856092723 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f795c0b2900 0x7f795c0b4dc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7964005fd0 tx=0x7f7964005d90 comp rx=0 tx=0).stop 2024-05-21T03:22:25.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.949+0000 7f7978b08700 1 -- 172.21.15.27:0/1856092723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f797406bbe0 msgr2=0x7f797419fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:25.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.949+0000 7f7978b08700 1 --2- 172.21.15.27:0/1856092723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f797406bbe0 0x7f797419fda0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7968000c00 tx=0x7f79680047b0 comp rx=0 tx=0).stop 2024-05-21T03:22:25.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.949+0000 7f7978b08700 1 -- 172.21.15.27:0/1856092723 shutdown_connections 2024-05-21T03:22:25.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.949+0000 7f7978b08700 1 --2- 172.21.15.27:0/1856092723 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f795c0b2900 0x7f795c0b4dc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:25.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.949+0000 7f7978b08700 1 --2- 172.21.15.27:0/1856092723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79741095b0 0x7f7974199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:25.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.949+0000 7f7978b08700 1 --2- 172.21.15.27:0/1856092723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f797406b220 0x7f797419f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:25.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.949+0000 7f7978b08700 1 --2- 172.21.15.27:0/1856092723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f797406bbe0 0x7f797419fda0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:25.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.949+0000 7f7978b08700 1 -- 172.21.15.27:0/1856092723 >> 172.21.15.27:0/1856092723 conn(0x7f79740793f0 msgr2=0x7f79741011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:25.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.950+0000 7f7978b08700 1 -- 172.21.15.27:0/1856092723 shutdown_connections 2024-05-21T03:22:25.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:25.950+0000 7f7978b08700 1 -- 172.21.15.27:0/1856092723 wait complete. 2024-05-21T03:22:26.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:26 smithi039 ceph-mon[36429]: pgmap v348: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-21T03:22:26.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:26 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/1856092723' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-21T03:22:26.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:26 smithi165 ceph-mon[36269]: pgmap v348: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-21T03:22:26.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:26 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/1856092723' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-21T03:22:26.781 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:26 smithi027 ceph-mon[32179]: pgmap v348: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-21T03:22:26.781 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:26 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/1856092723' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-21T03:22:27.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.193+0000 7f2df8648700 1 -- 172.21.15.27:0/317098931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df0109680 msgr2=0x7f2df0109a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:27.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.193+0000 7f2df8648700 1 --2- 172.21.15.27:0/317098931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df0109680 0x7f2df0109a60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2dec0099c0 tx=0x7f2dec009cd0 comp rx=0 tx=0).stop 2024-05-21T03:22:27.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.193+0000 7f2df8648700 1 -- 172.21.15.27:0/317098931 shutdown_connections 2024-05-21T03:22:27.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.193+0000 7f2df8648700 1 --2- 172.21.15.27:0/317098931 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2df0103fa0 0x7f2df0110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:27.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.193+0000 7f2df8648700 1 --2- 172.21.15.27:0/317098931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df0109680 0x7f2df0109a60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:27.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.193+0000 7f2df8648700 1 --2- 172.21.15.27:0/317098931 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2df01035e0 0x7f2df0103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:27.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.193+0000 7f2df8648700 1 -- 172.21.15.27:0/317098931 >> 172.21.15.27:0/317098931 conn(0x7f2df007a6f0 msgr2=0x7f2df007ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:27.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.193+0000 7f2df8648700 1 -- 172.21.15.27:0/317098931 shutdown_connections 2024-05-21T03:22:27.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.193+0000 7f2df8648700 1 -- 172.21.15.27:0/317098931 wait complete. 2024-05-21T03:22:27.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.194+0000 7f2df8648700 1 Processor -- start 2024-05-21T03:22:27.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.194+0000 7f2df8648700 1 -- start start 2024-05-21T03:22:27.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.194+0000 7f2df8648700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2df01035e0 0x7f2df019bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:27.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.194+0000 7f2df8648700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2df0103fa0 0x7f2df019c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:27.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.194+0000 7f2df8648700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df0109680 0x7f2df01a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:27.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.194+0000 7f2df8648700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2df01137a0 con 0x7f2df0109680 2024-05-21T03:22:27.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.194+0000 7f2df8648700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2df0113620 con 0x7f2df0103fa0 2024-05-21T03:22:27.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.194+0000 7f2df8648700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2df0113920 con 0x7f2df01035e0 2024-05-21T03:22:27.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.194+0000 7f2df63e4700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2df01035e0 0x7f2df019bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:27.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.194+0000 7f2df6be5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df0109680 0x7f2df01a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:27.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.194+0000 7f2df5be3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2df0103fa0 0x7f2df019c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:27.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.194+0000 7f2df63e4700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2df01035e0 0x7f2df019bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.27:60000/0 (socket says 172.21.15.27:60000) 2024-05-21T03:22:27.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.194+0000 7f2df63e4700 1 -- 172.21.15.27:0/221376410 learned_addr learned my addr 172.21.15.27:0/221376410 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:27.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.195+0000 7f2df6be5700 1 -- 172.21.15.27:0/221376410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2df01035e0 msgr2=0x7f2df019bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:27.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.195+0000 7f2df6be5700 1 --2- 172.21.15.27:0/221376410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2df01035e0 0x7f2df019bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:27.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.195+0000 7f2df6be5700 1 -- 172.21.15.27:0/221376410 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2df0103fa0 msgr2=0x7f2df019c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:27.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.195+0000 7f2df6be5700 1 --2- 172.21.15.27:0/221376410 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2df0103fa0 0x7f2df019c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:27.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.195+0000 7f2df6be5700 1 -- 172.21.15.27:0/221376410 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df01a0dc0 con 0x7f2df0109680 2024-05-21T03:22:27.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.195+0000 7f2df5be3700 1 --2- 172.21.15.27:0/221376410 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2df0103fa0 0x7f2df019c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:22:27.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.195+0000 7f2df63e4700 1 --2- 172.21.15.27:0/221376410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2df01035e0 0x7f2df019bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:22:27.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.195+0000 7f2df6be5700 1 --2- 172.21.15.27:0/221376410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df0109680 0x7f2df01a0680 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2de8007f10 tx=0x7f2de800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:27.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.195+0000 7f2de37fe700 1 -- 172.21.15.27:0/221376410 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8004db0 con 0x7f2df0109680 2024-05-21T03:22:27.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.195+0000 7f2de37fe700 1 -- 172.21.15.27:0/221376410 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2de8009e20 con 0x7f2df0109680 2024-05-21T03:22:27.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.195+0000 7f2df8648700 1 -- 172.21.15.27:0/221376410 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df01a10b0 con 0x7f2df0109680 2024-05-21T03:22:27.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.196+0000 7f2de37fe700 1 -- 172.21.15.27:0/221376410 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8015400 con 0x7f2df0109680 2024-05-21T03:22:27.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.196+0000 7f2df8648700 1 -- 172.21.15.27:0/221376410 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df01a8930 con 0x7f2df0109680 2024-05-21T03:22:27.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.197+0000 7f2df8648700 1 -- 172.21.15.27:0/221376410 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2df0104cf0 con 0x7f2df0109680 2024-05-21T03:22:27.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.198+0000 7f2de37fe700 1 -- 172.21.15.27:0/221376410 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2de800db20 con 0x7f2df0109680 2024-05-21T03:22:27.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.199+0000 7f2de37fe700 1 --2- 172.21.15.27:0/221376410 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f2ddc0b2750 0x7f2ddc0b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:27.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.199+0000 7f2df63e4700 1 --2- 172.21.15.27:0/221376410 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f2ddc0b2750 0x7f2ddc0b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:27.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.199+0000 7f2de37fe700 1 -- 172.21.15.27:0/221376410 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f2de80e1b20 con 0x7f2df0109680 2024-05-21T03:22:27.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.200+0000 7f2df63e4700 1 --2- 172.21.15.27:0/221376410 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f2ddc0b2750 0x7f2ddc0b4c10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2dec003800 tx=0x7f2dec005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:27.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.204+0000 7f2de37fe700 1 -- 172.21.15.27:0/221376410 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2de8010070 con 0x7f2df0109680 2024-05-21T03:22:27.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.437+0000 7f2df8648700 1 -- 172.21.15.27:0/221376410 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f2df019d110 con 0x7f2df0109680 2024-05-21T03:22:27.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.438+0000 7f2de37fe700 1 -- 172.21.15.27:0/221376410 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f2de80a9820 con 0x7f2df0109680 2024-05-21T03:22:27.438 INFO:teuthology.orchestra.run.smithi027.stdout:309237645350 2024-05-21T03:22:27.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.440+0000 7f2df8648700 1 -- 172.21.15.27:0/221376410 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f2ddc0b2750 msgr2=0x7f2ddc0b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:27.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.440+0000 7f2df8648700 1 --2- 172.21.15.27:0/221376410 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f2ddc0b2750 0x7f2ddc0b4c10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2dec003800 tx=0x7f2dec005fb0 comp rx=0 tx=0).stop 2024-05-21T03:22:27.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.440+0000 7f2df8648700 1 -- 172.21.15.27:0/221376410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df0109680 msgr2=0x7f2df01a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:27.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.440+0000 7f2df8648700 1 --2- 172.21.15.27:0/221376410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df0109680 0x7f2df01a0680 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2de8007f10 tx=0x7f2de800d3b0 comp rx=0 tx=0).stop 2024-05-21T03:22:27.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.440+0000 7f2df8648700 1 -- 172.21.15.27:0/221376410 shutdown_connections 2024-05-21T03:22:27.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.440+0000 7f2df8648700 1 --2- 172.21.15.27:0/221376410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df0109680 0x7f2df01a0680 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:27.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.440+0000 7f2df8648700 1 --2- 172.21.15.27:0/221376410 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f2ddc0b2750 0x7f2ddc0b4c10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:27.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.440+0000 7f2df8648700 1 --2- 172.21.15.27:0/221376410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2df01035e0 0x7f2df019bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:27.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.440+0000 7f2df8648700 1 --2- 172.21.15.27:0/221376410 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2df0103fa0 0x7f2df019c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:27.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.440+0000 7f2df8648700 1 -- 172.21.15.27:0/221376410 >> 172.21.15.27:0/221376410 conn(0x7f2df007a6f0 msgr2=0x7f2df010eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:27.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.440+0000 7f2df8648700 1 -- 172.21.15.27:0/221376410 shutdown_connections 2024-05-21T03:22:27.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.440+0000 7f2df8648700 1 -- 172.21.15.27:0/221376410 wait complete. 2024-05-21T03:22:27.630 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:27 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/4102691400' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-21T03:22:27.630 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:27 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/221376410' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-21T03:22:27.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:27 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/4102691400' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-21T03:22:27.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:27 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/221376410' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-21T03:22:27.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:27 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/4102691400' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-21T03:22:27.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:27 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/221376410' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-21T03:22:28.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.479+0000 7fce2d80f700 1 -- 172.21.15.27:0/1186196168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce281095b0 msgr2=0x7fce28109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:28.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.479+0000 7fce2d80f700 1 --2- 172.21.15.27:0/1186196168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce281095b0 0x7fce28109990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fce18009990 tx=0x7fce18009ca0 comp rx=0 tx=0).stop 2024-05-21T03:22:28.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.479+0000 7fce2d80f700 1 -- 172.21.15.27:0/1186196168 shutdown_connections 2024-05-21T03:22:28.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.479+0000 7fce2d80f700 1 --2- 172.21.15.27:0/1186196168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce2806bbe0 0x7fce281103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:28.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.479+0000 7fce2d80f700 1 --2- 172.21.15.27:0/1186196168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce281095b0 0x7fce28109990 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:28.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.479+0000 7fce2d80f700 1 --2- 172.21.15.27:0/1186196168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce2806b220 0x7fce2806b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:28.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.479+0000 7fce2d80f700 1 -- 172.21.15.27:0/1186196168 >> 172.21.15.27:0/1186196168 conn(0x7fce280793f0 msgr2=0x7fce28079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:28.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.479+0000 7fce2d80f700 1 -- 172.21.15.27:0/1186196168 shutdown_connections 2024-05-21T03:22:28.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.479+0000 7fce2d80f700 1 -- 172.21.15.27:0/1186196168 wait complete. 2024-05-21T03:22:28.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.480+0000 7fce2d80f700 1 Processor -- start 2024-05-21T03:22:28.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.480+0000 7fce2d80f700 1 -- start start 2024-05-21T03:22:28.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.480+0000 7fce2d80f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce2806b220 0x7fce2819f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:28.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.480+0000 7fce2d80f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce2806bbe0 0x7fce2819fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:28.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.480+0000 7fce2d80f700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce281095b0 0x7fce28199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:28.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.480+0000 7fce2d80f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fce281136c0 con 0x7fce2806b220 2024-05-21T03:22:28.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.480+0000 7fce2d80f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fce28113540 con 0x7fce2806bbe0 2024-05-21T03:22:28.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.480+0000 7fce2d80f700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fce28113840 con 0x7fce281095b0 2024-05-21T03:22:28.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.480+0000 7fce26ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce2806b220 0x7fce2819f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:28.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.481+0000 7fce26ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce2806b220 0x7fce2819f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36564/0 (socket says 172.21.15.27:36564) 2024-05-21T03:22:28.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.481+0000 7fce26ffd700 1 -- 172.21.15.27:0/2045582209 learned_addr learned my addr 172.21.15.27:0/2045582209 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:28.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.481+0000 7fce267fc700 1 --2- 172.21.15.27:0/2045582209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce2806bbe0 0x7fce2819fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:28.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.481+0000 7fce277fe700 1 --2- 172.21.15.27:0/2045582209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce281095b0 0x7fce28199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:28.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.481+0000 7fce26ffd700 1 -- 172.21.15.27:0/2045582209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce281095b0 msgr2=0x7fce28199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:28.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.481+0000 7fce26ffd700 1 --2- 172.21.15.27:0/2045582209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce281095b0 0x7fce28199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:28.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.481+0000 7fce26ffd700 1 -- 172.21.15.27:0/2045582209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce2806bbe0 msgr2=0x7fce2819fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:28.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.481+0000 7fce26ffd700 1 --2- 172.21.15.27:0/2045582209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce2806bbe0 0x7fce2819fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:28.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.481+0000 7fce26ffd700 1 -- 172.21.15.27:0/2045582209 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce2819a160 con 0x7fce2806b220 2024-05-21T03:22:28.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.481+0000 7fce267fc700 1 --2- 172.21.15.27:0/2045582209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce2806bbe0 0x7fce2819fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:22:28.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.481+0000 7fce26ffd700 1 --2- 172.21.15.27:0/2045582209 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce2806b220 0x7fce2819f840 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fce18000c00 tx=0x7fce180047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:28.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.481+0000 7fce2c80d700 1 -- 172.21.15.27:0/2045582209 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce18027070 con 0x7fce2806b220 2024-05-21T03:22:28.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.481+0000 7fce2c80d700 1 -- 172.21.15.27:0/2045582209 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce18003fa0 con 0x7fce2806b220 2024-05-21T03:22:28.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.481+0000 7fce2d80f700 1 -- 172.21.15.27:0/2045582209 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce2819a3f0 con 0x7fce2806b220 2024-05-21T03:22:28.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.481+0000 7fce2c80d700 1 -- 172.21.15.27:0/2045582209 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce18022db0 con 0x7fce2806b220 2024-05-21T03:22:28.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.481+0000 7fce2d80f700 1 -- 172.21.15.27:0/2045582209 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce281a6710 con 0x7fce2806b220 2024-05-21T03:22:28.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.482+0000 7fce2d80f700 1 -- 172.21.15.27:0/2045582209 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce2810ba90 con 0x7fce2806b220 2024-05-21T03:22:28.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.485+0000 7fce2c80d700 1 -- 172.21.15.27:0/2045582209 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fce1802a070 con 0x7fce2806b220 2024-05-21T03:22:28.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.486+0000 7fce2c80d700 1 --2- 172.21.15.27:0/2045582209 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fce140b2880 0x7fce140b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:28.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.486+0000 7fce267fc700 1 --2- 172.21.15.27:0/2045582209 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fce140b2880 0x7fce140b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:28.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.486+0000 7fce2c80d700 1 -- 172.21.15.27:0/2045582209 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fce180e32f0 con 0x7fce2806b220 2024-05-21T03:22:28.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.486+0000 7fce267fc700 1 --2- 172.21.15.27:0/2045582209 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fce140b2880 0x7fce140b4d40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fce10005fd0 tx=0x7fce10005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:28.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.489+0000 7fce2c80d700 1 -- 172.21.15.27:0/2045582209 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fce18013070 con 0x7fce2806b220 2024-05-21T03:22:28.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:28 smithi027 ceph-mon[32179]: pgmap v349: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-21T03:22:28.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:28 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3490047978' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-21T03:22:28.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.711+0000 7fce2d80f700 1 -- 172.21.15.27:0/2045582209 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fce2819af30 con 0x7fce2806b220 2024-05-21T03:22:28.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.711+0000 7fce2c80d700 1 -- 172.21.15.27:0/2045582209 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fce180ab080 con 0x7fce2806b220 2024-05-21T03:22:28.711 INFO:teuthology.orchestra.run.smithi027.stdout:210453397564 2024-05-21T03:22:28.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.713+0000 7fce2d80f700 1 -- 172.21.15.27:0/2045582209 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fce140b2880 msgr2=0x7fce140b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:28.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.713+0000 7fce2d80f700 1 --2- 172.21.15.27:0/2045582209 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fce140b2880 0x7fce140b4d40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fce10005fd0 tx=0x7fce10005e30 comp rx=0 tx=0).stop 2024-05-21T03:22:28.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.713+0000 7fce2d80f700 1 -- 172.21.15.27:0/2045582209 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce2806b220 msgr2=0x7fce2819f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:28.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.713+0000 7fce2d80f700 1 --2- 172.21.15.27:0/2045582209 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce2806b220 0x7fce2819f840 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fce18000c00 tx=0x7fce180047b0 comp rx=0 tx=0).stop 2024-05-21T03:22:28.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.713+0000 7fce2d80f700 1 -- 172.21.15.27:0/2045582209 shutdown_connections 2024-05-21T03:22:28.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.713+0000 7fce2d80f700 1 --2- 172.21.15.27:0/2045582209 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fce140b2880 0x7fce140b4d40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:28.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.713+0000 7fce2d80f700 1 --2- 172.21.15.27:0/2045582209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce281095b0 0x7fce28199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:28.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.713+0000 7fce2d80f700 1 --2- 172.21.15.27:0/2045582209 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce2806b220 0x7fce2819f840 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:28.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.713+0000 7fce2d80f700 1 --2- 172.21.15.27:0/2045582209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce2806bbe0 0x7fce2819fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:28.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.713+0000 7fce2d80f700 1 -- 172.21.15.27:0/2045582209 >> 172.21.15.27:0/2045582209 conn(0x7fce280793f0 msgr2=0x7fce28100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:28.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.714+0000 7fce2d80f700 1 -- 172.21.15.27:0/2045582209 shutdown_connections 2024-05-21T03:22:28.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.714+0000 7fce2d80f700 1 -- 172.21.15.27:0/2045582209 wait complete. 2024-05-21T03:22:28.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:28 smithi039 ceph-mon[36429]: pgmap v349: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-21T03:22:28.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:28 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/3490047978' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-21T03:22:28.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:28 smithi165 ceph-mon[36269]: pgmap v349: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-21T03:22:28.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:28 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/3490047978' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-21T03:22:29.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.039+0000 7fec98848700 1 -- 172.21.15.27:0/3537736874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec94101f00 msgr2=0x7fec94102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:29.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.039+0000 7fec98848700 1 --2- 172.21.15.27:0/3537736874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec94101f00 0x7fec94102380 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fec88009990 tx=0x7fec88009ca0 comp rx=0 tx=0).stop 2024-05-21T03:22:29.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.039+0000 7fec98848700 1 -- 172.21.15.27:0/3537736874 shutdown_connections 2024-05-21T03:22:29.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.039+0000 7fec98848700 1 --2- 172.21.15.27:0/3537736874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec941028c0 0x7fec94110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:29.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.039+0000 7fec98848700 1 --2- 172.21.15.27:0/3537736874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec94108620 0x7fec94108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:29.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.039+0000 7fec98848700 1 --2- 172.21.15.27:0/3537736874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec94101f00 0x7fec94102380 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:29.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.039+0000 7fec98848700 1 -- 172.21.15.27:0/3537736874 >> 172.21.15.27:0/3537736874 conn(0x7fec94077780 msgr2=0x7fec94077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:29.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.039+0000 7fec98848700 1 -- 172.21.15.27:0/3537736874 shutdown_connections 2024-05-21T03:22:29.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.039+0000 7fec98848700 1 -- 172.21.15.27:0/3537736874 wait complete. 2024-05-21T03:22:29.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.040+0000 7fec98848700 1 Processor -- start 2024-05-21T03:22:29.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.040+0000 7fec98848700 1 -- start start 2024-05-21T03:22:29.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.040+0000 7fec98848700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec94101f00 0x7fec9419f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:29.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.040+0000 7fec98848700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec941028c0 0x7fec9419fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:29.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.040+0000 7fec98848700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec94108620 0x7fec94199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:29.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.040+0000 7fec98848700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fec941137b0 con 0x7fec94101f00 2024-05-21T03:22:29.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.040+0000 7fec98848700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fec94113630 con 0x7fec941028c0 2024-05-21T03:22:29.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.040+0000 7fec98848700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fec94113930 con 0x7fec94108620 2024-05-21T03:22:29.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.040+0000 7fec9259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec94101f00 0x7fec9419f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:29.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.040+0000 7fec9259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec94101f00 0x7fec9419f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36592/0 (socket says 172.21.15.27:36592) 2024-05-21T03:22:29.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.040+0000 7fec9259c700 1 -- 172.21.15.27:0/972285685 learned_addr learned my addr 172.21.15.27:0/972285685 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:29.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.040+0000 7fec92d9d700 1 --2- 172.21.15.27:0/972285685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec94108620 0x7fec94199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:29.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.040+0000 7fec91d9b700 1 --2- 172.21.15.27:0/972285685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec941028c0 0x7fec9419fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:29.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.041+0000 7fec9259c700 1 -- 172.21.15.27:0/972285685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec94108620 msgr2=0x7fec94199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:29.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.041+0000 7fec9259c700 1 --2- 172.21.15.27:0/972285685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec94108620 0x7fec94199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:29.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.041+0000 7fec9259c700 1 -- 172.21.15.27:0/972285685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec941028c0 msgr2=0x7fec9419fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:29.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.041+0000 7fec9259c700 1 --2- 172.21.15.27:0/972285685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec941028c0 0x7fec9419fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:29.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.041+0000 7fec9259c700 1 -- 172.21.15.27:0/972285685 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9419a250 con 0x7fec94101f00 2024-05-21T03:22:29.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.041+0000 7fec9259c700 1 --2- 172.21.15.27:0/972285685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec94101f00 0x7fec9419f930 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fec8400e950 tx=0x7fec8400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:29.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.041+0000 7fec837fe700 1 -- 172.21.15.27:0/972285685 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8400eea0 con 0x7fec94101f00 2024-05-21T03:22:29.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.041+0000 7fec837fe700 1 -- 172.21.15.27:0/972285685 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec840044e0 con 0x7fec94101f00 2024-05-21T03:22:29.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.041+0000 7fec98848700 1 -- 172.21.15.27:0/972285685 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec9419a540 con 0x7fec94101f00 2024-05-21T03:22:29.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.041+0000 7fec837fe700 1 -- 172.21.15.27:0/972285685 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec84003e00 con 0x7fec94101f00 2024-05-21T03:22:29.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.041+0000 7fec98848700 1 -- 172.21.15.27:0/972285685 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec941a6800 con 0x7fec94101f00 2024-05-21T03:22:29.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.042+0000 7fec98848700 1 -- 172.21.15.27:0/972285685 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec9406a230 con 0x7fec94101f00 2024-05-21T03:22:29.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.043+0000 7fec837fe700 1 -- 172.21.15.27:0/972285685 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fec84003fa0 con 0x7fec94101f00 2024-05-21T03:22:29.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.044+0000 7fec837fe700 1 --2- 172.21.15.27:0/972285685 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fec7c0b2880 0x7fec7c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:29.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.044+0000 7fec91d9b700 1 --2- 172.21.15.27:0/972285685 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fec7c0b2880 0x7fec7c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:29.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.044+0000 7fec837fe700 1 -- 172.21.15.27:0/972285685 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fec840e2590 con 0x7fec94101f00 2024-05-21T03:22:29.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.045+0000 7fec91d9b700 1 --2- 172.21.15.27:0/972285685 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fec7c0b2880 0x7fec7c0b4d40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fec88006010 tx=0x7fec8801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:29.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.048+0000 7fec837fe700 1 -- 172.21.15.27:0/972285685 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fec840aa290 con 0x7fec94101f00 2024-05-21T03:22:29.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.275+0000 7fec98848700 1 -- 172.21.15.27:0/972285685 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fec9402e560 con 0x7fec94101f00 2024-05-21T03:22:29.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.275+0000 7fec837fe700 1 -- 172.21.15.27:0/972285685 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fec840a9b90 con 0x7fec94101f00 2024-05-21T03:22:29.276 INFO:teuthology.orchestra.run.smithi027.stdout:34359738463 2024-05-21T03:22:29.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.278+0000 7fec98848700 1 -- 172.21.15.27:0/972285685 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fec7c0b2880 msgr2=0x7fec7c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:29.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.278+0000 7fec98848700 1 --2- 172.21.15.27:0/972285685 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fec7c0b2880 0x7fec7c0b4d40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fec88006010 tx=0x7fec8801a040 comp rx=0 tx=0).stop 2024-05-21T03:22:29.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.278+0000 7fec98848700 1 -- 172.21.15.27:0/972285685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec94101f00 msgr2=0x7fec9419f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:29.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.278+0000 7fec98848700 1 --2- 172.21.15.27:0/972285685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec94101f00 0x7fec9419f930 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fec8400e950 tx=0x7fec8400ec60 comp rx=0 tx=0).stop 2024-05-21T03:22:29.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.278+0000 7fec98848700 1 -- 172.21.15.27:0/972285685 shutdown_connections 2024-05-21T03:22:29.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.278+0000 7fec98848700 1 --2- 172.21.15.27:0/972285685 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fec7c0b2880 0x7fec7c0b4d40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:29.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.278+0000 7fec98848700 1 --2- 172.21.15.27:0/972285685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec94108620 0x7fec94199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:29.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.278+0000 7fec98848700 1 --2- 172.21.15.27:0/972285685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec94101f00 0x7fec9419f930 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:29.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.278+0000 7fec98848700 1 --2- 172.21.15.27:0/972285685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec941028c0 0x7fec9419fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:29.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.278+0000 7fec98848700 1 -- 172.21.15.27:0/972285685 >> 172.21.15.27:0/972285685 conn(0x7fec94077780 msgr2=0x7fec94100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:29.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.278+0000 7fec98848700 1 -- 172.21.15.27:0/972285685 shutdown_connections 2024-05-21T03:22:29.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:29.278+0000 7fec98848700 1 -- 172.21.15.27:0/972285685 wait complete. 2024-05-21T03:22:29.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.100+0000 7f67bcc58700 1 -- 172.21.15.27:0/2853347385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67b8109680 msgr2=0x7f67b8109a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:29.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.100+0000 7f67bcc58700 1 --2- 172.21.15.27:0/2853347385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67b8109680 0x7f67b8109a60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f67ac0099c0 tx=0x7f67ac009cd0 comp rx=0 tx=0).stop 2024-05-21T03:22:29.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.100+0000 7f67bcc58700 1 -- 172.21.15.27:0/2853347385 shutdown_connections 2024-05-21T03:22:29.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.100+0000 7f67bcc58700 1 --2- 172.21.15.27:0/2853347385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67b8103fa0 0x7f67b8110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:29.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.100+0000 7f67bcc58700 1 --2- 172.21.15.27:0/2853347385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67b8109680 0x7f67b8109a60 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:29.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.100+0000 7f67bcc58700 1 --2- 172.21.15.27:0/2853347385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67b81035e0 0x7f67b8103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:29.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.100+0000 7f67bcc58700 1 -- 172.21.15.27:0/2853347385 >> 172.21.15.27:0/2853347385 conn(0x7f67b807a6f0 msgr2=0x7f67b807ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:29.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.100+0000 7f67bcc58700 1 -- 172.21.15.27:0/2853347385 shutdown_connections 2024-05-21T03:22:29.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.100+0000 7f67bcc58700 1 -- 172.21.15.27:0/2853347385 wait complete. 2024-05-21T03:22:29.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.101+0000 7f67bcc58700 1 Processor -- start 2024-05-21T03:22:29.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.101+0000 7f67bcc58700 1 -- start start 2024-05-21T03:22:29.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.101+0000 7f67bcc58700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67b81035e0 0x7f67b8075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:29.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.101+0000 7f67bcc58700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67b8103fa0 0x7f67b8075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:29.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.101+0000 7f67bcc58700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67b8109680 0x7f67b80762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:29.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.101+0000 7f67bcc58700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f67b8113860 con 0x7f67b81035e0 2024-05-21T03:22:29.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.101+0000 7f67bcc58700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f67b81136e0 con 0x7f67b8103fa0 2024-05-21T03:22:29.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.101+0000 7f67bcc58700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f67b81139e0 con 0x7f67b8109680 2024-05-21T03:22:29.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.101+0000 7f67b659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67b81035e0 0x7f67b8075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:29.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.101+0000 7f67b659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67b81035e0 0x7f67b8075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36502/0 (socket says 172.21.15.27:36502) 2024-05-21T03:22:29.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.101+0000 7f67b659c700 1 -- 172.21.15.27:0/4102691400 learned_addr learned my addr 172.21.15.27:0/4102691400 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:29.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.101+0000 7f67b5d9b700 1 --2- 172.21.15.27:0/4102691400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67b8103fa0 0x7f67b8075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:29.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.101+0000 7f67b6d9d700 1 --2- 172.21.15.27:0/4102691400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67b8109680 0x7f67b80762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:29.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.101+0000 7f67b659c700 1 -- 172.21.15.27:0/4102691400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67b8109680 msgr2=0x7f67b80762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:29.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.101+0000 7f67b659c700 1 --2- 172.21.15.27:0/4102691400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67b8109680 0x7f67b80762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:29.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.101+0000 7f67b659c700 1 -- 172.21.15.27:0/4102691400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67b8103fa0 msgr2=0x7f67b8075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:29.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.101+0000 7f67b659c700 1 --2- 172.21.15.27:0/4102691400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67b8103fa0 0x7f67b8075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:29.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.101+0000 7f67b659c700 1 -- 172.21.15.27:0/4102691400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b81a81d0 con 0x7f67b81035e0 2024-05-21T03:22:29.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.101+0000 7f67b5d9b700 1 --2- 172.21.15.27:0/4102691400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67b8103fa0 0x7f67b8075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:22:29.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.102+0000 7f67b659c700 1 --2- 172.21.15.27:0/4102691400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67b81035e0 0x7f67b8075840 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00b580 tx=0x7f67ac0037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:29.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.102+0000 7f67a77fe700 1 -- 172.21.15.27:0/4102691400 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac027070 con 0x7f67b81035e0 2024-05-21T03:22:29.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.102+0000 7f67a77fe700 1 -- 172.21.15.27:0/4102691400 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67ac004110 con 0x7f67b81035e0 2024-05-21T03:22:29.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.102+0000 7f67bcc58700 1 -- 172.21.15.27:0/4102691400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b81a8460 con 0x7f67b81035e0 2024-05-21T03:22:29.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.102+0000 7f67a77fe700 1 -- 172.21.15.27:0/4102691400 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac00f460 con 0x7f67b81035e0 2024-05-21T03:22:29.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.102+0000 7f67bcc58700 1 -- 172.21.15.27:0/4102691400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b81a88b0 con 0x7f67b81035e0 2024-05-21T03:22:29.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.103+0000 7f67bcc58700 1 -- 172.21.15.27:0/4102691400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67b810bb10 con 0x7f67b81035e0 2024-05-21T03:22:29.351 INFO:teuthology.orchestra.run.smithi027.stdout:111669149775 2024-05-21T03:22:29.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.104+0000 7f67a77fe700 1 -- 172.21.15.27:0/4102691400 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f67ac02a070 con 0x7f67b81035e0 2024-05-21T03:22:29.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.105+0000 7f67a77fe700 1 --2- 172.21.15.27:0/4102691400 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f67a00b2880 0x7f67a00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:29.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.105+0000 7f67b5d9b700 1 --2- 172.21.15.27:0/4102691400 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f67a00b2880 0x7f67a00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:29.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.105+0000 7f67a77fe700 1 -- 172.21.15.27:0/4102691400 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f67ac0e3270 con 0x7f67b81035e0 2024-05-21T03:22:29.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.106+0000 7f67b5d9b700 1 --2- 172.21.15.27:0/4102691400 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f67a00b2880 0x7f67a00b4d40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f67b807acf0 tx=0x7f67a8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:29.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.110+0000 7f67a77fe700 1 -- 172.21.15.27:0/4102691400 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f67ac05fdf0 con 0x7f67b81035e0 2024-05-21T03:22:29.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.336+0000 7f67bcc58700 1 -- 172.21.15.27:0/4102691400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f67b810b4f0 con 0x7f67b81035e0 2024-05-21T03:22:29.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.337+0000 7f67a77fe700 1 -- 172.21.15.27:0/4102691400 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f67ac013070 con 0x7f67b81035e0 2024-05-21T03:22:29.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.339+0000 7f67bcc58700 1 -- 172.21.15.27:0/4102691400 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f67a00b2880 msgr2=0x7f67a00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:29.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.339+0000 7f67bcc58700 1 --2- 172.21.15.27:0/4102691400 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f67a00b2880 0x7f67a00b4d40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f67b807acf0 tx=0x7f67a8008040 comp rx=0 tx=0).stop 2024-05-21T03:22:29.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.339+0000 7f67bcc58700 1 -- 172.21.15.27:0/4102691400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67b81035e0 msgr2=0x7f67b8075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:29.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.339+0000 7f67bcc58700 1 --2- 172.21.15.27:0/4102691400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67b81035e0 0x7f67b8075840 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00b580 tx=0x7f67ac0037c0 comp rx=0 tx=0).stop 2024-05-21T03:22:29.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.339+0000 7f67bcc58700 1 -- 172.21.15.27:0/4102691400 shutdown_connections 2024-05-21T03:22:29.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.339+0000 7f67bcc58700 1 --2- 172.21.15.27:0/4102691400 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f67a00b2880 0x7f67a00b4d40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:29.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.339+0000 7f67bcc58700 1 --2- 172.21.15.27:0/4102691400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67b8109680 0x7f67b80762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:29.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.339+0000 7f67bcc58700 1 --2- 172.21.15.27:0/4102691400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67b81035e0 0x7f67b8075840 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:29.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.339+0000 7f67bcc58700 1 --2- 172.21.15.27:0/4102691400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67b8103fa0 0x7f67b8075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:29.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.339+0000 7f67bcc58700 1 -- 172.21.15.27:0/4102691400 >> 172.21.15.27:0/4102691400 conn(0x7f67b807a6f0 msgr2=0x7f67b8102240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:29.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.339+0000 7f67bcc58700 1 -- 172.21.15.27:0/4102691400 shutdown_connections 2024-05-21T03:22:29.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:27.339+0000 7f67bcc58700 1 -- 172.21.15.27:0/4102691400 wait complete. 2024-05-21T03:22:29.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:29 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2045582209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-21T03:22:29.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:29 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/972285685' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-21T03:22:29.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:29 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/2045582209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-21T03:22:29.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:29 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/972285685' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-21T03:22:29.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:29 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/2045582209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-21T03:22:29.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:29 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/972285685' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-21T03:22:29.985 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149770 got 111669149775 for osd.3 2024-05-21T03:22:29.985 DEBUG:teuthology.parallel:result is None 2024-05-21T03:22:30.614 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:30 smithi027 ceph-mon[32179]: pgmap v350: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-21T03:22:30.646 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397559 got 210453397564 for osd.7 2024-05-21T03:22:30.646 DEBUG:teuthology.parallel:result is None 2024-05-21T03:22:30.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:30 smithi039 ceph-mon[36429]: pgmap v350: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-21T03:22:30.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:30 smithi165 ceph-mon[36269]: pgmap v350: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-21T03:22:31.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.165+0000 7fb33a677700 1 -- 172.21.15.27:0/1001336630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3341030e0 msgr2=0x7fb33410f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:31.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.165+0000 7fb33a677700 1 --2- 172.21.15.27:0/1001336630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3341030e0 0x7fb33410f320 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00b0a0 tx=0x7fb32c00b3b0 comp rx=0 tx=0).stop 2024-05-21T03:22:31.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.166+0000 7fb33a677700 1 -- 172.21.15.27:0/1001336630 shutdown_connections 2024-05-21T03:22:31.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.166+0000 7fb33a677700 1 --2- 172.21.15.27:0/1001336630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3341030e0 0x7fb33410f320 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:31.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.166+0000 7fb33a677700 1 --2- 172.21.15.27:0/1001336630 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb334107460 0x7fb334107840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:31.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.166+0000 7fb33a677700 1 --2- 172.21.15.27:0/1001336630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb334102720 0x7fb334102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:31.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.166+0000 7fb33a677700 1 -- 172.21.15.27:0/1001336630 >> 172.21.15.27:0/1001336630 conn(0x7fb334079300 msgr2=0x7fb334079710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:31.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.166+0000 7fb33a677700 1 -- 172.21.15.27:0/1001336630 shutdown_connections 2024-05-21T03:22:31.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.166+0000 7fb33a677700 1 -- 172.21.15.27:0/1001336630 wait complete. 2024-05-21T03:22:31.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.166+0000 7fb33a677700 1 Processor -- start 2024-05-21T03:22:31.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.166+0000 7fb33a677700 1 -- start start 2024-05-21T03:22:31.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.167+0000 7fb33a677700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb334102720 0x7fb33419e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:31.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.167+0000 7fb33a677700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3341030e0 0x7fb33419ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:31.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.167+0000 7fb33a677700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb334107460 0x7fb3341988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:31.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.167+0000 7fb33a677700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb3341127a0 con 0x7fb3341030e0 2024-05-21T03:22:31.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.167+0000 7fb33a677700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb334112620 con 0x7fb334107460 2024-05-21T03:22:31.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.167+0000 7fb33a677700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb334112920 con 0x7fb334102720 2024-05-21T03:22:31.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.167+0000 7fb3337fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3341030e0 0x7fb33419ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:31.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.167+0000 7fb3337fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3341030e0 0x7fb33419ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36538/0 (socket says 172.21.15.27:36538) 2024-05-21T03:22:31.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.167+0000 7fb3337fe700 1 -- 172.21.15.27:0/3490047978 learned_addr learned my addr 172.21.15.27:0/3490047978 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:31.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.167+0000 7fb333fff700 1 --2- 172.21.15.27:0/3490047978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb334102720 0x7fb33419e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:31.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.167+0000 7fb3337fe700 1 -- 172.21.15.27:0/3490047978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb334102720 msgr2=0x7fb33419e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:31.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.167+0000 7fb3337fe700 1 --2- 172.21.15.27:0/3490047978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb334102720 0x7fb33419e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:31.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.167+0000 7fb3337fe700 1 -- 172.21.15.27:0/3490047978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb334107460 msgr2=0x7fb3341988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:31.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.167+0000 7fb3337fe700 1 --2- 172.21.15.27:0/3490047978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb334107460 0x7fb3341988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:31.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.167+0000 7fb3337fe700 1 -- 172.21.15.27:0/3490047978 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb334199040 con 0x7fb3341030e0 2024-05-21T03:22:31.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.167+0000 7fb333fff700 1 --2- 172.21.15.27:0/3490047978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb334102720 0x7fb33419e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:22:31.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.168+0000 7fb3337fe700 1 --2- 172.21.15.27:0/3490047978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3341030e0 0x7fb33419ec60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb32800c9b0 tx=0x7fb32800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:31.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.168+0000 7fb3317fa700 1 -- 172.21.15.27:0/3490047978 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb328007b00 con 0x7fb3341030e0 2024-05-21T03:22:31.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.168+0000 7fb3317fa700 1 -- 172.21.15.27:0/3490047978 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb328007ca0 con 0x7fb3341030e0 2024-05-21T03:22:31.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.168+0000 7fb33a677700 1 -- 172.21.15.27:0/3490047978 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb334199330 con 0x7fb3341030e0 2024-05-21T03:22:31.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.168+0000 7fb3317fa700 1 -- 172.21.15.27:0/3490047978 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb328018730 con 0x7fb3341030e0 2024-05-21T03:22:31.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.168+0000 7fb33a677700 1 -- 172.21.15.27:0/3490047978 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3341a55f0 con 0x7fb3341030e0 2024-05-21T03:22:31.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.169+0000 7fb33a677700 1 -- 172.21.15.27:0/3490047978 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb33406a230 con 0x7fb3341030e0 2024-05-21T03:22:31.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.170+0000 7fb3317fa700 1 -- 172.21.15.27:0/3490047978 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb32800f450 con 0x7fb3341030e0 2024-05-21T03:22:31.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.171+0000 7fb3317fa700 1 --2- 172.21.15.27:0/3490047978 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb31c0b2900 0x7fb31c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:31.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.171+0000 7fb333fff700 1 --2- 172.21.15.27:0/3490047978 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb31c0b2900 0x7fb31c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:31.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.171+0000 7fb3317fa700 1 -- 172.21.15.27:0/3490047978 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fb3280e2540 con 0x7fb3341030e0 2024-05-21T03:22:31.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.171+0000 7fb333fff700 1 --2- 172.21.15.27:0/3490047978 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb31c0b2900 0x7fb31c0b4dc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb334199a20 tx=0x7fb32400b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:31.582 INFO:teuthology.orchestra.run.smithi027.stdout:283467841579 2024-05-21T03:22:31.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.174+0000 7fb3317fa700 1 -- 172.21.15.27:0/3490047978 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb328014030 con 0x7fb3341030e0 2024-05-21T03:22:31.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.393+0000 7fb33a677700 1 -- 172.21.15.27:0/3490047978 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fb334199e10 con 0x7fb3341030e0 2024-05-21T03:22:31.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.393+0000 7fb3317fa700 1 -- 172.21.15.27:0/3490047978 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fb3280aa350 con 0x7fb3341030e0 2024-05-21T03:22:31.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.396+0000 7fb33a677700 1 -- 172.21.15.27:0/3490047978 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb31c0b2900 msgr2=0x7fb31c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:31.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.396+0000 7fb33a677700 1 --2- 172.21.15.27:0/3490047978 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb31c0b2900 0x7fb31c0b4dc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb334199a20 tx=0x7fb32400b2c0 comp rx=0 tx=0).stop 2024-05-21T03:22:31.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.396+0000 7fb33a677700 1 -- 172.21.15.27:0/3490047978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3341030e0 msgr2=0x7fb33419ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:31.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.396+0000 7fb33a677700 1 --2- 172.21.15.27:0/3490047978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3341030e0 0x7fb33419ec60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb32800c9b0 tx=0x7fb32800cd70 comp rx=0 tx=0).stop 2024-05-21T03:22:31.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.396+0000 7fb33a677700 1 -- 172.21.15.27:0/3490047978 shutdown_connections 2024-05-21T03:22:31.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.396+0000 7fb33a677700 1 --2- 172.21.15.27:0/3490047978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb334107460 0x7fb3341988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:31.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.396+0000 7fb33a677700 1 --2- 172.21.15.27:0/3490047978 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb31c0b2900 0x7fb31c0b4dc0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:31.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.396+0000 7fb33a677700 1 --2- 172.21.15.27:0/3490047978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb334102720 0x7fb33419e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:31.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.396+0000 7fb33a677700 1 --2- 172.21.15.27:0/3490047978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3341030e0 0x7fb33419ec60 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:31.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.396+0000 7fb33a677700 1 -- 172.21.15.27:0/3490047978 >> 172.21.15.27:0/3490047978 conn(0x7fb334079300 msgr2=0x7fb334100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:31.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.396+0000 7fb33a677700 1 -- 172.21.15.27:0/3490047978 shutdown_connections 2024-05-21T03:22:31.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:28.396+0000 7fb33a677700 1 -- 172.21.15.27:0/3490047978 wait complete. 2024-05-21T03:22:32.301 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841576 got 283467841579 for osd.10 2024-05-21T03:22:32.301 DEBUG:teuthology.parallel:result is None 2024-05-21T03:22:32.579 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:32 smithi027 ceph-mon[32179]: pgmap v351: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:22:32.579 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:32 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:22:32.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:32 smithi039 ceph-mon[36429]: pgmap v351: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:22:32.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:32 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:22:32.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:32 smithi165 ceph-mon[36269]: pgmap v351: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:22:32.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:32 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:22:32.915 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757314 got 163208757317 for osd.5 2024-05-21T03:22:32.915 DEBUG:teuthology.parallel:result is None 2024-05-21T03:22:33.469 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645345 got 309237645350 for osd.11 2024-05-21T03:22:33.469 DEBUG:teuthology.parallel:result is None 2024-05-21T03:22:33.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.131+0000 7fa3f9447700 1 -- 172.21.15.27:0/1525401904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3f4102470 msgr2=0x7fa3f41028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:33.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.131+0000 7fa3f9447700 1 --2- 172.21.15.27:0/1525401904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3f4102470 0x7fa3f41028f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8009990 tx=0x7fa3e8009ca0 comp rx=0 tx=0).stop 2024-05-21T03:22:33.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.131+0000 7fa3f9447700 1 -- 172.21.15.27:0/1525401904 shutdown_connections 2024-05-21T03:22:33.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.131+0000 7fa3f9447700 1 --2- 172.21.15.27:0/1525401904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3f4102e30 0x7fa3f410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:33.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.131+0000 7fa3f9447700 1 --2- 172.21.15.27:0/1525401904 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3f41084b0 0x7fa3f4108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:33.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.131+0000 7fa3f9447700 1 --2- 172.21.15.27:0/1525401904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3f4102470 0x7fa3f41028f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:33.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.131+0000 7fa3f9447700 1 -- 172.21.15.27:0/1525401904 >> 172.21.15.27:0/1525401904 conn(0x7fa3f40793f0 msgr2=0x7fa3f4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:33.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.132+0000 7fa3f9447700 1 -- 172.21.15.27:0/1525401904 shutdown_connections 2024-05-21T03:22:33.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.132+0000 7fa3f9447700 1 -- 172.21.15.27:0/1525401904 wait complete. 2024-05-21T03:22:33.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.132+0000 7fa3f9447700 1 Processor -- start 2024-05-21T03:22:33.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.132+0000 7fa3f9447700 1 -- start start 2024-05-21T03:22:33.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.133+0000 7fa3f9447700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3f4102470 0x7fa3f419f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:33.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.133+0000 7fa3f9447700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3f4102e30 0x7fa3f419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:33.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.133+0000 7fa3f9447700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3f41084b0 0x7fa3f4199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:33.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.133+0000 7fa3f9447700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f41125c0 con 0x7fa3f41084b0 2024-05-21T03:22:33.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.133+0000 7fa3f9447700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f4112440 con 0x7fa3f4102e30 2024-05-21T03:22:33.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.133+0000 7fa3f9447700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f4112740 con 0x7fa3f4102470 2024-05-21T03:22:33.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.133+0000 7fa3f37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3f41084b0 0x7fa3f4199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:33.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.133+0000 7fa3f27fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3f4102e30 0x7fa3f419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:33.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.133+0000 7fa3f2ffd700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3f4102470 0x7fa3f419f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:33.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.133+0000 7fa3f37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3f41084b0 0x7fa3f4199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46036/0 (socket says 172.21.15.27:46036) 2024-05-21T03:22:33.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.133+0000 7fa3f37fe700 1 -- 172.21.15.27:0/2943793207 learned_addr learned my addr 172.21.15.27:0/2943793207 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:33.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.133+0000 7fa3f37fe700 1 -- 172.21.15.27:0/2943793207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3f4102470 msgr2=0x7fa3f419f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:33.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.133+0000 7fa3f37fe700 1 --2- 172.21.15.27:0/2943793207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3f4102470 0x7fa3f419f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:33.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.133+0000 7fa3f37fe700 1 -- 172.21.15.27:0/2943793207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3f4102e30 msgr2=0x7fa3f419fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:33.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.133+0000 7fa3f37fe700 1 --2- 172.21.15.27:0/2943793207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3f4102e30 0x7fa3f419fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:33.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.133+0000 7fa3f37fe700 1 -- 172.21.15.27:0/2943793207 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f419a1b0 con 0x7fa3f41084b0 2024-05-21T03:22:33.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.133+0000 7fa3f27fc700 1 --2- 172.21.15.27:0/2943793207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3f4102e30 0x7fa3f419fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:22:33.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.133+0000 7fa3f2ffd700 1 --2- 172.21.15.27:0/2943793207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3f4102470 0x7fa3f419f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:22:33.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.133+0000 7fa3f37fe700 1 --2- 172.21.15.27:0/2943793207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3f41084b0 0x7fa3f4199a50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec007ee0 tx=0x7fa3ec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:33.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.133+0000 7fa3dbfff700 1 -- 172.21.15.27:0/2943793207 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec00dd20 con 0x7fa3f41084b0 2024-05-21T03:22:33.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.134+0000 7fa3dbfff700 1 -- 172.21.15.27:0/2943793207 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3ec004020 con 0x7fa3f41084b0 2024-05-21T03:22:33.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.134+0000 7fa3f9447700 1 -- 172.21.15.27:0/2943793207 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f419a4a0 con 0x7fa3f41084b0 2024-05-21T03:22:33.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.134+0000 7fa3dbfff700 1 -- 172.21.15.27:0/2943793207 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec015660 con 0x7fa3f41084b0 2024-05-21T03:22:33.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.134+0000 7fa3f9447700 1 -- 172.21.15.27:0/2943793207 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f41a6760 con 0x7fa3f41084b0 2024-05-21T03:22:33.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.134+0000 7fa3f9447700 1 -- 172.21.15.27:0/2943793207 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3f406a230 con 0x7fa3f41084b0 2024-05-21T03:22:33.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.137+0000 7fa3dbfff700 1 -- 172.21.15.27:0/2943793207 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa3ec0041d0 con 0x7fa3f41084b0 2024-05-21T03:22:33.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.137+0000 7fa3dbfff700 1 --2- 172.21.15.27:0/2943793207 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fa3dc0b2880 0x7fa3dc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:33.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.138+0000 7fa3f2ffd700 1 --2- 172.21.15.27:0/2943793207 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fa3dc0b2880 0x7fa3dc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:33.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.138+0000 7fa3dbfff700 1 -- 172.21.15.27:0/2943793207 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fa3ec0e2420 con 0x7fa3f41084b0 2024-05-21T03:22:33.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.138+0000 7fa3f2ffd700 1 --2- 172.21.15.27:0/2943793207 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fa3dc0b2880 0x7fa3dc0b4d40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa3f419ab90 tx=0x7fa3e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:33.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.141+0000 7fa3dbfff700 1 -- 172.21.15.27:0/2943793207 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa3ec010070 con 0x7fa3f41084b0 2024-05-21T03:22:33.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.360+0000 7fa3f9447700 1 -- 172.21.15.27:0/2943793207 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fa3f402e560 con 0x7fa3f41084b0 2024-05-21T03:22:33.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.360+0000 7fa3dbfff700 1 -- 172.21.15.27:0/2943793207 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa3ec0aa120 con 0x7fa3f41084b0 2024-05-21T03:22:33.555 INFO:teuthology.orchestra.run.smithi027.stdout:55834574939 2024-05-21T03:22:33.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.363+0000 7fa3f9447700 1 -- 172.21.15.27:0/2943793207 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fa3dc0b2880 msgr2=0x7fa3dc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:33.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.363+0000 7fa3f9447700 1 --2- 172.21.15.27:0/2943793207 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fa3dc0b2880 0x7fa3dc0b4d40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa3f419ab90 tx=0x7fa3e4009260 comp rx=0 tx=0).stop 2024-05-21T03:22:33.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.363+0000 7fa3f9447700 1 -- 172.21.15.27:0/2943793207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3f41084b0 msgr2=0x7fa3f4199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:33.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.363+0000 7fa3f9447700 1 --2- 172.21.15.27:0/2943793207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3f41084b0 0x7fa3f4199a50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec007ee0 tx=0x7fa3ec00d3b0 comp rx=0 tx=0).stop 2024-05-21T03:22:33.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.363+0000 7fa3f9447700 1 -- 172.21.15.27:0/2943793207 shutdown_connections 2024-05-21T03:22:33.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.363+0000 7fa3f9447700 1 --2- 172.21.15.27:0/2943793207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3f41084b0 0x7fa3f4199a50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:33.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.363+0000 7fa3f9447700 1 --2- 172.21.15.27:0/2943793207 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fa3dc0b2880 0x7fa3dc0b4d40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:33.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.363+0000 7fa3f9447700 1 --2- 172.21.15.27:0/2943793207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3f4102470 0x7fa3f419f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:33.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.363+0000 7fa3f9447700 1 --2- 172.21.15.27:0/2943793207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3f4102e30 0x7fa3f419fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:33.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.363+0000 7fa3f9447700 1 -- 172.21.15.27:0/2943793207 >> 172.21.15.27:0/2943793207 conn(0x7fa3f40793f0 msgr2=0x7fa3f406bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:33.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.363+0000 7fa3f9447700 1 -- 172.21.15.27:0/2943793207 shutdown_connections 2024-05-21T03:22:33.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:33.363+0000 7fa3f9447700 1 -- 172.21.15.27:0/2943793207 wait complete. 2024-05-21T03:22:34.114 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574934 got 55834574939 for osd.1 2024-05-21T03:22:34.114 DEBUG:teuthology.parallel:result is None 2024-05-21T03:22:34.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:34 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:22:34.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:34 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:22:34.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:34 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2943793207' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-21T03:22:34.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:34 smithi027 ceph-mon[32179]: pgmap v352: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:22:34.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:34 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:22:34.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:34 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:22:34.492 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738459 got 34359738463 for osd.0 2024-05-21T03:22:34.492 DEBUG:teuthology.parallel:result is None 2024-05-21T03:22:34.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:34 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:22:34.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:34 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:22:34.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:34 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/2943793207' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-21T03:22:34.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:34 smithi039 ceph-mon[36429]: pgmap v352: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:22:34.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:34 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:22:34.770 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:34 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:22:34.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:34 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:22:34.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:34 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:22:34.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:34 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/2943793207' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-21T03:22:34.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:34 smithi165 ceph-mon[36269]: pgmap v352: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:22:34.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:34 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:22:34.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:34 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:22:34.864 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005100 got 261993005105 for osd.9 2024-05-21T03:22:34.865 DEBUG:teuthology.parallel:result is None 2024-05-21T03:22:35.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.042+0000 7f804d455700 1 -- 172.21.15.27:0/133070775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80481084d0 msgr2=0x7f80481088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:35.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.042+0000 7f804d455700 1 --2- 172.21.15.27:0/133070775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80481084d0 0x7f80481088b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8038009990 tx=0x7f8038009ca0 comp rx=0 tx=0).stop 2024-05-21T03:22:35.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.042+0000 7f804d455700 1 -- 172.21.15.27:0/133070775 shutdown_connections 2024-05-21T03:22:35.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.042+0000 7f804d455700 1 --2- 172.21.15.27:0/133070775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8048102e90 0x7f804810f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:35.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.042+0000 7f804d455700 1 --2- 172.21.15.27:0/133070775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80481084d0 0x7f80481088b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:35.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.042+0000 7f804d455700 1 --2- 172.21.15.27:0/133070775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80481024d0 0x7f8048102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:35.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.042+0000 7f804d455700 1 -- 172.21.15.27:0/133070775 >> 172.21.15.27:0/133070775 conn(0x7f8048078490 msgr2=0x7f80480788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:35.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.042+0000 7f804d455700 1 -- 172.21.15.27:0/133070775 shutdown_connections 2024-05-21T03:22:35.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.042+0000 7f804d455700 1 -- 172.21.15.27:0/133070775 wait complete. 2024-05-21T03:22:35.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.043+0000 7f804d455700 1 Processor -- start 2024-05-21T03:22:35.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.043+0000 7f804d455700 1 -- start start 2024-05-21T03:22:35.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.043+0000 7f804d455700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80481024d0 0x7f804819f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:35.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.043+0000 7f804d455700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8048102e90 0x7f804819fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:35.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.043+0000 7f804d455700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80481084d0 0x7f8048199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:35.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.043+0000 7f804d455700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8048112660 con 0x7f80481084d0 2024-05-21T03:22:35.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.043+0000 7f804d455700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f80481124e0 con 0x7f8048102e90 2024-05-21T03:22:35.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.043+0000 7f804d455700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f80481127e0 con 0x7f80481024d0 2024-05-21T03:22:35.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.043+0000 7f80477fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80481084d0 0x7f8048199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:35.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.043+0000 7f80477fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80481084d0 0x7f8048199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46058/0 (socket says 172.21.15.27:46058) 2024-05-21T03:22:35.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.043+0000 7f80477fe700 1 -- 172.21.15.27:0/2234652517 learned_addr learned my addr 172.21.15.27:0/2234652517 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:35.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.043+0000 7f803ffff700 1 --2- 172.21.15.27:0/2234652517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8048102e90 0x7f804819fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:35.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.043+0000 7f80477fe700 1 -- 172.21.15.27:0/2234652517 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80481024d0 msgr2=0x7f804819f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:35.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.043+0000 7f80477fe700 1 --2- 172.21.15.27:0/2234652517 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80481024d0 0x7f804819f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:35.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.043+0000 7f80477fe700 1 -- 172.21.15.27:0/2234652517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8048102e90 msgr2=0x7f804819fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:35.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.043+0000 7f80477fe700 1 --2- 172.21.15.27:0/2234652517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8048102e90 0x7f804819fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:35.045 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.043+0000 7f80477fe700 1 -- 172.21.15.27:0/2234652517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804819a1d0 con 0x7f80481084d0 2024-05-21T03:22:35.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.043+0000 7f80477fe700 1 --2- 172.21.15.27:0/2234652517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80481084d0 0x7f8048199a70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8040007ee0 tx=0x7f804000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:35.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.044+0000 7f8044ff9700 1 -- 172.21.15.27:0/2234652517 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804000dbd0 con 0x7f80481084d0 2024-05-21T03:22:35.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.044+0000 7f804d455700 1 -- 172.21.15.27:0/2234652517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f804819a4c0 con 0x7f80481084d0 2024-05-21T03:22:35.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.044+0000 7f8044ff9700 1 -- 172.21.15.27:0/2234652517 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f804000dd70 con 0x7f80481084d0 2024-05-21T03:22:35.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.044+0000 7f8044ff9700 1 -- 172.21.15.27:0/2234652517 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8040005990 con 0x7f80481084d0 2024-05-21T03:22:35.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.044+0000 7f804d455700 1 -- 172.21.15.27:0/2234652517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80481a6780 con 0x7f80481084d0 2024-05-21T03:22:35.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.045+0000 7f804d455700 1 -- 172.21.15.27:0/2234652517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f804806a230 con 0x7f80481084d0 2024-05-21T03:22:35.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.046+0000 7f8044ff9700 1 -- 172.21.15.27:0/2234652517 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f804001a020 con 0x7f80481084d0 2024-05-21T03:22:35.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.047+0000 7f8044ff9700 1 --2- 172.21.15.27:0/2234652517 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f80340b2880 0x7f80340b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:35.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.047+0000 7f8046ffd700 1 --2- 172.21.15.27:0/2234652517 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f80340b2880 0x7f80340b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:35.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.047+0000 7f8044ff9700 1 -- 172.21.15.27:0/2234652517 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f80400e20c0 con 0x7f80481084d0 2024-05-21T03:22:35.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.047+0000 7f8046ffd700 1 --2- 172.21.15.27:0/2234652517 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f80340b2880 0x7f80340b4d40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f80380098e0 tx=0x7f8038005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:35.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.050+0000 7f8044ff9700 1 -- 172.21.15.27:0/2234652517 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8040010070 con 0x7f80481084d0 2024-05-21T03:22:35.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.267+0000 7f804d455700 1 -- 172.21.15.27:0/2234652517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f804802e560 con 0x7f80481084d0 2024-05-21T03:22:35.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.267+0000 7f8044ff9700 1 -- 172.21.15.27:0/2234652517 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f80400a9dc0 con 0x7f80481084d0 2024-05-21T03:22:35.268 INFO:teuthology.orchestra.run.smithi027.stdout:77309411414 2024-05-21T03:22:35.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.270+0000 7f804d455700 1 -- 172.21.15.27:0/2234652517 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f80340b2880 msgr2=0x7f80340b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:35.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.270+0000 7f804d455700 1 --2- 172.21.15.27:0/2234652517 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f80340b2880 0x7f80340b4d40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f80380098e0 tx=0x7f8038005fb0 comp rx=0 tx=0).stop 2024-05-21T03:22:35.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.270+0000 7f804d455700 1 -- 172.21.15.27:0/2234652517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80481084d0 msgr2=0x7f8048199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:35.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.270+0000 7f804d455700 1 --2- 172.21.15.27:0/2234652517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80481084d0 0x7f8048199a70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8040007ee0 tx=0x7f804000d3b0 comp rx=0 tx=0).stop 2024-05-21T03:22:35.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.270+0000 7f804d455700 1 -- 172.21.15.27:0/2234652517 shutdown_connections 2024-05-21T03:22:35.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.270+0000 7f804d455700 1 --2- 172.21.15.27:0/2234652517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80481084d0 0x7f8048199a70 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:35.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.270+0000 7f804d455700 1 --2- 172.21.15.27:0/2234652517 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f80340b2880 0x7f80340b4d40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:35.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.270+0000 7f804d455700 1 --2- 172.21.15.27:0/2234652517 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80481024d0 0x7f804819f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:35.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.270+0000 7f804d455700 1 --2- 172.21.15.27:0/2234652517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8048102e90 0x7f804819fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:35.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.270+0000 7f804d455700 1 -- 172.21.15.27:0/2234652517 >> 172.21.15.27:0/2234652517 conn(0x7f8048078490 msgr2=0x7f804806bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:35.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.270+0000 7f804d455700 1 -- 172.21.15.27:0/2234652517 shutdown_connections 2024-05-21T03:22:35.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:35.270+0000 7f804d455700 1 -- 172.21.15.27:0/2234652517 wait complete. 2024-05-21T03:22:35.591 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:35 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2234652517' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-21T03:22:35.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:35 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/2234652517' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-21T03:22:35.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:35 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/2234652517' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-21T03:22:36.198 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411406 got 77309411414 for osd.2 2024-05-21T03:22:36.198 DEBUG:teuthology.parallel:result is None 2024-05-21T03:22:36.425 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:36 smithi027 ceph-mon[32179]: pgmap v353: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:22:36.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.641+0000 7efeca8c1700 1 -- 172.21.15.27:0/1802365560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efec4108520 msgr2=0x7efec4108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:36.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.641+0000 7efeca8c1700 1 --2- 172.21.15.27:0/1802365560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efec4108520 0x7efec4108900 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7efeb4009990 tx=0x7efeb4009ca0 comp rx=0 tx=0).stop 2024-05-21T03:22:36.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.641+0000 7efeca8c1700 1 -- 172.21.15.27:0/1802365560 shutdown_connections 2024-05-21T03:22:36.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.641+0000 7efeca8c1700 1 --2- 172.21.15.27:0/1802365560 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efec4102e00 0x7efec410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:36.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.641+0000 7efeca8c1700 1 --2- 172.21.15.27:0/1802365560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efec4108520 0x7efec4108900 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:36.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.641+0000 7efeca8c1700 1 --2- 172.21.15.27:0/1802365560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efec4102440 0x7efec41028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:36.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.641+0000 7efeca8c1700 1 -- 172.21.15.27:0/1802365560 >> 172.21.15.27:0/1802365560 conn(0x7efec40793f0 msgr2=0x7efec4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:36.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.641+0000 7efeca8c1700 1 -- 172.21.15.27:0/1802365560 shutdown_connections 2024-05-21T03:22:36.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.641+0000 7efeca8c1700 1 -- 172.21.15.27:0/1802365560 wait complete. 2024-05-21T03:22:36.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.642+0000 7efeca8c1700 1 Processor -- start 2024-05-21T03:22:36.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.642+0000 7efeca8c1700 1 -- start start 2024-05-21T03:22:36.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.642+0000 7efeca8c1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efec4102440 0x7efec4104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:36.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.642+0000 7efeca8c1700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efec4102e00 0x7efec41045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:36.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.642+0000 7efeca8c1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efec4108520 0x7efec4104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:36.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.643+0000 7efeca8c1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efec4112640 con 0x7efec4102440 2024-05-21T03:22:36.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.643+0000 7efeca8c1700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efec41124c0 con 0x7efec4108520 2024-05-21T03:22:36.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.643+0000 7efec3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efec4102440 0x7efec4104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:36.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.643+0000 7efec3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efec4102440 0x7efec4104080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46076/0 (socket says 172.21.15.27:46076) 2024-05-21T03:22:36.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.643+0000 7efec3fff700 1 -- 172.21.15.27:0/259798869 learned_addr learned my addr 172.21.15.27:0/259798869 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:36.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.643+0000 7efeca8c1700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efec41127c0 con 0x7efec4102e00 2024-05-21T03:22:36.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.643+0000 7efec37fe700 1 --2- 172.21.15.27:0/259798869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efec4102e00 0x7efec41045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:36.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.643+0000 7efec8e5e700 1 --2- 172.21.15.27:0/259798869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efec4108520 0x7efec4104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:36.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.643+0000 7efec3fff700 1 -- 172.21.15.27:0/259798869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efec4102e00 msgr2=0x7efec41045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:36.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.643+0000 7efec3fff700 1 --2- 172.21.15.27:0/259798869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efec4102e00 0x7efec41045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:36.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.643+0000 7efec3fff700 1 -- 172.21.15.27:0/259798869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efec4108520 msgr2=0x7efec4104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:36.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.643+0000 7efec3fff700 1 --2- 172.21.15.27:0/259798869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efec4108520 0x7efec4104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:36.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.643+0000 7efec3fff700 1 -- 172.21.15.27:0/259798869 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec41a8260 con 0x7efec4102440 2024-05-21T03:22:36.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.643+0000 7efec8e5e700 1 --2- 172.21.15.27:0/259798869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efec4108520 0x7efec4104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:22:36.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.643+0000 7efec3fff700 1 --2- 172.21.15.27:0/259798869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efec4102440 0x7efec4104080 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efeb4005910 tx=0x7efeb40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:36.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.643+0000 7efec17fa700 1 -- 172.21.15.27:0/259798869 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb4027070 con 0x7efec4102440 2024-05-21T03:22:36.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.643+0000 7efeca8c1700 1 -- 172.21.15.27:0/259798869 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec41a84f0 con 0x7efec4102440 2024-05-21T03:22:36.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.644+0000 7efeca8c1700 1 -- 172.21.15.27:0/259798869 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efec41a89a0 con 0x7efec4102440 2024-05-21T03:22:36.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.644+0000 7efec17fa700 1 -- 172.21.15.27:0/259798869 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efeb4003f20 con 0x7efec4102440 2024-05-21T03:22:36.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.644+0000 7efec17fa700 1 -- 172.21.15.27:0/259798869 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb400fc50 con 0x7efec4102440 2024-05-21T03:22:36.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.645+0000 7efeca8c1700 1 -- 172.21.15.27:0/259798869 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeb0005330 con 0x7efec4102440 2024-05-21T03:22:36.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.646+0000 7efec17fa700 1 -- 172.21.15.27:0/259798869 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7efeb402a070 con 0x7efec4102440 2024-05-21T03:22:36.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.647+0000 7efec17fa700 1 --2- 172.21.15.27:0/259798869 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7efeac0b2900 0x7efeac0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:36.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.647+0000 7efec37fe700 1 --2- 172.21.15.27:0/259798869 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7efeac0b2900 0x7efeac0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:36.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.647+0000 7efec17fa700 1 -- 172.21.15.27:0/259798869 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7efeb40e2d00 con 0x7efec4102440 2024-05-21T03:22:36.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.648+0000 7efec37fe700 1 --2- 172.21.15.27:0/259798869 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7efeac0b2900 0x7efeac0b4dc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efec4079af0 tx=0x7efeb80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:36.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.651+0000 7efec17fa700 1 -- 172.21.15.27:0/259798869 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efeb4060df0 con 0x7efec4102440 2024-05-21T03:22:36.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:36 smithi039 ceph-mon[36429]: pgmap v353: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:22:36.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:36 smithi165 ceph-mon[36269]: pgmap v353: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:22:36.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.877+0000 7efeca8c1700 1 -- 172.21.15.27:0/259798869 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7efeb00058f0 con 0x7efec4102440 2024-05-21T03:22:36.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.878+0000 7efec17fa700 1 -- 172.21.15.27:0/259798869 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7efeb4013070 con 0x7efec4102440 2024-05-21T03:22:36.878 INFO:teuthology.orchestra.run.smithi027.stdout:188978561089 2024-05-21T03:22:36.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.881+0000 7efeca8c1700 1 -- 172.21.15.27:0/259798869 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7efeac0b2900 msgr2=0x7efeac0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:36.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.881+0000 7efeca8c1700 1 --2- 172.21.15.27:0/259798869 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7efeac0b2900 0x7efeac0b4dc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efec4079af0 tx=0x7efeb80073d0 comp rx=0 tx=0).stop 2024-05-21T03:22:36.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.881+0000 7efeca8c1700 1 -- 172.21.15.27:0/259798869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efec4102440 msgr2=0x7efec4104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:36.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.881+0000 7efeca8c1700 1 --2- 172.21.15.27:0/259798869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efec4102440 0x7efec4104080 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efeb4005910 tx=0x7efeb40047b0 comp rx=0 tx=0).stop 2024-05-21T03:22:36.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.881+0000 7efeca8c1700 1 -- 172.21.15.27:0/259798869 shutdown_connections 2024-05-21T03:22:36.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.881+0000 7efeca8c1700 1 --2- 172.21.15.27:0/259798869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efec4108520 0x7efec4104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:36.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.881+0000 7efeca8c1700 1 --2- 172.21.15.27:0/259798869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efec4102440 0x7efec4104080 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:36.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.881+0000 7efeca8c1700 1 --2- 172.21.15.27:0/259798869 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7efeac0b2900 0x7efeac0b4dc0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:36.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.881+0000 7efeca8c1700 1 --2- 172.21.15.27:0/259798869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efec4102e00 0x7efec41045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:36.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.881+0000 7efeca8c1700 1 -- 172.21.15.27:0/259798869 >> 172.21.15.27:0/259798869 conn(0x7efec40793f0 msgr2=0x7efec406b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:36.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.881+0000 7efeca8c1700 1 -- 172.21.15.27:0/259798869 shutdown_connections 2024-05-21T03:22:36.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:36.881+0000 7efeca8c1700 1 -- 172.21.15.27:0/259798869 wait complete. 2024-05-21T03:22:37.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:37 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/259798869' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-21T03:22:37.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:37 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/259798869' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-21T03:22:37.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:37 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/259798869' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-21T03:22:38.136 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561083 got 188978561089 for osd.6 2024-05-21T03:22:38.137 DEBUG:teuthology.parallel:result is None 2024-05-21T03:22:38.137 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-21T03:22:38.137 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-21T03:22:38.421 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:22:38.426 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:38 smithi027 ceph-mon[32179]: pgmap v354: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:22:38.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:38 smithi039 ceph-mon[36429]: pgmap v354: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:22:38.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:38 smithi165 ceph-mon[36269]: pgmap v354: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:22:40.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.064+0000 7f1cf9bbe700 1 -- 172.21.15.27:0/1959657853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cf406b250 msgr2=0x7f1cf406b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:40.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.064+0000 7f1cf9bbe700 1 --2- 172.21.15.27:0/1959657853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cf406b250 0x7f1cf406b630 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400b0a0 tx=0x7f1ce400b3b0 comp rx=0 tx=0).stop 2024-05-21T03:22:40.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.064+0000 7f1cf9bbe700 1 -- 172.21.15.27:0/1959657853 shutdown_connections 2024-05-21T03:22:40.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.064+0000 7f1cf9bbe700 1 --2- 172.21.15.27:0/1959657853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cf410fff0 0x7f1cf41123e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:40.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.064+0000 7f1cf9bbe700 1 --2- 172.21.15.27:0/1959657853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cf406b250 0x7f1cf406b630 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:40.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.064+0000 7f1cf9bbe700 1 --2- 172.21.15.27:0/1959657853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cf406bb70 0x7f1cf410fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:40.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.064+0000 7f1cf9bbe700 1 -- 172.21.15.27:0/1959657853 >> 172.21.15.27:0/1959657853 conn(0x7f1cf40793f0 msgr2=0x7f1cf4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:40.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.064+0000 7f1cf9bbe700 1 -- 172.21.15.27:0/1959657853 shutdown_connections 2024-05-21T03:22:40.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.064+0000 7f1cf9bbe700 1 -- 172.21.15.27:0/1959657853 wait complete. 2024-05-21T03:22:40.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.065+0000 7f1cf9bbe700 1 Processor -- start 2024-05-21T03:22:40.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.065+0000 7f1cf9bbe700 1 -- start start 2024-05-21T03:22:40.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.065+0000 7f1cf9bbe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cf406b250 0x7f1cf419be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:40.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.065+0000 7f1cf9bbe700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cf406bb70 0x7f1cf419c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:40.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.065+0000 7f1cf9bbe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cf410fff0 0x7f1cf41a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:40.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.065+0000 7f1cf9bbe700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf41158f0 con 0x7f1cf406b250 2024-05-21T03:22:40.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.065+0000 7f1cf9bbe700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf4115770 con 0x7f1cf410fff0 2024-05-21T03:22:40.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.065+0000 7f1cf9bbe700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf4115a70 con 0x7f1cf406bb70 2024-05-21T03:22:40.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.065+0000 7f1cf37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cf406b250 0x7f1cf419be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:40.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.065+0000 7f1cf37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cf406b250 0x7f1cf419be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46096/0 (socket says 172.21.15.27:46096) 2024-05-21T03:22:40.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.065+0000 7f1cf37fe700 1 -- 172.21.15.27:0/4025372484 learned_addr learned my addr 172.21.15.27:0/4025372484 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:40.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.065+0000 7f1cf3fff700 1 --2- 172.21.15.27:0/4025372484 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cf410fff0 0x7f1cf41a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:40.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.066+0000 7f1cf2ffd700 1 --2- 172.21.15.27:0/4025372484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cf406bb70 0x7f1cf419c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:40.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.066+0000 7f1cf37fe700 1 -- 172.21.15.27:0/4025372484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cf406bb70 msgr2=0x7f1cf419c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:40.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.066+0000 7f1cf37fe700 1 --2- 172.21.15.27:0/4025372484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cf406bb70 0x7f1cf419c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:40.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.066+0000 7f1cf37fe700 1 -- 172.21.15.27:0/4025372484 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cf410fff0 msgr2=0x7f1cf41a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:40.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.066+0000 7f1cf37fe700 1 --2- 172.21.15.27:0/4025372484 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cf410fff0 0x7f1cf41a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:40.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.066+0000 7f1cf37fe700 1 -- 172.21.15.27:0/4025372484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf41a0e50 con 0x7f1cf406b250 2024-05-21T03:22:40.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.066+0000 7f1cf2ffd700 1 --2- 172.21.15.27:0/4025372484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cf406bb70 0x7f1cf419c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:22:40.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.066+0000 7f1cf37fe700 1 --2- 172.21.15.27:0/4025372484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cf406b250 0x7f1cf419be40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400b070 tx=0x7f1ce4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:40.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.066+0000 7f1cf0ff9700 1 -- 172.21.15.27:0/4025372484 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4021070 con 0x7f1cf406b250 2024-05-21T03:22:40.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.066+0000 7f1cf9bbe700 1 -- 172.21.15.27:0/4025372484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf41a10e0 con 0x7f1cf406b250 2024-05-21T03:22:40.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.066+0000 7f1cf0ff9700 1 -- 172.21.15.27:0/4025372484 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ce4007e30 con 0x7f1cf406b250 2024-05-21T03:22:40.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.066+0000 7f1cf0ff9700 1 -- 172.21.15.27:0/4025372484 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4004840 con 0x7f1cf406b250 2024-05-21T03:22:40.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.066+0000 7f1cf9bbe700 1 -- 172.21.15.27:0/4025372484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf41a89c0 con 0x7f1cf406b250 2024-05-21T03:22:40.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.067+0000 7f1cf9bbe700 1 -- 172.21.15.27:0/4025372484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cf406a230 con 0x7f1cf406b250 2024-05-21T03:22:40.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.069+0000 7f1cf0ff9700 1 -- 172.21.15.27:0/4025372484 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1ce4007980 con 0x7f1cf406b250 2024-05-21T03:22:40.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.070+0000 7f1cf0ff9700 1 --2- 172.21.15.27:0/4025372484 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f1ce00b2880 0x7f1ce00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:40.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.070+0000 7f1cf2ffd700 1 --2- 172.21.15.27:0/4025372484 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f1ce00b2880 0x7f1ce00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:40.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.070+0000 7f1cf0ff9700 1 -- 172.21.15.27:0/4025372484 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f1ce400f020 con 0x7f1cf406b250 2024-05-21T03:22:40.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.070+0000 7f1cf2ffd700 1 --2- 172.21.15.27:0/4025372484 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f1ce00b2880 0x7f1ce00b4d40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc005fd0 tx=0x7f1cdc005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:40.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.074+0000 7f1cf0ff9700 1 -- 172.21.15.27:0/4025372484 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1ce402a020 con 0x7f1cf406b250 2024-05-21T03:22:40.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:40 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:22:40.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:40 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:22:40.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:40 smithi027 ceph-mon[32179]: pgmap v355: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:22:40.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:40 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:22:40.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:40 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:22:40.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:40 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:22:40.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.285+0000 7f1cf9bbe700 1 -- 172.21.15.27:0/4025372484 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1cf419d1a0 con 0x7f1ce00b2880 2024-05-21T03:22:40.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.292+0000 7f1cf0ff9700 1 -- 172.21.15.27:0/4025372484 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258887 (secure 0 0 0) 0x7f1cf419d1a0 con 0x7f1ce00b2880 2024-05-21T03:22:40.295 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:22:40.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.295+0000 7f1cf9bbe700 1 -- 172.21.15.27:0/4025372484 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f1ce00b2880 msgr2=0x7f1ce00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:40.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.295+0000 7f1cf9bbe700 1 --2- 172.21.15.27:0/4025372484 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f1ce00b2880 0x7f1ce00b4d40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc005fd0 tx=0x7f1cdc005ee0 comp rx=0 tx=0).stop 2024-05-21T03:22:40.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.295+0000 7f1cf9bbe700 1 -- 172.21.15.27:0/4025372484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cf406b250 msgr2=0x7f1cf419be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:40.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.295+0000 7f1cf9bbe700 1 --2- 172.21.15.27:0/4025372484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cf406b250 0x7f1cf419be40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400b070 tx=0x7f1ce4004d00 comp rx=0 tx=0).stop 2024-05-21T03:22:40.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.295+0000 7f1cf9bbe700 1 -- 172.21.15.27:0/4025372484 shutdown_connections 2024-05-21T03:22:40.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.295+0000 7f1cf9bbe700 1 --2- 172.21.15.27:0/4025372484 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cf410fff0 0x7f1cf41a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:40.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.295+0000 7f1cf9bbe700 1 --2- 172.21.15.27:0/4025372484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cf406b250 0x7f1cf419be40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:40.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.295+0000 7f1cf9bbe700 1 --2- 172.21.15.27:0/4025372484 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f1ce00b2880 0x7f1ce00b4d40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:40.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.295+0000 7f1cf9bbe700 1 --2- 172.21.15.27:0/4025372484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cf406bb70 0x7f1cf419c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:40.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.295+0000 7f1cf9bbe700 1 -- 172.21.15.27:0/4025372484 >> 172.21.15.27:0/4025372484 conn(0x7f1cf40793f0 msgr2=0x7f1cf41115a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:40.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.295+0000 7f1cf9bbe700 1 -- 172.21.15.27:0/4025372484 shutdown_connections 2024-05-21T03:22:40.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:40.295+0000 7f1cf9bbe700 1 -- 172.21.15.27:0/4025372484 wait complete. 2024-05-21T03:22:40.299 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-05-21T03:22:40.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:40 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:22:40.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:40 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:22:40.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:40 smithi039 ceph-mon[36429]: pgmap v355: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:22:40.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:40 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:22:40.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:40 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:22:40.520 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:40 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:22:40.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:40 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:22:40.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:40 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:22:40.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:40 smithi165 ceph-mon[36269]: pgmap v355: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:22:40.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:40 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:22:40.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:40 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:22:40.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:40 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:22:40.950 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-05-21T03:22:39.370290+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658631,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1177,"num_read_kb":703,"num_write":380,"num_write_kb":2296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2953518,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":814,"ondisk_log_size":814,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357724,"kb_used_data":12732,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584100,"statfs":{"total":1151940427776,"available":1151574118400,"internally_reserved":0,"allocated":13037568,"data_stored":7163097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19139,"internal_metadata":353154365},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":125,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002986"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.404523+0000","last_change":"2024-05-21T03:20:32.149961+0000","last_active":"2024-05-21T03:22:13.404523+0000","last_peered":"2024-05-21T03:22:13.404523+0000","last_clean":"2024-05-21T03:22:13.404523+0000","last_became_active":"2024-05-21T03:20:32.149552+0000","last_became_peered":"2024-05-21T03:20:32.149552+0000","last_unstale":"2024-05-21T03:22:13.404523+0000","last_undegraded":"2024-05-21T03:22:13.404523+0000","last_fullsized":"2024-05-21T03:22:13.404523+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:20:38.543551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.410926+0000","last_change":"2024-05-21T03:20:32.146413+0000","last_active":"2024-05-21T03:22:13.410926+0000","last_peered":"2024-05-21T03:22:13.410926+0000","last_clean":"2024-05-21T03:22:13.410926+0000","last_became_active":"2024-05-21T03:20:32.146006+0000","last_became_peered":"2024-05-21T03:20:32.146006+0000","last_unstale":"2024-05-21T03:22:13.410926+0000","last_undegraded":"2024-05-21T03:22:13.410926+0000","last_fullsized":"2024-05-21T03:22:13.410926+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:05:12.888550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.416235+0000","last_change":"2024-05-21T03:20:32.153654+0000","last_active":"2024-05-21T03:22:13.416235+0000","last_peered":"2024-05-21T03:22:13.416235+0000","last_clean":"2024-05-21T03:22:13.416235+0000","last_became_active":"2024-05-21T03:20:32.153296+0000","last_became_peered":"2024-05-21T03:20:32.153296+0000","last_unstale":"2024-05-21T03:22:13.416235+0000","last_undegraded":"2024-05-21T03:22:13.416235+0000","last_fullsized":"2024-05-21T03:22:13.416235+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:16:09.423236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.372040+0000","last_change":"2024-05-21T03:20:32.138672+0000","last_active":"2024-05-21T03:22:13.372040+0000","last_peered":"2024-05-21T03:22:13.372040+0000","last_clean":"2024-05-21T03:22:13.372040+0000","last_became_active":"2024-05-21T03:20:32.138163+0000","last_became_peered":"2024-05-21T03:20:32.138163+0000","last_unstale":"2024-05-21T03:22:13.372040+0000","last_undegraded":"2024-05-21T03:22:13.372040+0000","last_fullsized":"2024-05-21T03:22:13.372040+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:26:55.012897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.380094+0000","last_change":"2024-05-21T03:20:32.153950+0000","last_active":"2024-05-21T03:22:13.380094+0000","last_peered":"2024-05-21T03:22:13.380094+0000","last_clean":"2024-05-21T03:22:13.380094+0000","last_became_active":"2024-05-21T03:20:32.153567+0000","last_became_peered":"2024-05-21T03:20:32.153567+0000","last_unstale":"2024-05-21T03:22:13.380094+0000","last_undegraded":"2024-05-21T03:22:13.380094+0000","last_fullsized":"2024-05-21T03:22:13.380094+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:28:34.621193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.395805+0000","last_change":"2024-05-21T03:20:32.140038+0000","last_active":"2024-05-21T03:22:13.395805+0000","last_peered":"2024-05-21T03:22:13.395805+0000","last_clean":"2024-05-21T03:22:13.395805+0000","last_became_active":"2024-05-21T03:20:32.139690+0000","last_became_peered":"2024-05-21T03:20:32.139690+0000","last_unstale":"2024-05-21T03:22:13.395805+0000","last_undegraded":"2024-05-21T03:22:13.395805+0000","last_fullsized":"2024-05-21T03:22:13.395805+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:31:24.270755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.387191+0000","last_change":"2024-05-21T03:20:32.143735+0000","last_active":"2024-05-21T03:22:13.387191+0000","last_peered":"2024-05-21T03:22:13.387191+0000","last_clean":"2024-05-21T03:22:13.387191+0000","last_became_active":"2024-05-21T03:20:32.143397+0000","last_became_peered":"2024-05-21T03:20:32.143397+0000","last_unstale":"2024-05-21T03:22:13.387191+0000","last_undegraded":"2024-05-21T03:22:13.387191+0000","last_fullsized":"2024-05-21T03:22:13.387191+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:40:01.951608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.402887+0000","last_change":"2024-05-21T03:20:32.156999+0000","last_active":"2024-05-21T03:22:13.402887+0000","last_peered":"2024-05-21T03:22:13.402887+0000","last_clean":"2024-05-21T03:22:13.402887+0000","last_became_active":"2024-05-21T03:20:32.156677+0000","last_became_peered":"2024-05-21T03:20:32.156677+0000","last_unstale":"2024-05-21T03:22:13.402887+0000","last_undegraded":"2024-05-21T03:22:13.402887+0000","last_fullsized":"2024-05-21T03:22:13.402887+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:33:24.225667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.384207+0000","last_change":"2024-05-21T03:20:32.145039+0000","last_active":"2024-05-21T03:22:13.384207+0000","last_peered":"2024-05-21T03:22:13.384207+0000","last_clean":"2024-05-21T03:22:13.384207+0000","last_became_active":"2024-05-21T03:20:32.140926+0000","last_became_peered":"2024-05-21T03:20:32.140926+0000","last_unstale":"2024-05-21T03:22:13.384207+0000","last_undegraded":"2024-05-21T03:22:13.384207+0000","last_fullsized":"2024-05-21T03:22:13.384207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:11:27.049426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.371393+0000","last_change":"2024-05-21T03:20:32.145840+0000","last_active":"2024-05-21T03:22:13.371393+0000","last_peered":"2024-05-21T03:22:13.371393+0000","last_clean":"2024-05-21T03:22:13.371393+0000","last_became_active":"2024-05-21T03:20:32.145291+0000","last_became_peered":"2024-05-21T03:20:32.145291+0000","last_unstale":"2024-05-21T03:22:13.371393+0000","last_undegraded":"2024-05-21T03:22:13.371393+0000","last_fullsized":"2024-05-21T03:22:13.371393+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:22:27.696588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.389055+0000","last_change":"2024-05-21T03:20:32.142522+0000","last_active":"2024-05-21T03:22:13.389055+0000","last_peered":"2024-05-21T03:22:13.389055+0000","last_clean":"2024-05-21T03:22:13.389055+0000","last_became_active":"2024-05-21T03:20:32.142165+0000","last_became_peered":"2024-05-21T03:20:32.142165+0000","last_unstale":"2024-05-21T03:22:13.389055+0000","last_undegraded":"2024-05-21T03:22:13.389055+0000","last_fullsized":"2024-05-21T03:22:13.389055+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:58:39.085834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.371212+0000","last_change":"2024-05-21T03:20:32.142526+0000","last_active":"2024-05-21T03:22:13.371212+0000","last_peered":"2024-05-21T03:22:13.371212+0000","last_clean":"2024-05-21T03:22:13.371212+0000","last_became_active":"2024-05-21T03:20:32.141143+0000","last_became_peered":"2024-05-21T03:20:32.141143+0000","last_unstale":"2024-05-21T03:22:13.371212+0000","last_undegraded":"2024-05-21T03:22:13.371212+0000","last_fullsized":"2024-05-21T03:22:13.371212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:35:11.336796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.408485+0000","last_change":"2024-05-21T03:20:32.135631+0000","last_active":"2024-05-21T03:22:13.408485+0000","last_peered":"2024-05-21T03:22:13.408485+0000","last_clean":"2024-05-21T03:22:13.408485+0000","last_became_active":"2024-05-21T03:20:32.135302+0000","last_became_peered":"2024-05-21T03:20:32.135302+0000","last_unstale":"2024-05-21T03:22:13.408485+0000","last_undegraded":"2024-05-21T03:22:13.408485+0000","last_fullsized":"2024-05-21T03:22:13.408485+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:05:55.000690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.413140+0000","last_change":"2024-05-21T03:20:32.146580+0000","last_active":"2024-05-21T03:22:13.413140+0000","last_peered":"2024-05-21T03:22:13.413140+0000","last_clean":"2024-05-21T03:22:13.413140+0000","last_became_active":"2024-05-21T03:20:32.145803+0000","last_became_peered":"2024-05-21T03:20:32.145803+0000","last_unstale":"2024-05-21T03:22:13.413140+0000","last_undegraded":"2024-05-21T03:22:13.413140+0000","last_fullsized":"2024-05-21T03:22:13.413140+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:34:09.108799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.415151+0000","last_change":"2024-05-21T03:21:14.559498+0000","last_active":"2024-05-21T03:22:13.415151+0000","last_peered":"2024-05-21T03:22:13.415151+0000","last_clean":"2024-05-21T03:22:13.415151+0000","last_became_active":"2024-05-21T03:21:14.559039+0000","last_became_peered":"2024-05-21T03:21:14.559039+0000","last_unstale":"2024-05-21T03:22:13.415151+0000","last_undegraded":"2024-05-21T03:22:13.415151+0000","last_fullsized":"2024-05-21T03:22:13.415151+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:04:36.676857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.405267+0000","last_change":"2024-05-21T03:20:32.148944+0000","last_active":"2024-05-21T03:22:13.405267+0000","last_peered":"2024-05-21T03:22:13.405267+0000","last_clean":"2024-05-21T03:22:13.405267+0000","last_became_active":"2024-05-21T03:20:32.148502+0000","last_became_peered":"2024-05-21T03:20:32.148502+0000","last_unstale":"2024-05-21T03:22:13.405267+0000","last_undegraded":"2024-05-21T03:22:13.405267+0000","last_fullsized":"2024-05-21T03:22:13.405267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:11:00.178105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.404451+0000","last_change":"2024-05-21T03:20:32.150609+0000","last_active":"2024-05-21T03:22:13.404451+0000","last_peered":"2024-05-21T03:22:13.404451+0000","last_clean":"2024-05-21T03:22:13.404451+0000","last_became_active":"2024-05-21T03:20:32.150314+0000","last_became_peered":"2024-05-21T03:20:32.150314+0000","last_unstale":"2024-05-21T03:22:13.404451+0000","last_undegraded":"2024-05-21T03:22:13.404451+0000","last_fullsized":"2024-05-21T03:22:13.404451+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:34:21.869304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.396652+0000","last_change":"2024-05-21T03:20:32.142472+0000","last_active":"2024-05-21T03:22:13.396652+0000","last_peered":"2024-05-21T03:22:13.396652+0000","last_clean":"2024-05-21T03:22:13.396652+0000","last_became_active":"2024-05-21T03:20:32.142161+0000","last_became_peered":"2024-05-21T03:20:32.142161+0000","last_unstale":"2024-05-21T03:22:13.396652+0000","last_undegraded":"2024-05-21T03:22:13.396652+0000","last_fullsized":"2024-05-21T03:22:13.396652+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:26:57.281655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.380693+0000","last_change":"2024-05-21T03:20:32.153987+0000","last_active":"2024-05-21T03:22:13.380693+0000","last_peered":"2024-05-21T03:22:13.380693+0000","last_clean":"2024-05-21T03:22:13.380693+0000","last_became_active":"2024-05-21T03:20:32.153617+0000","last_became_peered":"2024-05-21T03:20:32.153617+0000","last_unstale":"2024-05-21T03:22:13.380693+0000","last_undegraded":"2024-05-21T03:22:13.380693+0000","last_fullsized":"2024-05-21T03:22:13.380693+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:57:07.341758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":149,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:36.645523+0000","last_change":"2024-05-21T03:20:29.099928+0000","last_active":"2024-05-21T03:22:36.645523+0000","last_peered":"2024-05-21T03:22:36.645523+0000","last_clean":"2024-05-21T03:22:36.645523+0000","last_became_active":"2024-05-21T03:20:29.099592+0000","last_became_peered":"2024-05-21T03:20:29.099592+0000","last_unstale":"2024-05-21T03:22:36.645523+0000","last_undegraded":"2024-05-21T03:22:36.645523+0000","last_fullsized":"2024-05-21T03:22:36.645523+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:35:34.843325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.401190+0000","last_change":"2024-05-21T03:20:32.154962+0000","last_active":"2024-05-21T03:22:13.401190+0000","last_peered":"2024-05-21T03:22:13.401190+0000","last_clean":"2024-05-21T03:22:13.401190+0000","last_became_active":"2024-05-21T03:20:32.154620+0000","last_became_peered":"2024-05-21T03:20:32.154620+0000","last_unstale":"2024-05-21T03:22:13.401190+0000","last_undegraded":"2024-05-21T03:22:13.401190+0000","last_fullsized":"2024-05-21T03:22:13.401190+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:56:21.553272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561791+0000","last_change":"2024-05-21T03:20:29.097312+0000","last_active":"2024-05-21T03:21:14.561791+0000","last_peered":"2024-05-21T03:21:14.561791+0000","last_clean":"2024-05-21T03:21:14.561791+0000","last_became_active":"2024-05-21T03:20:29.096977+0000","last_became_peered":"2024-05-21T03:20:29.096977+0000","last_unstale":"2024-05-21T03:21:14.561791+0000","last_undegraded":"2024-05-21T03:21:14.561791+0000","last_fullsized":"2024-05-21T03:21:14.561791+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:28:45.467446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.415671+0000","last_change":"2024-05-21T03:20:32.150035+0000","last_active":"2024-05-21T03:22:13.415671+0000","last_peered":"2024-05-21T03:22:13.415671+0000","last_clean":"2024-05-21T03:22:13.415671+0000","last_became_active":"2024-05-21T03:20:32.149688+0000","last_became_peered":"2024-05-21T03:20:32.149688+0000","last_unstale":"2024-05-21T03:22:13.415671+0000","last_undegraded":"2024-05-21T03:22:13.415671+0000","last_fullsized":"2024-05-21T03:22:13.415671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:27:00.410770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.573309+0000","last_change":"2024-05-21T03:21:14.572832+0000","last_active":"2024-05-21T03:21:14.573309+0000","last_peered":"2024-05-21T03:21:14.573309+0000","last_clean":"2024-05-21T03:21:14.573309+0000","last_became_active":"2024-05-21T03:21:14.559381+0000","last_became_peered":"2024-05-21T03:21:14.559381+0000","last_unstale":"2024-05-21T03:21:14.573309+0000","last_undegraded":"2024-05-21T03:21:14.573309+0000","last_fullsized":"2024-05-21T03:21:14.573309+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:02:53.004031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.410752+0000","last_change":"2024-05-21T03:20:32.150301+0000","last_active":"2024-05-21T03:22:13.410752+0000","last_peered":"2024-05-21T03:22:13.410752+0000","last_clean":"2024-05-21T03:22:13.410752+0000","last_became_active":"2024-05-21T03:20:32.148475+0000","last_became_peered":"2024-05-21T03:20:32.148475+0000","last_unstale":"2024-05-21T03:22:13.410752+0000","last_undegraded":"2024-05-21T03:22:13.410752+0000","last_fullsized":"2024-05-21T03:22:13.410752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:59:30.339660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556899+0000","last_change":"2024-05-21T03:20:29.098861+0000","last_active":"2024-05-21T03:21:14.556899+0000","last_peered":"2024-05-21T03:21:14.556899+0000","last_clean":"2024-05-21T03:21:14.556899+0000","last_became_active":"2024-05-21T03:20:29.097874+0000","last_became_peered":"2024-05-21T03:20:29.097874+0000","last_unstale":"2024-05-21T03:21:14.556899+0000","last_undegraded":"2024-05-21T03:21:14.556899+0000","last_fullsized":"2024-05-21T03:21:14.556899+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:25:36.227339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.371721+0000","last_change":"2024-05-21T03:20:32.142820+0000","last_active":"2024-05-21T03:22:13.371721+0000","last_peered":"2024-05-21T03:22:13.371721+0000","last_clean":"2024-05-21T03:22:13.371721+0000","last_became_active":"2024-05-21T03:20:32.142396+0000","last_became_peered":"2024-05-21T03:20:32.142396+0000","last_unstale":"2024-05-21T03:22:13.371721+0000","last_undegraded":"2024-05-21T03:22:13.371721+0000","last_fullsized":"2024-05-21T03:22:13.371721+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:30:14.452389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":140,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:35.629032+0000","last_change":"2024-05-21T03:20:29.103153+0000","last_active":"2024-05-21T03:22:35.629032+0000","last_peered":"2024-05-21T03:22:35.629032+0000","last_clean":"2024-05-21T03:22:35.629032+0000","last_became_active":"2024-05-21T03:20:29.102819+0000","last_became_peered":"2024-05-21T03:20:29.102819+0000","last_unstale":"2024-05-21T03:22:35.629032+0000","last_undegraded":"2024-05-21T03:22:35.629032+0000","last_fullsized":"2024-05-21T03:22:35.629032+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:47:49.843926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.366995+0000","last_change":"2024-05-21T03:20:32.142263+0000","last_active":"2024-05-21T03:22:13.366995+0000","last_peered":"2024-05-21T03:22:13.366995+0000","last_clean":"2024-05-21T03:22:13.366995+0000","last_became_active":"2024-05-21T03:20:32.141820+0000","last_became_peered":"2024-05-21T03:20:32.141820+0000","last_unstale":"2024-05-21T03:22:13.366995+0000","last_undegraded":"2024-05-21T03:22:13.366995+0000","last_fullsized":"2024-05-21T03:22:13.366995+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:45:20.021237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.554011+0000","last_change":"2024-05-21T03:20:29.100088+0000","last_active":"2024-05-21T03:21:14.554011+0000","last_peered":"2024-05-21T03:21:14.554011+0000","last_clean":"2024-05-21T03:21:14.554011+0000","last_became_active":"2024-05-21T03:20:29.097343+0000","last_became_peered":"2024-05-21T03:20:29.097343+0000","last_unstale":"2024-05-21T03:21:14.554011+0000","last_undegraded":"2024-05-21T03:21:14.554011+0000","last_fullsized":"2024-05-21T03:21:14.554011+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:00:35.682393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.408104+0000","last_change":"2024-05-21T03:21:14.562281+0000","last_active":"2024-05-21T03:22:13.408104+0000","last_peered":"2024-05-21T03:22:13.408104+0000","last_clean":"2024-05-21T03:22:13.408104+0000","last_became_active":"2024-05-21T03:21:14.561857+0000","last_became_peered":"2024-05-21T03:21:14.561857+0000","last_unstale":"2024-05-21T03:22:13.408104+0000","last_undegraded":"2024-05-21T03:22:13.408104+0000","last_fullsized":"2024-05-21T03:22:13.408104+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:37:35.283529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552729+0000","last_change":"2024-05-21T03:20:29.097945+0000","last_active":"2024-05-21T03:21:14.552729+0000","last_peered":"2024-05-21T03:21:14.552729+0000","last_clean":"2024-05-21T03:21:14.552729+0000","last_became_active":"2024-05-21T03:20:29.097624+0000","last_became_peered":"2024-05-21T03:20:29.097624+0000","last_unstale":"2024-05-21T03:21:14.552729+0000","last_undegraded":"2024-05-21T03:21:14.552729+0000","last_fullsized":"2024-05-21T03:21:14.552729+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:01:01.694893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.411465+0000","last_change":"2024-05-21T03:20:32.148955+0000","last_active":"2024-05-21T03:22:13.411465+0000","last_peered":"2024-05-21T03:22:13.411465+0000","last_clean":"2024-05-21T03:22:13.411465+0000","last_became_active":"2024-05-21T03:20:32.148631+0000","last_became_peered":"2024-05-21T03:20:32.148631+0000","last_unstale":"2024-05-21T03:22:13.411465+0000","last_undegraded":"2024-05-21T03:22:13.411465+0000","last_fullsized":"2024-05-21T03:22:13.411465+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:36:25.820580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561173+0000","last_change":"2024-05-21T03:20:29.100902+0000","last_active":"2024-05-21T03:21:14.561173+0000","last_peered":"2024-05-21T03:21:14.561173+0000","last_clean":"2024-05-21T03:21:14.561173+0000","last_became_active":"2024-05-21T03:20:29.100570+0000","last_became_peered":"2024-05-21T03:20:29.100570+0000","last_unstale":"2024-05-21T03:21:14.561173+0000","last_undegraded":"2024-05-21T03:21:14.561173+0000","last_fullsized":"2024-05-21T03:21:14.561173+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:58:36.651338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.406823+0000","last_change":"2024-05-21T03:20:32.150724+0000","last_active":"2024-05-21T03:22:13.406823+0000","last_peered":"2024-05-21T03:22:13.406823+0000","last_clean":"2024-05-21T03:22:13.406823+0000","last_became_active":"2024-05-21T03:20:32.150371+0000","last_became_peered":"2024-05-21T03:20:32.150371+0000","last_unstale":"2024-05-21T03:22:13.406823+0000","last_undegraded":"2024-05-21T03:22:13.406823+0000","last_fullsized":"2024-05-21T03:22:13.406823+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:44:32.321192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'5","reported_seq":247,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:37.631094+0000","last_change":"2024-05-21T03:20:29.095045+0000","last_active":"2024-05-21T03:22:37.631094+0000","last_peered":"2024-05-21T03:22:37.631094+0000","last_clean":"2024-05-21T03:22:37.631094+0000","last_became_active":"2024-05-21T03:20:29.094552+0000","last_became_peered":"2024-05-21T03:20:29.094552+0000","last_unstale":"2024-05-21T03:22:37.631094+0000","last_undegraded":"2024-05-21T03:22:37.631094+0000","last_fullsized":"2024-05-21T03:22:37.631094+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:36:15.155125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":216,"num_read_kb":216,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.411077+0000","last_change":"2024-05-21T03:20:32.149813+0000","last_active":"2024-05-21T03:22:13.411077+0000","last_peered":"2024-05-21T03:22:13.411077+0000","last_clean":"2024-05-21T03:22:13.411077+0000","last_became_active":"2024-05-21T03:20:32.148422+0000","last_became_peered":"2024-05-21T03:20:32.148422+0000","last_unstale":"2024-05-21T03:22:13.411077+0000","last_undegraded":"2024-05-21T03:22:13.411077+0000","last_fullsized":"2024-05-21T03:22:13.411077+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:55:35.115463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558265+0000","last_change":"2024-05-21T03:20:29.101463+0000","last_active":"2024-05-21T03:21:14.558265+0000","last_peered":"2024-05-21T03:21:14.558265+0000","last_clean":"2024-05-21T03:21:14.558265+0000","last_became_active":"2024-05-21T03:20:29.101127+0000","last_became_peered":"2024-05-21T03:20:29.101127+0000","last_unstale":"2024-05-21T03:21:14.558265+0000","last_undegraded":"2024-05-21T03:21:14.558265+0000","last_fullsized":"2024-05-21T03:21:14.558265+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:58:17.408315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.411937+0000","last_change":"2024-05-21T03:21:14.563434+0000","last_active":"2024-05-21T03:22:13.411937+0000","last_peered":"2024-05-21T03:22:13.411937+0000","last_clean":"2024-05-21T03:22:13.411937+0000","last_became_active":"2024-05-21T03:21:14.563081+0000","last_became_peered":"2024-05-21T03:21:14.563081+0000","last_unstale":"2024-05-21T03:22:13.411937+0000","last_undegraded":"2024-05-21T03:22:13.411937+0000","last_fullsized":"2024-05-21T03:22:13.411937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:28:55.497338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.562216+0000","last_change":"2024-05-21T03:20:29.098725+0000","last_active":"2024-05-21T03:21:14.562216+0000","last_peered":"2024-05-21T03:21:14.562216+0000","last_clean":"2024-05-21T03:21:14.562216+0000","last_became_active":"2024-05-21T03:20:29.098471+0000","last_became_peered":"2024-05-21T03:20:29.098471+0000","last_unstale":"2024-05-21T03:21:14.562216+0000","last_undegraded":"2024-05-21T03:21:14.562216+0000","last_fullsized":"2024-05-21T03:21:14.562216+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:17:11.059825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.388349+0000","last_change":"2024-05-21T03:21:14.560975+0000","last_active":"2024-05-21T03:22:13.388349+0000","last_peered":"2024-05-21T03:22:13.388349+0000","last_clean":"2024-05-21T03:22:13.388349+0000","last_became_active":"2024-05-21T03:21:14.560528+0000","last_became_peered":"2024-05-21T03:21:14.560528+0000","last_unstale":"2024-05-21T03:22:13.388349+0000","last_undegraded":"2024-05-21T03:22:13.388349+0000","last_fullsized":"2024-05-21T03:22:13.388349+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:51:47.469291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'34","reported_seq":286,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:37.631033+0000","last_change":"2024-05-21T03:20:29.099968+0000","last_active":"2024-05-21T03:22:37.631033+0000","last_peered":"2024-05-21T03:22:37.631033+0000","last_clean":"2024-05-21T03:22:37.631033+0000","last_became_active":"2024-05-21T03:20:29.099181+0000","last_became_peered":"2024-05-21T03:20:29.099181+0000","last_unstale":"2024-05-21T03:22:37.631033+0000","last_undegraded":"2024-05-21T03:22:37.631033+0000","last_fullsized":"2024-05-21T03:22:37.631033+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:35:33.934412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60011,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":226,"num_read_kb":171,"num_write":36,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.381167+0000","last_change":"2024-05-21T03:20:32.144024+0000","last_active":"2024-05-21T03:22:13.381167+0000","last_peered":"2024-05-21T03:22:13.381167+0000","last_clean":"2024-05-21T03:22:13.381167+0000","last_became_active":"2024-05-21T03:20:32.143668+0000","last_became_peered":"2024-05-21T03:20:32.143668+0000","last_unstale":"2024-05-21T03:22:13.381167+0000","last_undegraded":"2024-05-21T03:22:13.381167+0000","last_fullsized":"2024-05-21T03:22:13.381167+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:11:31.100802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556933+0000","last_change":"2024-05-21T03:20:29.101770+0000","last_active":"2024-05-21T03:21:14.556933+0000","last_peered":"2024-05-21T03:21:14.556933+0000","last_clean":"2024-05-21T03:21:14.556933+0000","last_became_active":"2024-05-21T03:20:29.100407+0000","last_became_peered":"2024-05-21T03:20:29.100407+0000","last_unstale":"2024-05-21T03:21:14.556933+0000","last_undegraded":"2024-05-21T03:21:14.556933+0000","last_fullsized":"2024-05-21T03:21:14.556933+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:01:33.991246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.415145+0000","last_change":"2024-05-21T03:20:32.147479+0000","last_active":"2024-05-21T03:22:13.415145+0000","last_peered":"2024-05-21T03:22:13.415145+0000","last_clean":"2024-05-21T03:22:13.415145+0000","last_became_active":"2024-05-21T03:20:32.147160+0000","last_became_peered":"2024-05-21T03:20:32.147160+0000","last_unstale":"2024-05-21T03:22:13.415145+0000","last_undegraded":"2024-05-21T03:22:13.415145+0000","last_fullsized":"2024-05-21T03:22:13.415145+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:51:59.343380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.562216+0000","last_change":"2024-05-21T03:20:29.101875+0000","last_active":"2024-05-21T03:21:14.562216+0000","last_peered":"2024-05-21T03:21:14.562216+0000","last_clean":"2024-05-21T03:21:14.562216+0000","last_became_active":"2024-05-21T03:20:29.101468+0000","last_became_peered":"2024-05-21T03:20:29.101468+0000","last_unstale":"2024-05-21T03:21:14.562216+0000","last_undegraded":"2024-05-21T03:21:14.562216+0000","last_fullsized":"2024-05-21T03:21:14.562216+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:10:19.577836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.410660+0000","last_change":"2024-05-21T03:20:32.139002+0000","last_active":"2024-05-21T03:22:13.410660+0000","last_peered":"2024-05-21T03:22:13.410660+0000","last_clean":"2024-05-21T03:22:13.410660+0000","last_became_active":"2024-05-21T03:20:32.138619+0000","last_became_peered":"2024-05-21T03:20:32.138619+0000","last_unstale":"2024-05-21T03:22:13.410660+0000","last_undegraded":"2024-05-21T03:22:13.410660+0000","last_fullsized":"2024-05-21T03:22:13.410660+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:21:36.136283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":143,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:35.628592+0000","last_change":"2024-05-21T03:20:29.101532+0000","last_active":"2024-05-21T03:22:35.628592+0000","last_peered":"2024-05-21T03:22:35.628592+0000","last_clean":"2024-05-21T03:22:35.628592+0000","last_became_active":"2024-05-21T03:20:29.101206+0000","last_became_peered":"2024-05-21T03:20:29.101206+0000","last_unstale":"2024-05-21T03:22:35.628592+0000","last_undegraded":"2024-05-21T03:22:35.628592+0000","last_fullsized":"2024-05-21T03:22:35.628592+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:47:38.555995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.385079+0000","last_change":"2024-05-21T03:20:32.146866+0000","last_active":"2024-05-21T03:22:13.385079+0000","last_peered":"2024-05-21T03:22:13.385079+0000","last_clean":"2024-05-21T03:22:13.385079+0000","last_became_active":"2024-05-21T03:20:32.146463+0000","last_became_peered":"2024-05-21T03:20:32.146463+0000","last_unstale":"2024-05-21T03:22:13.385079+0000","last_undegraded":"2024-05-21T03:22:13.385079+0000","last_fullsized":"2024-05-21T03:22:13.385079+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:06:54.359922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556379+0000","last_change":"2024-05-21T03:20:29.103910+0000","last_active":"2024-05-21T03:21:14.556379+0000","last_peered":"2024-05-21T03:21:14.556379+0000","last_clean":"2024-05-21T03:21:14.556379+0000","last_became_active":"2024-05-21T03:20:29.103583+0000","last_became_peered":"2024-05-21T03:20:29.103583+0000","last_unstale":"2024-05-21T03:21:14.556379+0000","last_undegraded":"2024-05-21T03:21:14.556379+0000","last_fullsized":"2024-05-21T03:21:14.556379+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:57:15.665158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.366845+0000","last_change":"2024-05-21T03:20:32.141242+0000","last_active":"2024-05-21T03:22:13.366845+0000","last_peered":"2024-05-21T03:22:13.366845+0000","last_clean":"2024-05-21T03:22:13.366845+0000","last_became_active":"2024-05-21T03:20:32.140892+0000","last_became_peered":"2024-05-21T03:20:32.140892+0000","last_unstale":"2024-05-21T03:22:13.366845+0000","last_undegraded":"2024-05-21T03:22:13.366845+0000","last_fullsized":"2024-05-21T03:22:13.366845+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:42:13.997708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.557026+0000","last_change":"2024-05-21T03:20:29.098973+0000","last_active":"2024-05-21T03:21:14.557026+0000","last_peered":"2024-05-21T03:21:14.557026+0000","last_clean":"2024-05-21T03:21:14.557026+0000","last_became_active":"2024-05-21T03:20:29.098694+0000","last_became_peered":"2024-05-21T03:20:29.098694+0000","last_unstale":"2024-05-21T03:21:14.557026+0000","last_undegraded":"2024-05-21T03:21:14.557026+0000","last_fullsized":"2024-05-21T03:21:14.557026+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:55:17.915820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.376336+0000","last_change":"2024-05-21T03:20:32.147465+0000","last_active":"2024-05-21T03:22:13.376336+0000","last_peered":"2024-05-21T03:22:13.376336+0000","last_clean":"2024-05-21T03:22:13.376336+0000","last_became_active":"2024-05-21T03:20:32.147073+0000","last_became_peered":"2024-05-21T03:20:32.147073+0000","last_unstale":"2024-05-21T03:22:13.376336+0000","last_undegraded":"2024-05-21T03:22:13.376336+0000","last_fullsized":"2024-05-21T03:22:13.376336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:29:05.451370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561052+0000","last_change":"2024-05-21T03:21:14.560568+0000","last_active":"2024-05-21T03:21:14.561052+0000","last_peered":"2024-05-21T03:21:14.561052+0000","last_clean":"2024-05-21T03:21:14.561052+0000","last_became_active":"2024-05-21T03:21:14.560093+0000","last_became_peered":"2024-05-21T03:21:14.560093+0000","last_unstale":"2024-05-21T03:21:14.561052+0000","last_undegraded":"2024-05-21T03:21:14.561052+0000","last_fullsized":"2024-05-21T03:21:14.561052+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:55:39.839942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.410846+0000","last_change":"2024-05-21T03:20:32.146456+0000","last_active":"2024-05-21T03:22:13.410846+0000","last_peered":"2024-05-21T03:22:13.410846+0000","last_clean":"2024-05-21T03:22:13.410846+0000","last_became_active":"2024-05-21T03:20:32.146105+0000","last_became_peered":"2024-05-21T03:20:32.146105+0000","last_unstale":"2024-05-21T03:22:13.410846+0000","last_undegraded":"2024-05-21T03:22:13.410846+0000","last_fullsized":"2024-05-21T03:22:13.410846+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:00:30.550178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'7","reported_seq":143,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:36.645969+0000","last_change":"2024-05-21T03:20:29.100262+0000","last_active":"2024-05-21T03:22:36.645969+0000","last_peered":"2024-05-21T03:22:36.645969+0000","last_clean":"2024-05-21T03:22:36.645969+0000","last_became_active":"2024-05-21T03:20:29.099917+0000","last_became_peered":"2024-05-21T03:20:29.099917+0000","last_unstale":"2024-05-21T03:22:36.645969+0000","last_undegraded":"2024-05-21T03:22:36.645969+0000","last_fullsized":"2024-05-21T03:22:36.645969+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:55:48.608341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.380411+0000","last_change":"2024-05-21T03:20:32.142776+0000","last_active":"2024-05-21T03:22:13.380411+0000","last_peered":"2024-05-21T03:22:13.380411+0000","last_clean":"2024-05-21T03:22:13.380411+0000","last_became_active":"2024-05-21T03:20:32.142430+0000","last_became_peered":"2024-05-21T03:20:32.142430+0000","last_unstale":"2024-05-21T03:22:13.380411+0000","last_undegraded":"2024-05-21T03:22:13.380411+0000","last_fullsized":"2024-05-21T03:22:13.380411+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:54:28.682343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.375317+0000","last_change":"2024-05-21T03:20:32.139506+0000","last_active":"2024-05-21T03:22:13.375317+0000","last_peered":"2024-05-21T03:22:13.375317+0000","last_clean":"2024-05-21T03:22:13.375317+0000","last_became_active":"2024-05-21T03:20:32.138136+0000","last_became_peered":"2024-05-21T03:20:32.138136+0000","last_unstale":"2024-05-21T03:22:13.375317+0000","last_undegraded":"2024-05-21T03:22:13.375317+0000","last_fullsized":"2024-05-21T03:22:13.375317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:31:24.996276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":140,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:35.628033+0000","last_change":"2024-05-21T03:20:29.096546+0000","last_active":"2024-05-21T03:22:35.628033+0000","last_peered":"2024-05-21T03:22:35.628033+0000","last_clean":"2024-05-21T03:22:35.628033+0000","last_became_active":"2024-05-21T03:20:29.096184+0000","last_became_peered":"2024-05-21T03:20:29.096184+0000","last_unstale":"2024-05-21T03:22:35.628033+0000","last_undegraded":"2024-05-21T03:22:35.628033+0000","last_fullsized":"2024-05-21T03:22:35.628033+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:09:12.832745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":107,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.386503+0000","last_change":"2024-05-21T03:20:32.156231+0000","last_active":"2024-05-21T03:22:13.386503+0000","last_peered":"2024-05-21T03:22:13.386503+0000","last_clean":"2024-05-21T03:22:13.386503+0000","last_became_active":"2024-05-21T03:20:32.154035+0000","last_became_peered":"2024-05-21T03:20:32.154035+0000","last_unstale":"2024-05-21T03:22:13.386503+0000","last_undegraded":"2024-05-21T03:22:13.386503+0000","last_fullsized":"2024-05-21T03:22:13.386503+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:30:53.820292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556193+0000","last_change":"2024-05-21T03:20:29.100422+0000","last_active":"2024-05-21T03:21:14.556193+0000","last_peered":"2024-05-21T03:21:14.556193+0000","last_clean":"2024-05-21T03:21:14.556193+0000","last_became_active":"2024-05-21T03:20:29.099993+0000","last_became_peered":"2024-05-21T03:20:29.099993+0000","last_unstale":"2024-05-21T03:21:14.556193+0000","last_undegraded":"2024-05-21T03:21:14.556193+0000","last_fullsized":"2024-05-21T03:21:14.556193+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:03:15.456447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":99,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556851+0000","last_change":"2024-05-21T03:19:05.647808+0000","last_active":"2024-05-21T03:21:14.556851+0000","last_peered":"2024-05-21T03:21:14.556851+0000","last_clean":"2024-05-21T03:21:14.556851+0000","last_became_active":"2024-05-21T03:19:05.641865+0000","last_became_peered":"2024-05-21T03:19:05.641865+0000","last_unstale":"2024-05-21T03:21:14.556851+0000","last_undegraded":"2024-05-21T03:21:14.556851+0000","last_fullsized":"2024-05-21T03:21:14.556851+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:15:42.008493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:15:42.008493+0000","last_clean_scrub_stamp":"2024-05-21T03:15:42.008493+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:26:58.040196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.398618+0000","last_change":"2024-05-21T03:20:32.138182+0000","last_active":"2024-05-21T03:22:13.398618+0000","last_peered":"2024-05-21T03:22:13.398618+0000","last_clean":"2024-05-21T03:22:13.398618+0000","last_became_active":"2024-05-21T03:20:32.137842+0000","last_became_peered":"2024-05-21T03:20:32.137842+0000","last_unstale":"2024-05-21T03:22:13.398618+0000","last_undegraded":"2024-05-21T03:22:13.398618+0000","last_fullsized":"2024-05-21T03:22:13.398618+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:43:50.070160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.557047+0000","last_change":"2024-05-21T03:20:29.104246+0000","last_active":"2024-05-21T03:21:14.557047+0000","last_peered":"2024-05-21T03:21:14.557047+0000","last_clean":"2024-05-21T03:21:14.557047+0000","last_became_active":"2024-05-21T03:20:29.103910+0000","last_became_peered":"2024-05-21T03:20:29.103910+0000","last_unstale":"2024-05-21T03:21:14.557047+0000","last_undegraded":"2024-05-21T03:21:14.557047+0000","last_fullsized":"2024-05-21T03:21:14.557047+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:39:28.029283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556776+0000","last_change":"2024-05-21T03:20:29.102020+0000","last_active":"2024-05-21T03:21:14.556776+0000","last_peered":"2024-05-21T03:21:14.556776+0000","last_clean":"2024-05-21T03:21:14.556776+0000","last_became_active":"2024-05-21T03:20:29.100429+0000","last_became_peered":"2024-05-21T03:20:29.100429+0000","last_unstale":"2024-05-21T03:21:14.556776+0000","last_undegraded":"2024-05-21T03:21:14.556776+0000","last_fullsized":"2024-05-21T03:21:14.556776+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:09:01.621074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.418537+0000","last_change":"2024-05-21T03:20:32.137130+0000","last_active":"2024-05-21T03:22:13.418537+0000","last_peered":"2024-05-21T03:22:13.418537+0000","last_clean":"2024-05-21T03:22:13.418537+0000","last_became_active":"2024-05-21T03:20:32.136821+0000","last_became_peered":"2024-05-21T03:20:32.136821+0000","last_unstale":"2024-05-21T03:22:13.418537+0000","last_undegraded":"2024-05-21T03:22:13.418537+0000","last_fullsized":"2024-05-21T03:22:13.418537+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:53:41.290370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'15","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:14.284507+0000","last_change":"2024-05-21T03:20:29.099869+0000","last_active":"2024-05-21T03:22:14.284507+0000","last_peered":"2024-05-21T03:22:14.284507+0000","last_clean":"2024-05-21T03:22:14.284507+0000","last_became_active":"2024-05-21T03:20:29.099534+0000","last_became_peered":"2024-05-21T03:20:29.099534+0000","last_unstale":"2024-05-21T03:22:14.284507+0000","last_undegraded":"2024-05-21T03:22:14.284507+0000","last_fullsized":"2024-05-21T03:22:14.284507+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:01:11.281544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.389637+0000","last_change":"2024-05-21T03:20:32.153921+0000","last_active":"2024-05-21T03:22:13.389637+0000","last_peered":"2024-05-21T03:22:13.389637+0000","last_clean":"2024-05-21T03:22:13.389637+0000","last_became_active":"2024-05-21T03:20:32.153594+0000","last_became_peered":"2024-05-21T03:20:32.153594+0000","last_unstale":"2024-05-21T03:22:13.389637+0000","last_undegraded":"2024-05-21T03:22:13.389637+0000","last_fullsized":"2024-05-21T03:22:13.389637+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:27:59.426942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556915+0000","last_change":"2024-05-21T03:20:29.097399+0000","last_active":"2024-05-21T03:21:14.556915+0000","last_peered":"2024-05-21T03:21:14.556915+0000","last_clean":"2024-05-21T03:21:14.556915+0000","last_became_active":"2024-05-21T03:20:29.097010+0000","last_became_peered":"2024-05-21T03:20:29.097010+0000","last_unstale":"2024-05-21T03:21:14.556915+0000","last_undegraded":"2024-05-21T03:21:14.556915+0000","last_fullsized":"2024-05-21T03:21:14.556915+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:54:53.273947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.412963+0000","last_change":"2024-05-21T03:21:14.559609+0000","last_active":"2024-05-21T03:22:13.412963+0000","last_peered":"2024-05-21T03:22:13.412963+0000","last_clean":"2024-05-21T03:22:13.412963+0000","last_became_active":"2024-05-21T03:21:14.559218+0000","last_became_peered":"2024-05-21T03:21:14.559218+0000","last_unstale":"2024-05-21T03:22:13.412963+0000","last_undegraded":"2024-05-21T03:22:13.412963+0000","last_fullsized":"2024-05-21T03:22:13.412963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:32:11.156037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,4],"acting":[0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.557892+0000","last_change":"2024-05-21T03:20:29.103526+0000","last_active":"2024-05-21T03:21:14.557892+0000","last_peered":"2024-05-21T03:21:14.557892+0000","last_clean":"2024-05-21T03:21:14.557892+0000","last_became_active":"2024-05-21T03:20:29.103067+0000","last_became_peered":"2024-05-21T03:20:29.103067+0000","last_unstale":"2024-05-21T03:21:14.557892+0000","last_undegraded":"2024-05-21T03:21:14.557892+0000","last_fullsized":"2024-05-21T03:21:14.557892+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:54:47.566949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.339758+0000","last_change":"2024-05-21T03:20:32.141088+0000","last_active":"2024-05-21T03:22:13.339758+0000","last_peered":"2024-05-21T03:22:13.339758+0000","last_clean":"2024-05-21T03:22:13.339758+0000","last_became_active":"2024-05-21T03:20:32.140744+0000","last_became_peered":"2024-05-21T03:20:32.140744+0000","last_unstale":"2024-05-21T03:22:13.339758+0000","last_undegraded":"2024-05-21T03:22:13.339758+0000","last_fullsized":"2024-05-21T03:22:13.339758+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:57:35.591164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.560757+0000","last_change":"2024-05-21T03:20:29.093440+0000","last_active":"2024-05-21T03:21:14.560757+0000","last_peered":"2024-05-21T03:21:14.560757+0000","last_clean":"2024-05-21T03:21:14.560757+0000","last_became_active":"2024-05-21T03:20:29.092935+0000","last_became_peered":"2024-05-21T03:20:29.092935+0000","last_unstale":"2024-05-21T03:21:14.560757+0000","last_undegraded":"2024-05-21T03:21:14.560757+0000","last_fullsized":"2024-05-21T03:21:14.560757+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:18:00.690465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.404572+0000","last_change":"2024-05-21T03:20:32.143781+0000","last_active":"2024-05-21T03:22:13.404572+0000","last_peered":"2024-05-21T03:22:13.404572+0000","last_clean":"2024-05-21T03:22:13.404572+0000","last_became_active":"2024-05-21T03:20:32.143391+0000","last_became_peered":"2024-05-21T03:20:32.143391+0000","last_unstale":"2024-05-21T03:22:13.404572+0000","last_undegraded":"2024-05-21T03:22:13.404572+0000","last_fullsized":"2024-05-21T03:22:13.404572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:00:41.506561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556256+0000","last_change":"2024-05-21T03:20:29.101342+0000","last_active":"2024-05-21T03:21:14.556256+0000","last_peered":"2024-05-21T03:21:14.556256+0000","last_clean":"2024-05-21T03:21:14.556256+0000","last_became_active":"2024-05-21T03:20:29.100997+0000","last_became_peered":"2024-05-21T03:20:29.100997+0000","last_unstale":"2024-05-21T03:21:14.556256+0000","last_undegraded":"2024-05-21T03:21:14.556256+0000","last_fullsized":"2024-05-21T03:21:14.556256+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:05:31.801669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.373332+0000","last_change":"2024-05-21T03:20:32.150563+0000","last_active":"2024-05-21T03:22:13.373332+0000","last_peered":"2024-05-21T03:22:13.373332+0000","last_clean":"2024-05-21T03:22:13.373332+0000","last_became_active":"2024-05-21T03:20:32.150231+0000","last_became_peered":"2024-05-21T03:20:32.150231+0000","last_unstale":"2024-05-21T03:22:13.373332+0000","last_undegraded":"2024-05-21T03:22:13.373332+0000","last_fullsized":"2024-05-21T03:22:13.373332+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:10:00.251284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556609+0000","last_change":"2024-05-21T03:20:29.101576+0000","last_active":"2024-05-21T03:21:14.556609+0000","last_peered":"2024-05-21T03:21:14.556609+0000","last_clean":"2024-05-21T03:21:14.556609+0000","last_became_active":"2024-05-21T03:20:29.101270+0000","last_became_peered":"2024-05-21T03:20:29.101270+0000","last_unstale":"2024-05-21T03:21:14.556609+0000","last_undegraded":"2024-05-21T03:21:14.556609+0000","last_fullsized":"2024-05-21T03:21:14.556609+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:50:10.579901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.403247+0000","last_change":"2024-05-21T03:20:32.141340+0000","last_active":"2024-05-21T03:22:13.403247+0000","last_peered":"2024-05-21T03:22:13.403247+0000","last_clean":"2024-05-21T03:22:13.403247+0000","last_became_active":"2024-05-21T03:20:32.140792+0000","last_became_peered":"2024-05-21T03:20:32.140792+0000","last_unstale":"2024-05-21T03:22:13.403247+0000","last_undegraded":"2024-05-21T03:22:13.403247+0000","last_fullsized":"2024-05-21T03:22:13.403247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:35:05.842929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'7","reported_seq":104,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:35.680398+0000","last_change":"2024-05-21T03:21:14.566886+0000","last_active":"2024-05-21T03:22:35.680398+0000","last_peered":"2024-05-21T03:22:35.680398+0000","last_clean":"2024-05-21T03:22:35.680398+0000","last_became_active":"2024-05-21T03:21:14.559816+0000","last_became_peered":"2024-05-21T03:21:14.559816+0000","last_unstale":"2024-05-21T03:22:35.680398+0000","last_undegraded":"2024-05-21T03:22:35.680398+0000","last_fullsized":"2024-05-21T03:22:35.680398+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:34:43.109424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":89,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1561,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.389190+0000","last_change":"2024-05-21T03:20:32.146674+0000","last_active":"2024-05-21T03:22:13.389190+0000","last_peered":"2024-05-21T03:22:13.389190+0000","last_clean":"2024-05-21T03:22:13.389190+0000","last_became_active":"2024-05-21T03:20:32.146150+0000","last_became_peered":"2024-05-21T03:20:32.146150+0000","last_unstale":"2024-05-21T03:22:13.389190+0000","last_undegraded":"2024-05-21T03:22:13.389190+0000","last_fullsized":"2024-05-21T03:22:13.389190+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:39:09.905640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561667+0000","last_change":"2024-05-21T03:20:29.097613+0000","last_active":"2024-05-21T03:21:14.561667+0000","last_peered":"2024-05-21T03:21:14.561667+0000","last_clean":"2024-05-21T03:21:14.561667+0000","last_became_active":"2024-05-21T03:20:29.097331+0000","last_became_peered":"2024-05-21T03:20:29.097331+0000","last_unstale":"2024-05-21T03:21:14.561667+0000","last_undegraded":"2024-05-21T03:21:14.561667+0000","last_fullsized":"2024-05-21T03:21:14.561667+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:56:14.720420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.395548+0000","last_change":"2024-05-21T03:21:14.566406+0000","last_active":"2024-05-21T03:22:13.395548+0000","last_peered":"2024-05-21T03:22:13.395548+0000","last_clean":"2024-05-21T03:22:13.395548+0000","last_became_active":"2024-05-21T03:21:14.563023+0000","last_became_peered":"2024-05-21T03:21:14.563023+0000","last_unstale":"2024-05-21T03:22:13.395548+0000","last_undegraded":"2024-05-21T03:22:13.395548+0000","last_fullsized":"2024-05-21T03:22:13.395548+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:12:53.125136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552818+0000","last_change":"2024-05-21T03:20:29.103456+0000","last_active":"2024-05-21T03:21:14.552818+0000","last_peered":"2024-05-21T03:21:14.552818+0000","last_clean":"2024-05-21T03:21:14.552818+0000","last_became_active":"2024-05-21T03:20:29.103008+0000","last_became_peered":"2024-05-21T03:20:29.103008+0000","last_unstale":"2024-05-21T03:21:14.552818+0000","last_undegraded":"2024-05-21T03:21:14.552818+0000","last_fullsized":"2024-05-21T03:21:14.552818+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:02:37.951347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.387395+0000","last_change":"2024-05-21T03:20:32.146986+0000","last_active":"2024-05-21T03:22:13.387395+0000","last_peered":"2024-05-21T03:22:13.387395+0000","last_clean":"2024-05-21T03:22:13.387395+0000","last_became_active":"2024-05-21T03:20:32.146355+0000","last_became_peered":"2024-05-21T03:20:32.146355+0000","last_unstale":"2024-05-21T03:22:13.387395+0000","last_undegraded":"2024-05-21T03:22:13.387395+0000","last_fullsized":"2024-05-21T03:22:13.387395+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:36:29.503099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.410905+0000","last_change":"2024-05-21T03:21:14.559658+0000","last_active":"2024-05-21T03:22:13.410905+0000","last_peered":"2024-05-21T03:22:13.410905+0000","last_clean":"2024-05-21T03:22:13.410905+0000","last_became_active":"2024-05-21T03:21:14.559194+0000","last_became_peered":"2024-05-21T03:21:14.559194+0000","last_unstale":"2024-05-21T03:22:13.410905+0000","last_undegraded":"2024-05-21T03:22:13.410905+0000","last_fullsized":"2024-05-21T03:22:13.410905+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:56:31.709545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.372831+0000","last_change":"2024-05-21T03:20:32.137532+0000","last_active":"2024-05-21T03:22:13.372831+0000","last_peered":"2024-05-21T03:22:13.372831+0000","last_clean":"2024-05-21T03:22:13.372831+0000","last_became_active":"2024-05-21T03:20:32.137182+0000","last_became_peered":"2024-05-21T03:20:32.137182+0000","last_unstale":"2024-05-21T03:22:13.372831+0000","last_undegraded":"2024-05-21T03:22:13.372831+0000","last_fullsized":"2024-05-21T03:22:13.372831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:25:44.207866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.403495+0000","last_change":"2024-05-21T03:20:32.153912+0000","last_active":"2024-05-21T03:22:13.403495+0000","last_peered":"2024-05-21T03:22:13.403495+0000","last_clean":"2024-05-21T03:22:13.403495+0000","last_became_active":"2024-05-21T03:20:32.153541+0000","last_became_peered":"2024-05-21T03:20:32.153541+0000","last_unstale":"2024-05-21T03:22:13.403495+0000","last_undegraded":"2024-05-21T03:22:13.403495+0000","last_fullsized":"2024-05-21T03:22:13.403495+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:02:58.688378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.413482+0000","last_change":"2024-05-21T03:20:32.143689+0000","last_active":"2024-05-21T03:22:13.413482+0000","last_peered":"2024-05-21T03:22:13.413482+0000","last_clean":"2024-05-21T03:22:13.413482+0000","last_became_active":"2024-05-21T03:20:32.143399+0000","last_became_peered":"2024-05-21T03:20:32.143399+0000","last_unstale":"2024-05-21T03:22:13.413482+0000","last_undegraded":"2024-05-21T03:22:13.413482+0000","last_fullsized":"2024-05-21T03:22:13.413482+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:27:55.116599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.414090+0000","last_change":"2024-05-21T03:20:32.141355+0000","last_active":"2024-05-21T03:22:13.414090+0000","last_peered":"2024-05-21T03:22:13.414090+0000","last_clean":"2024-05-21T03:22:13.414090+0000","last_became_active":"2024-05-21T03:20:32.140960+0000","last_became_peered":"2024-05-21T03:20:32.140960+0000","last_unstale":"2024-05-21T03:22:13.414090+0000","last_undegraded":"2024-05-21T03:22:13.414090+0000","last_fullsized":"2024-05-21T03:22:13.414090+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:23:50.287864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.376470+0000","last_change":"2024-05-21T03:20:32.142145+0000","last_active":"2024-05-21T03:22:13.376470+0000","last_peered":"2024-05-21T03:22:13.376470+0000","last_clean":"2024-05-21T03:22:13.376470+0000","last_became_active":"2024-05-21T03:20:32.141802+0000","last_became_peered":"2024-05-21T03:20:32.141802+0000","last_unstale":"2024-05-21T03:22:13.376470+0000","last_undegraded":"2024-05-21T03:22:13.376470+0000","last_fullsized":"2024-05-21T03:22:13.376470+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:26:28.707616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.371412+0000","last_change":"2024-05-21T03:20:32.141606+0000","last_active":"2024-05-21T03:22:13.371412+0000","last_peered":"2024-05-21T03:22:13.371412+0000","last_clean":"2024-05-21T03:22:13.371412+0000","last_became_active":"2024-05-21T03:20:32.141258+0000","last_became_peered":"2024-05-21T03:20:32.141258+0000","last_unstale":"2024-05-21T03:22:13.371412+0000","last_undegraded":"2024-05-21T03:22:13.371412+0000","last_fullsized":"2024-05-21T03:22:13.371412+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:22.372723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.396785+0000","last_change":"2024-05-21T03:20:32.151621+0000","last_active":"2024-05-21T03:22:13.396785+0000","last_peered":"2024-05-21T03:22:13.396785+0000","last_clean":"2024-05-21T03:22:13.396785+0000","last_became_active":"2024-05-21T03:20:32.151260+0000","last_became_peered":"2024-05-21T03:20:32.151260+0000","last_unstale":"2024-05-21T03:22:13.396785+0000","last_undegraded":"2024-05-21T03:22:13.396785+0000","last_fullsized":"2024-05-21T03:22:13.396785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:48:59.751179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.407694+0000","last_change":"2024-05-21T03:20:32.153843+0000","last_active":"2024-05-21T03:22:13.407694+0000","last_peered":"2024-05-21T03:22:13.407694+0000","last_clean":"2024-05-21T03:22:13.407694+0000","last_became_active":"2024-05-21T03:20:32.153418+0000","last_became_peered":"2024-05-21T03:20:32.153418+0000","last_unstale":"2024-05-21T03:22:13.407694+0000","last_undegraded":"2024-05-21T03:22:13.407694+0000","last_fullsized":"2024-05-21T03:22:13.407694+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:26:29.663966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.398192+0000","last_change":"2024-05-21T03:20:32.153942+0000","last_active":"2024-05-21T03:22:13.398192+0000","last_peered":"2024-05-21T03:22:13.398192+0000","last_clean":"2024-05-21T03:22:13.398192+0000","last_became_active":"2024-05-21T03:20:32.153566+0000","last_became_peered":"2024-05-21T03:20:32.153566+0000","last_unstale":"2024-05-21T03:22:13.398192+0000","last_undegraded":"2024-05-21T03:22:13.398192+0000","last_fullsized":"2024-05-21T03:22:13.398192+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:06.899039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.404233+0000","last_change":"2024-05-21T03:20:32.153243+0000","last_active":"2024-05-21T03:22:13.404233+0000","last_peered":"2024-05-21T03:22:13.404233+0000","last_clean":"2024-05-21T03:22:13.404233+0000","last_became_active":"2024-05-21T03:20:32.152386+0000","last_became_peered":"2024-05-21T03:20:32.152386+0000","last_unstale":"2024-05-21T03:22:13.404233+0000","last_undegraded":"2024-05-21T03:22:13.404233+0000","last_fullsized":"2024-05-21T03:22:13.404233+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:25:28.935183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.401142+0000","last_change":"2024-05-21T03:20:32.142536+0000","last_active":"2024-05-21T03:22:13.401142+0000","last_peered":"2024-05-21T03:22:13.401142+0000","last_clean":"2024-05-21T03:22:13.401142+0000","last_became_active":"2024-05-21T03:20:32.141793+0000","last_became_peered":"2024-05-21T03:20:32.141793+0000","last_unstale":"2024-05-21T03:22:13.401142+0000","last_undegraded":"2024-05-21T03:22:13.401142+0000","last_fullsized":"2024-05-21T03:22:13.401142+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:21:43.547103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.372845+0000","last_change":"2024-05-21T03:20:32.140368+0000","last_active":"2024-05-21T03:22:13.372845+0000","last_peered":"2024-05-21T03:22:13.372845+0000","last_clean":"2024-05-21T03:22:13.372845+0000","last_became_active":"2024-05-21T03:20:32.139747+0000","last_became_peered":"2024-05-21T03:20:32.139747+0000","last_unstale":"2024-05-21T03:22:13.372845+0000","last_undegraded":"2024-05-21T03:22:13.372845+0000","last_fullsized":"2024-05-21T03:22:13.372845+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:05:44.851191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68029,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1096,"num_read_kb":634,"num_write":255,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1561,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":204087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645353,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":433991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841582,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1601536,"data_stored":1085031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005108,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":433506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201335,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":433767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397566,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32660,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712492,"statfs":{"total":95995035648,"available":95961591808,"internally_reserved":0,"allocated":1519616,"data_stored":1026958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561090,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":433638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":494562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":434977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":493080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149777,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":432814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953549,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":917504,"data_stored":435201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757320,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32672,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712480,"statfs":{"total":95995035648,"available":95961579520,"internally_reserved":0,"allocated":1531904,"data_stored":1025572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T03:22:40.955 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-21T03:22:41.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:41 smithi027 ceph-mon[32179]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:22:41.208 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:22:41.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:41 smithi039 ceph-mon[36429]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:22:41.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:41 smithi165 ceph-mon[36269]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:22:42.337 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:42 smithi027 ceph-mon[32179]: pgmap v356: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:22:42.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:42 smithi039 ceph-mon[36429]: pgmap v356: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:22:42.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:42 smithi165 ceph-mon[36269]: pgmap v356: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:22:42.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.695+0000 7fbf28f14700 1 -- 172.21.15.27:0/1080017683 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf24102460 msgr2=0x7fbf241028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:42.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.695+0000 7fbf28f14700 1 --2- 172.21.15.27:0/1080017683 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf24102460 0x7fbf241028e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009990 tx=0x7fbf14009ca0 comp rx=0 tx=0).stop 2024-05-21T03:22:42.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.696+0000 7fbf28f14700 1 -- 172.21.15.27:0/1080017683 shutdown_connections 2024-05-21T03:22:42.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.696+0000 7fbf28f14700 1 --2- 172.21.15.27:0/1080017683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf24102e20 0x7fbf2410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:42.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.696+0000 7fbf28f14700 1 --2- 172.21.15.27:0/1080017683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf241084f0 0x7fbf241088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:42.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.696+0000 7fbf28f14700 1 --2- 172.21.15.27:0/1080017683 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf24102460 0x7fbf241028e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:42.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.696+0000 7fbf28f14700 1 -- 172.21.15.27:0/1080017683 >> 172.21.15.27:0/1080017683 conn(0x7fbf240794e0 msgr2=0x7fbf240798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:42.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.696+0000 7fbf28f14700 1 -- 172.21.15.27:0/1080017683 shutdown_connections 2024-05-21T03:22:42.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.696+0000 7fbf28f14700 1 -- 172.21.15.27:0/1080017683 wait complete. 2024-05-21T03:22:42.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.696+0000 7fbf28f14700 1 Processor -- start 2024-05-21T03:22:42.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.697+0000 7fbf28f14700 1 -- start start 2024-05-21T03:22:42.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.697+0000 7fbf28f14700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf24102460 0x7fbf2419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.697+0000 7fbf28f14700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf24102e20 0x7fbf2419afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.697+0000 7fbf28f14700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf241084f0 0x7fbf241a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.697+0000 7fbf28f14700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbf24112630 con 0x7fbf24102e20 2024-05-21T03:22:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.697+0000 7fbf28f14700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241124b0 con 0x7fbf241084f0 2024-05-21T03:22:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.697+0000 7fbf28f14700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241127b0 con 0x7fbf24102460 2024-05-21T03:22:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.697+0000 7fbf21d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf24102e20 0x7fbf2419afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.697+0000 7fbf21d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf24102e20 0x7fbf2419afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47912/0 (socket says 172.21.15.27:47912) 2024-05-21T03:22:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.697+0000 7fbf21d9b700 1 -- 172.21.15.27:0/2870527126 learned_addr learned my addr 172.21.15.27:0/2870527126 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.697+0000 7fbf22d9d700 1 --2- 172.21.15.27:0/2870527126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf241084f0 0x7fbf241a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.697+0000 7fbf2259c700 1 --2- 172.21.15.27:0/2870527126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf24102460 0x7fbf2419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.697+0000 7fbf21d9b700 1 -- 172.21.15.27:0/2870527126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf24102460 msgr2=0x7fbf2419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.697+0000 7fbf21d9b700 1 --2- 172.21.15.27:0/2870527126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf24102460 0x7fbf2419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:42.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.697+0000 7fbf21d9b700 1 -- 172.21.15.27:0/2870527126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf241084f0 msgr2=0x7fbf241a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:42.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.697+0000 7fbf21d9b700 1 --2- 172.21.15.27:0/2870527126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf241084f0 0x7fbf241a5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:42.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.697+0000 7fbf21d9b700 1 -- 172.21.15.27:0/2870527126 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf241a6030 con 0x7fbf24102e20 2024-05-21T03:22:42.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.697+0000 7fbf2259c700 1 --2- 172.21.15.27:0/2870527126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf24102460 0x7fbf2419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:22:42.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.698+0000 7fbf21d9b700 1 --2- 172.21.15.27:0/2870527126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf24102e20 0x7fbf2419afb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbf14000c00 tx=0x7fbf140047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:42.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.698+0000 7fbf1b7fe700 1 -- 172.21.15.27:0/2870527126 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf14027070 con 0x7fbf24102e20 2024-05-21T03:22:42.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.698+0000 7fbf28f14700 1 -- 172.21.15.27:0/2870527126 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf241a6200 con 0x7fbf24102e20 2024-05-21T03:22:42.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.698+0000 7fbf1b7fe700 1 -- 172.21.15.27:0/2870527126 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf14003f20 con 0x7fbf24102e20 2024-05-21T03:22:42.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.698+0000 7fbf1b7fe700 1 -- 172.21.15.27:0/2870527126 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf14022cf0 con 0x7fbf24102e20 2024-05-21T03:22:42.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.698+0000 7fbf28f14700 1 -- 172.21.15.27:0/2870527126 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf241a66b0 con 0x7fbf24102e20 2024-05-21T03:22:42.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.699+0000 7fbf28f14700 1 -- 172.21.15.27:0/2870527126 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf2406a230 con 0x7fbf24102e20 2024-05-21T03:22:42.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.700+0000 7fbf1b7fe700 1 -- 172.21.15.27:0/2870527126 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbf1402a070 con 0x7fbf24102e20 2024-05-21T03:22:42.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.701+0000 7fbf1b7fe700 1 --2- 172.21.15.27:0/2870527126 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fbf100b2880 0x7fbf100b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:42.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.702+0000 7fbf2259c700 1 --2- 172.21.15.27:0/2870527126 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fbf100b2880 0x7fbf100b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:42.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.702+0000 7fbf1b7fe700 1 -- 172.21.15.27:0/2870527126 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fbf140e3320 con 0x7fbf24102e20 2024-05-21T03:22:42.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.702+0000 7fbf2259c700 1 --2- 172.21.15.27:0/2870527126 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fbf100b2880 0x7fbf100b4d40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c005e10 tx=0x7fbf0c005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:42.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.705+0000 7fbf1b7fe700 1 -- 172.21.15.27:0/2870527126 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbf14013070 con 0x7fbf24102e20 2024-05-21T03:22:42.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.914+0000 7fbf28f14700 1 -- 172.21.15.27:0/2870527126 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbf2410a9e0 con 0x7fbf100b2880 2024-05-21T03:22:42.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.921+0000 7fbf1b7fe700 1 -- 172.21.15.27:0/2870527126 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258887 (secure 0 0 0) 0x7fbf2410a9e0 con 0x7fbf100b2880 2024-05-21T03:22:42.924 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:22:42.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.925+0000 7fbf28f14700 1 -- 172.21.15.27:0/2870527126 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fbf100b2880 msgr2=0x7fbf100b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:42.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.925+0000 7fbf28f14700 1 --2- 172.21.15.27:0/2870527126 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fbf100b2880 0x7fbf100b4d40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c005e10 tx=0x7fbf0c005d60 comp rx=0 tx=0).stop 2024-05-21T03:22:42.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.925+0000 7fbf28f14700 1 -- 172.21.15.27:0/2870527126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf24102e20 msgr2=0x7fbf2419afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:42.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.925+0000 7fbf28f14700 1 --2- 172.21.15.27:0/2870527126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf24102e20 0x7fbf2419afb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbf14000c00 tx=0x7fbf140047b0 comp rx=0 tx=0).stop 2024-05-21T03:22:42.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.925+0000 7fbf28f14700 1 -- 172.21.15.27:0/2870527126 shutdown_connections 2024-05-21T03:22:42.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.925+0000 7fbf28f14700 1 --2- 172.21.15.27:0/2870527126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf241084f0 0x7fbf241a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:42.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.925+0000 7fbf28f14700 1 --2- 172.21.15.27:0/2870527126 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fbf100b2880 0x7fbf100b4d40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:42.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.925+0000 7fbf28f14700 1 --2- 172.21.15.27:0/2870527126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf24102460 0x7fbf2419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:42.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.925+0000 7fbf28f14700 1 --2- 172.21.15.27:0/2870527126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf24102e20 0x7fbf2419afb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:42.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.925+0000 7fbf28f14700 1 -- 172.21.15.27:0/2870527126 >> 172.21.15.27:0/2870527126 conn(0x7fbf240794e0 msgr2=0x7fbf24105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:42.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.925+0000 7fbf28f14700 1 -- 172.21.15.27:0/2870527126 shutdown_connections 2024-05-21T03:22:42.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:42.925+0000 7fbf28f14700 1 -- 172.21.15.27:0/2870527126 wait complete. 2024-05-21T03:22:42.926 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-05-21T03:22:43.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:43 smithi027 ceph-mon[32179]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:22:43.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:43 smithi039 ceph-mon[36429]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:22:43.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:43 smithi165 ceph-mon[36269]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:22:43.588 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2024-05-21T03:22:41.370697+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658631,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1182,"num_read_kb":708,"num_write":380,"num_write_kb":2296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2953518,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":814,"ondisk_log_size":814,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357724,"kb_used_data":12732,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584100,"statfs":{"total":1151940427776,"available":1151574118400,"internally_reserved":0,"allocated":13037568,"data_stored":7163097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19139,"internal_metadata":353154365},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002846"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.404523+0000","last_change":"2024-05-21T03:20:32.149961+0000","last_active":"2024-05-21T03:22:13.404523+0000","last_peered":"2024-05-21T03:22:13.404523+0000","last_clean":"2024-05-21T03:22:13.404523+0000","last_became_active":"2024-05-21T03:20:32.149552+0000","last_became_peered":"2024-05-21T03:20:32.149552+0000","last_unstale":"2024-05-21T03:22:13.404523+0000","last_undegraded":"2024-05-21T03:22:13.404523+0000","last_fullsized":"2024-05-21T03:22:13.404523+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:20:38.543551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.410926+0000","last_change":"2024-05-21T03:20:32.146413+0000","last_active":"2024-05-21T03:22:13.410926+0000","last_peered":"2024-05-21T03:22:13.410926+0000","last_clean":"2024-05-21T03:22:13.410926+0000","last_became_active":"2024-05-21T03:20:32.146006+0000","last_became_peered":"2024-05-21T03:20:32.146006+0000","last_unstale":"2024-05-21T03:22:13.410926+0000","last_undegraded":"2024-05-21T03:22:13.410926+0000","last_fullsized":"2024-05-21T03:22:13.410926+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:05:12.888550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.416235+0000","last_change":"2024-05-21T03:20:32.153654+0000","last_active":"2024-05-21T03:22:13.416235+0000","last_peered":"2024-05-21T03:22:13.416235+0000","last_clean":"2024-05-21T03:22:13.416235+0000","last_became_active":"2024-05-21T03:20:32.153296+0000","last_became_peered":"2024-05-21T03:20:32.153296+0000","last_unstale":"2024-05-21T03:22:13.416235+0000","last_undegraded":"2024-05-21T03:22:13.416235+0000","last_fullsized":"2024-05-21T03:22:13.416235+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:16:09.423236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.372040+0000","last_change":"2024-05-21T03:20:32.138672+0000","last_active":"2024-05-21T03:22:13.372040+0000","last_peered":"2024-05-21T03:22:13.372040+0000","last_clean":"2024-05-21T03:22:13.372040+0000","last_became_active":"2024-05-21T03:20:32.138163+0000","last_became_peered":"2024-05-21T03:20:32.138163+0000","last_unstale":"2024-05-21T03:22:13.372040+0000","last_undegraded":"2024-05-21T03:22:13.372040+0000","last_fullsized":"2024-05-21T03:22:13.372040+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:26:55.012897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.380094+0000","last_change":"2024-05-21T03:20:32.153950+0000","last_active":"2024-05-21T03:22:13.380094+0000","last_peered":"2024-05-21T03:22:13.380094+0000","last_clean":"2024-05-21T03:22:13.380094+0000","last_became_active":"2024-05-21T03:20:32.153567+0000","last_became_peered":"2024-05-21T03:20:32.153567+0000","last_unstale":"2024-05-21T03:22:13.380094+0000","last_undegraded":"2024-05-21T03:22:13.380094+0000","last_fullsized":"2024-05-21T03:22:13.380094+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:28:34.621193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.395805+0000","last_change":"2024-05-21T03:20:32.140038+0000","last_active":"2024-05-21T03:22:13.395805+0000","last_peered":"2024-05-21T03:22:13.395805+0000","last_clean":"2024-05-21T03:22:13.395805+0000","last_became_active":"2024-05-21T03:20:32.139690+0000","last_became_peered":"2024-05-21T03:20:32.139690+0000","last_unstale":"2024-05-21T03:22:13.395805+0000","last_undegraded":"2024-05-21T03:22:13.395805+0000","last_fullsized":"2024-05-21T03:22:13.395805+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:31:24.270755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.387191+0000","last_change":"2024-05-21T03:20:32.143735+0000","last_active":"2024-05-21T03:22:13.387191+0000","last_peered":"2024-05-21T03:22:13.387191+0000","last_clean":"2024-05-21T03:22:13.387191+0000","last_became_active":"2024-05-21T03:20:32.143397+0000","last_became_peered":"2024-05-21T03:20:32.143397+0000","last_unstale":"2024-05-21T03:22:13.387191+0000","last_undegraded":"2024-05-21T03:22:13.387191+0000","last_fullsized":"2024-05-21T03:22:13.387191+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:40:01.951608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.402887+0000","last_change":"2024-05-21T03:20:32.156999+0000","last_active":"2024-05-21T03:22:13.402887+0000","last_peered":"2024-05-21T03:22:13.402887+0000","last_clean":"2024-05-21T03:22:13.402887+0000","last_became_active":"2024-05-21T03:20:32.156677+0000","last_became_peered":"2024-05-21T03:20:32.156677+0000","last_unstale":"2024-05-21T03:22:13.402887+0000","last_undegraded":"2024-05-21T03:22:13.402887+0000","last_fullsized":"2024-05-21T03:22:13.402887+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:33:24.225667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.384207+0000","last_change":"2024-05-21T03:20:32.145039+0000","last_active":"2024-05-21T03:22:13.384207+0000","last_peered":"2024-05-21T03:22:13.384207+0000","last_clean":"2024-05-21T03:22:13.384207+0000","last_became_active":"2024-05-21T03:20:32.140926+0000","last_became_peered":"2024-05-21T03:20:32.140926+0000","last_unstale":"2024-05-21T03:22:13.384207+0000","last_undegraded":"2024-05-21T03:22:13.384207+0000","last_fullsized":"2024-05-21T03:22:13.384207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:11:27.049426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.371393+0000","last_change":"2024-05-21T03:20:32.145840+0000","last_active":"2024-05-21T03:22:13.371393+0000","last_peered":"2024-05-21T03:22:13.371393+0000","last_clean":"2024-05-21T03:22:13.371393+0000","last_became_active":"2024-05-21T03:20:32.145291+0000","last_became_peered":"2024-05-21T03:20:32.145291+0000","last_unstale":"2024-05-21T03:22:13.371393+0000","last_undegraded":"2024-05-21T03:22:13.371393+0000","last_fullsized":"2024-05-21T03:22:13.371393+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:22:27.696588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.389055+0000","last_change":"2024-05-21T03:20:32.142522+0000","last_active":"2024-05-21T03:22:13.389055+0000","last_peered":"2024-05-21T03:22:13.389055+0000","last_clean":"2024-05-21T03:22:13.389055+0000","last_became_active":"2024-05-21T03:20:32.142165+0000","last_became_peered":"2024-05-21T03:20:32.142165+0000","last_unstale":"2024-05-21T03:22:13.389055+0000","last_undegraded":"2024-05-21T03:22:13.389055+0000","last_fullsized":"2024-05-21T03:22:13.389055+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:58:39.085834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.371212+0000","last_change":"2024-05-21T03:20:32.142526+0000","last_active":"2024-05-21T03:22:13.371212+0000","last_peered":"2024-05-21T03:22:13.371212+0000","last_clean":"2024-05-21T03:22:13.371212+0000","last_became_active":"2024-05-21T03:20:32.141143+0000","last_became_peered":"2024-05-21T03:20:32.141143+0000","last_unstale":"2024-05-21T03:22:13.371212+0000","last_undegraded":"2024-05-21T03:22:13.371212+0000","last_fullsized":"2024-05-21T03:22:13.371212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:35:11.336796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.408485+0000","last_change":"2024-05-21T03:20:32.135631+0000","last_active":"2024-05-21T03:22:13.408485+0000","last_peered":"2024-05-21T03:22:13.408485+0000","last_clean":"2024-05-21T03:22:13.408485+0000","last_became_active":"2024-05-21T03:20:32.135302+0000","last_became_peered":"2024-05-21T03:20:32.135302+0000","last_unstale":"2024-05-21T03:22:13.408485+0000","last_undegraded":"2024-05-21T03:22:13.408485+0000","last_fullsized":"2024-05-21T03:22:13.408485+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:05:55.000690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.413140+0000","last_change":"2024-05-21T03:20:32.146580+0000","last_active":"2024-05-21T03:22:13.413140+0000","last_peered":"2024-05-21T03:22:13.413140+0000","last_clean":"2024-05-21T03:22:13.413140+0000","last_became_active":"2024-05-21T03:20:32.145803+0000","last_became_peered":"2024-05-21T03:20:32.145803+0000","last_unstale":"2024-05-21T03:22:13.413140+0000","last_undegraded":"2024-05-21T03:22:13.413140+0000","last_fullsized":"2024-05-21T03:22:13.413140+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:34:09.108799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.415151+0000","last_change":"2024-05-21T03:21:14.559498+0000","last_active":"2024-05-21T03:22:13.415151+0000","last_peered":"2024-05-21T03:22:13.415151+0000","last_clean":"2024-05-21T03:22:13.415151+0000","last_became_active":"2024-05-21T03:21:14.559039+0000","last_became_peered":"2024-05-21T03:21:14.559039+0000","last_unstale":"2024-05-21T03:22:13.415151+0000","last_undegraded":"2024-05-21T03:22:13.415151+0000","last_fullsized":"2024-05-21T03:22:13.415151+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:04:36.676857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.405267+0000","last_change":"2024-05-21T03:20:32.148944+0000","last_active":"2024-05-21T03:22:13.405267+0000","last_peered":"2024-05-21T03:22:13.405267+0000","last_clean":"2024-05-21T03:22:13.405267+0000","last_became_active":"2024-05-21T03:20:32.148502+0000","last_became_peered":"2024-05-21T03:20:32.148502+0000","last_unstale":"2024-05-21T03:22:13.405267+0000","last_undegraded":"2024-05-21T03:22:13.405267+0000","last_fullsized":"2024-05-21T03:22:13.405267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:11:00.178105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.404451+0000","last_change":"2024-05-21T03:20:32.150609+0000","last_active":"2024-05-21T03:22:13.404451+0000","last_peered":"2024-05-21T03:22:13.404451+0000","last_clean":"2024-05-21T03:22:13.404451+0000","last_became_active":"2024-05-21T03:20:32.150314+0000","last_became_peered":"2024-05-21T03:20:32.150314+0000","last_unstale":"2024-05-21T03:22:13.404451+0000","last_undegraded":"2024-05-21T03:22:13.404451+0000","last_fullsized":"2024-05-21T03:22:13.404451+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:34:21.869304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.396652+0000","last_change":"2024-05-21T03:20:32.142472+0000","last_active":"2024-05-21T03:22:13.396652+0000","last_peered":"2024-05-21T03:22:13.396652+0000","last_clean":"2024-05-21T03:22:13.396652+0000","last_became_active":"2024-05-21T03:20:32.142161+0000","last_became_peered":"2024-05-21T03:20:32.142161+0000","last_unstale":"2024-05-21T03:22:13.396652+0000","last_undegraded":"2024-05-21T03:22:13.396652+0000","last_fullsized":"2024-05-21T03:22:13.396652+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:26:57.281655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.380693+0000","last_change":"2024-05-21T03:20:32.153987+0000","last_active":"2024-05-21T03:22:13.380693+0000","last_peered":"2024-05-21T03:22:13.380693+0000","last_clean":"2024-05-21T03:22:13.380693+0000","last_became_active":"2024-05-21T03:20:32.153617+0000","last_became_peered":"2024-05-21T03:20:32.153617+0000","last_unstale":"2024-05-21T03:22:13.380693+0000","last_undegraded":"2024-05-21T03:22:13.380693+0000","last_fullsized":"2024-05-21T03:22:13.380693+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:57:07.341758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":149,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:36.645523+0000","last_change":"2024-05-21T03:20:29.099928+0000","last_active":"2024-05-21T03:22:36.645523+0000","last_peered":"2024-05-21T03:22:36.645523+0000","last_clean":"2024-05-21T03:22:36.645523+0000","last_became_active":"2024-05-21T03:20:29.099592+0000","last_became_peered":"2024-05-21T03:20:29.099592+0000","last_unstale":"2024-05-21T03:22:36.645523+0000","last_undegraded":"2024-05-21T03:22:36.645523+0000","last_fullsized":"2024-05-21T03:22:36.645523+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:35:34.843325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.401190+0000","last_change":"2024-05-21T03:20:32.154962+0000","last_active":"2024-05-21T03:22:13.401190+0000","last_peered":"2024-05-21T03:22:13.401190+0000","last_clean":"2024-05-21T03:22:13.401190+0000","last_became_active":"2024-05-21T03:20:32.154620+0000","last_became_peered":"2024-05-21T03:20:32.154620+0000","last_unstale":"2024-05-21T03:22:13.401190+0000","last_undegraded":"2024-05-21T03:22:13.401190+0000","last_fullsized":"2024-05-21T03:22:13.401190+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:56:21.553272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561791+0000","last_change":"2024-05-21T03:20:29.097312+0000","last_active":"2024-05-21T03:21:14.561791+0000","last_peered":"2024-05-21T03:21:14.561791+0000","last_clean":"2024-05-21T03:21:14.561791+0000","last_became_active":"2024-05-21T03:20:29.096977+0000","last_became_peered":"2024-05-21T03:20:29.096977+0000","last_unstale":"2024-05-21T03:21:14.561791+0000","last_undegraded":"2024-05-21T03:21:14.561791+0000","last_fullsized":"2024-05-21T03:21:14.561791+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:28:45.467446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.415671+0000","last_change":"2024-05-21T03:20:32.150035+0000","last_active":"2024-05-21T03:22:13.415671+0000","last_peered":"2024-05-21T03:22:13.415671+0000","last_clean":"2024-05-21T03:22:13.415671+0000","last_became_active":"2024-05-21T03:20:32.149688+0000","last_became_peered":"2024-05-21T03:20:32.149688+0000","last_unstale":"2024-05-21T03:22:13.415671+0000","last_undegraded":"2024-05-21T03:22:13.415671+0000","last_fullsized":"2024-05-21T03:22:13.415671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:27:00.410770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.573309+0000","last_change":"2024-05-21T03:21:14.572832+0000","last_active":"2024-05-21T03:21:14.573309+0000","last_peered":"2024-05-21T03:21:14.573309+0000","last_clean":"2024-05-21T03:21:14.573309+0000","last_became_active":"2024-05-21T03:21:14.559381+0000","last_became_peered":"2024-05-21T03:21:14.559381+0000","last_unstale":"2024-05-21T03:21:14.573309+0000","last_undegraded":"2024-05-21T03:21:14.573309+0000","last_fullsized":"2024-05-21T03:21:14.573309+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:02:53.004031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.410752+0000","last_change":"2024-05-21T03:20:32.150301+0000","last_active":"2024-05-21T03:22:13.410752+0000","last_peered":"2024-05-21T03:22:13.410752+0000","last_clean":"2024-05-21T03:22:13.410752+0000","last_became_active":"2024-05-21T03:20:32.148475+0000","last_became_peered":"2024-05-21T03:20:32.148475+0000","last_unstale":"2024-05-21T03:22:13.410752+0000","last_undegraded":"2024-05-21T03:22:13.410752+0000","last_fullsized":"2024-05-21T03:22:13.410752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:59:30.339660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556899+0000","last_change":"2024-05-21T03:20:29.098861+0000","last_active":"2024-05-21T03:21:14.556899+0000","last_peered":"2024-05-21T03:21:14.556899+0000","last_clean":"2024-05-21T03:21:14.556899+0000","last_became_active":"2024-05-21T03:20:29.097874+0000","last_became_peered":"2024-05-21T03:20:29.097874+0000","last_unstale":"2024-05-21T03:21:14.556899+0000","last_undegraded":"2024-05-21T03:21:14.556899+0000","last_fullsized":"2024-05-21T03:21:14.556899+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:25:36.227339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.371721+0000","last_change":"2024-05-21T03:20:32.142820+0000","last_active":"2024-05-21T03:22:13.371721+0000","last_peered":"2024-05-21T03:22:13.371721+0000","last_clean":"2024-05-21T03:22:13.371721+0000","last_became_active":"2024-05-21T03:20:32.142396+0000","last_became_peered":"2024-05-21T03:20:32.142396+0000","last_unstale":"2024-05-21T03:22:13.371721+0000","last_undegraded":"2024-05-21T03:22:13.371721+0000","last_fullsized":"2024-05-21T03:22:13.371721+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:30:14.452389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":140,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:35.629032+0000","last_change":"2024-05-21T03:20:29.103153+0000","last_active":"2024-05-21T03:22:35.629032+0000","last_peered":"2024-05-21T03:22:35.629032+0000","last_clean":"2024-05-21T03:22:35.629032+0000","last_became_active":"2024-05-21T03:20:29.102819+0000","last_became_peered":"2024-05-21T03:20:29.102819+0000","last_unstale":"2024-05-21T03:22:35.629032+0000","last_undegraded":"2024-05-21T03:22:35.629032+0000","last_fullsized":"2024-05-21T03:22:35.629032+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:47:49.843926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.366995+0000","last_change":"2024-05-21T03:20:32.142263+0000","last_active":"2024-05-21T03:22:13.366995+0000","last_peered":"2024-05-21T03:22:13.366995+0000","last_clean":"2024-05-21T03:22:13.366995+0000","last_became_active":"2024-05-21T03:20:32.141820+0000","last_became_peered":"2024-05-21T03:20:32.141820+0000","last_unstale":"2024-05-21T03:22:13.366995+0000","last_undegraded":"2024-05-21T03:22:13.366995+0000","last_fullsized":"2024-05-21T03:22:13.366995+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:45:20.021237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.554011+0000","last_change":"2024-05-21T03:20:29.100088+0000","last_active":"2024-05-21T03:21:14.554011+0000","last_peered":"2024-05-21T03:21:14.554011+0000","last_clean":"2024-05-21T03:21:14.554011+0000","last_became_active":"2024-05-21T03:20:29.097343+0000","last_became_peered":"2024-05-21T03:20:29.097343+0000","last_unstale":"2024-05-21T03:21:14.554011+0000","last_undegraded":"2024-05-21T03:21:14.554011+0000","last_fullsized":"2024-05-21T03:21:14.554011+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:00:35.682393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.408104+0000","last_change":"2024-05-21T03:21:14.562281+0000","last_active":"2024-05-21T03:22:13.408104+0000","last_peered":"2024-05-21T03:22:13.408104+0000","last_clean":"2024-05-21T03:22:13.408104+0000","last_became_active":"2024-05-21T03:21:14.561857+0000","last_became_peered":"2024-05-21T03:21:14.561857+0000","last_unstale":"2024-05-21T03:22:13.408104+0000","last_undegraded":"2024-05-21T03:22:13.408104+0000","last_fullsized":"2024-05-21T03:22:13.408104+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:37:35.283529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552729+0000","last_change":"2024-05-21T03:20:29.097945+0000","last_active":"2024-05-21T03:21:14.552729+0000","last_peered":"2024-05-21T03:21:14.552729+0000","last_clean":"2024-05-21T03:21:14.552729+0000","last_became_active":"2024-05-21T03:20:29.097624+0000","last_became_peered":"2024-05-21T03:20:29.097624+0000","last_unstale":"2024-05-21T03:21:14.552729+0000","last_undegraded":"2024-05-21T03:21:14.552729+0000","last_fullsized":"2024-05-21T03:21:14.552729+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:01:01.694893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.411465+0000","last_change":"2024-05-21T03:20:32.148955+0000","last_active":"2024-05-21T03:22:13.411465+0000","last_peered":"2024-05-21T03:22:13.411465+0000","last_clean":"2024-05-21T03:22:13.411465+0000","last_became_active":"2024-05-21T03:20:32.148631+0000","last_became_peered":"2024-05-21T03:20:32.148631+0000","last_unstale":"2024-05-21T03:22:13.411465+0000","last_undegraded":"2024-05-21T03:22:13.411465+0000","last_fullsized":"2024-05-21T03:22:13.411465+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:36:25.820580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561173+0000","last_change":"2024-05-21T03:20:29.100902+0000","last_active":"2024-05-21T03:21:14.561173+0000","last_peered":"2024-05-21T03:21:14.561173+0000","last_clean":"2024-05-21T03:21:14.561173+0000","last_became_active":"2024-05-21T03:20:29.100570+0000","last_became_peered":"2024-05-21T03:20:29.100570+0000","last_unstale":"2024-05-21T03:21:14.561173+0000","last_undegraded":"2024-05-21T03:21:14.561173+0000","last_fullsized":"2024-05-21T03:21:14.561173+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:58:36.651338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.406823+0000","last_change":"2024-05-21T03:20:32.150724+0000","last_active":"2024-05-21T03:22:13.406823+0000","last_peered":"2024-05-21T03:22:13.406823+0000","last_clean":"2024-05-21T03:22:13.406823+0000","last_became_active":"2024-05-21T03:20:32.150371+0000","last_became_peered":"2024-05-21T03:20:32.150371+0000","last_unstale":"2024-05-21T03:22:13.406823+0000","last_undegraded":"2024-05-21T03:22:13.406823+0000","last_fullsized":"2024-05-21T03:22:13.406823+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:44:32.321192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'5","reported_seq":247,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:37.631094+0000","last_change":"2024-05-21T03:20:29.095045+0000","last_active":"2024-05-21T03:22:37.631094+0000","last_peered":"2024-05-21T03:22:37.631094+0000","last_clean":"2024-05-21T03:22:37.631094+0000","last_became_active":"2024-05-21T03:20:29.094552+0000","last_became_peered":"2024-05-21T03:20:29.094552+0000","last_unstale":"2024-05-21T03:22:37.631094+0000","last_undegraded":"2024-05-21T03:22:37.631094+0000","last_fullsized":"2024-05-21T03:22:37.631094+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:36:15.155125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":216,"num_read_kb":216,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.411077+0000","last_change":"2024-05-21T03:20:32.149813+0000","last_active":"2024-05-21T03:22:13.411077+0000","last_peered":"2024-05-21T03:22:13.411077+0000","last_clean":"2024-05-21T03:22:13.411077+0000","last_became_active":"2024-05-21T03:20:32.148422+0000","last_became_peered":"2024-05-21T03:20:32.148422+0000","last_unstale":"2024-05-21T03:22:13.411077+0000","last_undegraded":"2024-05-21T03:22:13.411077+0000","last_fullsized":"2024-05-21T03:22:13.411077+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:55:35.115463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.558265+0000","last_change":"2024-05-21T03:20:29.101463+0000","last_active":"2024-05-21T03:21:14.558265+0000","last_peered":"2024-05-21T03:21:14.558265+0000","last_clean":"2024-05-21T03:21:14.558265+0000","last_became_active":"2024-05-21T03:20:29.101127+0000","last_became_peered":"2024-05-21T03:20:29.101127+0000","last_unstale":"2024-05-21T03:21:14.558265+0000","last_undegraded":"2024-05-21T03:21:14.558265+0000","last_fullsized":"2024-05-21T03:21:14.558265+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:58:17.408315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.411937+0000","last_change":"2024-05-21T03:21:14.563434+0000","last_active":"2024-05-21T03:22:13.411937+0000","last_peered":"2024-05-21T03:22:13.411937+0000","last_clean":"2024-05-21T03:22:13.411937+0000","last_became_active":"2024-05-21T03:21:14.563081+0000","last_became_peered":"2024-05-21T03:21:14.563081+0000","last_unstale":"2024-05-21T03:22:13.411937+0000","last_undegraded":"2024-05-21T03:22:13.411937+0000","last_fullsized":"2024-05-21T03:22:13.411937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:28:55.497338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.562216+0000","last_change":"2024-05-21T03:20:29.098725+0000","last_active":"2024-05-21T03:21:14.562216+0000","last_peered":"2024-05-21T03:21:14.562216+0000","last_clean":"2024-05-21T03:21:14.562216+0000","last_became_active":"2024-05-21T03:20:29.098471+0000","last_became_peered":"2024-05-21T03:20:29.098471+0000","last_unstale":"2024-05-21T03:21:14.562216+0000","last_undegraded":"2024-05-21T03:21:14.562216+0000","last_fullsized":"2024-05-21T03:21:14.562216+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:17:11.059825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.388349+0000","last_change":"2024-05-21T03:21:14.560975+0000","last_active":"2024-05-21T03:22:13.388349+0000","last_peered":"2024-05-21T03:22:13.388349+0000","last_clean":"2024-05-21T03:22:13.388349+0000","last_became_active":"2024-05-21T03:21:14.560528+0000","last_became_peered":"2024-05-21T03:21:14.560528+0000","last_unstale":"2024-05-21T03:22:13.388349+0000","last_undegraded":"2024-05-21T03:22:13.388349+0000","last_fullsized":"2024-05-21T03:22:13.388349+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:51:47.469291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'34","reported_seq":286,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:37.631033+0000","last_change":"2024-05-21T03:20:29.099968+0000","last_active":"2024-05-21T03:22:37.631033+0000","last_peered":"2024-05-21T03:22:37.631033+0000","last_clean":"2024-05-21T03:22:37.631033+0000","last_became_active":"2024-05-21T03:20:29.099181+0000","last_became_peered":"2024-05-21T03:20:29.099181+0000","last_unstale":"2024-05-21T03:22:37.631033+0000","last_undegraded":"2024-05-21T03:22:37.631033+0000","last_fullsized":"2024-05-21T03:22:37.631033+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:35:33.934412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60011,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":226,"num_read_kb":171,"num_write":36,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.381167+0000","last_change":"2024-05-21T03:20:32.144024+0000","last_active":"2024-05-21T03:22:13.381167+0000","last_peered":"2024-05-21T03:22:13.381167+0000","last_clean":"2024-05-21T03:22:13.381167+0000","last_became_active":"2024-05-21T03:20:32.143668+0000","last_became_peered":"2024-05-21T03:20:32.143668+0000","last_unstale":"2024-05-21T03:22:13.381167+0000","last_undegraded":"2024-05-21T03:22:13.381167+0000","last_fullsized":"2024-05-21T03:22:13.381167+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:11:31.100802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556933+0000","last_change":"2024-05-21T03:20:29.101770+0000","last_active":"2024-05-21T03:21:14.556933+0000","last_peered":"2024-05-21T03:21:14.556933+0000","last_clean":"2024-05-21T03:21:14.556933+0000","last_became_active":"2024-05-21T03:20:29.100407+0000","last_became_peered":"2024-05-21T03:20:29.100407+0000","last_unstale":"2024-05-21T03:21:14.556933+0000","last_undegraded":"2024-05-21T03:21:14.556933+0000","last_fullsized":"2024-05-21T03:21:14.556933+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:01:33.991246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.415145+0000","last_change":"2024-05-21T03:20:32.147479+0000","last_active":"2024-05-21T03:22:13.415145+0000","last_peered":"2024-05-21T03:22:13.415145+0000","last_clean":"2024-05-21T03:22:13.415145+0000","last_became_active":"2024-05-21T03:20:32.147160+0000","last_became_peered":"2024-05-21T03:20:32.147160+0000","last_unstale":"2024-05-21T03:22:13.415145+0000","last_undegraded":"2024-05-21T03:22:13.415145+0000","last_fullsized":"2024-05-21T03:22:13.415145+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:51:59.343380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.562216+0000","last_change":"2024-05-21T03:20:29.101875+0000","last_active":"2024-05-21T03:21:14.562216+0000","last_peered":"2024-05-21T03:21:14.562216+0000","last_clean":"2024-05-21T03:21:14.562216+0000","last_became_active":"2024-05-21T03:20:29.101468+0000","last_became_peered":"2024-05-21T03:20:29.101468+0000","last_unstale":"2024-05-21T03:21:14.562216+0000","last_undegraded":"2024-05-21T03:21:14.562216+0000","last_fullsized":"2024-05-21T03:21:14.562216+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:10:19.577836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.410660+0000","last_change":"2024-05-21T03:20:32.139002+0000","last_active":"2024-05-21T03:22:13.410660+0000","last_peered":"2024-05-21T03:22:13.410660+0000","last_clean":"2024-05-21T03:22:13.410660+0000","last_became_active":"2024-05-21T03:20:32.138619+0000","last_became_peered":"2024-05-21T03:20:32.138619+0000","last_unstale":"2024-05-21T03:22:13.410660+0000","last_undegraded":"2024-05-21T03:22:13.410660+0000","last_fullsized":"2024-05-21T03:22:13.410660+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:21:36.136283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":143,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:35.628592+0000","last_change":"2024-05-21T03:20:29.101532+0000","last_active":"2024-05-21T03:22:35.628592+0000","last_peered":"2024-05-21T03:22:35.628592+0000","last_clean":"2024-05-21T03:22:35.628592+0000","last_became_active":"2024-05-21T03:20:29.101206+0000","last_became_peered":"2024-05-21T03:20:29.101206+0000","last_unstale":"2024-05-21T03:22:35.628592+0000","last_undegraded":"2024-05-21T03:22:35.628592+0000","last_fullsized":"2024-05-21T03:22:35.628592+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:47:38.555995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.385079+0000","last_change":"2024-05-21T03:20:32.146866+0000","last_active":"2024-05-21T03:22:13.385079+0000","last_peered":"2024-05-21T03:22:13.385079+0000","last_clean":"2024-05-21T03:22:13.385079+0000","last_became_active":"2024-05-21T03:20:32.146463+0000","last_became_peered":"2024-05-21T03:20:32.146463+0000","last_unstale":"2024-05-21T03:22:13.385079+0000","last_undegraded":"2024-05-21T03:22:13.385079+0000","last_fullsized":"2024-05-21T03:22:13.385079+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:06:54.359922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556379+0000","last_change":"2024-05-21T03:20:29.103910+0000","last_active":"2024-05-21T03:21:14.556379+0000","last_peered":"2024-05-21T03:21:14.556379+0000","last_clean":"2024-05-21T03:21:14.556379+0000","last_became_active":"2024-05-21T03:20:29.103583+0000","last_became_peered":"2024-05-21T03:20:29.103583+0000","last_unstale":"2024-05-21T03:21:14.556379+0000","last_undegraded":"2024-05-21T03:21:14.556379+0000","last_fullsized":"2024-05-21T03:21:14.556379+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:57:15.665158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.366845+0000","last_change":"2024-05-21T03:20:32.141242+0000","last_active":"2024-05-21T03:22:13.366845+0000","last_peered":"2024-05-21T03:22:13.366845+0000","last_clean":"2024-05-21T03:22:13.366845+0000","last_became_active":"2024-05-21T03:20:32.140892+0000","last_became_peered":"2024-05-21T03:20:32.140892+0000","last_unstale":"2024-05-21T03:22:13.366845+0000","last_undegraded":"2024-05-21T03:22:13.366845+0000","last_fullsized":"2024-05-21T03:22:13.366845+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:42:13.997708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.557026+0000","last_change":"2024-05-21T03:20:29.098973+0000","last_active":"2024-05-21T03:21:14.557026+0000","last_peered":"2024-05-21T03:21:14.557026+0000","last_clean":"2024-05-21T03:21:14.557026+0000","last_became_active":"2024-05-21T03:20:29.098694+0000","last_became_peered":"2024-05-21T03:20:29.098694+0000","last_unstale":"2024-05-21T03:21:14.557026+0000","last_undegraded":"2024-05-21T03:21:14.557026+0000","last_fullsized":"2024-05-21T03:21:14.557026+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:55:17.915820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.376336+0000","last_change":"2024-05-21T03:20:32.147465+0000","last_active":"2024-05-21T03:22:13.376336+0000","last_peered":"2024-05-21T03:22:13.376336+0000","last_clean":"2024-05-21T03:22:13.376336+0000","last_became_active":"2024-05-21T03:20:32.147073+0000","last_became_peered":"2024-05-21T03:20:32.147073+0000","last_unstale":"2024-05-21T03:22:13.376336+0000","last_undegraded":"2024-05-21T03:22:13.376336+0000","last_fullsized":"2024-05-21T03:22:13.376336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:29:05.451370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561052+0000","last_change":"2024-05-21T03:21:14.560568+0000","last_active":"2024-05-21T03:21:14.561052+0000","last_peered":"2024-05-21T03:21:14.561052+0000","last_clean":"2024-05-21T03:21:14.561052+0000","last_became_active":"2024-05-21T03:21:14.560093+0000","last_became_peered":"2024-05-21T03:21:14.560093+0000","last_unstale":"2024-05-21T03:21:14.561052+0000","last_undegraded":"2024-05-21T03:21:14.561052+0000","last_fullsized":"2024-05-21T03:21:14.561052+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:55:39.839942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.410846+0000","last_change":"2024-05-21T03:20:32.146456+0000","last_active":"2024-05-21T03:22:13.410846+0000","last_peered":"2024-05-21T03:22:13.410846+0000","last_clean":"2024-05-21T03:22:13.410846+0000","last_became_active":"2024-05-21T03:20:32.146105+0000","last_became_peered":"2024-05-21T03:20:32.146105+0000","last_unstale":"2024-05-21T03:22:13.410846+0000","last_undegraded":"2024-05-21T03:22:13.410846+0000","last_fullsized":"2024-05-21T03:22:13.410846+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:00:30.550178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'7","reported_seq":143,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:36.645969+0000","last_change":"2024-05-21T03:20:29.100262+0000","last_active":"2024-05-21T03:22:36.645969+0000","last_peered":"2024-05-21T03:22:36.645969+0000","last_clean":"2024-05-21T03:22:36.645969+0000","last_became_active":"2024-05-21T03:20:29.099917+0000","last_became_peered":"2024-05-21T03:20:29.099917+0000","last_unstale":"2024-05-21T03:22:36.645969+0000","last_undegraded":"2024-05-21T03:22:36.645969+0000","last_fullsized":"2024-05-21T03:22:36.645969+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:55:48.608341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.380411+0000","last_change":"2024-05-21T03:20:32.142776+0000","last_active":"2024-05-21T03:22:13.380411+0000","last_peered":"2024-05-21T03:22:13.380411+0000","last_clean":"2024-05-21T03:22:13.380411+0000","last_became_active":"2024-05-21T03:20:32.142430+0000","last_became_peered":"2024-05-21T03:20:32.142430+0000","last_unstale":"2024-05-21T03:22:13.380411+0000","last_undegraded":"2024-05-21T03:22:13.380411+0000","last_fullsized":"2024-05-21T03:22:13.380411+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:54:28.682343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.375317+0000","last_change":"2024-05-21T03:20:32.139506+0000","last_active":"2024-05-21T03:22:13.375317+0000","last_peered":"2024-05-21T03:22:13.375317+0000","last_clean":"2024-05-21T03:22:13.375317+0000","last_became_active":"2024-05-21T03:20:32.138136+0000","last_became_peered":"2024-05-21T03:20:32.138136+0000","last_unstale":"2024-05-21T03:22:13.375317+0000","last_undegraded":"2024-05-21T03:22:13.375317+0000","last_fullsized":"2024-05-21T03:22:13.375317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:31:24.996276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":145,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:39.633605+0000","last_change":"2024-05-21T03:20:29.096546+0000","last_active":"2024-05-21T03:22:39.633605+0000","last_peered":"2024-05-21T03:22:39.633605+0000","last_clean":"2024-05-21T03:22:39.633605+0000","last_became_active":"2024-05-21T03:20:29.096184+0000","last_became_peered":"2024-05-21T03:20:29.096184+0000","last_unstale":"2024-05-21T03:22:39.633605+0000","last_undegraded":"2024-05-21T03:22:39.633605+0000","last_fullsized":"2024-05-21T03:22:39.633605+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:09:12.832745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.386638+0000","last_change":"2024-05-21T03:20:32.156231+0000","last_active":"2024-05-21T03:22:13.386638+0000","last_peered":"2024-05-21T03:22:13.386638+0000","last_clean":"2024-05-21T03:22:13.386638+0000","last_became_active":"2024-05-21T03:20:32.154035+0000","last_became_peered":"2024-05-21T03:20:32.154035+0000","last_unstale":"2024-05-21T03:22:13.386638+0000","last_undegraded":"2024-05-21T03:22:13.386638+0000","last_fullsized":"2024-05-21T03:22:13.386638+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:30:53.820292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556193+0000","last_change":"2024-05-21T03:20:29.100422+0000","last_active":"2024-05-21T03:21:14.556193+0000","last_peered":"2024-05-21T03:21:14.556193+0000","last_clean":"2024-05-21T03:21:14.556193+0000","last_became_active":"2024-05-21T03:20:29.099993+0000","last_became_peered":"2024-05-21T03:20:29.099993+0000","last_unstale":"2024-05-21T03:21:14.556193+0000","last_undegraded":"2024-05-21T03:21:14.556193+0000","last_fullsized":"2024-05-21T03:21:14.556193+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:03:15.456447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":99,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556851+0000","last_change":"2024-05-21T03:19:05.647808+0000","last_active":"2024-05-21T03:21:14.556851+0000","last_peered":"2024-05-21T03:21:14.556851+0000","last_clean":"2024-05-21T03:21:14.556851+0000","last_became_active":"2024-05-21T03:19:05.641865+0000","last_became_peered":"2024-05-21T03:19:05.641865+0000","last_unstale":"2024-05-21T03:21:14.556851+0000","last_undegraded":"2024-05-21T03:21:14.556851+0000","last_fullsized":"2024-05-21T03:21:14.556851+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:15:42.008493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:15:42.008493+0000","last_clean_scrub_stamp":"2024-05-21T03:15:42.008493+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:26:58.040196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.403388+0000","last_change":"2024-05-21T03:20:32.138182+0000","last_active":"2024-05-21T03:22:13.403388+0000","last_peered":"2024-05-21T03:22:13.403388+0000","last_clean":"2024-05-21T03:22:13.403388+0000","last_became_active":"2024-05-21T03:20:32.137842+0000","last_became_peered":"2024-05-21T03:20:32.137842+0000","last_unstale":"2024-05-21T03:22:13.403388+0000","last_undegraded":"2024-05-21T03:22:13.403388+0000","last_fullsized":"2024-05-21T03:22:13.403388+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:43:50.070160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.557047+0000","last_change":"2024-05-21T03:20:29.104246+0000","last_active":"2024-05-21T03:21:14.557047+0000","last_peered":"2024-05-21T03:21:14.557047+0000","last_clean":"2024-05-21T03:21:14.557047+0000","last_became_active":"2024-05-21T03:20:29.103910+0000","last_became_peered":"2024-05-21T03:20:29.103910+0000","last_unstale":"2024-05-21T03:21:14.557047+0000","last_undegraded":"2024-05-21T03:21:14.557047+0000","last_fullsized":"2024-05-21T03:21:14.557047+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:39:28.029283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556776+0000","last_change":"2024-05-21T03:20:29.102020+0000","last_active":"2024-05-21T03:21:14.556776+0000","last_peered":"2024-05-21T03:21:14.556776+0000","last_clean":"2024-05-21T03:21:14.556776+0000","last_became_active":"2024-05-21T03:20:29.100429+0000","last_became_peered":"2024-05-21T03:20:29.100429+0000","last_unstale":"2024-05-21T03:21:14.556776+0000","last_undegraded":"2024-05-21T03:21:14.556776+0000","last_fullsized":"2024-05-21T03:21:14.556776+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:09:01.621074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.418662+0000","last_change":"2024-05-21T03:20:32.137130+0000","last_active":"2024-05-21T03:22:13.418662+0000","last_peered":"2024-05-21T03:22:13.418662+0000","last_clean":"2024-05-21T03:22:13.418662+0000","last_became_active":"2024-05-21T03:20:32.136821+0000","last_became_peered":"2024-05-21T03:20:32.136821+0000","last_unstale":"2024-05-21T03:22:13.418662+0000","last_undegraded":"2024-05-21T03:22:13.418662+0000","last_fullsized":"2024-05-21T03:22:13.418662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:53:41.290370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'15","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:14.284507+0000","last_change":"2024-05-21T03:20:29.099869+0000","last_active":"2024-05-21T03:22:14.284507+0000","last_peered":"2024-05-21T03:22:14.284507+0000","last_clean":"2024-05-21T03:22:14.284507+0000","last_became_active":"2024-05-21T03:20:29.099534+0000","last_became_peered":"2024-05-21T03:20:29.099534+0000","last_unstale":"2024-05-21T03:22:14.284507+0000","last_undegraded":"2024-05-21T03:22:14.284507+0000","last_fullsized":"2024-05-21T03:22:14.284507+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:01:11.281544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.389637+0000","last_change":"2024-05-21T03:20:32.153921+0000","last_active":"2024-05-21T03:22:13.389637+0000","last_peered":"2024-05-21T03:22:13.389637+0000","last_clean":"2024-05-21T03:22:13.389637+0000","last_became_active":"2024-05-21T03:20:32.153594+0000","last_became_peered":"2024-05-21T03:20:32.153594+0000","last_unstale":"2024-05-21T03:22:13.389637+0000","last_undegraded":"2024-05-21T03:22:13.389637+0000","last_fullsized":"2024-05-21T03:22:13.389637+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:27:59.426942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556915+0000","last_change":"2024-05-21T03:20:29.097399+0000","last_active":"2024-05-21T03:21:14.556915+0000","last_peered":"2024-05-21T03:21:14.556915+0000","last_clean":"2024-05-21T03:21:14.556915+0000","last_became_active":"2024-05-21T03:20:29.097010+0000","last_became_peered":"2024-05-21T03:20:29.097010+0000","last_unstale":"2024-05-21T03:21:14.556915+0000","last_undegraded":"2024-05-21T03:21:14.556915+0000","last_fullsized":"2024-05-21T03:21:14.556915+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:54:53.273947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.412963+0000","last_change":"2024-05-21T03:21:14.559609+0000","last_active":"2024-05-21T03:22:13.412963+0000","last_peered":"2024-05-21T03:22:13.412963+0000","last_clean":"2024-05-21T03:22:13.412963+0000","last_became_active":"2024-05-21T03:21:14.559218+0000","last_became_peered":"2024-05-21T03:21:14.559218+0000","last_unstale":"2024-05-21T03:22:13.412963+0000","last_undegraded":"2024-05-21T03:22:13.412963+0000","last_fullsized":"2024-05-21T03:22:13.412963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:32:11.156037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,4],"acting":[0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.557957+0000","last_change":"2024-05-21T03:20:29.103526+0000","last_active":"2024-05-21T03:21:14.557957+0000","last_peered":"2024-05-21T03:21:14.557957+0000","last_clean":"2024-05-21T03:21:14.557957+0000","last_became_active":"2024-05-21T03:20:29.103067+0000","last_became_peered":"2024-05-21T03:20:29.103067+0000","last_unstale":"2024-05-21T03:21:14.557957+0000","last_undegraded":"2024-05-21T03:21:14.557957+0000","last_fullsized":"2024-05-21T03:21:14.557957+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:54:47.566949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.339758+0000","last_change":"2024-05-21T03:20:32.141088+0000","last_active":"2024-05-21T03:22:13.339758+0000","last_peered":"2024-05-21T03:22:13.339758+0000","last_clean":"2024-05-21T03:22:13.339758+0000","last_became_active":"2024-05-21T03:20:32.140744+0000","last_became_peered":"2024-05-21T03:20:32.140744+0000","last_unstale":"2024-05-21T03:22:13.339758+0000","last_undegraded":"2024-05-21T03:22:13.339758+0000","last_fullsized":"2024-05-21T03:22:13.339758+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:57:35.591164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.560810+0000","last_change":"2024-05-21T03:20:29.093440+0000","last_active":"2024-05-21T03:21:14.560810+0000","last_peered":"2024-05-21T03:21:14.560810+0000","last_clean":"2024-05-21T03:21:14.560810+0000","last_became_active":"2024-05-21T03:20:29.092935+0000","last_became_peered":"2024-05-21T03:20:29.092935+0000","last_unstale":"2024-05-21T03:21:14.560810+0000","last_undegraded":"2024-05-21T03:21:14.560810+0000","last_fullsized":"2024-05-21T03:21:14.560810+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:18:00.690465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.404572+0000","last_change":"2024-05-21T03:20:32.143781+0000","last_active":"2024-05-21T03:22:13.404572+0000","last_peered":"2024-05-21T03:22:13.404572+0000","last_clean":"2024-05-21T03:22:13.404572+0000","last_became_active":"2024-05-21T03:20:32.143391+0000","last_became_peered":"2024-05-21T03:20:32.143391+0000","last_unstale":"2024-05-21T03:22:13.404572+0000","last_undegraded":"2024-05-21T03:22:13.404572+0000","last_fullsized":"2024-05-21T03:22:13.404572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:00:41.506561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556323+0000","last_change":"2024-05-21T03:20:29.101342+0000","last_active":"2024-05-21T03:21:14.556323+0000","last_peered":"2024-05-21T03:21:14.556323+0000","last_clean":"2024-05-21T03:21:14.556323+0000","last_became_active":"2024-05-21T03:20:29.100997+0000","last_became_peered":"2024-05-21T03:20:29.100997+0000","last_unstale":"2024-05-21T03:21:14.556323+0000","last_undegraded":"2024-05-21T03:21:14.556323+0000","last_fullsized":"2024-05-21T03:21:14.556323+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:05:31.801669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.373332+0000","last_change":"2024-05-21T03:20:32.150563+0000","last_active":"2024-05-21T03:22:13.373332+0000","last_peered":"2024-05-21T03:22:13.373332+0000","last_clean":"2024-05-21T03:22:13.373332+0000","last_became_active":"2024-05-21T03:20:32.150231+0000","last_became_peered":"2024-05-21T03:20:32.150231+0000","last_unstale":"2024-05-21T03:22:13.373332+0000","last_undegraded":"2024-05-21T03:22:13.373332+0000","last_fullsized":"2024-05-21T03:22:13.373332+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:10:00.251284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.556609+0000","last_change":"2024-05-21T03:20:29.101576+0000","last_active":"2024-05-21T03:21:14.556609+0000","last_peered":"2024-05-21T03:21:14.556609+0000","last_clean":"2024-05-21T03:21:14.556609+0000","last_became_active":"2024-05-21T03:20:29.101270+0000","last_became_peered":"2024-05-21T03:20:29.101270+0000","last_unstale":"2024-05-21T03:21:14.556609+0000","last_undegraded":"2024-05-21T03:21:14.556609+0000","last_fullsized":"2024-05-21T03:21:14.556609+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:50:10.579901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.403247+0000","last_change":"2024-05-21T03:20:32.141340+0000","last_active":"2024-05-21T03:22:13.403247+0000","last_peered":"2024-05-21T03:22:13.403247+0000","last_clean":"2024-05-21T03:22:13.403247+0000","last_became_active":"2024-05-21T03:20:32.140792+0000","last_became_peered":"2024-05-21T03:20:32.140792+0000","last_unstale":"2024-05-21T03:22:13.403247+0000","last_undegraded":"2024-05-21T03:22:13.403247+0000","last_fullsized":"2024-05-21T03:22:13.403247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:35:05.842929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'7","reported_seq":104,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:35.680398+0000","last_change":"2024-05-21T03:21:14.566886+0000","last_active":"2024-05-21T03:22:35.680398+0000","last_peered":"2024-05-21T03:22:35.680398+0000","last_clean":"2024-05-21T03:22:35.680398+0000","last_became_active":"2024-05-21T03:21:14.559816+0000","last_became_peered":"2024-05-21T03:21:14.559816+0000","last_unstale":"2024-05-21T03:22:35.680398+0000","last_undegraded":"2024-05-21T03:22:35.680398+0000","last_fullsized":"2024-05-21T03:22:35.680398+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:34:43.109424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":89,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1561,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.389190+0000","last_change":"2024-05-21T03:20:32.146674+0000","last_active":"2024-05-21T03:22:13.389190+0000","last_peered":"2024-05-21T03:22:13.389190+0000","last_clean":"2024-05-21T03:22:13.389190+0000","last_became_active":"2024-05-21T03:20:32.146150+0000","last_became_peered":"2024-05-21T03:20:32.146150+0000","last_unstale":"2024-05-21T03:22:13.389190+0000","last_undegraded":"2024-05-21T03:22:13.389190+0000","last_fullsized":"2024-05-21T03:22:13.389190+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:39:09.905640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.561667+0000","last_change":"2024-05-21T03:20:29.097613+0000","last_active":"2024-05-21T03:21:14.561667+0000","last_peered":"2024-05-21T03:21:14.561667+0000","last_clean":"2024-05-21T03:21:14.561667+0000","last_became_active":"2024-05-21T03:20:29.097331+0000","last_became_peered":"2024-05-21T03:20:29.097331+0000","last_unstale":"2024-05-21T03:21:14.561667+0000","last_undegraded":"2024-05-21T03:21:14.561667+0000","last_fullsized":"2024-05-21T03:21:14.561667+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:56:14.720420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.395548+0000","last_change":"2024-05-21T03:21:14.566406+0000","last_active":"2024-05-21T03:22:13.395548+0000","last_peered":"2024-05-21T03:22:13.395548+0000","last_clean":"2024-05-21T03:22:13.395548+0000","last_became_active":"2024-05-21T03:21:14.563023+0000","last_became_peered":"2024-05-21T03:21:14.563023+0000","last_unstale":"2024-05-21T03:22:13.395548+0000","last_undegraded":"2024-05-21T03:22:13.395548+0000","last_fullsized":"2024-05-21T03:22:13.395548+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:12:53.125136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:21:14.552818+0000","last_change":"2024-05-21T03:20:29.103456+0000","last_active":"2024-05-21T03:21:14.552818+0000","last_peered":"2024-05-21T03:21:14.552818+0000","last_clean":"2024-05-21T03:21:14.552818+0000","last_became_active":"2024-05-21T03:20:29.103008+0000","last_became_peered":"2024-05-21T03:20:29.103008+0000","last_unstale":"2024-05-21T03:21:14.552818+0000","last_undegraded":"2024-05-21T03:21:14.552818+0000","last_fullsized":"2024-05-21T03:21:14.552818+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:02:37.951347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.387395+0000","last_change":"2024-05-21T03:20:32.146986+0000","last_active":"2024-05-21T03:22:13.387395+0000","last_peered":"2024-05-21T03:22:13.387395+0000","last_clean":"2024-05-21T03:22:13.387395+0000","last_became_active":"2024-05-21T03:20:32.146355+0000","last_became_peered":"2024-05-21T03:20:32.146355+0000","last_unstale":"2024-05-21T03:22:13.387395+0000","last_undegraded":"2024-05-21T03:22:13.387395+0000","last_fullsized":"2024-05-21T03:22:13.387395+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:36:29.503099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.410905+0000","last_change":"2024-05-21T03:21:14.559658+0000","last_active":"2024-05-21T03:22:13.410905+0000","last_peered":"2024-05-21T03:22:13.410905+0000","last_clean":"2024-05-21T03:22:13.410905+0000","last_became_active":"2024-05-21T03:21:14.559194+0000","last_became_peered":"2024-05-21T03:21:14.559194+0000","last_unstale":"2024-05-21T03:22:13.410905+0000","last_undegraded":"2024-05-21T03:22:13.410905+0000","last_fullsized":"2024-05-21T03:22:13.410905+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:56:31.709545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.373021+0000","last_change":"2024-05-21T03:20:32.137532+0000","last_active":"2024-05-21T03:22:13.373021+0000","last_peered":"2024-05-21T03:22:13.373021+0000","last_clean":"2024-05-21T03:22:13.373021+0000","last_became_active":"2024-05-21T03:20:32.137182+0000","last_became_peered":"2024-05-21T03:20:32.137182+0000","last_unstale":"2024-05-21T03:22:13.373021+0000","last_undegraded":"2024-05-21T03:22:13.373021+0000","last_fullsized":"2024-05-21T03:22:13.373021+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:25:44.207866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.403832+0000","last_change":"2024-05-21T03:20:32.153912+0000","last_active":"2024-05-21T03:22:13.403832+0000","last_peered":"2024-05-21T03:22:13.403832+0000","last_clean":"2024-05-21T03:22:13.403832+0000","last_became_active":"2024-05-21T03:20:32.153541+0000","last_became_peered":"2024-05-21T03:20:32.153541+0000","last_unstale":"2024-05-21T03:22:13.403832+0000","last_undegraded":"2024-05-21T03:22:13.403832+0000","last_fullsized":"2024-05-21T03:22:13.403832+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:02:58.688378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.413482+0000","last_change":"2024-05-21T03:20:32.143689+0000","last_active":"2024-05-21T03:22:13.413482+0000","last_peered":"2024-05-21T03:22:13.413482+0000","last_clean":"2024-05-21T03:22:13.413482+0000","last_became_active":"2024-05-21T03:20:32.143399+0000","last_became_peered":"2024-05-21T03:20:32.143399+0000","last_unstale":"2024-05-21T03:22:13.413482+0000","last_undegraded":"2024-05-21T03:22:13.413482+0000","last_fullsized":"2024-05-21T03:22:13.413482+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:27:55.116599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.414090+0000","last_change":"2024-05-21T03:20:32.141355+0000","last_active":"2024-05-21T03:22:13.414090+0000","last_peered":"2024-05-21T03:22:13.414090+0000","last_clean":"2024-05-21T03:22:13.414090+0000","last_became_active":"2024-05-21T03:20:32.140960+0000","last_became_peered":"2024-05-21T03:20:32.140960+0000","last_unstale":"2024-05-21T03:22:13.414090+0000","last_undegraded":"2024-05-21T03:22:13.414090+0000","last_fullsized":"2024-05-21T03:22:13.414090+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:23:50.287864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.376470+0000","last_change":"2024-05-21T03:20:32.142145+0000","last_active":"2024-05-21T03:22:13.376470+0000","last_peered":"2024-05-21T03:22:13.376470+0000","last_clean":"2024-05-21T03:22:13.376470+0000","last_became_active":"2024-05-21T03:20:32.141802+0000","last_became_peered":"2024-05-21T03:20:32.141802+0000","last_unstale":"2024-05-21T03:22:13.376470+0000","last_undegraded":"2024-05-21T03:22:13.376470+0000","last_fullsized":"2024-05-21T03:22:13.376470+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:26:28.707616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.371412+0000","last_change":"2024-05-21T03:20:32.141606+0000","last_active":"2024-05-21T03:22:13.371412+0000","last_peered":"2024-05-21T03:22:13.371412+0000","last_clean":"2024-05-21T03:22:13.371412+0000","last_became_active":"2024-05-21T03:20:32.141258+0000","last_became_peered":"2024-05-21T03:20:32.141258+0000","last_unstale":"2024-05-21T03:22:13.371412+0000","last_undegraded":"2024-05-21T03:22:13.371412+0000","last_fullsized":"2024-05-21T03:22:13.371412+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:22.372723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.396785+0000","last_change":"2024-05-21T03:20:32.151621+0000","last_active":"2024-05-21T03:22:13.396785+0000","last_peered":"2024-05-21T03:22:13.396785+0000","last_clean":"2024-05-21T03:22:13.396785+0000","last_became_active":"2024-05-21T03:20:32.151260+0000","last_became_peered":"2024-05-21T03:20:32.151260+0000","last_unstale":"2024-05-21T03:22:13.396785+0000","last_undegraded":"2024-05-21T03:22:13.396785+0000","last_fullsized":"2024-05-21T03:22:13.396785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:48:59.751179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.407694+0000","last_change":"2024-05-21T03:20:32.153843+0000","last_active":"2024-05-21T03:22:13.407694+0000","last_peered":"2024-05-21T03:22:13.407694+0000","last_clean":"2024-05-21T03:22:13.407694+0000","last_became_active":"2024-05-21T03:20:32.153418+0000","last_became_peered":"2024-05-21T03:20:32.153418+0000","last_unstale":"2024-05-21T03:22:13.407694+0000","last_undegraded":"2024-05-21T03:22:13.407694+0000","last_fullsized":"2024-05-21T03:22:13.407694+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:26:29.663966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.398192+0000","last_change":"2024-05-21T03:20:32.153942+0000","last_active":"2024-05-21T03:22:13.398192+0000","last_peered":"2024-05-21T03:22:13.398192+0000","last_clean":"2024-05-21T03:22:13.398192+0000","last_became_active":"2024-05-21T03:20:32.153566+0000","last_became_peered":"2024-05-21T03:20:32.153566+0000","last_unstale":"2024-05-21T03:22:13.398192+0000","last_undegraded":"2024-05-21T03:22:13.398192+0000","last_fullsized":"2024-05-21T03:22:13.398192+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:06.899039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.404233+0000","last_change":"2024-05-21T03:20:32.153243+0000","last_active":"2024-05-21T03:22:13.404233+0000","last_peered":"2024-05-21T03:22:13.404233+0000","last_clean":"2024-05-21T03:22:13.404233+0000","last_became_active":"2024-05-21T03:20:32.152386+0000","last_became_peered":"2024-05-21T03:20:32.152386+0000","last_unstale":"2024-05-21T03:22:13.404233+0000","last_undegraded":"2024-05-21T03:22:13.404233+0000","last_fullsized":"2024-05-21T03:22:13.404233+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:25:28.935183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.401142+0000","last_change":"2024-05-21T03:20:32.142536+0000","last_active":"2024-05-21T03:22:13.401142+0000","last_peered":"2024-05-21T03:22:13.401142+0000","last_clean":"2024-05-21T03:22:13.401142+0000","last_became_active":"2024-05-21T03:20:32.141793+0000","last_became_peered":"2024-05-21T03:20:32.141793+0000","last_unstale":"2024-05-21T03:22:13.401142+0000","last_undegraded":"2024-05-21T03:22:13.401142+0000","last_fullsized":"2024-05-21T03:22:13.401142+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:21:43.547103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T03:22:13.372845+0000","last_change":"2024-05-21T03:20:32.140368+0000","last_active":"2024-05-21T03:22:13.372845+0000","last_peered":"2024-05-21T03:22:13.372845+0000","last_clean":"2024-05-21T03:22:13.372845+0000","last_became_active":"2024-05-21T03:20:32.139747+0000","last_became_peered":"2024-05-21T03:20:32.139747+0000","last_unstale":"2024-05-21T03:22:13.372845+0000","last_undegraded":"2024-05-21T03:22:13.372845+0000","last_fullsized":"2024-05-21T03:22:13.372845+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:05:44.851191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68029,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1101,"num_read_kb":639,"num_write":255,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1561,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":204087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645353,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":433991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841582,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1601536,"data_stored":1085031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005108,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":433506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201336,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":433767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397566,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32660,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712492,"statfs":{"total":95995035648,"available":95961591808,"internally_reserved":0,"allocated":1519616,"data_stored":1026958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561091,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":433638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":494562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":434977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":493080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149778,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":432814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953549,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":917504,"data_stored":435201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757320,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32672,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712480,"statfs":{"total":95995035648,"available":95961579520,"internally_reserved":0,"allocated":1531904,"data_stored":1025572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T03:22:43.593 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-21T03:22:43.593 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-21T03:22:43.593 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-21T03:22:43.594 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph health --format=json 2024-05-21T03:22:43.844 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:22:44.291 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:44 smithi027 ceph-mon[32179]: pgmap v357: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:22:44.519 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:44 smithi039 ceph-mon[36429]: pgmap v357: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:22:44.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:44 smithi165 ceph-mon[36269]: pgmap v357: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:22:45.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.471+0000 7f1ec640e700 1 -- 172.21.15.27:0/3788965652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec0101e60 msgr2=0x7f1ec0102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:45.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.471+0000 7f1ec640e700 1 --2- 172.21.15.27:0/3788965652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec0101e60 0x7f1ec0102240 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1eb00099c0 tx=0x7f1eb0009cd0 comp rx=0 tx=0).stop 2024-05-21T03:22:45.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.471+0000 7f1ec640e700 1 -- 172.21.15.27:0/3788965652 shutdown_connections 2024-05-21T03:22:45.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.471+0000 7f1ec640e700 1 --2- 172.21.15.27:0/3788965652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ec0106ef0 0x7f1ec010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:45.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.471+0000 7f1ec640e700 1 --2- 172.21.15.27:0/3788965652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec0101e60 0x7f1ec0102240 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:45.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.471+0000 7f1ec640e700 1 --2- 172.21.15.27:0/3788965652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ec0102810 0x7f1ec0106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:45.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.471+0000 7f1ec640e700 1 -- 172.21.15.27:0/3788965652 >> 172.21.15.27:0/3788965652 conn(0x7f1ec0078ab0 msgr2=0x7f1ec00ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:45.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.471+0000 7f1ec640e700 1 -- 172.21.15.27:0/3788965652 shutdown_connections 2024-05-21T03:22:45.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.471+0000 7f1ec640e700 1 -- 172.21.15.27:0/3788965652 wait complete. 2024-05-21T03:22:45.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.472+0000 7f1ec640e700 1 Processor -- start 2024-05-21T03:22:45.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.472+0000 7f1ec640e700 1 -- start start 2024-05-21T03:22:45.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.472+0000 7f1ec640e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ec0101e60 0x7f1ec019bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:45.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.472+0000 7f1ec640e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec0102810 0x7f1ec019c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:45.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.472+0000 7f1ec640e700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ec0106ef0 0x7f1ec01a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:45.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.472+0000 7f1ec640e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec0112620 con 0x7f1ec0102810 2024-05-21T03:22:45.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.472+0000 7f1ec640e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec01124a0 con 0x7f1ec0101e60 2024-05-21T03:22:45.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.472+0000 7f1ec640e700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec01127a0 con 0x7f1ec0106ef0 2024-05-21T03:22:45.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.472+0000 7f1eb7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec0102810 0x7f1ec019c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:45.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.472+0000 7f1eb7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec0102810 0x7f1ec019c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47922/0 (socket says 172.21.15.27:47922) 2024-05-21T03:22:45.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.472+0000 7f1eb7fff700 1 -- 172.21.15.27:0/425438589 learned_addr learned my addr 172.21.15.27:0/425438589 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:45.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.472+0000 7f1ebffff700 1 --2- 172.21.15.27:0/425438589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ec0101e60 0x7f1ec019bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:45.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.472+0000 7f1ec49ab700 1 --2- 172.21.15.27:0/425438589 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ec0106ef0 0x7f1ec01a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:45.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.472+0000 7f1eb7fff700 1 -- 172.21.15.27:0/425438589 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ec0106ef0 msgr2=0x7f1ec01a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:45.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.472+0000 7f1eb7fff700 1 --2- 172.21.15.27:0/425438589 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ec0106ef0 0x7f1ec01a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:45.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.472+0000 7f1eb7fff700 1 -- 172.21.15.27:0/425438589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ec0101e60 msgr2=0x7f1ec019bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:45.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.472+0000 7f1eb7fff700 1 --2- 172.21.15.27:0/425438589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ec0101e60 0x7f1ec019bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:45.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.472+0000 7f1eb7fff700 1 -- 172.21.15.27:0/425438589 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec01a0d30 con 0x7f1ec0102810 2024-05-21T03:22:45.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.473+0000 7f1eb7fff700 1 --2- 172.21.15.27:0/425438589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec0102810 0x7f1ec019c260 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1ea800c9e0 tx=0x7f1ea800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:45.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.473+0000 7f1ebdffb700 1 -- 172.21.15.27:0/425438589 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8007aa0 con 0x7f1ec0102810 2024-05-21T03:22:45.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.473+0000 7f1ebdffb700 1 -- 172.21.15.27:0/425438589 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ea8007c40 con 0x7f1ec0102810 2024-05-21T03:22:45.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.473+0000 7f1ec640e700 1 -- 172.21.15.27:0/425438589 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec01a1020 con 0x7f1ec0102810 2024-05-21T03:22:45.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.473+0000 7f1ebdffb700 1 -- 172.21.15.27:0/425438589 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8018660 con 0x7f1ec0102810 2024-05-21T03:22:45.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.473+0000 7f1ec640e700 1 -- 172.21.15.27:0/425438589 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec01a88a0 con 0x7f1ec0102810 2024-05-21T03:22:45.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.474+0000 7f1ec640e700 1 -- 172.21.15.27:0/425438589 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ec01032f0 con 0x7f1ec0102810 2024-05-21T03:22:45.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.475+0000 7f1ebdffb700 1 -- 172.21.15.27:0/425438589 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1ea8018800 con 0x7f1ec0102810 2024-05-21T03:22:45.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.476+0000 7f1ebdffb700 1 --2- 172.21.15.27:0/425438589 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f1eac0b2880 0x7f1eac0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:45.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.476+0000 7f1ebffff700 1 --2- 172.21.15.27:0/425438589 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f1eac0b2880 0x7f1eac0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:45.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.476+0000 7f1ebdffb700 1 -- 172.21.15.27:0/425438589 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f1ea80e24a0 con 0x7f1ec0102810 2024-05-21T03:22:45.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.476+0000 7f1ebffff700 1 --2- 172.21.15.27:0/425438589 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f1eac0b2880 0x7f1eac0b4d40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0009990 tx=0x7f1eb000b870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:45.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.479+0000 7f1ebdffb700 1 -- 172.21.15.27:0/425438589 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1ea8014030 con 0x7f1ec0102810 2024-05-21T03:22:45.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.749+0000 7f1ec640e700 1 -- 172.21.15.27:0/425438589 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1ec002e560 con 0x7f1ec0102810 2024-05-21T03:22:45.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.749+0000 7f1ebdffb700 1 -- 172.21.15.27:0/425438589 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f1ea80aa230 con 0x7f1ec0102810 2024-05-21T03:22:45.748 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:22:45.748 INFO:teuthology.orchestra.run.smithi027.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-21T03:22:45.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.751+0000 7f1ec640e700 1 -- 172.21.15.27:0/425438589 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f1eac0b2880 msgr2=0x7f1eac0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:45.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.751+0000 7f1ec640e700 1 --2- 172.21.15.27:0/425438589 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f1eac0b2880 0x7f1eac0b4d40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0009990 tx=0x7f1eb000b870 comp rx=0 tx=0).stop 2024-05-21T03:22:45.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.751+0000 7f1ec640e700 1 -- 172.21.15.27:0/425438589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec0102810 msgr2=0x7f1ec019c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:45.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.751+0000 7f1ec640e700 1 --2- 172.21.15.27:0/425438589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec0102810 0x7f1ec019c260 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1ea800c9e0 tx=0x7f1ea800ccf0 comp rx=0 tx=0).stop 2024-05-21T03:22:45.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.751+0000 7f1ec640e700 1 -- 172.21.15.27:0/425438589 shutdown_connections 2024-05-21T03:22:45.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.751+0000 7f1ec640e700 1 --2- 172.21.15.27:0/425438589 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f1eac0b2880 0x7f1eac0b4d40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:45.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.751+0000 7f1ec640e700 1 --2- 172.21.15.27:0/425438589 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ec0106ef0 0x7f1ec01a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:45.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.751+0000 7f1ec640e700 1 --2- 172.21.15.27:0/425438589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ec0101e60 0x7f1ec019bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:45.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.751+0000 7f1ec640e700 1 --2- 172.21.15.27:0/425438589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec0102810 0x7f1ec019c260 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:45.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.751+0000 7f1ec640e700 1 -- 172.21.15.27:0/425438589 >> 172.21.15.27:0/425438589 conn(0x7f1ec0078ab0 msgr2=0x7f1ec010f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:45.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.752+0000 7f1ec640e700 1 -- 172.21.15.27:0/425438589 shutdown_connections 2024-05-21T03:22:45.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:45.752+0000 7f1ec640e700 1 -- 172.21.15.27:0/425438589 wait complete. 2024-05-21T03:22:46.397 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-21T03:22:46.398 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:22:46.658 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:22:46.665 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:46 smithi027 ceph-mon[32179]: pgmap v358: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:22:46.665 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:46 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/425438589' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-21T03:22:46.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:46 smithi039 ceph-mon[36429]: pgmap v358: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:22:46.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:46 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/425438589' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-21T03:22:46.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:46 smithi165 ceph-mon[36269]: pgmap v358: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:22:46.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:46 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/425438589' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-21T03:22:48.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.245+0000 7f6b90ce3700 1 -- 172.21.15.27:0/2829633140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b8c06c6d0 msgr2=0x7f6b8c110400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:48.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.245+0000 7f6b90ce3700 1 --2- 172.21.15.27:0/2829633140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b8c06c6d0 0x7f6b8c110400 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6b840099c0 tx=0x7f6b84009cd0 comp rx=0 tx=0).stop 2024-05-21T03:22:48.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.245+0000 7f6b90ce3700 1 -- 172.21.15.27:0/2829633140 shutdown_connections 2024-05-21T03:22:48.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.245+0000 7f6b90ce3700 1 --2- 172.21.15.27:0/2829633140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b8c06c6d0 0x7f6b8c110400 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:48.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.245+0000 7f6b90ce3700 1 --2- 172.21.15.27:0/2829633140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b8c109630 0x7f6b8c109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:48.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.245+0000 7f6b90ce3700 1 --2- 172.21.15.27:0/2829633140 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b8c06bd10 0x7f6b8c06c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:48.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.245+0000 7f6b90ce3700 1 -- 172.21.15.27:0/2829633140 >> 172.21.15.27:0/2829633140 conn(0x7f6b8c0794e0 msgr2=0x7f6b8c0798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:48.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.245+0000 7f6b90ce3700 1 -- 172.21.15.27:0/2829633140 shutdown_connections 2024-05-21T03:22:48.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.246+0000 7f6b90ce3700 1 -- 172.21.15.27:0/2829633140 wait complete. 2024-05-21T03:22:48.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.246+0000 7f6b90ce3700 1 Processor -- start 2024-05-21T03:22:48.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.246+0000 7f6b90ce3700 1 -- start start 2024-05-21T03:22:48.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.246+0000 7f6b90ce3700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b8c06bd10 0x7f6b8c1024b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:48.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.246+0000 7f6b90ce3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b8c06c6d0 0x7f6b8c1029f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:48.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.247+0000 7f6b90ce3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b8c109630 0x7f6b8c102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:48.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.247+0000 7f6b90ce3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c113860 con 0x7f6b8c109630 2024-05-21T03:22:48.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.247+0000 7f6b90ce3700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c1136e0 con 0x7f6b8c06c6d0 2024-05-21T03:22:48.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.247+0000 7f6b90ce3700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c1139e0 con 0x7f6b8c06bd10 2024-05-21T03:22:48.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.247+0000 7f6b8ad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b8c109630 0x7f6b8c102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:48.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.247+0000 7f6b8ad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b8c109630 0x7f6b8c102f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47948/0 (socket says 172.21.15.27:47948) 2024-05-21T03:22:48.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.247+0000 7f6b8ad9d700 1 -- 172.21.15.27:0/4101289256 learned_addr learned my addr 172.21.15.27:0/4101289256 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:48.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.247+0000 7f6b8a59c700 1 --2- 172.21.15.27:0/4101289256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b8c06bd10 0x7f6b8c1024b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:48.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.247+0000 7f6b89d9b700 1 --2- 172.21.15.27:0/4101289256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b8c06c6d0 0x7f6b8c1029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:48.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.247+0000 7f6b8ad9d700 1 -- 172.21.15.27:0/4101289256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b8c06bd10 msgr2=0x7f6b8c1024b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:48.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.247+0000 7f6b8ad9d700 1 --2- 172.21.15.27:0/4101289256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b8c06bd10 0x7f6b8c1024b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:48.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.247+0000 7f6b8ad9d700 1 -- 172.21.15.27:0/4101289256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b8c06c6d0 msgr2=0x7f6b8c1029f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:48.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.247+0000 7f6b8ad9d700 1 --2- 172.21.15.27:0/4101289256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b8c06c6d0 0x7f6b8c1029f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:48.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.247+0000 7f6b8ad9d700 1 -- 172.21.15.27:0/4101289256 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b8c1a81b0 con 0x7f6b8c109630 2024-05-21T03:22:48.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.247+0000 7f6b89d9b700 1 --2- 172.21.15.27:0/4101289256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b8c06c6d0 0x7f6b8c1029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:22:48.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.247+0000 7f6b8a59c700 1 --2- 172.21.15.27:0/4101289256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b8c06bd10 0x7f6b8c1024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:22:48.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.247+0000 7f6b8ad9d700 1 --2- 172.21.15.27:0/4101289256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b8c109630 0x7f6b8c102f30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6b84009990 tx=0x7f6b840036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:48.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.247+0000 7f6b837fe700 1 -- 172.21.15.27:0/4101289256 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b84027070 con 0x7f6b8c109630 2024-05-21T03:22:48.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.247+0000 7f6b90ce3700 1 -- 172.21.15.27:0/4101289256 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b8c1a8440 con 0x7f6b8c109630 2024-05-21T03:22:48.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.247+0000 7f6b837fe700 1 -- 172.21.15.27:0/4101289256 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b84004020 con 0x7f6b8c109630 2024-05-21T03:22:48.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.247+0000 7f6b837fe700 1 -- 172.21.15.27:0/4101289256 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b84022d40 con 0x7f6b8c109630 2024-05-21T03:22:48.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.248+0000 7f6b90ce3700 1 -- 172.21.15.27:0/4101289256 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b8c1a8840 con 0x7f6b8c109630 2024-05-21T03:22:48.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.248+0000 7f6b90ce3700 1 -- 172.21.15.27:0/4101289256 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b8c10bac0 con 0x7f6b8c109630 2024-05-21T03:22:48.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.250+0000 7f6b837fe700 1 -- 172.21.15.27:0/4101289256 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6b8402a070 con 0x7f6b8c109630 2024-05-21T03:22:48.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.250+0000 7f6b837fe700 1 --2- 172.21.15.27:0/4101289256 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f6b780b2880 0x7f6b780b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:48.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.251+0000 7f6b8a59c700 1 --2- 172.21.15.27:0/4101289256 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f6b780b2880 0x7f6b780b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:48.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.251+0000 7f6b837fe700 1 -- 172.21.15.27:0/4101289256 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f6b840e3a20 con 0x7f6b8c109630 2024-05-21T03:22:48.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.251+0000 7f6b8a59c700 1 --2- 172.21.15.27:0/4101289256 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f6b780b2880 0x7f6b780b4d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c009fa0 tx=0x7f6b7c009570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:48.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.254+0000 7f6b837fe700 1 -- 172.21.15.27:0/4101289256 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6b84031020 con 0x7f6b8c109630 2024-05-21T03:22:48.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.516+0000 7f6b90ce3700 1 -- 172.21.15.27:0/4101289256 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b8c06a230 con 0x7f6b8c109630 2024-05-21T03:22:48.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.516+0000 7f6b837fe700 1 -- 172.21.15.27:0/4101289256 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11924 (secure 0 0 0) 0x7f6b84013070 con 0x7f6b8c109630 2024-05-21T03:22:48.516 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:22:48.516 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/148854869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":148854869},{"type":"v1","addr":"172.21.15.27:6835","nonce":148854869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/4145544730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":4145544730},{"type":"v1","addr":"172.21.15.27:6837","nonce":4145544730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T03:20:34.055985+0000","modified":"2024-05-21T03:20:52.467919+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24472,"mds_2":24457,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/745486764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":745486764},{"type":"v1","addr":"172.21.15.27:6839","nonce":745486764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6841/3982752351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3982752351},{"type":"v1","addr":"172.21.15.27:6841","nonce":3982752351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/3497521608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3497521608},{"type":"v1","addr":"172.21.15.39:6835","nonce":3497521608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.165:6835/3576910218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3576910218},{"type":"v1","addr":"172.21.15.165:6835","nonce":3576910218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/3673846852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3673846852},{"type":"v1","addr":"172.21.15.39:6837","nonce":3673846852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6837/691356139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":691356139},{"type":"v1","addr":"172.21.15.165:6837","nonce":691356139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6839/4158170814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4158170814},{"type":"v1","addr":"172.21.15.165:6839","nonce":4158170814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6839/287607819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":287607819},{"type":"v1","addr":"172.21.15.39:6839","nonce":287607819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24451,24472,24452,24457,24476]},"id":1}]} 2024-05-21T03:22:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.519+0000 7f6b90ce3700 1 -- 172.21.15.27:0/4101289256 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f6b780b2880 msgr2=0x7f6b780b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.519+0000 7f6b90ce3700 1 --2- 172.21.15.27:0/4101289256 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f6b780b2880 0x7f6b780b4d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c009fa0 tx=0x7f6b7c009570 comp rx=0 tx=0).stop 2024-05-21T03:22:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.519+0000 7f6b90ce3700 1 -- 172.21.15.27:0/4101289256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b8c109630 msgr2=0x7f6b8c102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:48.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.519+0000 7f6b90ce3700 1 --2- 172.21.15.27:0/4101289256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b8c109630 0x7f6b8c102f30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6b84009990 tx=0x7f6b840036f0 comp rx=0 tx=0).stop 2024-05-21T03:22:48.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.519+0000 7f6b90ce3700 1 -- 172.21.15.27:0/4101289256 shutdown_connections 2024-05-21T03:22:48.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.519+0000 7f6b90ce3700 1 --2- 172.21.15.27:0/4101289256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b8c109630 0x7f6b8c102f30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:48.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.519+0000 7f6b90ce3700 1 --2- 172.21.15.27:0/4101289256 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f6b780b2880 0x7f6b780b4d40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:48.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.519+0000 7f6b90ce3700 1 --2- 172.21.15.27:0/4101289256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b8c06bd10 0x7f6b8c1024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:48.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.519+0000 7f6b90ce3700 1 --2- 172.21.15.27:0/4101289256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b8c06c6d0 0x7f6b8c1029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:48.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.519+0000 7f6b90ce3700 1 -- 172.21.15.27:0/4101289256 >> 172.21.15.27:0/4101289256 conn(0x7f6b8c0794e0 msgr2=0x7f6b8c100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:48.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.519+0000 7f6b90ce3700 1 -- 172.21.15.27:0/4101289256 shutdown_connections 2024-05-21T03:22:48.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:48.519+0000 7f6b90ce3700 1 -- 172.21.15.27:0/4101289256 wait complete. 2024-05-21T03:22:48.519 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 28 2024-05-21T03:22:48.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:48 smithi039 ceph-mon[36429]: pgmap v359: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:22:48.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:48 smithi165 ceph-mon[36269]: pgmap v359: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:22:48.874 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:48 smithi027 ceph-mon[32179]: pgmap v359: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:22:49.182 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:22:49.437 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:22:49.699 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:49 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/4101289256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:22:49.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:49 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/4101289256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:22:49.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:49 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/4101289256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:22:50.540 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:50 smithi027 ceph-mon[32179]: pgmap v360: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:22:50.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:50 smithi039 ceph-mon[36429]: pgmap v360: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:22:50.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:50 smithi165 ceph-mon[36269]: pgmap v360: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:22:51.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.074+0000 7fb746ec9700 1 -- 172.21.15.27:0/1257394559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7401026a0 msgr2=0x7fb740108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:51.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.074+0000 7fb746ec9700 1 --2- 172.21.15.27:0/1257394559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7401026a0 0x7fb740108200 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb72c0099c0 tx=0x7fb72c009cd0 comp rx=0 tx=0).stop 2024-05-21T03:22:51.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.074+0000 7fb746ec9700 1 -- 172.21.15.27:0/1257394559 shutdown_connections 2024-05-21T03:22:51.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.074+0000 7fb746ec9700 1 --2- 172.21.15.27:0/1257394559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb740108930 0x7fb7401104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:51.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.074+0000 7fb746ec9700 1 --2- 172.21.15.27:0/1257394559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb740101cf0 0x7fb7401020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:51.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.074+0000 7fb746ec9700 1 --2- 172.21.15.27:0/1257394559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7401026a0 0x7fb740108200 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:51.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.074+0000 7fb746ec9700 1 -- 172.21.15.27:0/1257394559 >> 172.21.15.27:0/1257394559 conn(0x7fb740078a40 msgr2=0x7fb7400ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:51.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.074+0000 7fb746ec9700 1 -- 172.21.15.27:0/1257394559 shutdown_connections 2024-05-21T03:22:51.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.074+0000 7fb746ec9700 1 -- 172.21.15.27:0/1257394559 wait complete. 2024-05-21T03:22:51.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.075+0000 7fb746ec9700 1 Processor -- start 2024-05-21T03:22:51.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.075+0000 7fb746ec9700 1 -- start start 2024-05-21T03:22:51.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.075+0000 7fb746ec9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb740101cf0 0x7fb74019f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:51.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.075+0000 7fb746ec9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7401026a0 0x7fb74019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:51.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.075+0000 7fb746ec9700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb740108930 0x7fb740199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:51.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.075+0000 7fb746ec9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb740113710 con 0x7fb740101cf0 2024-05-21T03:22:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.075+0000 7fb746ec9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb740113590 con 0x7fb7401026a0 2024-05-21T03:22:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.075+0000 7fb746ec9700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb740113890 con 0x7fb740108930 2024-05-21T03:22:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.075+0000 7fb744c65700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb740101cf0 0x7fb74019f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.075+0000 7fb744c65700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb740101cf0 0x7fb74019f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47968/0 (socket says 172.21.15.27:47968) 2024-05-21T03:22:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.075+0000 7fb744c65700 1 -- 172.21.15.27:0/1731198583 learned_addr learned my addr 172.21.15.27:0/1731198583 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.075+0000 7fb73ffff700 1 --2- 172.21.15.27:0/1731198583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7401026a0 0x7fb74019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.075+0000 7fb745466700 1 --2- 172.21.15.27:0/1731198583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb740108930 0x7fb740199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.075+0000 7fb744c65700 1 -- 172.21.15.27:0/1731198583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb740108930 msgr2=0x7fb740199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.075+0000 7fb744c65700 1 --2- 172.21.15.27:0/1731198583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb740108930 0x7fb740199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.075+0000 7fb744c65700 1 -- 172.21.15.27:0/1731198583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7401026a0 msgr2=0x7fb74019fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.075+0000 7fb744c65700 1 --2- 172.21.15.27:0/1731198583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7401026a0 0x7fb74019fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.075+0000 7fb744c65700 1 -- 172.21.15.27:0/1731198583 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb74019a0f0 con 0x7fb740101cf0 2024-05-21T03:22:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.076+0000 7fb744c65700 1 --2- 172.21.15.27:0/1731198583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb740101cf0 0x7fb74019f7d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb73400e9a0 tx=0x7fb73400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.076+0000 7fb73dffb700 1 -- 172.21.15.27:0/1731198583 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73400ca10 con 0x7fb740101cf0 2024-05-21T03:22:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.076+0000 7fb73dffb700 1 -- 172.21.15.27:0/1731198583 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb73400cbb0 con 0x7fb740101cf0 2024-05-21T03:22:51.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.076+0000 7fb73dffb700 1 -- 172.21.15.27:0/1731198583 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb734018680 con 0x7fb740101cf0 2024-05-21T03:22:51.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.076+0000 7fb746ec9700 1 -- 172.21.15.27:0/1731198583 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb74019a3e0 con 0x7fb740101cf0 2024-05-21T03:22:51.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.076+0000 7fb746ec9700 1 -- 172.21.15.27:0/1731198583 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7401a66a0 con 0x7fb740101cf0 2024-05-21T03:22:51.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.077+0000 7fb746ec9700 1 -- 172.21.15.27:0/1731198583 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb74006a230 con 0x7fb740101cf0 2024-05-21T03:22:51.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.079+0000 7fb73dffb700 1 -- 172.21.15.27:0/1731198583 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb734010430 con 0x7fb740101cf0 2024-05-21T03:22:51.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.079+0000 7fb73dffb700 1 --2- 172.21.15.27:0/1731198583 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb7300b2900 0x7fb7300b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:51.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.079+0000 7fb73ffff700 1 --2- 172.21.15.27:0/1731198583 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb7300b2900 0x7fb7300b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:51.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.079+0000 7fb73dffb700 1 -- 172.21.15.27:0/1731198583 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fb7340e2660 con 0x7fb740101cf0 2024-05-21T03:22:51.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.080+0000 7fb73ffff700 1 --2- 172.21.15.27:0/1731198583 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb7300b2900 0x7fb7300b4dc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb740100eb0 tx=0x7fb72c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:51.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.083+0000 7fb73dffb700 1 -- 172.21.15.27:0/1731198583 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb7340aa360 con 0x7fb740101cf0 2024-05-21T03:22:51.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.342+0000 7fb746ec9700 1 -- 172.21.15.27:0/1731198583 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb74002e560 con 0x7fb740101cf0 2024-05-21T03:22:51.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.343+0000 7fb73dffb700 1 -- 172.21.15.27:0/1731198583 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11924 (secure 0 0 0) 0x7fb7340b23e0 con 0x7fb740101cf0 2024-05-21T03:22:51.342 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:22:51.343 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/148854869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":148854869},{"type":"v1","addr":"172.21.15.27:6835","nonce":148854869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/4145544730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":4145544730},{"type":"v1","addr":"172.21.15.27:6837","nonce":4145544730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T03:20:34.055985+0000","modified":"2024-05-21T03:20:52.467919+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24472,"mds_2":24457,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/745486764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":745486764},{"type":"v1","addr":"172.21.15.27:6839","nonce":745486764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6841/3982752351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3982752351},{"type":"v1","addr":"172.21.15.27:6841","nonce":3982752351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/3497521608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3497521608},{"type":"v1","addr":"172.21.15.39:6835","nonce":3497521608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.165:6835/3576910218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3576910218},{"type":"v1","addr":"172.21.15.165:6835","nonce":3576910218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/3673846852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3673846852},{"type":"v1","addr":"172.21.15.39:6837","nonce":3673846852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6837/691356139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":691356139},{"type":"v1","addr":"172.21.15.165:6837","nonce":691356139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6839/4158170814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4158170814},{"type":"v1","addr":"172.21.15.165:6839","nonce":4158170814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6839/287607819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":287607819},{"type":"v1","addr":"172.21.15.39:6839","nonce":287607819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24451,24472,24452,24457,24476]},"id":1}]} 2024-05-21T03:22:51.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.345+0000 7fb746ec9700 1 -- 172.21.15.27:0/1731198583 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb7300b2900 msgr2=0x7fb7300b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:51.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.345+0000 7fb746ec9700 1 --2- 172.21.15.27:0/1731198583 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb7300b2900 0x7fb7300b4dc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb740100eb0 tx=0x7fb72c005fb0 comp rx=0 tx=0).stop 2024-05-21T03:22:51.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.345+0000 7fb746ec9700 1 -- 172.21.15.27:0/1731198583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb740101cf0 msgr2=0x7fb74019f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:51.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.345+0000 7fb746ec9700 1 --2- 172.21.15.27:0/1731198583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb740101cf0 0x7fb74019f7d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb73400e9a0 tx=0x7fb73400ecb0 comp rx=0 tx=0).stop 2024-05-21T03:22:51.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.345+0000 7fb746ec9700 1 -- 172.21.15.27:0/1731198583 shutdown_connections 2024-05-21T03:22:51.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.345+0000 7fb746ec9700 1 --2- 172.21.15.27:0/1731198583 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fb7300b2900 0x7fb7300b4dc0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:51.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.345+0000 7fb746ec9700 1 --2- 172.21.15.27:0/1731198583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb740108930 0x7fb740199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:51.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.345+0000 7fb746ec9700 1 --2- 172.21.15.27:0/1731198583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb740101cf0 0x7fb74019f7d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:51.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.345+0000 7fb746ec9700 1 --2- 172.21.15.27:0/1731198583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7401026a0 0x7fb74019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:51.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.345+0000 7fb746ec9700 1 -- 172.21.15.27:0/1731198583 >> 172.21.15.27:0/1731198583 conn(0x7fb740078a40 msgr2=0x7fb740105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:51.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.345+0000 7fb746ec9700 1 -- 172.21.15.27:0/1731198583 shutdown_connections 2024-05-21T03:22:51.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:51.345+0000 7fb746ec9700 1 -- 172.21.15.27:0/1731198583 wait complete. 2024-05-21T03:22:51.346 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 28 2024-05-21T03:22:51.637 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:51 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/1731198583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:22:51.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:51 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/1731198583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:22:51.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:51 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/1731198583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:22:51.934 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-21T03:22:52.185 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:22:52.650 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:52 smithi027 ceph-mon[32179]: pgmap v361: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:22:52.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:52 smithi039 ceph-mon[36429]: pgmap v361: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:22:52.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:52 smithi165 ceph-mon[36269]: pgmap v361: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:22:53.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.807+0000 7f883b2c9700 1 -- 172.21.15.27:0/3364381125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f883406c590 msgr2=0x7f883410fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:53.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.807+0000 7f883b2c9700 1 --2- 172.21.15.27:0/3364381125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f883406c590 0x7f883410fae0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8824009990 tx=0x7f8824009ca0 comp rx=0 tx=0).stop 2024-05-21T03:22:53.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.807+0000 7f883b2c9700 1 -- 172.21.15.27:0/3364381125 shutdown_connections 2024-05-21T03:22:53.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.807+0000 7f883b2c9700 1 --2- 172.21.15.27:0/3364381125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8834110020 0x7f8834112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:53.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.807+0000 7f883b2c9700 1 --2- 172.21.15.27:0/3364381125 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f883406bc70 0x7f883406c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:53.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.807+0000 7f883b2c9700 1 --2- 172.21.15.27:0/3364381125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f883406c590 0x7f883410fae0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:53.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.807+0000 7f883b2c9700 1 -- 172.21.15.27:0/3364381125 >> 172.21.15.27:0/3364381125 conn(0x7f88340793f0 msgr2=0x7f8834079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:53.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.807+0000 7f883b2c9700 1 -- 172.21.15.27:0/3364381125 shutdown_connections 2024-05-21T03:22:53.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.807+0000 7f883b2c9700 1 -- 172.21.15.27:0/3364381125 wait complete. 2024-05-21T03:22:53.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.808+0000 7f883b2c9700 1 Processor -- start 2024-05-21T03:22:53.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.808+0000 7f883b2c9700 1 -- start start 2024-05-21T03:22:53.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.808+0000 7f883b2c9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f883406bc70 0x7f883419bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:53.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.808+0000 7f883b2c9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f883406c590 0x7f883419c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:53.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.808+0000 7f883b2c9700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8834110020 0x7f88341a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:53.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.808+0000 7f883b2c9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8834115970 con 0x7f883406c590 2024-05-21T03:22:53.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.808+0000 7f883b2c9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f88341157f0 con 0x7f883406bc70 2024-05-21T03:22:53.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.808+0000 7f883b2c9700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8834115af0 con 0x7f8834110020 2024-05-21T03:22:53.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.808+0000 7f8838864700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f883406c590 0x7f883419c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:53.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.808+0000 7f8838864700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f883406c590 0x7f883419c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43178/0 (socket says 172.21.15.27:43178) 2024-05-21T03:22:53.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.808+0000 7f8838864700 1 -- 172.21.15.27:0/3025549823 learned_addr learned my addr 172.21.15.27:0/3025549823 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:53.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.808+0000 7f8839065700 1 --2- 172.21.15.27:0/3025549823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f883406bc70 0x7f883419bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:53.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.808+0000 7f8839866700 1 --2- 172.21.15.27:0/3025549823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8834110020 0x7f88341a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:53.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.808+0000 7f8838864700 1 -- 172.21.15.27:0/3025549823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8834110020 msgr2=0x7f88341a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:53.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.808+0000 7f8838864700 1 --2- 172.21.15.27:0/3025549823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8834110020 0x7f88341a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:53.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.808+0000 7f8838864700 1 -- 172.21.15.27:0/3025549823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f883406bc70 msgr2=0x7f883419bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:53.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.808+0000 7f8838864700 1 --2- 172.21.15.27:0/3025549823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f883406bc70 0x7f883419bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:53.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.809+0000 7f8838864700 1 -- 172.21.15.27:0/3025549823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88341a0ed0 con 0x7f883406c590 2024-05-21T03:22:53.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.809+0000 7f8838864700 1 --2- 172.21.15.27:0/3025549823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f883406c590 0x7f883419c400 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f88240057c0 tx=0x7f88240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:53.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.809+0000 7f882a7fc700 1 -- 172.21.15.27:0/3025549823 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8824027070 con 0x7f883406c590 2024-05-21T03:22:53.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.809+0000 7f882a7fc700 1 -- 172.21.15.27:0/3025549823 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8824003f20 con 0x7f883406c590 2024-05-21T03:22:53.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.809+0000 7f883b2c9700 1 -- 172.21.15.27:0/3025549823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88341a1160 con 0x7f883406c590 2024-05-21T03:22:53.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.809+0000 7f882a7fc700 1 -- 172.21.15.27:0/3025549823 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8824022ba0 con 0x7f883406c590 2024-05-21T03:22:53.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.809+0000 7f883b2c9700 1 -- 172.21.15.27:0/3025549823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88341a8a40 con 0x7f883406c590 2024-05-21T03:22:53.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.810+0000 7f883b2c9700 1 -- 172.21.15.27:0/3025549823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f883406a230 con 0x7f883406c590 2024-05-21T03:22:53.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.812+0000 7f882a7fc700 1 -- 172.21.15.27:0/3025549823 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f882402a070 con 0x7f883406c590 2024-05-21T03:22:53.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.812+0000 7f882a7fc700 1 --2- 172.21.15.27:0/3025549823 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f88200b2880 0x7f88200b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:53.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.812+0000 7f8839065700 1 --2- 172.21.15.27:0/3025549823 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f88200b2880 0x7f88200b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:53.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.812+0000 7f882a7fc700 1 -- 172.21.15.27:0/3025549823 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f88240e3ab0 con 0x7f883406c590 2024-05-21T03:22:53.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.813+0000 7f8839065700 1 --2- 172.21.15.27:0/3025549823 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f88200b2880 0x7f88200b4d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8830005950 tx=0x7f883000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:53.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:53.816+0000 7f882a7fc700 1 -- 172.21.15.27:0/3025549823 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8824013070 con 0x7f883406c590 2024-05-21T03:22:54.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:54.034+0000 7f883b2c9700 1 -- 172.21.15.27:0/3025549823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f883402e560 con 0x7f883406c590 2024-05-21T03:22:54.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:54.035+0000 7f882a7fc700 1 -- 172.21.15.27:0/3025549823 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22683 (secure 0 0 0) 0x7f88240ab840 con 0x7f883406c590 2024-05-21T03:22:54.035 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:22:54.035 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":82,"fsid":"c9ca94be-171f-11ef-bc9a-c7b262605968","created":"2024-05-21T03:12:18.217236+0000","modified":"2024-05-21T03:21:14.540989+0000","last_up_change":"2024-05-21T03:19:30.382150+0000","last_in_change":"2024-05-21T03:19:09.058249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T03:15:41.340194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T03:20:27.507346+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T03:20:30.830169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c745c09f-9a76-4d05-af75-c7554f6d5e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6803","nonce":1399996449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6805","nonce":1399996449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6809","nonce":1399996449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6807","nonce":1399996449}]},"public_addr":"172.21.15.27:6803/1399996449","cluster_addr":"172.21.15.27:6805/1399996449","heartbeat_back_addr":"172.21.15.27:6809/1399996449","heartbeat_front_addr":"172.21.15.27:6807/1399996449","state":["exists","up"]},{"osd":1,"uuid":"8b73aa00-4364-438f-9722-53caa4e9bec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6801","nonce":1776318847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6803","nonce":1776318847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6807","nonce":1776318847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6805","nonce":1776318847}]},"public_addr":"172.21.15.39:6801/1776318847","cluster_addr":"172.21.15.39:6803/1776318847","heartbeat_back_addr":"172.21.15.39:6807/1776318847","heartbeat_front_addr":"172.21.15.39:6805/1776318847","state":["exists","up"]},{"osd":2,"uuid":"0822043e-7d3c-4e98-9dc0-94b98b4d562a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6801","nonce":703322625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6803","nonce":703322625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6807","nonce":703322625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6805","nonce":703322625}]},"public_addr":"172.21.15.165:6801/703322625","cluster_addr":"172.21.15.165:6803/703322625","heartbeat_back_addr":"172.21.15.165:6807/703322625","heartbeat_front_addr":"172.21.15.165:6805/703322625","state":["exists","up"]},{"osd":3,"uuid":"c563b1b2-efcb-4f8c-8121-c1938f83131e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6811","nonce":1933403135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6813","nonce":1933403135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6817","nonce":1933403135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6815","nonce":1933403135}]},"public_addr":"172.21.15.27:6811/1933403135","cluster_addr":"172.21.15.27:6813/1933403135","heartbeat_back_addr":"172.21.15.27:6817/1933403135","heartbeat_front_addr":"172.21.15.27:6815/1933403135","state":["exists","up"]},{"osd":4,"uuid":"0c793b87-b045-4b4a-89e7-87ec36deeb0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6809","nonce":155579331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6811","nonce":155579331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6815","nonce":155579331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6813","nonce":155579331}]},"public_addr":"172.21.15.39:6809/155579331","cluster_addr":"172.21.15.39:6811/155579331","heartbeat_back_addr":"172.21.15.39:6815/155579331","heartbeat_front_addr":"172.21.15.39:6813/155579331","state":["exists","up"]},{"osd":5,"uuid":"367eeb5a-9d75-4644-bf46-317c03f33d8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6809","nonce":386850244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6811","nonce":386850244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6815","nonce":386850244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6813","nonce":386850244}]},"public_addr":"172.21.15.165:6809/386850244","cluster_addr":"172.21.15.165:6811/386850244","heartbeat_back_addr":"172.21.15.165:6815/386850244","heartbeat_front_addr":"172.21.15.165:6813/386850244","state":["exists","up"]},{"osd":6,"uuid":"e000167f-24fd-4115-83f0-8afbfa8d8991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6819","nonce":3531376679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6821","nonce":3531376679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6825","nonce":3531376679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6823","nonce":3531376679}]},"public_addr":"172.21.15.27:6819/3531376679","cluster_addr":"172.21.15.27:6821/3531376679","heartbeat_back_addr":"172.21.15.27:6825/3531376679","heartbeat_front_addr":"172.21.15.27:6823/3531376679","state":["exists","up"]},{"osd":7,"uuid":"4b8d486b-868e-4f33-98a1-b99686adeaaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6817","nonce":1550035852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6819","nonce":1550035852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6823","nonce":1550035852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6821","nonce":1550035852}]},"public_addr":"172.21.15.39:6817/1550035852","cluster_addr":"172.21.15.39:6819/1550035852","heartbeat_back_addr":"172.21.15.39:6823/1550035852","heartbeat_front_addr":"172.21.15.39:6821/1550035852","state":["exists","up"]},{"osd":8,"uuid":"2259f35c-2edd-4331-9a63-8c0c1acbdc71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6817","nonce":3184704971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6819","nonce":3184704971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6823","nonce":3184704971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6821","nonce":3184704971}]},"public_addr":"172.21.15.165:6817/3184704971","cluster_addr":"172.21.15.165:6819/3184704971","heartbeat_back_addr":"172.21.15.165:6823/3184704971","heartbeat_front_addr":"172.21.15.165:6821/3184704971","state":["exists","up"]},{"osd":9,"uuid":"10307ecc-8270-4be2-b62b-9708e09bb207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6827","nonce":2800048793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6829","nonce":2800048793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6833","nonce":2800048793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6831","nonce":2800048793}]},"public_addr":"172.21.15.27:6827/2800048793","cluster_addr":"172.21.15.27:6829/2800048793","heartbeat_back_addr":"172.21.15.27:6833/2800048793","heartbeat_front_addr":"172.21.15.27:6831/2800048793","state":["exists","up"]},{"osd":10,"uuid":"dfe1b8ca-ff8a-4b7f-ade5-ad4f00cf60cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6825","nonce":363500811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6827","nonce":363500811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6831","nonce":363500811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6829","nonce":363500811}]},"public_addr":"172.21.15.39:6825/363500811","cluster_addr":"172.21.15.39:6827/363500811","heartbeat_back_addr":"172.21.15.39:6831/363500811","heartbeat_front_addr":"172.21.15.39:6829/363500811","state":["exists","up"]},{"osd":11,"uuid":"f9531145-5d09-4dbd-a050-3159ee09db24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6825","nonce":2274332243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6827","nonce":2274332243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6831","nonce":2274332243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6829","nonce":2274332243}]},"public_addr":"172.21.15.165:6825/2274332243","cluster_addr":"172.21.15.165:6827/2274332243","heartbeat_back_addr":"172.21.15.165:6831/2274332243","heartbeat_front_addr":"172.21.15.165:6829/2274332243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:14:47.077860+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:15:13.272879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:15:38.046651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:03.376483+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:29.179280+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:53.391261+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:17:18.472829+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:17:44.349925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:18:08.652193+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:18:35.282669+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:19:01.637151+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:19:27.461566+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6800/1185518061":"2024-05-22T03:13:13.249220+0000","172.21.15.27:6801/1185518061":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/4194431212":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/1890032101":"2024-05-22T03:13:13.249220+0000","172.21.15.27:6800/175229478":"2024-05-22T03:12:45.739195+0000","172.21.15.27:0/999325706":"2024-05-22T03:12:45.739195+0000","172.21.15.27:6801/175229478":"2024-05-22T03:12:45.739195+0000","172.21.15.27:0/4184128736":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/3000197506":"2024-05-22T03:12:45.739195+0000","172.21.15.27:0/972677319":"2024-05-22T03:12:45.739195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T03:22:54.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:54.038+0000 7f883b2c9700 1 -- 172.21.15.27:0/3025549823 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f88200b2880 msgr2=0x7f88200b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:54.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:54.038+0000 7f883b2c9700 1 --2- 172.21.15.27:0/3025549823 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f88200b2880 0x7f88200b4d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8830005950 tx=0x7f883000b410 comp rx=0 tx=0).stop 2024-05-21T03:22:54.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:54.038+0000 7f883b2c9700 1 -- 172.21.15.27:0/3025549823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f883406c590 msgr2=0x7f883419c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:54.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:54.038+0000 7f883b2c9700 1 --2- 172.21.15.27:0/3025549823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f883406c590 0x7f883419c400 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f88240057c0 tx=0x7f88240057f0 comp rx=0 tx=0).stop 2024-05-21T03:22:54.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:54.038+0000 7f883b2c9700 1 -- 172.21.15.27:0/3025549823 shutdown_connections 2024-05-21T03:22:54.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:54.038+0000 7f883b2c9700 1 --2- 172.21.15.27:0/3025549823 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f88200b2880 0x7f88200b4d40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:54.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:54.038+0000 7f883b2c9700 1 --2- 172.21.15.27:0/3025549823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8834110020 0x7f88341a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:54.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:54.038+0000 7f883b2c9700 1 --2- 172.21.15.27:0/3025549823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f883406bc70 0x7f883419bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:54.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:54.038+0000 7f883b2c9700 1 --2- 172.21.15.27:0/3025549823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f883406c590 0x7f883419c400 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:54.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:54.038+0000 7f883b2c9700 1 -- 172.21.15.27:0/3025549823 >> 172.21.15.27:0/3025549823 conn(0x7f88340793f0 msgr2=0x7f8834111590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:54.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:54.038+0000 7f883b2c9700 1 -- 172.21.15.27:0/3025549823 shutdown_connections 2024-05-21T03:22:54.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:54.038+0000 7f883b2c9700 1 -- 172.21.15.27:0/3025549823 wait complete. 2024-05-21T03:22:54.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:54 smithi027 ceph-mon[32179]: pgmap v362: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:22:54.662 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:54 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3025549823' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:22:54.691 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:22:54.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:54 smithi039 ceph-mon[36429]: pgmap v362: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:22:54.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:54 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/3025549823' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:22:54.773 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:54 smithi165 ceph-mon[36269]: pgmap v362: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:22:54.773 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:54 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/3025549823' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:22:54.942 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:22:56.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.602+0000 7f26fad85700 1 -- 172.21.15.27:0/3416472269 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26f41096a0 msgr2=0x7f26f4109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:56.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.602+0000 7f26fad85700 1 --2- 172.21.15.27:0/3416472269 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26f41096a0 0x7f26f4109a80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f26e400b0a0 tx=0x7f26e400b3b0 comp rx=0 tx=0).stop 2024-05-21T03:22:56.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.602+0000 7f26fad85700 1 -- 172.21.15.27:0/3416472269 shutdown_connections 2024-05-21T03:22:56.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.602+0000 7f26fad85700 1 --2- 172.21.15.27:0/3416472269 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26f4103f50 0x7f26f4110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:56.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.602+0000 7f26fad85700 1 --2- 172.21.15.27:0/3416472269 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26f41096a0 0x7f26f4109a80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:56.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.602+0000 7f26fad85700 1 --2- 172.21.15.27:0/3416472269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26f4103590 0x7f26f4103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:56.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.602+0000 7f26fad85700 1 -- 172.21.15.27:0/3416472269 >> 172.21.15.27:0/3416472269 conn(0x7f26f407a6f0 msgr2=0x7f26f407ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:56.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.602+0000 7f26fad85700 1 -- 172.21.15.27:0/3416472269 shutdown_connections 2024-05-21T03:22:56.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.602+0000 7f26fad85700 1 -- 172.21.15.27:0/3416472269 wait complete. 2024-05-21T03:22:56.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.603+0000 7f26fad85700 1 Processor -- start 2024-05-21T03:22:56.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.603+0000 7f26fad85700 1 -- start start 2024-05-21T03:22:56.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.603+0000 7f26fad85700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26f4103590 0x7f26f419f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:56.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.603+0000 7f26fad85700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26f4103f50 0x7f26f419fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:56.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.603+0000 7f26fad85700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26f41096a0 0x7f26f4199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:56.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.603+0000 7f26fad85700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f26f4113790 con 0x7f26f41096a0 2024-05-21T03:22:56.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.603+0000 7f26fad85700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f26f4113610 con 0x7f26f4103f50 2024-05-21T03:22:56.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.603+0000 7f26fad85700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f26f4113910 con 0x7f26f4103590 2024-05-21T03:22:56.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.603+0000 7f26f9322700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26f41096a0 0x7f26f4199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:56.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.603+0000 7f26f9322700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26f41096a0 0x7f26f4199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43208/0 (socket says 172.21.15.27:43208) 2024-05-21T03:22:56.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.603+0000 7f26f9322700 1 -- 172.21.15.27:0/3217909925 learned_addr learned my addr 172.21.15.27:0/3217909925 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:56.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.603+0000 7f26f8b21700 1 --2- 172.21.15.27:0/3217909925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26f4103590 0x7f26f419f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:56.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.603+0000 7f26f3fff700 1 --2- 172.21.15.27:0/3217909925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26f4103f50 0x7f26f419fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:56.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.604+0000 7f26f9322700 1 -- 172.21.15.27:0/3217909925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26f4103590 msgr2=0x7f26f419f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:56.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.604+0000 7f26f9322700 1 --2- 172.21.15.27:0/3217909925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26f4103590 0x7f26f419f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:56.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.604+0000 7f26f9322700 1 -- 172.21.15.27:0/3217909925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26f4103f50 msgr2=0x7f26f419fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:56.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.604+0000 7f26f9322700 1 --2- 172.21.15.27:0/3217909925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26f4103f50 0x7f26f419fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:56.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.604+0000 7f26f9322700 1 -- 172.21.15.27:0/3217909925 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f419a1f0 con 0x7f26f41096a0 2024-05-21T03:22:56.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.604+0000 7f26f3fff700 1 --2- 172.21.15.27:0/3217909925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26f4103f50 0x7f26f419fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:22:56.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.604+0000 7f26f9322700 1 --2- 172.21.15.27:0/3217909925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26f41096a0 0x7f26f4199a90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00b410 tx=0x7f26ec00b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:56.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.604+0000 7f26f1ffb700 1 -- 172.21.15.27:0/3217909925 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec015020 con 0x7f26f41096a0 2024-05-21T03:22:56.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.604+0000 7f26fad85700 1 -- 172.21.15.27:0/3217909925 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f419a4e0 con 0x7f26f41096a0 2024-05-21T03:22:56.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.604+0000 7f26f1ffb700 1 -- 172.21.15.27:0/3217909925 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26ec011ce0 con 0x7f26f41096a0 2024-05-21T03:22:56.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.604+0000 7f26f1ffb700 1 -- 172.21.15.27:0/3217909925 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec010cb0 con 0x7f26f41096a0 2024-05-21T03:22:56.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.604+0000 7f26fad85700 1 -- 172.21.15.27:0/3217909925 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f41a6750 con 0x7f26f41096a0 2024-05-21T03:22:56.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.605+0000 7f26fad85700 1 -- 172.21.15.27:0/3217909925 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26f410baf0 con 0x7f26f41096a0 2024-05-21T03:22:56.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.606+0000 7f26f1ffb700 1 -- 172.21.15.27:0/3217909925 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f26ec018020 con 0x7f26f41096a0 2024-05-21T03:22:56.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.607+0000 7f26f1ffb700 1 --2- 172.21.15.27:0/3217909925 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f26e00b2880 0x7f26e00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:56.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.607+0000 7f26f8b21700 1 --2- 172.21.15.27:0/3217909925 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f26e00b2880 0x7f26e00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:56.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.607+0000 7f26f1ffb700 1 -- 172.21.15.27:0/3217909925 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f26ec0e18b0 con 0x7f26f41096a0 2024-05-21T03:22:56.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.608+0000 7f26f8b21700 1 --2- 172.21.15.27:0/3217909925 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f26e00b2880 0x7f26e00b4d40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009fd0 tx=0x7f26e400b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:56.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.612+0000 7f26f1ffb700 1 -- 172.21.15.27:0/3217909925 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f26ec01f020 con 0x7f26f41096a0 2024-05-21T03:22:56.613 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:56 smithi027 ceph-mon[32179]: pgmap v363: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:22:56.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:56 smithi039 ceph-mon[36429]: pgmap v363: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:22:56.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:56 smithi165 ceph-mon[36269]: pgmap v363: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:22:56.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.874+0000 7f26fad85700 1 -- 172.21.15.27:0/3217909925 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26f410b4d0 con 0x7f26f41096a0 2024-05-21T03:22:56.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.874+0000 7f26f1ffb700 1 -- 172.21.15.27:0/3217909925 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11924 (secure 0 0 0) 0x7f26ec0a95b0 con 0x7f26f41096a0 2024-05-21T03:22:56.875 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:22:56.875 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/148854869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":148854869},{"type":"v1","addr":"172.21.15.27:6835","nonce":148854869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/4145544730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":4145544730},{"type":"v1","addr":"172.21.15.27:6837","nonce":4145544730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T03:20:34.055985+0000","modified":"2024-05-21T03:20:52.467919+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24472,"mds_2":24457,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/745486764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":745486764},{"type":"v1","addr":"172.21.15.27:6839","nonce":745486764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6841/3982752351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3982752351},{"type":"v1","addr":"172.21.15.27:6841","nonce":3982752351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/3497521608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3497521608},{"type":"v1","addr":"172.21.15.39:6835","nonce":3497521608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.165:6835/3576910218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3576910218},{"type":"v1","addr":"172.21.15.165:6835","nonce":3576910218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/3673846852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3673846852},{"type":"v1","addr":"172.21.15.39:6837","nonce":3673846852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6837/691356139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":691356139},{"type":"v1","addr":"172.21.15.165:6837","nonce":691356139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6839/4158170814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4158170814},{"type":"v1","addr":"172.21.15.165:6839","nonce":4158170814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6839/287607819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":287607819},{"type":"v1","addr":"172.21.15.39:6839","nonce":287607819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24451,24472,24452,24457,24476]},"id":1}]} 2024-05-21T03:22:56.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.877+0000 7f26fad85700 1 -- 172.21.15.27:0/3217909925 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f26e00b2880 msgr2=0x7f26e00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:56.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.877+0000 7f26fad85700 1 --2- 172.21.15.27:0/3217909925 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f26e00b2880 0x7f26e00b4d40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009fd0 tx=0x7f26e400b7b0 comp rx=0 tx=0).stop 2024-05-21T03:22:56.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.877+0000 7f26fad85700 1 -- 172.21.15.27:0/3217909925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26f41096a0 msgr2=0x7f26f4199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:56.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.877+0000 7f26fad85700 1 --2- 172.21.15.27:0/3217909925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26f41096a0 0x7f26f4199a90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00b410 tx=0x7f26ec00b720 comp rx=0 tx=0).stop 2024-05-21T03:22:56.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.877+0000 7f26fad85700 1 -- 172.21.15.27:0/3217909925 shutdown_connections 2024-05-21T03:22:56.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.877+0000 7f26fad85700 1 --2- 172.21.15.27:0/3217909925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26f41096a0 0x7f26f4199a90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:56.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.877+0000 7f26fad85700 1 --2- 172.21.15.27:0/3217909925 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f26e00b2880 0x7f26e00b4d40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:56.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.877+0000 7f26fad85700 1 --2- 172.21.15.27:0/3217909925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26f4103590 0x7f26f419f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:56.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.877+0000 7f26fad85700 1 --2- 172.21.15.27:0/3217909925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26f4103f50 0x7f26f419fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:56.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.877+0000 7f26fad85700 1 -- 172.21.15.27:0/3217909925 >> 172.21.15.27:0/3217909925 conn(0x7f26f407a6f0 msgr2=0x7f26f4102270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:56.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.877+0000 7f26fad85700 1 -- 172.21.15.27:0/3217909925 shutdown_connections 2024-05-21T03:22:56.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:56.877+0000 7f26fad85700 1 -- 172.21.15.27:0/3217909925 wait complete. 2024-05-21T03:22:56.878 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 28 2024-05-21T03:22:57.513 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-21T03:20:34.055985+0000', 'modified': '2024-05-21T03:20:52.467919+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24476, 'mds_1': 24472, 'mds_2': 24457, 'mds_3': 24452, 'mds_4': 24451}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14613': {'gid': 14613, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.27:6839/745486764', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6838', 'nonce': 745486764}, {'type': 'v1', 'addr': '172.21.15.27:6839', 'nonce': 745486764}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14625': {'gid': 14625, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.27:6841/3982752351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6840', 'nonce': 3982752351}, {'type': 'v1', 'addr': '172.21.15.27:6841', 'nonce': 3982752351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24401': {'gid': 24401, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.39:6833/1102873763', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 1102873763}, {'type': 'v1', 'addr': '172.21.15.39:6833', 'nonce': 1102873763}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24418': {'gid': 24418, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6833/4176762898', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6832', 'nonce': 4176762898}, {'type': 'v1', 'addr': '172.21.15.165:6833', 'nonce': 4176762898}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.39:6835/3497521608', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6834', 'nonce': 3497521608}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 3497521608}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.165:6835/3576910218', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6834', 'nonce': 3576910218}, {'type': 'v1', 'addr': '172.21.15.165:6835', 'nonce': 3576910218}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'h', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.39:6837/3673846852', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6836', 'nonce': 3673846852}, {'type': 'v1', 'addr': '172.21.15.39:6837', 'nonce': 3673846852}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.165:6837/691356139', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6836', 'nonce': 691356139}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 691356139}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.165:6839/4158170814', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6838', 'nonce': 4158170814}, {'type': 'v1', 'addr': '172.21.15.165:6839', 'nonce': 4158170814}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6839/287607819', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6838', 'nonce': 287607819}, {'type': 'v1', 'addr': '172.21.15.39:6839', 'nonce': 287607819}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24476, 'qdb_cluster': [24451, 24472, 24452, 24457, 24476]} 2024-05-21T03:22:57.513 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-21T03:22:57.514 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell mds.f status 2024-05-21T03:22:57.765 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:22:57.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:57 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/3217909925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:22:57.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:57 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/3217909925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:22:57.885 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:57 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3217909925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:22:58.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:22:58 smithi039 ceph-mon[36429]: pgmap v364: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:22:58.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:22:58 smithi165 ceph-mon[36269]: pgmap v364: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:22:58.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:22:58 smithi027 ceph-mon[32179]: pgmap v364: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:22:59.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.444+0000 7f0b3e256700 1 -- 172.21.15.27:0/3417036429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b38102e00 msgr2=0x7f0b3810f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:59.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.444+0000 7f0b3e256700 1 --2- 172.21.15.27:0/3417036429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b38102e00 0x7f0b3810f2f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0b30009990 tx=0x7f0b30009ca0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.444+0000 7f0b3e256700 1 -- 172.21.15.27:0/3417036429 shutdown_connections 2024-05-21T03:22:59.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.444+0000 7f0b3e256700 1 --2- 172.21.15.27:0/3417036429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b38102e00 0x7f0b3810f2f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.444+0000 7f0b3e256700 1 --2- 172.21.15.27:0/3417036429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b38108520 0x7f0b38108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.444+0000 7f0b3e256700 1 --2- 172.21.15.27:0/3417036429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b38102440 0x7f0b381028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.444+0000 7f0b3e256700 1 -- 172.21.15.27:0/3417036429 >> 172.21.15.27:0/3417036429 conn(0x7f0b380793f0 msgr2=0x7f0b38079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:59.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.444+0000 7f0b3e256700 1 -- 172.21.15.27:0/3417036429 shutdown_connections 2024-05-21T03:22:59.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.444+0000 7f0b3e256700 1 -- 172.21.15.27:0/3417036429 wait complete. 2024-05-21T03:22:59.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.445+0000 7f0b3e256700 1 Processor -- start 2024-05-21T03:22:59.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.445+0000 7f0b3e256700 1 -- start start 2024-05-21T03:22:59.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.445+0000 7f0b3e256700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b38102440 0x7f0b3819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:59.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.445+0000 7f0b3e256700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b38102e00 0x7f0b3819afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:59.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.445+0000 7f0b3e256700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b38108520 0x7f0b381a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:59.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.445+0000 7f0b3e256700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0b38112640 con 0x7f0b38102e00 2024-05-21T03:22:59.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.445+0000 7f0b3e256700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0b381124c0 con 0x7f0b38102440 2024-05-21T03:22:59.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.445+0000 7f0b3e256700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0b381127c0 con 0x7f0b38108520 2024-05-21T03:22:59.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.446+0000 7f0b36ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b38102e00 0x7f0b3819afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:59.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.446+0000 7f0b36ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b38102e00 0x7f0b3819afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43230/0 (socket says 172.21.15.27:43230) 2024-05-21T03:22:59.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.446+0000 7f0b36ffd700 1 -- 172.21.15.27:0/642385245 learned_addr learned my addr 172.21.15.27:0/642385245 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:59.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.446+0000 7f0b37fff700 1 --2- 172.21.15.27:0/642385245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b38108520 0x7f0b381a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:59.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.446+0000 7f0b377fe700 1 --2- 172.21.15.27:0/642385245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b38102440 0x7f0b3819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:59.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.446+0000 7f0b36ffd700 1 -- 172.21.15.27:0/642385245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b38108520 msgr2=0x7f0b381a5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:59.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.446+0000 7f0b36ffd700 1 --2- 172.21.15.27:0/642385245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b38108520 0x7f0b381a5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.446+0000 7f0b36ffd700 1 -- 172.21.15.27:0/642385245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b38102440 msgr2=0x7f0b3819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:59.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.446+0000 7f0b36ffd700 1 --2- 172.21.15.27:0/642385245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b38102440 0x7f0b3819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.446+0000 7f0b36ffd700 1 -- 172.21.15.27:0/642385245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b381a6040 con 0x7f0b38102e00 2024-05-21T03:22:59.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.446+0000 7f0b377fe700 1 --2- 172.21.15.27:0/642385245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b38102440 0x7f0b3819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:22:59.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.446+0000 7f0b36ffd700 1 --2- 172.21.15.27:0/642385245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b38102e00 0x7f0b3819afc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0b2000d950 tx=0x7f0b2000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:59.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.446+0000 7f0b34ff9700 1 -- 172.21.15.27:0/642385245 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b20016070 con 0x7f0b38102e00 2024-05-21T03:22:59.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.446+0000 7f0b34ff9700 1 -- 172.21.15.27:0/642385245 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b20009df0 con 0x7f0b38102e00 2024-05-21T03:22:59.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.446+0000 7f0b3e256700 1 -- 172.21.15.27:0/642385245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b381a6270 con 0x7f0b38102e00 2024-05-21T03:22:59.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.446+0000 7f0b34ff9700 1 -- 172.21.15.27:0/642385245 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2000f3c0 con 0x7f0b38102e00 2024-05-21T03:22:59.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.446+0000 7f0b3e256700 1 -- 172.21.15.27:0/642385245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b381a6780 con 0x7f0b38102e00 2024-05-21T03:22:59.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.449+0000 7f0b34ff9700 1 -- 172.21.15.27:0/642385245 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0b20019070 con 0x7f0b38102e00 2024-05-21T03:22:59.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.450+0000 7f0b34ff9700 1 --2- 172.21.15.27:0/642385245 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f0b240b2880 0x7f0b240b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:59.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.450+0000 7f0b377fe700 1 --2- 172.21.15.27:0/642385245 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f0b240b2880 0x7f0b240b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:59.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.450+0000 7f0b34ff9700 1 -- 172.21.15.27:0/642385245 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f0b200e1e50 con 0x7f0b38102e00 2024-05-21T03:22:59.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.450+0000 7f0b377fe700 1 --2- 172.21.15.27:0/642385245 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f0b240b2880 0x7f0b240b4d40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0b280093a0 tx=0x7f0b2800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:59.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.450+0000 7f0b48aecb80 1 Processor -- start 2024-05-21T03:22:59.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.450+0000 7f0b48aecb80 1 -- start start 2024-05-21T03:22:59.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.451+0000 7f0b48aecb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562db67055c0 0x562db67059a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:59.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.451+0000 7f0b48aecb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562db6705f70 0x562db66ffc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:59.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.451+0000 7f0b48aecb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562db6700280 0x562db67006e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:59.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.451+0000 7f0b48aecb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562db6659d20 con 0x562db67055c0 2024-05-21T03:22:59.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.451+0000 7f0b48aecb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562db66498c0 con 0x562db6700280 2024-05-21T03:22:59.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.451+0000 7f0b48aecb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562db664d900 con 0x562db6705f70 2024-05-21T03:22:59.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.451+0000 7f0b377fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562db67055c0 0x562db67059a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:59.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.451+0000 7f0b377fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562db67055c0 0x562db67059a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43244/0 (socket says 172.21.15.27:43244) 2024-05-21T03:22:59.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.451+0000 7f0b377fe700 1 -- 172.21.15.27:0/1865403861 learned_addr learned my addr 172.21.15.27:0/1865403861 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:59.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.451+0000 7f0b36ffd700 1 --2- 172.21.15.27:0/1865403861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562db6705f70 0x562db66ffc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:59.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.451+0000 7f0b37fff700 1 --2- 172.21.15.27:0/1865403861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562db6700280 0x562db67006e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:59.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.451+0000 7f0b377fe700 1 -- 172.21.15.27:0/1865403861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562db6705f70 msgr2=0x562db66ffc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:22:59.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.451+0000 7f0b377fe700 1 --2- 172.21.15.27:0/1865403861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562db6705f70 0x562db66ffc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.451+0000 7f0b377fe700 1 -- 172.21.15.27:0/1865403861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562db6700280 msgr2=0x562db67006e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:22:59.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.451+0000 7f0b377fe700 1 --2- 172.21.15.27:0/1865403861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562db6700280 0x562db67006e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.451+0000 7f0b377fe700 1 -- 172.21.15.27:0/1865403861 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562db65f2f60 con 0x562db67055c0 2024-05-21T03:22:59.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.451+0000 7f0b37fff700 1 --2- 172.21.15.27:0/1865403861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562db6700280 0x562db67006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:22:59.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.451+0000 7f0b377fe700 1 --2- 172.21.15.27:0/1865403861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562db67055c0 0x562db67059a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0b28015790 tx=0x7f0b28013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed37dc34280a8e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:59.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.451+0000 7f0b2dffb700 1 -- 172.21.15.27:0/1865403861 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2801c050 con 0x562db67055c0 2024-05-21T03:22:59.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.452+0000 7f0b2dffb700 1 -- 172.21.15.27:0/1865403861 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b28026070 con 0x562db67055c0 2024-05-21T03:22:59.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.452+0000 7f0b2dffb700 1 -- 172.21.15.27:0/1865403861 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28016a30 con 0x562db67055c0 2024-05-21T03:22:59.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.452+0000 7f0b48aecb80 1 -- 172.21.15.27:0/1865403861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562db67055c0 msgr2=0x562db67059a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:59.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.452+0000 7f0b48aecb80 1 --2- 172.21.15.27:0/1865403861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562db67055c0 0x562db67059a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0b28015790 tx=0x7f0b28013000 comp rx=0 tx=0).stop 2024-05-21T03:22:59.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.452+0000 7f0b48aecb80 1 -- 172.21.15.27:0/1865403861 shutdown_connections 2024-05-21T03:22:59.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.452+0000 7f0b48aecb80 1 --2- 172.21.15.27:0/1865403861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562db6700280 0x562db67006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.452+0000 7f0b48aecb80 1 --2- 172.21.15.27:0/1865403861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562db67055c0 0x562db67059a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.452+0000 7f0b48aecb80 1 --2- 172.21.15.27:0/1865403861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562db6705f70 0x562db66ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.452+0000 7f0b48aecb80 1 -- 172.21.15.27:0/1865403861 >> 172.21.15.27:0/1865403861 conn(0x562db66271f0 msgr2=0x562db66f6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:59.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.452+0000 7f0b48aecb80 1 -- 172.21.15.27:0/1865403861 shutdown_connections 2024-05-21T03:22:59.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.452+0000 7f0b48aecb80 1 -- 172.21.15.27:0/1865403861 wait complete. 2024-05-21T03:22:59.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.453+0000 7f0b48aecb80 1 -- start start 2024-05-21T03:22:59.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.453+0000 7f0b48aecb80 1 Processor -- start 2024-05-21T03:22:59.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.453+0000 7f0b48aecb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562db6700280 0x562db670bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:59.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.453+0000 7f0b48aecb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562db67055c0 0x562db6709130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:59.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.453+0000 7f0b48aecb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562db6705f70 0x562db6709670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:59.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.453+0000 7f0b48aecb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562db654ff50 con 0x562db67055c0 2024-05-21T03:22:59.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.453+0000 7f0b48aecb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562db65b1d50 con 0x562db6700280 2024-05-21T03:22:59.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.453+0000 7f0b48aecb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562db670ca50 con 0x562db6705f70 2024-05-21T03:22:59.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.453+0000 7f0b48aecb80 10 client.0 _ll_register_callbacks cb 0x562db65e4900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:22:59.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.453+0000 7f0b36ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562db67055c0 0x562db6709130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:59.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.453+0000 7f0b36ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562db67055c0 0x562db6709130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43258/0 (socket says 172.21.15.27:43258) 2024-05-21T03:22:59.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.453+0000 7f0b36ffd700 1 -- 172.21.15.27:0/2465199586 learned_addr learned my addr 172.21.15.27:0/2465199586 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:59.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.453+0000 7f0b37fff700 1 --2- 172.21.15.27:0/2465199586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562db6705f70 0x562db6709670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:59.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.454+0000 7f0b377fe700 1 --2- 172.21.15.27:0/2465199586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562db6700280 0x562db670bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:59.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.454+0000 7f0b36ffd700 1 -- 172.21.15.27:0/2465199586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562db6705f70 msgr2=0x562db6709670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:22:59.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.454+0000 7f0b36ffd700 1 --2- 172.21.15.27:0/2465199586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562db6705f70 0x562db6709670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.454+0000 7f0b36ffd700 1 -- 172.21.15.27:0/2465199586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562db6700280 msgr2=0x562db670bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:22:59.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.454+0000 7f0b36ffd700 1 --2- 172.21.15.27:0/2465199586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562db6700280 0x562db670bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.454+0000 7f0b36ffd700 1 -- 172.21.15.27:0/2465199586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562db6599880 con 0x562db67055c0 2024-05-21T03:22:59.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.454+0000 7f0b377fe700 1 --2- 172.21.15.27:0/2465199586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562db6700280 0x562db670bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:22:59.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.454+0000 7f0b36ffd700 1 --2- 172.21.15.27:0/2465199586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562db67055c0 0x562db6709130 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0b2000b2f0 tx=0x7f0b200a90c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6bf5a135e6b1ed8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:59.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.454+0000 7f0b16ffd700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:22:59.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.454+0000 7f0b16ffd700 1 -- 172.21.15.27:0/2465199586 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b20020070 con 0x562db67055c0 2024-05-21T03:22:59.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.454+0000 7f0b16ffd700 1 -- 172.21.15.27:0/2465199586 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f0b20010710 con 0x562db67055c0 2024-05-21T03:22:59.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.454+0000 7f0b48aecb80 1 -- 172.21.15.27:0/2465199586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b28026070 con 0x562db67055c0 2024-05-21T03:22:59.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.454+0000 7f0b16ffd700 1 -- 172.21.15.27:0/2465199586 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0b200ad800 con 0x562db67055c0 2024-05-21T03:22:59.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.454+0000 7f0b48aecb80 10 client.15003 fetch_fsmap learned FSMap version 28 2024-05-21T03:22:59.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.454+0000 7f0b48aecb80 1 -- 172.21.15.27:0/2465199586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562db6603710 con 0x562db67055c0 2024-05-21T03:22:59.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.454+0000 7f0b16ffd700 1 -- 172.21.15.27:0/2465199586 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b28026070 con 0x562db67055c0 2024-05-21T03:22:59.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.455+0000 7f0b16ffd700 1 -- 172.21.15.27:0/2465199586 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0b200b0b80 con 0x562db67055c0 2024-05-21T03:22:59.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.455+0000 7f0b48aecb80 10 client.15003 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T03:22:59.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.455+0000 7f0b48aecb80 10 client.15003 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-21T03:22:59.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.455+0000 7f0b48aecb80 20 client.15003 populate_metadata read hostname 'smithi027' 2024-05-21T03:22:59.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.455+0000 7f0b48aecb80 1 --2- 172.21.15.27:0/2465199586 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x562db67195d0 0x562db671aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:59.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.455+0000 7f0b48aecb80 4 client.15003 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:22:59.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.455+0000 7f0b48aecb80 1 -- 172.21.15.27:0/2465199586 --> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562db665bc70 con 0x562db67195d0 2024-05-21T03:22:59.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.455+0000 7f0b377fe700 1 --2- 172.21.15.27:0/2465199586 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x562db67195d0 0x562db671aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:59.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.455+0000 7f0b377fe700 1 --2- 172.21.15.27:0/2465199586 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x562db67195d0 0x562db671aa40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3283551a4e83e046 server_cookie=1cc715b031a5f5b1 in_seq=0 out_seq=0 2024-05-21T03:22:59.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.455+0000 7f0b16ffd700 10 client.15003 ms_handle_connect on v2:172.21.15.165:6834/3576910218 2024-05-21T03:22:59.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.456+0000 7f0b16ffd700 1 -- 172.21.15.27:0/2465199586 <== mds.4 v2:172.21.15.165:6834/3576910218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562db665bc70 con 0x562db67195d0 2024-05-21T03:22:59.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.456+0000 7f0b16ffd700 10 client.15003 handle_command_reply: tid=0 2024-05-21T03:22:59.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.456+0000 7f0b48aecb80 1 client.15003 shutdown 2024-05-21T03:22:59.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.456+0000 7f0b48aecb80 1 -- 172.21.15.27:0/2465199586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562db67055c0 msgr2=0x562db6709130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:59.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.456+0000 7f0b48aecb80 1 --2- 172.21.15.27:0/2465199586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562db67055c0 0x562db6709130 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0b2000b2f0 tx=0x7f0b200a90c0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.456+0000 7f0b48aecb80 1 -- 172.21.15.27:0/2465199586 shutdown_connections 2024-05-21T03:22:59.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.456+0000 7f0b48aecb80 1 --2- 172.21.15.27:0/2465199586 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x562db67195d0 0x562db671aa40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.456+0000 7f0b48aecb80 1 --2- 172.21.15.27:0/2465199586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562db6705f70 0x562db6709670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.456+0000 7f0b48aecb80 1 --2- 172.21.15.27:0/2465199586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562db6700280 0x562db670bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.456+0000 7f0b48aecb80 1 --2- 172.21.15.27:0/2465199586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562db67055c0 0x562db6709130 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.456+0000 7f0b16ffd700 0 client.15003 ms_handle_reset on v2:172.21.15.165:6834/3576910218 2024-05-21T03:22:59.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.456+0000 7f0b48aecb80 1 -- 172.21.15.27:0/2465199586 >> 172.21.15.27:0/2465199586 conn(0x562db66271f0 msgr2=0x562db66f5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:59.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.456+0000 7f0b48aecb80 1 -- 172.21.15.27:0/2465199586 shutdown_connections 2024-05-21T03:22:59.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.457+0000 7f0b48aecb80 1 -- 172.21.15.27:0/2465199586 wait complete. 2024-05-21T03:22:59.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.457+0000 7f0b48aecb80 20 client.15003 trim_cache size 0 max 16384 2024-05-21T03:22:59.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.478+0000 7f0b48aecb80 1 Processor -- start 2024-05-21T03:22:59.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.478+0000 7f0b48aecb80 1 -- start start 2024-05-21T03:22:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.478+0000 7f0b48aecb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562db6700280 0x562db6709800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.478+0000 7f0b48aecb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562db67055c0 0x562db6718f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.478+0000 7f0b48aecb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562db6705f70 0x562db66f6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.478+0000 7f0b48aecb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562db6718d80 con 0x562db6700280 2024-05-21T03:22:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.478+0000 7f0b48aecb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562db6719450 con 0x562db67055c0 2024-05-21T03:22:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.478+0000 7f0b48aecb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562db6719b20 con 0x562db6705f70 2024-05-21T03:22:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.479+0000 7f0b377fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562db6700280 0x562db6709800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.479+0000 7f0b377fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562db6700280 0x562db6709800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43270/0 (socket says 172.21.15.27:43270) 2024-05-21T03:22:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.479+0000 7f0b377fe700 1 -- 172.21.15.27:0/1946135272 learned_addr learned my addr 172.21.15.27:0/1946135272 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.479+0000 7f0b37fff700 1 --2- 172.21.15.27:0/1946135272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562db6705f70 0x562db66f6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.479+0000 7f0b36ffd700 1 --2- 172.21.15.27:0/1946135272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562db67055c0 0x562db6718f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.479+0000 7f0b377fe700 1 -- 172.21.15.27:0/1946135272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562db6705f70 msgr2=0x562db66f6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:22:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.479+0000 7f0b377fe700 1 --2- 172.21.15.27:0/1946135272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562db6705f70 0x562db66f6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.479+0000 7f0b377fe700 1 -- 172.21.15.27:0/1946135272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562db67055c0 msgr2=0x562db6718f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:22:59.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.479+0000 7f0b377fe700 1 --2- 172.21.15.27:0/1946135272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562db67055c0 0x562db6718f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.479+0000 7f0b377fe700 1 -- 172.21.15.27:0/1946135272 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562db67076a0 con 0x562db6700280 2024-05-21T03:22:59.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.479+0000 7f0b377fe700 1 --2- 172.21.15.27:0/1946135272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562db6700280 0x562db6709800 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0b28015d80 tx=0x7f0b28007fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3088de0611ec3c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:59.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.479+0000 7f0b2d7fa700 1 -- 172.21.15.27:0/1946135272 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2801c070 con 0x562db6700280 2024-05-21T03:22:59.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.479+0000 7f0b2d7fa700 1 -- 172.21.15.27:0/1946135272 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b28026040 con 0x562db6700280 2024-05-21T03:22:59.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.479+0000 7f0b2d7fa700 1 -- 172.21.15.27:0/1946135272 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28034600 con 0x562db6700280 2024-05-21T03:22:59.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.480+0000 7f0b48aecb80 1 -- 172.21.15.27:0/1946135272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562db6700280 msgr2=0x562db6709800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:59.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.480+0000 7f0b48aecb80 1 --2- 172.21.15.27:0/1946135272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562db6700280 0x562db6709800 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0b28015d80 tx=0x7f0b28007fb0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.480+0000 7f0b48aecb80 1 -- 172.21.15.27:0/1946135272 shutdown_connections 2024-05-21T03:22:59.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.480+0000 7f0b48aecb80 1 --2- 172.21.15.27:0/1946135272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562db6705f70 0x562db66f6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.480+0000 7f0b48aecb80 1 --2- 172.21.15.27:0/1946135272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562db6700280 0x562db6709800 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.480+0000 7f0b48aecb80 1 --2- 172.21.15.27:0/1946135272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562db67055c0 0x562db6718f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.480+0000 7f0b48aecb80 1 -- 172.21.15.27:0/1946135272 >> 172.21.15.27:0/1946135272 conn(0x562db66271f0 msgr2=0x562db67090f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:59.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.480+0000 7f0b48aecb80 1 -- 172.21.15.27:0/1946135272 shutdown_connections 2024-05-21T03:22:59.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.480+0000 7f0b48aecb80 1 -- 172.21.15.27:0/1946135272 wait complete. 2024-05-21T03:22:59.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.481+0000 7f0b48aecb80 1 -- start start 2024-05-21T03:22:59.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.481+0000 7f0b48aecb80 1 Processor -- start 2024-05-21T03:22:59.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.481+0000 7f0b48aecb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562db6700280 0x562db6723ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:59.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.481+0000 7f0b48aecb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562db67055c0 0x562db6724400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:59.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.481+0000 7f0b48aecb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562db6705f70 0x562db671ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:59.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.481+0000 7f0b48aecb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562db671a8c0 con 0x562db6705f70 2024-05-21T03:22:59.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.481+0000 7f0b48aecb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562db671a1f0 con 0x562db6700280 2024-05-21T03:22:59.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.481+0000 7f0b48aecb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562db671af90 con 0x562db67055c0 2024-05-21T03:22:59.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.481+0000 7f0b48aecb80 10 client.0 _ll_register_callbacks cb 0x562db65e4900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:22:59.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.481+0000 7f0b37fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562db6705f70 0x562db671ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:59.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.481+0000 7f0b36ffd700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562db67055c0 0x562db6724400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:59.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.481+0000 7f0b37fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562db6705f70 0x562db671ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43278/0 (socket says 172.21.15.27:43278) 2024-05-21T03:22:59.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.481+0000 7f0b37fff700 1 -- 172.21.15.27:0/573475468 learned_addr learned my addr 172.21.15.27:0/573475468 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:22:59.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.481+0000 7f0b377fe700 1 --2- 172.21.15.27:0/573475468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562db6700280 0x562db6723ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:59.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.482+0000 7f0b37fff700 1 -- 172.21.15.27:0/573475468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562db67055c0 msgr2=0x562db6724400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:22:59.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.482+0000 7f0b37fff700 1 --2- 172.21.15.27:0/573475468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562db67055c0 0x562db6724400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.482+0000 7f0b37fff700 1 -- 172.21.15.27:0/573475468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562db6700280 msgr2=0x562db6723ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:22:59.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.482+0000 7f0b37fff700 1 --2- 172.21.15.27:0/573475468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562db6700280 0x562db6723ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.482+0000 7f0b37fff700 1 -- 172.21.15.27:0/573475468 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562db66f6830 con 0x562db6705f70 2024-05-21T03:22:59.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.482+0000 7f0b37fff700 1 --2- 172.21.15.27:0/573475468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562db6705f70 0x562db671ee70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0b30009990 tx=0x7f0b300057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4ff5b5e5da7b6f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:22:59.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.482+0000 7f0b16ffd700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:22:59.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.482+0000 7f0b16ffd700 1 -- 172.21.15.27:0/573475468 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3000bb30 con 0x562db6705f70 2024-05-21T03:22:59.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.482+0000 7f0b16ffd700 1 -- 172.21.15.27:0/573475468 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f0b3000bcd0 con 0x562db6705f70 2024-05-21T03:22:59.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.482+0000 7f0b48aecb80 1 -- 172.21.15.27:0/573475468 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b28026040 con 0x562db6705f70 2024-05-21T03:22:59.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.482+0000 7f0b16ffd700 1 -- 172.21.15.27:0/573475468 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0b30034050 con 0x562db6705f70 2024-05-21T03:22:59.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.482+0000 7f0b48aecb80 10 client.15009 fetch_fsmap learned FSMap version 28 2024-05-21T03:22:59.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.482+0000 7f0b48aecb80 1 -- 172.21.15.27:0/573475468 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562db6785320 con 0x562db6705f70 2024-05-21T03:22:59.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.483+0000 7f0b16ffd700 1 -- 172.21.15.27:0/573475468 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b28026040 con 0x562db6705f70 2024-05-21T03:22:59.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.483+0000 7f0b16ffd700 1 -- 172.21.15.27:0/573475468 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0b3000f700 con 0x562db6705f70 2024-05-21T03:22:59.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.483+0000 7f0b48aecb80 10 client.15009 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T03:22:59.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.483+0000 7f0b48aecb80 10 client.15009 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-21T03:22:59.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.483+0000 7f0b48aecb80 20 client.15009 populate_metadata read hostname 'smithi027' 2024-05-21T03:22:59.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.483+0000 7f0b48aecb80 1 --2- 172.21.15.27:0/573475468 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x562db6789fe0 0x562db6788cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:22:59.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.483+0000 7f0b48aecb80 4 client.15009 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T03:22:59.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.483+0000 7f0b48aecb80 1 -- 172.21.15.27:0/573475468 --> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] -- command(tid 0: {"prefix": "status"}) -- 0x562db6655450 con 0x562db6789fe0 2024-05-21T03:22:59.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.483+0000 7f0b377fe700 1 --2- 172.21.15.27:0/573475468 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x562db6789fe0 0x562db6788cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:22:59.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.483+0000 7f0b16ffd700 10 client.15009 ms_handle_connect on v2:172.21.15.165:6834/3576910218 2024-05-21T03:22:59.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.483+0000 7f0b377fe700 1 --2- 172.21.15.27:0/573475468 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x562db6789fe0 0x562db6788cf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=faca2e88f44561fc server_cookie=de99eac41ba3df20 in_seq=0 out_seq=0 2024-05-21T03:22:59.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.484+0000 7f0b16ffd700 1 -- 172.21.15.27:0/573475468 <== mds.4 v2:172.21.15.165:6834/3576910218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562db6655450 con 0x562db6789fe0 2024-05-21T03:22:59.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.484+0000 7f0b16ffd700 10 client.15009 handle_command_reply: tid=0 2024-05-21T03:22:59.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.484+0000 7f0b48aecb80 1 client.15009 shutdown 2024-05-21T03:22:59.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.484+0000 7f0b48aecb80 1 -- 172.21.15.27:0/573475468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562db6705f70 msgr2=0x562db671ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:59.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.484+0000 7f0b48aecb80 1 --2- 172.21.15.27:0/573475468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562db6705f70 0x562db671ee70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0b30009990 tx=0x7f0b300057b0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.484+0000 7f0b48aecb80 1 -- 172.21.15.27:0/573475468 shutdown_connections 2024-05-21T03:22:59.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.484+0000 7f0b48aecb80 1 --2- 172.21.15.27:0/573475468 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x562db6789fe0 0x562db6788cf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.484+0000 7f0b48aecb80 1 --2- 172.21.15.27:0/573475468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562db6705f70 0x562db671ee70 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.484+0000 7f0b48aecb80 1 --2- 172.21.15.27:0/573475468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562db6700280 0x562db6723ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.484+0000 7f0b48aecb80 1 --2- 172.21.15.27:0/573475468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562db67055c0 0x562db6724400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.484+0000 7f0b48aecb80 1 -- 172.21.15.27:0/573475468 >> 172.21.15.27:0/573475468 conn(0x562db66271f0 msgr2=0x562db6703110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:59.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.484+0000 7f0b16ffd700 0 client.15009 ms_handle_reset on v2:172.21.15.165:6834/3576910218 2024-05-21T03:22:59.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.484+0000 7f0b48aecb80 1 -- 172.21.15.27:0/573475468 shutdown_connections 2024-05-21T03:22:59.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.484+0000 7f0b48aecb80 1 -- 172.21.15.27:0/573475468 wait complete. 2024-05-21T03:22:59.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.484+0000 7f0b48aecb80 20 client.15009 trim_cache size 0 max 16384 2024-05-21T03:22:59.489 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:22:59.490 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "c9ca94be-171f-11ef-bc9a-c7b262605968", 2024-05-21T03:22:59.490 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 4, 2024-05-21T03:22:59.490 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24451, 2024-05-21T03:22:59.490 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-05-21T03:22:59.490 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-05-21T03:22:59.490 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-05-21T03:22:59.490 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 130.02351810499999, 2024-05-21T03:22:59.490 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 28, 2024-05-21T03:22:59.490 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 82, 2024-05-21T03:22:59.490 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 80, 2024-05-21T03:22:59.490 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 177.891024824 2024-05-21T03:22:59.490 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:22:59.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.485+0000 7f0b3e256700 1 -- 172.21.15.27:0/642385245 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f0b240b2880 msgr2=0x7f0b240b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:59.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.485+0000 7f0b3e256700 1 --2- 172.21.15.27:0/642385245 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f0b240b2880 0x7f0b240b4d40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0b280093a0 tx=0x7f0b2800b040 comp rx=0 tx=0).stop 2024-05-21T03:22:59.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.485+0000 7f0b3e256700 1 -- 172.21.15.27:0/642385245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b38102e00 msgr2=0x7f0b3819afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:22:59.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.485+0000 7f0b3e256700 1 --2- 172.21.15.27:0/642385245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b38102e00 0x7f0b3819afc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0b2000d950 tx=0x7f0b2000dc60 comp rx=0 tx=0).stop 2024-05-21T03:22:59.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.485+0000 7f0b3e256700 1 -- 172.21.15.27:0/642385245 shutdown_connections 2024-05-21T03:22:59.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.485+0000 7f0b3e256700 1 --2- 172.21.15.27:0/642385245 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f0b240b2880 0x7f0b240b4d40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.485+0000 7f0b3e256700 1 --2- 172.21.15.27:0/642385245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b38108520 0x7f0b381a5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.485+0000 7f0b3e256700 1 --2- 172.21.15.27:0/642385245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b38102440 0x7f0b3819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.485+0000 7f0b3e256700 1 --2- 172.21.15.27:0/642385245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b38102e00 0x7f0b3819afc0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:22:59.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.485+0000 7f0b3e256700 1 -- 172.21.15.27:0/642385245 >> 172.21.15.27:0/642385245 conn(0x7f0b380793f0 msgr2=0x7f0b38100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:22:59.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.485+0000 7f0b3e256700 1 -- 172.21.15.27:0/642385245 shutdown_connections 2024-05-21T03:22:59.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:22:59.485+0000 7f0b3e256700 1 -- 172.21.15.27:0/642385245 wait complete. 2024-05-21T03:23:00.179 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell mds.h status 2024-05-21T03:23:00.431 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:23:00.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:00 smithi039 ceph-mon[36429]: pgmap v365: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:23:00.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:00 smithi165 ceph-mon[36269]: pgmap v365: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:23:00.785 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:00 smithi027 ceph-mon[32179]: pgmap v365: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:23:01.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.957+0000 7f5e877fd700 1 -- 172.21.15.27:0/2121558941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e801039d0 msgr2=0x7f5e8010f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:01.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.957+0000 7f5e877fd700 1 --2- 172.21.15.27:0/2121558941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e801039d0 0x7f5e8010f3e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5e780099c0 tx=0x7f5e78009cd0 comp rx=0 tx=0).stop 2024-05-21T03:23:01.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.957+0000 7f5e877fd700 1 -- 172.21.15.27:0/2121558941 shutdown_connections 2024-05-21T03:23:01.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.957+0000 7f5e877fd700 1 --2- 172.21.15.27:0/2121558941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e801039d0 0x7f5e8010f3e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:01.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.957+0000 7f5e877fd700 1 --2- 172.21.15.27:0/2121558941 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e80108730 0x7f5e80108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:01.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.957+0000 7f5e877fd700 1 --2- 172.21.15.27:0/2121558941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e80103010 0x7f5e80103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:01.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.957+0000 7f5e877fd700 1 -- 172.21.15.27:0/2121558941 >> 172.21.15.27:0/2121558941 conn(0x7f5e80077780 msgr2=0x7f5e80077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:01.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.957+0000 7f5e877fd700 1 -- 172.21.15.27:0/2121558941 shutdown_connections 2024-05-21T03:23:01.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.957+0000 7f5e877fd700 1 -- 172.21.15.27:0/2121558941 wait complete. 2024-05-21T03:23:01.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.958+0000 7f5e877fd700 1 Processor -- start 2024-05-21T03:23:01.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.958+0000 7f5e877fd700 1 -- start start 2024-05-21T03:23:01.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.958+0000 7f5e877fd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e80103010 0x7f5e8019fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:01.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.958+0000 7f5e877fd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e801039d0 0x7f5e8019ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:01.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.958+0000 7f5e877fd700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e80108730 0x7f5e80199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:01.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.958+0000 7f5e877fd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5e801127b0 con 0x7f5e80103010 2024-05-21T03:23:01.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.958+0000 7f5e877fd700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5e80112630 con 0x7f5e801039d0 2024-05-21T03:23:01.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.958+0000 7f5e877fd700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5e80112930 con 0x7f5e80108730 2024-05-21T03:23:01.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.958+0000 7f5e85599700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e80103010 0x7f5e8019fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:01.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.958+0000 7f5e85599700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e80103010 0x7f5e8019fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43296/0 (socket says 172.21.15.27:43296) 2024-05-21T03:23:01.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.958+0000 7f5e85599700 1 -- 172.21.15.27:0/4087182201 learned_addr learned my addr 172.21.15.27:0/4087182201 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:01.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.958+0000 7f5e85d9a700 1 --2- 172.21.15.27:0/4087182201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e80108730 0x7f5e80199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:01.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.959+0000 7f5e84d98700 1 --2- 172.21.15.27:0/4087182201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e801039d0 0x7f5e8019ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:01.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.959+0000 7f5e85599700 1 -- 172.21.15.27:0/4087182201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e80108730 msgr2=0x7f5e80199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:01.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.959+0000 7f5e85599700 1 --2- 172.21.15.27:0/4087182201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e80108730 0x7f5e80199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:01.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.959+0000 7f5e85599700 1 -- 172.21.15.27:0/4087182201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e801039d0 msgr2=0x7f5e8019ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:01.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.959+0000 7f5e85599700 1 --2- 172.21.15.27:0/4087182201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e801039d0 0x7f5e8019ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:01.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.959+0000 7f5e85599700 1 -- 172.21.15.27:0/4087182201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8019a460 con 0x7f5e80103010 2024-05-21T03:23:01.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.959+0000 7f5e85d9a700 1 --2- 172.21.15.27:0/4087182201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e80108730 0x7f5e80199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:23:01.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.959+0000 7f5e84d98700 1 --2- 172.21.15.27:0/4087182201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e801039d0 0x7f5e8019ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:23:01.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.959+0000 7f5e85599700 1 --2- 172.21.15.27:0/4087182201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e80103010 0x7f5e8019fa60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c0060b0 tx=0x7f5e7c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:01.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.959+0000 7f5e767fc700 1 -- 172.21.15.27:0/4087182201 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c01a070 con 0x7f5e80103010 2024-05-21T03:23:01.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.959+0000 7f5e877fd700 1 -- 172.21.15.27:0/4087182201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e8019a750 con 0x7f5e80103010 2024-05-21T03:23:01.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.959+0000 7f5e767fc700 1 -- 172.21.15.27:0/4087182201 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e7c00de20 con 0x7f5e80103010 2024-05-21T03:23:01.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.959+0000 7f5e767fc700 1 -- 172.21.15.27:0/4087182201 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c014720 con 0x7f5e80103010 2024-05-21T03:23:01.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.959+0000 7f5e877fd700 1 -- 172.21.15.27:0/4087182201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e801a69e0 con 0x7f5e80103010 2024-05-21T03:23:01.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.961+0000 7f5e767fc700 1 -- 172.21.15.27:0/4087182201 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5e7c01d070 con 0x7f5e80103010 2024-05-21T03:23:01.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.962+0000 7f5e767fc700 1 --2- 172.21.15.27:0/4087182201 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f5e6c0b2880 0x7f5e6c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:01.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.962+0000 7f5e84d98700 1 --2- 172.21.15.27:0/4087182201 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f5e6c0b2880 0x7f5e6c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:01.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.962+0000 7f5e767fc700 1 -- 172.21.15.27:0/4087182201 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f5e7c0e1a90 con 0x7f5e80103010 2024-05-21T03:23:01.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.963+0000 7f5e92093b80 1 Processor -- start 2024-05-21T03:23:01.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.963+0000 7f5e84d98700 1 --2- 172.21.15.27:0/4087182201 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f5e6c0b2880 0x7f5e6c0b4d40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5e70005fd0 tx=0x7f5e70005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:01.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.963+0000 7f5e92093b80 1 -- start start 2024-05-21T03:23:01.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.963+0000 7f5e92093b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf93bac20 0x55ddf93bb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:01.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.963+0000 7f5e92093b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddf93bb5d0 0x55ddf93bed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:01.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.963+0000 7f5e92093b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ddf93bf320 0x55ddf93bf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:01.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.963+0000 7f5e92093b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ddf9315c40 con 0x55ddf93bf320 2024-05-21T03:23:01.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.963+0000 7f5e92093b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ddf93057e0 con 0x55ddf93bac20 2024-05-21T03:23:01.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.963+0000 7f5e92093b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ddf9309820 con 0x55ddf93bb5d0 2024-05-21T03:23:01.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.963+0000 7f5e85d9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ddf93bf320 0x55ddf93bf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:01.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.963+0000 7f5e85d9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ddf93bf320 0x55ddf93bf760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43306/0 (socket says 172.21.15.27:43306) 2024-05-21T03:23:01.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.963+0000 7f5e85d9a700 1 -- 172.21.15.27:0/1004632642 learned_addr learned my addr 172.21.15.27:0/1004632642 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:01.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.963+0000 7f5e84d98700 1 --2- 172.21.15.27:0/1004632642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddf93bb5d0 0x55ddf93bed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:01.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.963+0000 7f5e85599700 1 --2- 172.21.15.27:0/1004632642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf93bac20 0x55ddf93bb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:01.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.964+0000 7f5e85d9a700 1 -- 172.21.15.27:0/1004632642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddf93bb5d0 msgr2=0x55ddf93bed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:01.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.964+0000 7f5e85d9a700 1 --2- 172.21.15.27:0/1004632642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddf93bb5d0 0x55ddf93bed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:01.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.964+0000 7f5e85d9a700 1 -- 172.21.15.27:0/1004632642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf93bac20 msgr2=0x55ddf93bb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:01.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.964+0000 7f5e85d9a700 1 --2- 172.21.15.27:0/1004632642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf93bac20 0x55ddf93bb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:01.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.964+0000 7f5e85d9a700 1 -- 172.21.15.27:0/1004632642 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddf9255880 con 0x55ddf93bf320 2024-05-21T03:23:01.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.964+0000 7f5e84d98700 1 --2- 172.21.15.27:0/1004632642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddf93bb5d0 0x55ddf93bed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:23:01.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.964+0000 7f5e85d9a700 1 --2- 172.21.15.27:0/1004632642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ddf93bf320 0x55ddf93bf760 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5e78006010 tx=0x7f5e78005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e719bad4f82a31 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:01.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.964+0000 7f5e6b7fe700 1 -- 172.21.15.27:0/1004632642 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78027070 con 0x55ddf93bf320 2024-05-21T03:23:01.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.964+0000 7f5e6b7fe700 1 -- 172.21.15.27:0/1004632642 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e780050d0 con 0x55ddf93bf320 2024-05-21T03:23:01.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.964+0000 7f5e6b7fe700 1 -- 172.21.15.27:0/1004632642 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7800ba90 con 0x55ddf93bf320 2024-05-21T03:23:01.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.965+0000 7f5e92093b80 1 -- 172.21.15.27:0/1004632642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ddf93bf320 msgr2=0x55ddf93bf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:01.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.965+0000 7f5e92093b80 1 --2- 172.21.15.27:0/1004632642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ddf93bf320 0x55ddf93bf760 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5e78006010 tx=0x7f5e78005c00 comp rx=0 tx=0).stop 2024-05-21T03:23:01.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.965+0000 7f5e92093b80 1 -- 172.21.15.27:0/1004632642 shutdown_connections 2024-05-21T03:23:01.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.965+0000 7f5e92093b80 1 --2- 172.21.15.27:0/1004632642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ddf93bf320 0x55ddf93bf760 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:01.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.965+0000 7f5e92093b80 1 --2- 172.21.15.27:0/1004632642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf93bac20 0x55ddf93bb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:01.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.965+0000 7f5e92093b80 1 --2- 172.21.15.27:0/1004632642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddf93bb5d0 0x55ddf93bed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:01.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.965+0000 7f5e92093b80 1 -- 172.21.15.27:0/1004632642 >> 172.21.15.27:0/1004632642 conn(0x55ddf92e4030 msgr2=0x55ddf93b0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:01.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.965+0000 7f5e92093b80 1 -- 172.21.15.27:0/1004632642 shutdown_connections 2024-05-21T03:23:01.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.965+0000 7f5e92093b80 1 -- 172.21.15.27:0/1004632642 wait complete. 2024-05-21T03:23:01.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.965+0000 7f5e92093b80 1 -- start start 2024-05-21T03:23:01.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.965+0000 7f5e92093b80 1 Processor -- start 2024-05-21T03:23:01.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.966+0000 7f5e92093b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ddf93bac20 0x55ddf93c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:01.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.966+0000 7f5e92093b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddf93bb5d0 0x55ddf93c3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:01.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.966+0000 7f5e92093b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf93bf320 0x55ddf93c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:01.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.966+0000 7f5e92093b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ddf920bf50 con 0x55ddf93bac20 2024-05-21T03:23:01.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.966+0000 7f5e92093b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ddf92a7770 con 0x55ddf93bf320 2024-05-21T03:23:01.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.966+0000 7f5e92093b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ddf93c7480 con 0x55ddf93bb5d0 2024-05-21T03:23:01.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.966+0000 7f5e92093b80 10 client.0 _ll_register_callbacks cb 0x55ddf92a08d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:23:01.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.966+0000 7f5e85599700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ddf93bac20 0x55ddf93c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:01.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.966+0000 7f5e85599700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ddf93bac20 0x55ddf93c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43320/0 (socket says 172.21.15.27:43320) 2024-05-21T03:23:01.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.966+0000 7f5e85599700 1 -- 172.21.15.27:0/1743708713 learned_addr learned my addr 172.21.15.27:0/1743708713 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:01.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.966+0000 7f5e85d9a700 1 --2- 172.21.15.27:0/1743708713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf93bf320 0x55ddf93c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:01.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.966+0000 7f5e84d98700 1 --2- 172.21.15.27:0/1743708713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddf93bb5d0 0x55ddf93c3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:01.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.966+0000 7f5e85599700 1 -- 172.21.15.27:0/1743708713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddf93bb5d0 msgr2=0x55ddf93c3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:01.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.966+0000 7f5e85599700 1 --2- 172.21.15.27:0/1743708713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddf93bb5d0 0x55ddf93c3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:01.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.966+0000 7f5e85599700 1 -- 172.21.15.27:0/1743708713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf93bf320 msgr2=0x55ddf93c3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:01.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.966+0000 7f5e85599700 1 --2- 172.21.15.27:0/1743708713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf93bf320 0x55ddf93c3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:01.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.966+0000 7f5e85599700 1 -- 172.21.15.27:0/1743708713 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddf92a4640 con 0x55ddf93bac20 2024-05-21T03:23:01.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.966+0000 7f5e84d98700 1 --2- 172.21.15.27:0/1743708713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddf93bb5d0 0x55ddf93c3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:23:01.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.966+0000 7f5e85599700 1 --2- 172.21.15.27:0/1743708713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ddf93bac20 0x55ddf93c6770 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c0df8e0 tx=0x7f5e7c015c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c916dcd9d3a28220 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:01.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.966+0000 7f5e68ff9700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:23:01.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.966+0000 7f5e68ff9700 1 -- 172.21.15.27:0/1743708713 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c0ad440 con 0x55ddf93bac20 2024-05-21T03:23:01.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.967+0000 7f5e68ff9700 1 -- 172.21.15.27:0/1743708713 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f5e7c0b1a60 con 0x55ddf93bac20 2024-05-21T03:23:01.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.967+0000 7f5e92093b80 1 -- 172.21.15.27:0/1743708713 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e780050d0 con 0x55ddf93bac20 2024-05-21T03:23:01.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.967+0000 7f5e68ff9700 1 -- 172.21.15.27:0/1743708713 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5e7c0b1ef0 con 0x55ddf93bac20 2024-05-21T03:23:01.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.967+0000 7f5e92093b80 10 client.15021 fetch_fsmap learned FSMap version 28 2024-05-21T03:23:01.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.967+0000 7f5e92093b80 1 -- 172.21.15.27:0/1743708713 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ddf93d1b80 con 0x55ddf93bac20 2024-05-21T03:23:01.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.967+0000 7f5e68ff9700 1 -- 172.21.15.27:0/1743708713 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e780050d0 con 0x55ddf93bac20 2024-05-21T03:23:01.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.967+0000 7f5e68ff9700 1 -- 172.21.15.27:0/1743708713 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5e7c004ad0 con 0x55ddf93bac20 2024-05-21T03:23:01.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.967+0000 7f5e92093b80 10 client.15021 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T03:23:01.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.967+0000 7f5e92093b80 10 client.15021 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-21T03:23:01.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.967+0000 7f5e92093b80 20 client.15021 populate_metadata read hostname 'smithi027' 2024-05-21T03:23:01.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.967+0000 7f5e92093b80 1 --2- 172.21.15.27:0/1743708713 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x55ddf93d41b0 0x55ddf93d5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:01.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.967+0000 7f5e92093b80 4 client.15021 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:23:01.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.967+0000 7f5e92093b80 1 -- 172.21.15.27:0/1743708713 --> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddf9382990 con 0x55ddf93d41b0 2024-05-21T03:23:01.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.967+0000 7f5e84d98700 1 --2- 172.21.15.27:0/1743708713 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x55ddf93d41b0 0x55ddf93d5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:01.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.968+0000 7f5e68ff9700 10 client.15021 ms_handle_connect on v2:172.21.15.39:6836/3673846852 2024-05-21T03:23:01.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.968+0000 7f5e84d98700 1 --2- 172.21.15.27:0/1743708713 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x55ddf93d41b0 0x55ddf93d5620 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cb4fd1c71d6ed98f server_cookie=40e3065eede37bf8 in_seq=0 out_seq=0 2024-05-21T03:23:01.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.969+0000 7f5e68ff9700 1 -- 172.21.15.27:0/1743708713 <== mds.3 v2:172.21.15.39:6836/3673846852 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ddf9382990 con 0x55ddf93d41b0 2024-05-21T03:23:01.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.969+0000 7f5e68ff9700 10 client.15021 handle_command_reply: tid=0 2024-05-21T03:23:01.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.969+0000 7f5e92093b80 1 client.15021 shutdown 2024-05-21T03:23:01.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.969+0000 7f5e92093b80 1 -- 172.21.15.27:0/1743708713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ddf93bac20 msgr2=0x55ddf93c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:01.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.969+0000 7f5e92093b80 1 --2- 172.21.15.27:0/1743708713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ddf93bac20 0x55ddf93c6770 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c0df8e0 tx=0x7f5e7c015c70 comp rx=0 tx=0).stop 2024-05-21T03:23:01.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.969+0000 7f5e92093b80 1 -- 172.21.15.27:0/1743708713 shutdown_connections 2024-05-21T03:23:01.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.969+0000 7f5e92093b80 1 --2- 172.21.15.27:0/1743708713 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x55ddf93d41b0 0x55ddf93d5620 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:01.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.969+0000 7f5e92093b80 1 --2- 172.21.15.27:0/1743708713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf93bf320 0x55ddf93c3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:01.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.969+0000 7f5e92093b80 1 --2- 172.21.15.27:0/1743708713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ddf93bac20 0x55ddf93c6770 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:01.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.969+0000 7f5e68ff9700 0 client.15021 ms_handle_reset on v2:172.21.15.39:6836/3673846852 2024-05-21T03:23:01.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.969+0000 7f5e92093b80 1 --2- 172.21.15.27:0/1743708713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddf93bb5d0 0x55ddf93c3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:01.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.969+0000 7f5e92093b80 1 -- 172.21.15.27:0/1743708713 >> 172.21.15.27:0/1743708713 conn(0x55ddf92e4030 msgr2=0x55ddf93b04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:01.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.969+0000 7f5e92093b80 1 -- 172.21.15.27:0/1743708713 shutdown_connections 2024-05-21T03:23:01.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.969+0000 7f5e92093b80 1 -- 172.21.15.27:0/1743708713 wait complete. 2024-05-21T03:23:01.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.969+0000 7f5e92093b80 20 client.15021 trim_cache size 0 max 16384 2024-05-21T03:23:01.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.991+0000 7f5e92093b80 1 Processor -- start 2024-05-21T03:23:01.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.991+0000 7f5e92093b80 1 -- start start 2024-05-21T03:23:01.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.991+0000 7f5e92093b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf93bac20 0x55ddf93b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:01.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.991+0000 7f5e92093b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddf93bb5d0 0x55ddf93b1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:01.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.991+0000 7f5e92093b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ddf93bf320 0x55ddf93dea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:01.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.991+0000 7f5e92093b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ddf93d3960 con 0x55ddf93bf320 2024-05-21T03:23:01.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.991+0000 7f5e92093b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ddf93d4030 con 0x55ddf93bac20 2024-05-21T03:23:01.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.991+0000 7f5e92093b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ddf93d4700 con 0x55ddf93bb5d0 2024-05-21T03:23:01.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.991+0000 7f5e85d9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ddf93bf320 0x55ddf93dea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:01.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.991+0000 7f5e85d9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ddf93bf320 0x55ddf93dea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43322/0 (socket says 172.21.15.27:43322) 2024-05-21T03:23:01.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.991+0000 7f5e85d9a700 1 -- 172.21.15.27:0/1216703880 learned_addr learned my addr 172.21.15.27:0/1216703880 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:01.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.991+0000 7f5e84d98700 1 --2- 172.21.15.27:0/1216703880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddf93bb5d0 0x55ddf93b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:01.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.992+0000 7f5e85599700 1 --2- 172.21.15.27:0/1216703880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf93bac20 0x55ddf93b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:01.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.992+0000 7f5e85d9a700 1 -- 172.21.15.27:0/1216703880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddf93bb5d0 msgr2=0x55ddf93b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:01.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.992+0000 7f5e85d9a700 1 --2- 172.21.15.27:0/1216703880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddf93bb5d0 0x55ddf93b1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:01.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.992+0000 7f5e85d9a700 1 -- 172.21.15.27:0/1216703880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf93bac20 msgr2=0x55ddf93b0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:01.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.992+0000 7f5e85d9a700 1 --2- 172.21.15.27:0/1216703880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf93bac20 0x55ddf93b0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:01.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.992+0000 7f5e85d9a700 1 -- 172.21.15.27:0/1216703880 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddf93b1cf0 con 0x55ddf93bf320 2024-05-21T03:23:01.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.992+0000 7f5e85599700 1 --2- 172.21.15.27:0/1216703880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf93bac20 0x55ddf93b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:23:01.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.992+0000 7f5e85d9a700 1 --2- 172.21.15.27:0/1216703880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ddf93bf320 0x55ddf93dea80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5e780099c0 tx=0x7f5e780039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c2403cdb06d62a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:01.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.992+0000 7f5e6affd700 1 -- 172.21.15.27:0/1216703880 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78032070 con 0x55ddf93bf320 2024-05-21T03:23:01.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.992+0000 7f5e6affd700 1 -- 172.21.15.27:0/1216703880 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e78003d60 con 0x55ddf93bf320 2024-05-21T03:23:01.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.992+0000 7f5e6affd700 1 -- 172.21.15.27:0/1216703880 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78027040 con 0x55ddf93bf320 2024-05-21T03:23:01.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.993+0000 7f5e92093b80 1 -- 172.21.15.27:0/1216703880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ddf93bf320 msgr2=0x55ddf93dea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:01.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.993+0000 7f5e92093b80 1 --2- 172.21.15.27:0/1216703880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ddf93bf320 0x55ddf93dea80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5e780099c0 tx=0x7f5e780039f0 comp rx=0 tx=0).stop 2024-05-21T03:23:01.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.993+0000 7f5e92093b80 1 -- 172.21.15.27:0/1216703880 shutdown_connections 2024-05-21T03:23:01.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.993+0000 7f5e92093b80 1 --2- 172.21.15.27:0/1216703880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ddf93bf320 0x55ddf93dea80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:01.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.993+0000 7f5e92093b80 1 --2- 172.21.15.27:0/1216703880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf93bac20 0x55ddf93b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:01.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.993+0000 7f5e92093b80 1 --2- 172.21.15.27:0/1216703880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddf93bb5d0 0x55ddf93b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:01.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.993+0000 7f5e92093b80 1 -- 172.21.15.27:0/1216703880 >> 172.21.15.27:0/1216703880 conn(0x55ddf92e4030 msgr2=0x55ddf93b5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:01.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.993+0000 7f5e92093b80 1 -- 172.21.15.27:0/1216703880 shutdown_connections 2024-05-21T03:23:01.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.993+0000 7f5e92093b80 1 -- 172.21.15.27:0/1216703880 wait complete. 2024-05-21T03:23:01.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.993+0000 7f5e92093b80 1 -- start start 2024-05-21T03:23:01.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.994+0000 7f5e92093b80 1 Processor -- start 2024-05-21T03:23:01.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.994+0000 7f5e92093b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf93bac20 0x55ddf93db060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:01.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.994+0000 7f5e92093b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ddf93bb5d0 0x55ddf93db5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:01.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.994+0000 7f5e92093b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddf93bf320 0x55ddf94389f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:01.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.994+0000 7f5e92093b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ddf93d54a0 con 0x55ddf93bb5d0 2024-05-21T03:23:01.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.994+0000 7f5e92093b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ddf93d4dd0 con 0x55ddf93bac20 2024-05-21T03:23:01.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.994+0000 7f5e92093b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ddf93d5b70 con 0x55ddf93bf320 2024-05-21T03:23:01.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.994+0000 7f5e92093b80 10 client.0 _ll_register_callbacks cb 0x55ddf92a08d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:23:01.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.994+0000 7f5e84d98700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ddf93bb5d0 0x55ddf93db5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:01.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.994+0000 7f5e84d98700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ddf93bb5d0 0x55ddf93db5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43326/0 (socket says 172.21.15.27:43326) 2024-05-21T03:23:01.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.994+0000 7f5e84d98700 1 -- 172.21.15.27:0/3666385579 learned_addr learned my addr 172.21.15.27:0/3666385579 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:01.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.994+0000 7f5e85d9a700 1 --2- 172.21.15.27:0/3666385579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddf93bf320 0x55ddf94389f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:01.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.994+0000 7f5e85599700 1 --2- 172.21.15.27:0/3666385579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf93bac20 0x55ddf93db060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:01.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.994+0000 7f5e84d98700 1 -- 172.21.15.27:0/3666385579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddf93bf320 msgr2=0x55ddf94389f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:01.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.994+0000 7f5e84d98700 1 --2- 172.21.15.27:0/3666385579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddf93bf320 0x55ddf94389f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:01.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.994+0000 7f5e84d98700 1 -- 172.21.15.27:0/3666385579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf93bac20 msgr2=0x55ddf93db060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:01.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.994+0000 7f5e84d98700 1 --2- 172.21.15.27:0/3666385579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf93bac20 0x55ddf93db060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:02.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.995+0000 7f5e84d98700 1 -- 172.21.15.27:0/3666385579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddf9439030 con 0x55ddf93bb5d0 2024-05-21T03:23:02.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.995+0000 7f5e85599700 1 --2- 172.21.15.27:0/3666385579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf93bac20 0x55ddf93db060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:23:02.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.995+0000 7f5e84d98700 1 --2- 172.21.15.27:0/3666385579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ddf93bb5d0 0x55ddf93db5a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5e8019b280 tx=0x7f5e700047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=647502bf6a3c377f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:02.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.995+0000 7f5e68ff9700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:23:02.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.995+0000 7f5e68ff9700 1 -- 172.21.15.27:0/3666385579 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7002b070 con 0x55ddf93bb5d0 2024-05-21T03:23:02.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.995+0000 7f5e68ff9700 1 -- 172.21.15.27:0/3666385579 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f5e70027220 con 0x55ddf93bb5d0 2024-05-21T03:23:02.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.995+0000 7f5e92093b80 1 -- 172.21.15.27:0/3666385579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e78003d60 con 0x55ddf93bb5d0 2024-05-21T03:23:02.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.995+0000 7f5e68ff9700 1 -- 172.21.15.27:0/3666385579 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5e70036070 con 0x55ddf93bb5d0 2024-05-21T03:23:02.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.995+0000 7f5e92093b80 10 client.15027 fetch_fsmap learned FSMap version 28 2024-05-21T03:23:02.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.995+0000 7f5e92093b80 1 -- 172.21.15.27:0/3666385579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ddf9440450 con 0x55ddf93bb5d0 2024-05-21T03:23:02.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.995+0000 7f5e68ff9700 1 -- 172.21.15.27:0/3666385579 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e78003d60 con 0x55ddf93bb5d0 2024-05-21T03:23:02.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.996+0000 7f5e68ff9700 1 -- 172.21.15.27:0/3666385579 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5e7000f040 con 0x55ddf93bb5d0 2024-05-21T03:23:02.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.996+0000 7f5e92093b80 10 client.15027 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T03:23:02.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.996+0000 7f5e92093b80 10 client.15027 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-21T03:23:02.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.996+0000 7f5e92093b80 20 client.15027 populate_metadata read hostname 'smithi027' 2024-05-21T03:23:02.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.996+0000 7f5e92093b80 1 --2- 172.21.15.27:0/3666385579 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x55ddf9445060 0x55ddf9443d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:02.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.996+0000 7f5e92093b80 4 client.15027 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T03:23:02.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.996+0000 7f5e92093b80 1 -- 172.21.15.27:0/3666385579 --> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] -- command(tid 0: {"prefix": "status"}) -- 0x55ddf9317b90 con 0x55ddf9445060 2024-05-21T03:23:02.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.996+0000 7f5e85599700 1 --2- 172.21.15.27:0/3666385579 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x55ddf9445060 0x55ddf9443d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:02.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.996+0000 7f5e68ff9700 10 client.15027 ms_handle_connect on v2:172.21.15.39:6836/3673846852 2024-05-21T03:23:02.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.996+0000 7f5e85599700 1 --2- 172.21.15.27:0/3666385579 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x55ddf9445060 0x55ddf9443d70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=130cbec5074f917 server_cookie=f8bc1ebfaa20f4c7 in_seq=0 out_seq=0 2024-05-21T03:23:02.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.997+0000 7f5e68ff9700 1 -- 172.21.15.27:0/3666385579 <== mds.3 v2:172.21.15.39:6836/3673846852 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55ddf9317b90 con 0x55ddf9445060 2024-05-21T03:23:02.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.997+0000 7f5e68ff9700 10 client.15027 handle_command_reply: tid=0 2024-05-21T03:23:02.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.997+0000 7f5e92093b80 1 client.15027 shutdown 2024-05-21T03:23:02.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.997+0000 7f5e92093b80 1 -- 172.21.15.27:0/3666385579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ddf93bb5d0 msgr2=0x55ddf93db5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:02.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.997+0000 7f5e92093b80 1 --2- 172.21.15.27:0/3666385579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ddf93bb5d0 0x55ddf93db5a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5e8019b280 tx=0x7f5e700047c0 comp rx=0 tx=0).stop 2024-05-21T03:23:02.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.997+0000 7f5e92093b80 1 -- 172.21.15.27:0/3666385579 shutdown_connections 2024-05-21T03:23:02.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.997+0000 7f5e92093b80 1 --2- 172.21.15.27:0/3666385579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddf93bf320 0x55ddf94389f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:02.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.997+0000 7f5e92093b80 1 --2- 172.21.15.27:0/3666385579 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x55ddf9445060 0x55ddf9443d70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:02.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.997+0000 7f5e68ff9700 0 client.15027 ms_handle_reset on v2:172.21.15.39:6836/3673846852 2024-05-21T03:23:02.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.997+0000 7f5e92093b80 1 --2- 172.21.15.27:0/3666385579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf93bac20 0x55ddf93db060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:02.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.997+0000 7f5e92093b80 1 --2- 172.21.15.27:0/3666385579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ddf93bb5d0 0x55ddf93db5a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:02.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.997+0000 7f5e92093b80 1 -- 172.21.15.27:0/3666385579 >> 172.21.15.27:0/3666385579 conn(0x55ddf92e4030 msgr2=0x55ddf93c2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:02.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.997+0000 7f5e92093b80 1 -- 172.21.15.27:0/3666385579 shutdown_connections 2024-05-21T03:23:02.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.997+0000 7f5e92093b80 1 -- 172.21.15.27:0/3666385579 wait complete. 2024-05-21T03:23:02.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.997+0000 7f5e92093b80 20 client.15027 trim_cache size 0 max 16384 2024-05-21T03:23:02.002 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:23:02.003 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "c9ca94be-171f-11ef-bc9a-c7b262605968", 2024-05-21T03:23:02.003 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 3, 2024-05-21T03:23:02.003 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24452, 2024-05-21T03:23:02.003 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-05-21T03:23:02.003 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-05-21T03:23:02.003 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-05-21T03:23:02.003 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 133.55304323999999, 2024-05-21T03:23:02.003 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 28, 2024-05-21T03:23:02.003 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 82, 2024-05-21T03:23:02.003 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 80, 2024-05-21T03:23:02.003 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 174.75055881 2024-05-21T03:23:02.003 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:23:02.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.998+0000 7f5e877fd700 1 -- 172.21.15.27:0/4087182201 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f5e6c0b2880 msgr2=0x7f5e6c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:02.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.998+0000 7f5e877fd700 1 --2- 172.21.15.27:0/4087182201 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f5e6c0b2880 0x7f5e6c0b4d40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5e70005fd0 tx=0x7f5e70005ee0 comp rx=0 tx=0).stop 2024-05-21T03:23:02.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.998+0000 7f5e877fd700 1 -- 172.21.15.27:0/4087182201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e80103010 msgr2=0x7f5e8019fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:02.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.998+0000 7f5e877fd700 1 --2- 172.21.15.27:0/4087182201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e80103010 0x7f5e8019fa60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c0060b0 tx=0x7f5e7c007f90 comp rx=0 tx=0).stop 2024-05-21T03:23:02.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.998+0000 7f5e877fd700 1 -- 172.21.15.27:0/4087182201 shutdown_connections 2024-05-21T03:23:02.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.998+0000 7f5e877fd700 1 --2- 172.21.15.27:0/4087182201 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f5e6c0b2880 0x7f5e6c0b4d40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:02.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.998+0000 7f5e877fd700 1 --2- 172.21.15.27:0/4087182201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e80108730 0x7f5e80199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:02.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.998+0000 7f5e877fd700 1 --2- 172.21.15.27:0/4087182201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e80103010 0x7f5e8019fa60 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:02.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.998+0000 7f5e877fd700 1 --2- 172.21.15.27:0/4087182201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e801039d0 0x7f5e8019ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:02.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.998+0000 7f5e877fd700 1 -- 172.21.15.27:0/4087182201 >> 172.21.15.27:0/4087182201 conn(0x7f5e80077780 msgr2=0x7f5e80102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:02.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.998+0000 7f5e877fd700 1 -- 172.21.15.27:0/4087182201 shutdown_connections 2024-05-21T03:23:02.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:01.998+0000 7f5e877fd700 1 -- 172.21.15.27:0/4087182201 wait complete. 2024-05-21T03:23:02.662 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell mds.c status 2024-05-21T03:23:02.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:02 smithi039 ceph-mon[36429]: pgmap v366: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:23:02.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:02 smithi165 ceph-mon[36269]: pgmap v366: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:23:02.916 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:23:02.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:02 smithi027 ceph-mon[32179]: pgmap v366: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:23:04.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.546+0000 7f9795438700 1 -- 172.21.15.27:0/3912198130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9790101a90 msgr2=0x7f9790101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:04.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.546+0000 7f9795438700 1 --2- 172.21.15.27:0/3912198130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9790101a90 0x7f9790101f10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f97780099c0 tx=0x7f9778009cd0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.546+0000 7f9795438700 1 -- 172.21.15.27:0/3912198130 shutdown_connections 2024-05-21T03:23:04.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.546+0000 7f9795438700 1 --2- 172.21.15.27:0/3912198130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9790102450 0x7f97901103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.546+0000 7f9795438700 1 --2- 172.21.15.27:0/3912198130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9790109610 0x7f97901099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.546+0000 7f9795438700 1 --2- 172.21.15.27:0/3912198130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9790101a90 0x7f9790101f10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.546+0000 7f9795438700 1 -- 172.21.15.27:0/3912198130 >> 172.21.15.27:0/3912198130 conn(0x7f9790078a40 msgr2=0x7f97900ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:04.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.546+0000 7f9795438700 1 -- 172.21.15.27:0/3912198130 shutdown_connections 2024-05-21T03:23:04.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.546+0000 7f9795438700 1 -- 172.21.15.27:0/3912198130 wait complete. 2024-05-21T03:23:04.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.546+0000 7f9795438700 1 Processor -- start 2024-05-21T03:23:04.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.547+0000 7f9795438700 1 -- start start 2024-05-21T03:23:04.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.547+0000 7f9795438700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9790101a90 0x7f9790102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:04.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.547+0000 7f9795438700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9790102450 0x7f97901031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:04.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.547+0000 7f9795438700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9790109610 0x7f97901036f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:04.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.547+0000 7f9795438700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9790113790 con 0x7f9790102450 2024-05-21T03:23:04.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.547+0000 7f9795438700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9790113610 con 0x7f9790101a90 2024-05-21T03:23:04.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.547+0000 7f9795438700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9790113910 con 0x7f9790109610 2024-05-21T03:23:04.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.547+0000 7f978e7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9790102450 0x7f97901031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:04.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.547+0000 7f978e7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9790102450 0x7f97901031b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37744/0 (socket says 172.21.15.27:37744) 2024-05-21T03:23:04.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.547+0000 7f978e7fc700 1 -- 172.21.15.27:0/4291200273 learned_addr learned my addr 172.21.15.27:0/4291200273 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:04.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.547+0000 7f978effd700 1 --2- 172.21.15.27:0/4291200273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9790101a90 0x7f9790102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:04.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.547+0000 7f978f7fe700 1 --2- 172.21.15.27:0/4291200273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9790109610 0x7f97901036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:04.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.547+0000 7f978e7fc700 1 -- 172.21.15.27:0/4291200273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9790109610 msgr2=0x7f97901036f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:04.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.547+0000 7f978e7fc700 1 --2- 172.21.15.27:0/4291200273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9790109610 0x7f97901036f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.547+0000 7f978e7fc700 1 -- 172.21.15.27:0/4291200273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9790101a90 msgr2=0x7f9790102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:04.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.547+0000 7f978e7fc700 1 --2- 172.21.15.27:0/4291200273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9790101a90 0x7f9790102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.547+0000 7f978e7fc700 1 -- 172.21.15.27:0/4291200273 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97901a8110 con 0x7f9790102450 2024-05-21T03:23:04.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.548+0000 7f978e7fc700 1 --2- 172.21.15.27:0/4291200273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9790102450 0x7f97901031b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f977800b580 tx=0x7f97780047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:04.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.548+0000 7f9787fff700 1 -- 172.21.15.27:0/4291200273 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9778027070 con 0x7f9790102450 2024-05-21T03:23:04.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.548+0000 7f9795438700 1 -- 172.21.15.27:0/4291200273 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97901a83a0 con 0x7f9790102450 2024-05-21T03:23:04.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.548+0000 7f9787fff700 1 -- 172.21.15.27:0/4291200273 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9778003f20 con 0x7f9790102450 2024-05-21T03:23:04.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.548+0000 7f9787fff700 1 -- 172.21.15.27:0/4291200273 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9778022d10 con 0x7f9790102450 2024-05-21T03:23:04.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.548+0000 7f9795438700 1 -- 172.21.15.27:0/4291200273 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97901a8800 con 0x7f9790102450 2024-05-21T03:23:04.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.550+0000 7f9787fff700 1 -- 172.21.15.27:0/4291200273 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9778022470 con 0x7f9790102450 2024-05-21T03:23:04.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.551+0000 7f9787fff700 1 --2- 172.21.15.27:0/4291200273 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f977c0b2880 0x7f977c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:04.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.551+0000 7f978effd700 1 --2- 172.21.15.27:0/4291200273 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f977c0b2880 0x7f977c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:04.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.551+0000 7f9787fff700 1 -- 172.21.15.27:0/4291200273 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f97780e2e90 con 0x7f9790102450 2024-05-21T03:23:04.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.551+0000 7f979fcceb80 1 Processor -- start 2024-05-21T03:23:04.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.552+0000 7f978effd700 1 --2- 172.21.15.27:0/4291200273 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f977c0b2880 0x7f977c0b4d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f978000a400 tx=0x7f978000d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:04.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.552+0000 7f979fcceb80 1 -- start start 2024-05-21T03:23:04.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.552+0000 7f979fcceb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4579452d0 0x55f4579456b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:04.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.552+0000 7f979fcceb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f45793f850 0x55f45793fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:04.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.552+0000 7f979fcceb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f457940210 0x55f45794c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:04.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.552+0000 7f979fcceb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f45789dd20 con 0x55f45793f850 2024-05-21T03:23:04.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.552+0000 7f979fcceb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f45788d8c0 con 0x55f4579452d0 2024-05-21T03:23:04.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.552+0000 7f979fcceb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f457891900 con 0x55f457940210 2024-05-21T03:23:04.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.552+0000 7f978e7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f45793f850 0x55f45793fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:04.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.552+0000 7f978e7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f45793f850 0x55f45793fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37748/0 (socket says 172.21.15.27:37748) 2024-05-21T03:23:04.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.552+0000 7f978e7fc700 1 -- 172.21.15.27:0/1989883921 learned_addr learned my addr 172.21.15.27:0/1989883921 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:04.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.552+0000 7f978effd700 1 --2- 172.21.15.27:0/1989883921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4579452d0 0x55f4579456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:04.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.552+0000 7f978f7fe700 1 --2- 172.21.15.27:0/1989883921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f457940210 0x55f45794c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:04.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.552+0000 7f978e7fc700 1 -- 172.21.15.27:0/1989883921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f457940210 msgr2=0x55f45794c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:04.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.552+0000 7f978e7fc700 1 --2- 172.21.15.27:0/1989883921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f457940210 0x55f45794c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.552+0000 7f978e7fc700 1 -- 172.21.15.27:0/1989883921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4579452d0 msgr2=0x55f4579456b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:04.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.552+0000 7f978e7fc700 1 --2- 172.21.15.27:0/1989883921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4579452d0 0x55f4579456b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.552+0000 7f978e7fc700 1 -- 172.21.15.27:0/1989883921 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f457836f60 con 0x55f45793f850 2024-05-21T03:23:04.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.553+0000 7f978e7fc700 1 --2- 172.21.15.27:0/1989883921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f45793f850 0x55f45793fcd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f97780aa220 tx=0x7f9778003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a165a08bac41a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:04.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.553+0000 7f97857fa700 1 -- 172.21.15.27:0/1989883921 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97780aac20 con 0x55f45793f850 2024-05-21T03:23:04.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.553+0000 7f97857fa700 1 -- 172.21.15.27:0/1989883921 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97780aa520 con 0x55f45793f850 2024-05-21T03:23:04.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.553+0000 7f97857fa700 1 -- 172.21.15.27:0/1989883921 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97780b3060 con 0x55f45793f850 2024-05-21T03:23:04.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.553+0000 7f979fcceb80 1 -- 172.21.15.27:0/1989883921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f45793f850 msgr2=0x55f45793fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:04.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.553+0000 7f979fcceb80 1 --2- 172.21.15.27:0/1989883921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f45793f850 0x55f45793fcd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f97780aa220 tx=0x7f9778003780 comp rx=0 tx=0).stop 2024-05-21T03:23:04.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.554+0000 7f979fcceb80 1 -- 172.21.15.27:0/1989883921 shutdown_connections 2024-05-21T03:23:04.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.554+0000 7f979fcceb80 1 --2- 172.21.15.27:0/1989883921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f457940210 0x55f45794c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.554+0000 7f979fcceb80 1 --2- 172.21.15.27:0/1989883921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4579452d0 0x55f4579456b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.554+0000 7f979fcceb80 1 --2- 172.21.15.27:0/1989883921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f45793f850 0x55f45793fcd0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.554+0000 7f979fcceb80 1 -- 172.21.15.27:0/1989883921 >> 172.21.15.27:0/1989883921 conn(0x55f45786b1f0 msgr2=0x55f45793a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:04.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.554+0000 7f979fcceb80 1 -- 172.21.15.27:0/1989883921 shutdown_connections 2024-05-21T03:23:04.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.554+0000 7f979fcceb80 1 -- 172.21.15.27:0/1989883921 wait complete. 2024-05-21T03:23:04.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.554+0000 7f979fcceb80 1 -- start start 2024-05-21T03:23:04.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.554+0000 7f979fcceb80 1 Processor -- start 2024-05-21T03:23:04.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.554+0000 7f979fcceb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f45793f850 0x55f45794bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:04.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.554+0000 7f979fcceb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f457940210 0x55f457948fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:04.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.554+0000 7f979fcceb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4579452d0 0x55f4579494e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:04.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.554+0000 7f979fcceb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f457793f50 con 0x55f45793f850 2024-05-21T03:23:04.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.554+0000 7f979fcceb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f4577f5d50 con 0x55f457940210 2024-05-21T03:23:04.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.554+0000 7f979fcceb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f45789dd20 con 0x55f4579452d0 2024-05-21T03:23:04.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.555+0000 7f978effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f45793f850 0x55f45794bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:04.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.555+0000 7f979fcceb80 10 client.0 _ll_register_callbacks cb 0x55f457828900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:23:04.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.555+0000 7f978effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f45793f850 0x55f45794bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37754/0 (socket says 172.21.15.27:37754) 2024-05-21T03:23:04.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.555+0000 7f978effd700 1 -- 172.21.15.27:0/4275865611 learned_addr learned my addr 172.21.15.27:0/4275865611 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:04.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.555+0000 7f978e7fc700 1 --2- 172.21.15.27:0/4275865611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f457940210 0x55f457948fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:04.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.555+0000 7f978f7fe700 1 --2- 172.21.15.27:0/4275865611 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4579452d0 0x55f4579494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:04.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.555+0000 7f978effd700 1 -- 172.21.15.27:0/4275865611 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4579452d0 msgr2=0x55f4579494e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:04.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.555+0000 7f978effd700 1 --2- 172.21.15.27:0/4275865611 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4579452d0 0x55f4579494e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.555+0000 7f978effd700 1 -- 172.21.15.27:0/4275865611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f457940210 msgr2=0x55f457948fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:04.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.555+0000 7f978effd700 1 --2- 172.21.15.27:0/4275865611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f457940210 0x55f457948fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.555+0000 7f978effd700 1 -- 172.21.15.27:0/4275865611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4577dd880 con 0x55f45793f850 2024-05-21T03:23:04.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.555+0000 7f978effd700 1 --2- 172.21.15.27:0/4275865611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f45793f850 0x55f45794bca0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9780016e70 tx=0x7f9780004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fdfc7a7d9df3bc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:04.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.555+0000 7f97767fc700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:23:04.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.555+0000 7f97767fc700 1 -- 172.21.15.27:0/4275865611 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9780022440 con 0x55f45793f850 2024-05-21T03:23:04.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.555+0000 7f97767fc700 1 -- 172.21.15.27:0/4275865611 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f97800225e0 con 0x55f45793f850 2024-05-21T03:23:04.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.555+0000 7f979fcceb80 1 -- 172.21.15.27:0/4275865611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97780aa520 con 0x55f45793f850 2024-05-21T03:23:04.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.555+0000 7f97767fc700 1 -- 172.21.15.27:0/4275865611 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f97800227e0 con 0x55f45793f850 2024-05-21T03:23:04.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.556+0000 7f979fcceb80 10 client.15039 fetch_fsmap learned FSMap version 28 2024-05-21T03:23:04.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.556+0000 7f979fcceb80 1 -- 172.21.15.27:0/4275865611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f457847710 con 0x55f45793f850 2024-05-21T03:23:04.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.556+0000 7f97767fc700 1 -- 172.21.15.27:0/4275865611 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97780aa520 con 0x55f45793f850 2024-05-21T03:23:04.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.556+0000 7f97767fc700 1 -- 172.21.15.27:0/4275865611 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f978002b9d0 con 0x55f45793f850 2024-05-21T03:23:04.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.556+0000 7f979fcceb80 10 client.15039 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T03:23:04.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.556+0000 7f979fcceb80 10 client.15039 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-21T03:23:04.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.556+0000 7f979fcceb80 20 client.15039 populate_metadata read hostname 'smithi027' 2024-05-21T03:23:04.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.556+0000 7f979fcceb80 1 --2- 172.21.15.27:0/4275865611 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x55f457958ee0 0x55f45795a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:04.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.556+0000 7f979fcceb80 4 client.15039 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:23:04.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.556+0000 7f979fcceb80 1 -- 172.21.15.27:0/4275865611 --> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f45789fc70 con 0x55f457958ee0 2024-05-21T03:23:04.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.556+0000 7f978e7fc700 1 --2- 172.21.15.27:0/4275865611 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x55f457958ee0 0x55f45795a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:04.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.557+0000 7f978e7fc700 1 --2- 172.21.15.27:0/4275865611 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x55f457958ee0 0x55f45795a350 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=23427b65365d01a9 server_cookie=8cd8b9c39db71907 in_seq=0 out_seq=0 2024-05-21T03:23:04.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.557+0000 7f97767fc700 10 client.15039 ms_handle_connect on v2:172.21.15.165:6836/691356139 2024-05-21T03:23:04.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.558+0000 7f97767fc700 1 -- 172.21.15.27:0/4275865611 <== mds.2 v2:172.21.15.165:6836/691356139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f45789fc70 con 0x55f457958ee0 2024-05-21T03:23:04.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.558+0000 7f97767fc700 10 client.15039 handle_command_reply: tid=0 2024-05-21T03:23:04.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.558+0000 7f979fcceb80 1 client.15039 shutdown 2024-05-21T03:23:04.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.558+0000 7f979fcceb80 1 -- 172.21.15.27:0/4275865611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f45793f850 msgr2=0x55f45794bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:04.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.558+0000 7f979fcceb80 1 --2- 172.21.15.27:0/4275865611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f45793f850 0x55f45794bca0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9780016e70 tx=0x7f9780004be0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.558+0000 7f979fcceb80 1 -- 172.21.15.27:0/4275865611 shutdown_connections 2024-05-21T03:23:04.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.558+0000 7f979fcceb80 1 --2- 172.21.15.27:0/4275865611 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x55f457958ee0 0x55f45795a350 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.558+0000 7f979fcceb80 1 --2- 172.21.15.27:0/4275865611 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4579452d0 0x55f4579494e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.558+0000 7f979fcceb80 1 --2- 172.21.15.27:0/4275865611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f45793f850 0x55f45794bca0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.558+0000 7f979fcceb80 1 --2- 172.21.15.27:0/4275865611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f457940210 0x55f457948fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.558+0000 7f979fcceb80 1 -- 172.21.15.27:0/4275865611 >> 172.21.15.27:0/4275865611 conn(0x55f45786b1f0 msgr2=0x55f457939aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:04.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.558+0000 7f97767fc700 0 client.15039 ms_handle_reset on v2:172.21.15.165:6836/691356139 2024-05-21T03:23:04.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.558+0000 7f979fcceb80 1 -- 172.21.15.27:0/4275865611 shutdown_connections 2024-05-21T03:23:04.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.558+0000 7f979fcceb80 1 -- 172.21.15.27:0/4275865611 wait complete. 2024-05-21T03:23:04.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.558+0000 7f979fcceb80 20 client.15039 trim_cache size 0 max 16384 2024-05-21T03:23:04.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.579+0000 7f979fcceb80 1 Processor -- start 2024-05-21T03:23:04.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.580+0000 7f979fcceb80 1 -- start start 2024-05-21T03:23:04.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.580+0000 7f979fcceb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f45793f850 0x55f45793ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:04.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.580+0000 7f979fcceb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f457940210 0x55f4579440d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.580+0000 7f979fcceb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4579452d0 0x55f457944610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.580+0000 7f979fcceb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f457958690 con 0x55f45793f850 2024-05-21T03:23:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.580+0000 7f979fcceb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f457958d60 con 0x55f4579452d0 2024-05-21T03:23:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.580+0000 7f979fcceb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f457959430 con 0x55f457940210 2024-05-21T03:23:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.580+0000 7f978effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f45793f850 0x55f45793ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.580+0000 7f978effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f45793f850 0x55f45793ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37768/0 (socket says 172.21.15.27:37768) 2024-05-21T03:23:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.580+0000 7f978effd700 1 -- 172.21.15.27:0/1229718848 learned_addr learned my addr 172.21.15.27:0/1229718848 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.580+0000 7f978e7fc700 1 --2- 172.21.15.27:0/1229718848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f457940210 0x55f4579440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.580+0000 7f978effd700 1 -- 172.21.15.27:0/1229718848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f457940210 msgr2=0x55f4579440d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.580+0000 7f978effd700 1 --2- 172.21.15.27:0/1229718848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f457940210 0x55f4579440d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.580+0000 7f978effd700 1 -- 172.21.15.27:0/1229718848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4579452d0 msgr2=0x55f457944610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.580+0000 7f978effd700 1 --2- 172.21.15.27:0/1229718848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4579452d0 0x55f457944610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.580+0000 7f978effd700 1 -- 172.21.15.27:0/1229718848 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f457947510 con 0x55f45793f850 2024-05-21T03:23:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.580+0000 7f978e7fc700 1 --2- 172.21.15.27:0/1229718848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f457940210 0x55f4579440d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:23:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.580+0000 7f978effd700 1 --2- 172.21.15.27:0/1229718848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f45793f850 0x55f45793ae60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9780009710 tx=0x7f9780009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd7ed2932c93a00 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.581+0000 7f9784ff9700 1 -- 172.21.15.27:0/1229718848 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9780022440 con 0x55f45793f850 2024-05-21T03:23:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.581+0000 7f9784ff9700 1 -- 172.21.15.27:0/1229718848 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f978003a070 con 0x55f45793f850 2024-05-21T03:23:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.581+0000 7f9784ff9700 1 -- 172.21.15.27:0/1229718848 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9780021810 con 0x55f45793f850 2024-05-21T03:23:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.581+0000 7f979fcceb80 1 -- 172.21.15.27:0/1229718848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f45793f850 msgr2=0x55f45793ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.581+0000 7f979fcceb80 1 --2- 172.21.15.27:0/1229718848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f45793f850 0x55f45793ae60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9780009710 tx=0x7f9780009d20 comp rx=0 tx=0).stop 2024-05-21T03:23:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.581+0000 7f979fcceb80 1 -- 172.21.15.27:0/1229718848 shutdown_connections 2024-05-21T03:23:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.581+0000 7f979fcceb80 1 --2- 172.21.15.27:0/1229718848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4579452d0 0x55f457944610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.581+0000 7f979fcceb80 1 --2- 172.21.15.27:0/1229718848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f45793f850 0x55f45793ae60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.581+0000 7f979fcceb80 1 --2- 172.21.15.27:0/1229718848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f457940210 0x55f4579440d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.581+0000 7f979fcceb80 1 -- 172.21.15.27:0/1229718848 >> 172.21.15.27:0/1229718848 conn(0x55f45786b1f0 msgr2=0x55f457936710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.581+0000 7f979fcceb80 1 -- 172.21.15.27:0/1229718848 shutdown_connections 2024-05-21T03:23:04.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.582+0000 7f979fcceb80 1 -- 172.21.15.27:0/1229718848 wait complete. 2024-05-21T03:23:04.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.582+0000 7f979fcceb80 1 -- start start 2024-05-21T03:23:04.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.582+0000 7f979fcceb80 1 Processor -- start 2024-05-21T03:23:04.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.582+0000 7f979fcceb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f45793f850 0x55f457961070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:04.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.582+0000 7f979fcceb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f457940210 0x55f4579615b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:04.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.582+0000 7f979fcceb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4579452d0 0x55f457961af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:04.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.582+0000 7f979fcceb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f45795a1d0 con 0x55f457940210 2024-05-21T03:23:04.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.582+0000 7f979fcceb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f457959b00 con 0x55f45793f850 2024-05-21T03:23:04.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.582+0000 7f979fcceb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f45795a8a0 con 0x55f4579452d0 2024-05-21T03:23:04.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.582+0000 7f979fcceb80 10 client.0 _ll_register_callbacks cb 0x55f457828900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:23:04.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.582+0000 7f978e7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f457940210 0x55f4579615b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:04.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.582+0000 7f978e7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f457940210 0x55f4579615b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37778/0 (socket says 172.21.15.27:37778) 2024-05-21T03:23:04.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.582+0000 7f978e7fc700 1 -- 172.21.15.27:0/3284524504 learned_addr learned my addr 172.21.15.27:0/3284524504 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:04.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.583+0000 7f978effd700 1 --2- 172.21.15.27:0/3284524504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f45793f850 0x55f457961070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:04.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.583+0000 7f978f7fe700 1 --2- 172.21.15.27:0/3284524504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4579452d0 0x55f457961af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:04.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.583+0000 7f978e7fc700 1 -- 172.21.15.27:0/3284524504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4579452d0 msgr2=0x55f457961af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:04.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.583+0000 7f978e7fc700 1 --2- 172.21.15.27:0/3284524504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4579452d0 0x55f457961af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.583+0000 7f978e7fc700 1 -- 172.21.15.27:0/3284524504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f45793f850 msgr2=0x55f457961070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:04.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.583+0000 7f978e7fc700 1 --2- 172.21.15.27:0/3284524504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f45793f850 0x55f457961070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.583+0000 7f978e7fc700 1 -- 172.21.15.27:0/3284524504 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f457939ac0 con 0x55f457940210 2024-05-21T03:23:04.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.583+0000 7f978f7fe700 1 --2- 172.21.15.27:0/3284524504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4579452d0 0x55f457961af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:23:04.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.583+0000 7f978e7fc700 1 --2- 172.21.15.27:0/3284524504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f457940210 0x55f4579615b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f97780aab80 tx=0x7f9778022780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07ac7c797bc3c81 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:04.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.583+0000 7f97767fc700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:23:04.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.583+0000 7f97767fc700 1 -- 172.21.15.27:0/3284524504 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97780aa520 con 0x55f457940210 2024-05-21T03:23:04.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.583+0000 7f97767fc700 1 -- 172.21.15.27:0/3284524504 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f9778092040 con 0x55f457940210 2024-05-21T03:23:04.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.583+0000 7f979fcceb80 1 -- 172.21.15.27:0/3284524504 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f978003a070 con 0x55f457940210 2024-05-21T03:23:04.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.583+0000 7f97767fc700 1 -- 172.21.15.27:0/3284524504 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f97780b2e60 con 0x55f457940210 2024-05-21T03:23:04.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.583+0000 7f979fcceb80 10 client.15045 fetch_fsmap learned FSMap version 28 2024-05-21T03:23:04.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.583+0000 7f979fcceb80 1 -- 172.21.15.27:0/3284524504 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f4579c4a50 con 0x55f457940210 2024-05-21T03:23:04.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.583+0000 7f97767fc700 1 -- 172.21.15.27:0/3284524504 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f978003a070 con 0x55f457940210 2024-05-21T03:23:04.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.584+0000 7f97767fc700 1 -- 172.21.15.27:0/3284524504 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f97780a9780 con 0x55f457940210 2024-05-21T03:23:04.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.584+0000 7f979fcceb80 10 client.15045 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T03:23:04.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.584+0000 7f979fcceb80 10 client.15045 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-21T03:23:04.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.584+0000 7f979fcceb80 20 client.15045 populate_metadata read hostname 'smithi027' 2024-05-21T03:23:04.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.584+0000 7f979fcceb80 1 --2- 172.21.15.27:0/3284524504 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x55f4579c9260 0x55f4579c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:04.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.584+0000 7f979fcceb80 4 client.15045 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T03:23:04.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.584+0000 7f979fcceb80 1 -- 172.21.15.27:0/3284524504 --> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] -- command(tid 0: {"prefix": "status"}) -- 0x55f457899450 con 0x55f4579c9260 2024-05-21T03:23:04.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.584+0000 7f978effd700 1 --2- 172.21.15.27:0/3284524504 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x55f4579c9260 0x55f4579c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:04.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.584+0000 7f978effd700 1 --2- 172.21.15.27:0/3284524504 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x55f4579c9260 0x55f4579c7f70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7554979234058872 server_cookie=1834d510084d211b in_seq=0 out_seq=0 2024-05-21T03:23:04.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.585+0000 7f97767fc700 10 client.15045 ms_handle_connect on v2:172.21.15.165:6836/691356139 2024-05-21T03:23:04.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.585+0000 7f97767fc700 1 -- 172.21.15.27:0/3284524504 <== mds.2 v2:172.21.15.165:6836/691356139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f457899450 con 0x55f4579c9260 2024-05-21T03:23:04.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.585+0000 7f97767fc700 10 client.15045 handle_command_reply: tid=0 2024-05-21T03:23:04.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.585+0000 7f979fcceb80 1 client.15045 shutdown 2024-05-21T03:23:04.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.585+0000 7f979fcceb80 1 -- 172.21.15.27:0/3284524504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f457940210 msgr2=0x55f4579615b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:04.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.585+0000 7f979fcceb80 1 --2- 172.21.15.27:0/3284524504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f457940210 0x55f4579615b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f97780aab80 tx=0x7f9778022780 comp rx=0 tx=0).stop 2024-05-21T03:23:04.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.585+0000 7f979fcceb80 1 -- 172.21.15.27:0/3284524504 shutdown_connections 2024-05-21T03:23:04.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.585+0000 7f979fcceb80 1 --2- 172.21.15.27:0/3284524504 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x55f4579c9260 0x55f4579c7f70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.585+0000 7f979fcceb80 1 --2- 172.21.15.27:0/3284524504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4579452d0 0x55f457961af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.585+0000 7f979fcceb80 1 --2- 172.21.15.27:0/3284524504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f45793f850 0x55f457961070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.585+0000 7f979fcceb80 1 --2- 172.21.15.27:0/3284524504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f457940210 0x55f4579615b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.585+0000 7f979fcceb80 1 -- 172.21.15.27:0/3284524504 >> 172.21.15.27:0/3284524504 conn(0x55f45786b1f0 msgr2=0x55f4579440d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:04.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.585+0000 7f97767fc700 0 client.15045 ms_handle_reset on v2:172.21.15.165:6836/691356139 2024-05-21T03:23:04.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.585+0000 7f979fcceb80 1 -- 172.21.15.27:0/3284524504 shutdown_connections 2024-05-21T03:23:04.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.585+0000 7f979fcceb80 1 -- 172.21.15.27:0/3284524504 wait complete. 2024-05-21T03:23:04.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.585+0000 7f979fcceb80 20 client.15045 trim_cache size 0 max 16384 2024-05-21T03:23:04.593 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:23:04.594 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "c9ca94be-171f-11ef-bc9a-c7b262605968", 2024-05-21T03:23:04.594 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 2, 2024-05-21T03:23:04.594 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24457, 2024-05-21T03:23:04.594 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-05-21T03:23:04.594 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-05-21T03:23:04.594 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-05-21T03:23:04.594 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 137.14884447700001, 2024-05-21T03:23:04.594 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 28, 2024-05-21T03:23:04.594 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 82, 2024-05-21T03:23:04.594 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 80, 2024-05-21T03:23:04.594 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 180.24819477700001 2024-05-21T03:23:04.594 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:23:04.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.586+0000 7f9795438700 1 -- 172.21.15.27:0/4291200273 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f977c0b2880 msgr2=0x7f977c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:04.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.586+0000 7f9795438700 1 --2- 172.21.15.27:0/4291200273 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f977c0b2880 0x7f977c0b4d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f978000a400 tx=0x7f978000d040 comp rx=0 tx=0).stop 2024-05-21T03:23:04.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.586+0000 7f9795438700 1 -- 172.21.15.27:0/4291200273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9790102450 msgr2=0x7f97901031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:04.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.586+0000 7f9795438700 1 --2- 172.21.15.27:0/4291200273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9790102450 0x7f97901031b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f977800b580 tx=0x7f97780047b0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.586+0000 7f9795438700 1 -- 172.21.15.27:0/4291200273 shutdown_connections 2024-05-21T03:23:04.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.586+0000 7f9795438700 1 --2- 172.21.15.27:0/4291200273 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f977c0b2880 0x7f977c0b4d40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.586+0000 7f9795438700 1 --2- 172.21.15.27:0/4291200273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9790109610 0x7f97901036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.586+0000 7f9795438700 1 --2- 172.21.15.27:0/4291200273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9790101a90 0x7f9790102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.586+0000 7f9795438700 1 --2- 172.21.15.27:0/4291200273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9790102450 0x7f97901031b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:04.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.586+0000 7f9795438700 1 -- 172.21.15.27:0/4291200273 >> 172.21.15.27:0/4291200273 conn(0x7f9790078a40 msgr2=0x7f9790100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:04.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.586+0000 7f9795438700 1 -- 172.21.15.27:0/4291200273 shutdown_connections 2024-05-21T03:23:04.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:04.586+0000 7f9795438700 1 -- 172.21.15.27:0/4291200273 wait complete. 2024-05-21T03:23:04.597 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:04 smithi027 ceph-mon[32179]: pgmap v367: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:23:04.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:04 smithi039 ceph-mon[36429]: pgmap v367: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:23:04.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:04 smithi165 ceph-mon[36269]: pgmap v367: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:23:05.358 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell mds.i status 2024-05-21T03:23:05.620 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:23:06.644 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:06 smithi027 ceph-mon[32179]: pgmap v368: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:23:06.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:06 smithi039 ceph-mon[36429]: pgmap v368: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:23:06.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:06 smithi165 ceph-mon[36269]: pgmap v368: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:23:07.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.203+0000 7ff1457de700 1 -- 172.21.15.27:0/3228475259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1401088f0 msgr2=0x7ff140110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:07.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.203+0000 7ff1457de700 1 --2- 172.21.15.27:0/3228475259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1401088f0 0x7ff140110470 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff1380099c0 tx=0x7ff138009cd0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.203+0000 7ff1457de700 1 -- 172.21.15.27:0/3228475259 shutdown_connections 2024-05-21T03:23:07.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.203+0000 7ff1457de700 1 --2- 172.21.15.27:0/3228475259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1401088f0 0x7ff140110470 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.203+0000 7ff1457de700 1 --2- 172.21.15.27:0/3228475259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1401037f0 0x7ff140103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.203+0000 7ff1457de700 1 --2- 172.21.15.27:0/3228475259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1401041a0 0x7ff1401081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.203+0000 7ff1457de700 1 -- 172.21.15.27:0/3228475259 >> 172.21.15.27:0/3228475259 conn(0x7ff14007a6a0 msgr2=0x7ff14007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:07.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.203+0000 7ff1457de700 1 -- 172.21.15.27:0/3228475259 shutdown_connections 2024-05-21T03:23:07.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.203+0000 7ff1457de700 1 -- 172.21.15.27:0/3228475259 wait complete. 2024-05-21T03:23:07.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.204+0000 7ff1457de700 1 Processor -- start 2024-05-21T03:23:07.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.204+0000 7ff1457de700 1 -- start start 2024-05-21T03:23:07.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.205+0000 7ff1457de700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1401037f0 0x7ff140075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:07.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.205+0000 7ff1457de700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1401041a0 0x7ff140075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:07.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.205+0000 7ff1457de700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1401088f0 0x7ff1400762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:07.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.205+0000 7ff1457de700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff140113740 con 0x7ff1401041a0 2024-05-21T03:23:07.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.205+0000 7ff1457de700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff1401135c0 con 0x7ff1401037f0 2024-05-21T03:23:07.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.205+0000 7ff1457de700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff1401138c0 con 0x7ff1401088f0 2024-05-21T03:23:07.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.205+0000 7ff13e7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1401041a0 0x7ff140075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:07.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.205+0000 7ff13e7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1401041a0 0x7ff140075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37800/0 (socket says 172.21.15.27:37800) 2024-05-21T03:23:07.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.205+0000 7ff13e7fc700 1 -- 172.21.15.27:0/2561230308 learned_addr learned my addr 172.21.15.27:0/2561230308 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:07.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.205+0000 7ff13f7fe700 1 --2- 172.21.15.27:0/2561230308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1401088f0 0x7ff1400762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:07.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.205+0000 7ff13effd700 1 --2- 172.21.15.27:0/2561230308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1401037f0 0x7ff140075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:07.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.205+0000 7ff13e7fc700 1 -- 172.21.15.27:0/2561230308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1401088f0 msgr2=0x7ff1400762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:07.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.205+0000 7ff13e7fc700 1 --2- 172.21.15.27:0/2561230308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1401088f0 0x7ff1400762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.205+0000 7ff13e7fc700 1 -- 172.21.15.27:0/2561230308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1401037f0 msgr2=0x7ff140075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:07.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.205+0000 7ff13e7fc700 1 --2- 172.21.15.27:0/2561230308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1401037f0 0x7ff140075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.205+0000 7ff13e7fc700 1 -- 172.21.15.27:0/2561230308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1401a80d0 con 0x7ff1401041a0 2024-05-21T03:23:07.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.205+0000 7ff13f7fe700 1 --2- 172.21.15.27:0/2561230308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1401088f0 0x7ff1400762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:23:07.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.205+0000 7ff13e7fc700 1 --2- 172.21.15.27:0/2561230308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1401041a0 0x7ff140075d80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff13400d950 tx=0x7ff13400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:07.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.205+0000 7ff127fff700 1 -- 172.21.15.27:0/2561230308 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff134016070 con 0x7ff1401041a0 2024-05-21T03:23:07.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.205+0000 7ff1457de700 1 -- 172.21.15.27:0/2561230308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1401a83c0 con 0x7ff1401041a0 2024-05-21T03:23:07.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.205+0000 7ff127fff700 1 -- 172.21.15.27:0/2561230308 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff134010460 con 0x7ff1401041a0 2024-05-21T03:23:07.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.205+0000 7ff127fff700 1 -- 172.21.15.27:0/2561230308 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13400f620 con 0x7ff1401041a0 2024-05-21T03:23:07.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.206+0000 7ff1457de700 1 -- 172.21.15.27:0/2561230308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1401a8950 con 0x7ff1401041a0 2024-05-21T03:23:07.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.208+0000 7ff127fff700 1 -- 172.21.15.27:0/2561230308 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff134019070 con 0x7ff1401041a0 2024-05-21T03:23:07.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.209+0000 7ff127fff700 1 --2- 172.21.15.27:0/2561230308 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7ff1280b2880 0x7ff1280b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:07.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.209+0000 7ff13effd700 1 --2- 172.21.15.27:0/2561230308 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7ff1280b2880 0x7ff1280b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:07.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.209+0000 7ff127fff700 1 -- 172.21.15.27:0/2561230308 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7ff1340e1ec0 con 0x7ff1401041a0 2024-05-21T03:23:07.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.209+0000 7ff13effd700 1 --2- 172.21.15.27:0/2561230308 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7ff1280b2880 0x7ff1280b4d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff130009f20 tx=0x7ff1300094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:07.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.210+0000 7ff150074b80 1 Processor -- start 2024-05-21T03:23:07.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.210+0000 7ff150074b80 1 -- start start 2024-05-21T03:23:07.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.210+0000 7ff150074b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64219d810 0x55b64219dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.210+0000 7ff150074b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b64219e1c0 0x55b6421a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.210+0000 7ff150074b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b6421a1f10 0x55b6421a2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.210+0000 7ff150074b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b6420fcc40 con 0x55b6421a1f10 2024-05-21T03:23:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.210+0000 7ff150074b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b6420ec7e0 con 0x55b64219d810 2024-05-21T03:23:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.210+0000 7ff150074b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b6420f0820 con 0x55b64219e1c0 2024-05-21T03:23:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.210+0000 7ff13f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b6421a1f10 0x55b6421a2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.210+0000 7ff13f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b6421a1f10 0x55b6421a2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37814/0 (socket says 172.21.15.27:37814) 2024-05-21T03:23:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.210+0000 7ff13f7fe700 1 -- 172.21.15.27:0/1173811473 learned_addr learned my addr 172.21.15.27:0/1173811473 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.210+0000 7ff13e7fc700 1 --2- 172.21.15.27:0/1173811473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b64219e1c0 0x55b6421a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.210+0000 7ff13effd700 1 --2- 172.21.15.27:0/1173811473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64219d810 0x55b64219dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.211+0000 7ff13f7fe700 1 -- 172.21.15.27:0/1173811473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b64219e1c0 msgr2=0x55b6421a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.211+0000 7ff13f7fe700 1 --2- 172.21.15.27:0/1173811473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b64219e1c0 0x55b6421a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.211+0000 7ff13f7fe700 1 -- 172.21.15.27:0/1173811473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64219d810 msgr2=0x55b64219dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:07.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.211+0000 7ff13f7fe700 1 --2- 172.21.15.27:0/1173811473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64219d810 0x55b64219dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.211+0000 7ff13f7fe700 1 -- 172.21.15.27:0/1173811473 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b64203c880 con 0x55b6421a1f10 2024-05-21T03:23:07.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.211+0000 7ff13f7fe700 1 --2- 172.21.15.27:0/1173811473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b6421a1f10 0x55b6421a2350 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff138006010 tx=0x7ff138009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec6912ae1b25c05 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:07.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.211+0000 7ff1257fa700 1 -- 172.21.15.27:0/1173811473 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff138027070 con 0x55b6421a1f10 2024-05-21T03:23:07.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.211+0000 7ff1257fa700 1 -- 172.21.15.27:0/1173811473 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1380050d0 con 0x55b6421a1f10 2024-05-21T03:23:07.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.211+0000 7ff1257fa700 1 -- 172.21.15.27:0/1173811473 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13800bc10 con 0x55b6421a1f10 2024-05-21T03:23:07.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.212+0000 7ff150074b80 1 -- 172.21.15.27:0/1173811473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b6421a1f10 msgr2=0x55b6421a2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:07.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.212+0000 7ff150074b80 1 --2- 172.21.15.27:0/1173811473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b6421a1f10 0x55b6421a2350 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff138006010 tx=0x7ff138009510 comp rx=0 tx=0).stop 2024-05-21T03:23:07.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.212+0000 7ff150074b80 1 -- 172.21.15.27:0/1173811473 shutdown_connections 2024-05-21T03:23:07.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.212+0000 7ff150074b80 1 --2- 172.21.15.27:0/1173811473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b6421a1f10 0x55b6421a2350 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.212+0000 7ff150074b80 1 --2- 172.21.15.27:0/1173811473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64219d810 0x55b64219dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.212+0000 7ff150074b80 1 --2- 172.21.15.27:0/1173811473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b64219e1c0 0x55b6421a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.212+0000 7ff150074b80 1 -- 172.21.15.27:0/1173811473 >> 172.21.15.27:0/1173811473 conn(0x55b6420cb030 msgr2=0x55b642197e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:07.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.212+0000 7ff150074b80 1 -- 172.21.15.27:0/1173811473 shutdown_connections 2024-05-21T03:23:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.212+0000 7ff150074b80 1 -- 172.21.15.27:0/1173811473 wait complete. 2024-05-21T03:23:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.212+0000 7ff150074b80 1 -- start start 2024-05-21T03:23:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.212+0000 7ff150074b80 1 Processor -- start 2024-05-21T03:23:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.213+0000 7ff150074b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64219d810 0x55b6421a9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.213+0000 7ff150074b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b64219e1c0 0x55b6421a6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.213+0000 7ff150074b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b6421a1f10 0x55b6421a6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.213+0000 7ff150074b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b641ff2f50 con 0x55b6421a1f10 2024-05-21T03:23:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.213+0000 7ff150074b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b64208e770 con 0x55b64219d810 2024-05-21T03:23:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.213+0000 7ff150074b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b6421aa070 con 0x55b64219e1c0 2024-05-21T03:23:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.213+0000 7ff150074b80 10 client.0 _ll_register_callbacks cb 0x55b6420878d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:23:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.213+0000 7ff13f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b6421a1f10 0x55b6421a6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.213+0000 7ff13f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b6421a1f10 0x55b6421a6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37830/0 (socket says 172.21.15.27:37830) 2024-05-21T03:23:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.213+0000 7ff13f7fe700 1 -- 172.21.15.27:0/3739157518 learned_addr learned my addr 172.21.15.27:0/3739157518 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:07.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.213+0000 7ff13effd700 1 --2- 172.21.15.27:0/3739157518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64219d810 0x55b6421a9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:07.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.213+0000 7ff13e7fc700 1 --2- 172.21.15.27:0/3739157518 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b64219e1c0 0x55b6421a6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:07.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.213+0000 7ff13f7fe700 1 -- 172.21.15.27:0/3739157518 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b64219e1c0 msgr2=0x55b6421a6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:07.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.213+0000 7ff13f7fe700 1 --2- 172.21.15.27:0/3739157518 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b64219e1c0 0x55b6421a6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.213+0000 7ff13f7fe700 1 -- 172.21.15.27:0/3739157518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64219d810 msgr2=0x55b6421a9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:07.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.213+0000 7ff13f7fe700 1 --2- 172.21.15.27:0/3739157518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64219d810 0x55b6421a9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.213+0000 7ff13f7fe700 1 -- 172.21.15.27:0/3739157518 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b64208b640 con 0x55b6421a1f10 2024-05-21T03:23:07.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.213+0000 7ff13effd700 1 --2- 172.21.15.27:0/3739157518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64219d810 0x55b6421a9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:23:07.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.213+0000 7ff13f7fe700 1 --2- 172.21.15.27:0/3739157518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b6421a1f10 0x55b6421a6bb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff1380098e0 tx=0x7ff138003860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72acff75acf93b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:07.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.213+0000 7ff122ff5700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:23:07.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.213+0000 7ff122ff5700 1 -- 172.21.15.27:0/3739157518 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff138030060 con 0x55b6421a1f10 2024-05-21T03:23:07.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.214+0000 7ff150074b80 1 -- 172.21.15.27:0/3739157518 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1380050d0 con 0x55b6421a1f10 2024-05-21T03:23:07.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.214+0000 7ff122ff5700 1 -- 172.21.15.27:0/3739157518 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7ff13800bab0 con 0x55b6421a1f10 2024-05-21T03:23:07.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.214+0000 7ff122ff5700 1 -- 172.21.15.27:0/3739157518 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff138027070 con 0x55b6421a1f10 2024-05-21T03:23:07.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.214+0000 7ff150074b80 10 client.15057 fetch_fsmap learned FSMap version 28 2024-05-21T03:23:07.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.214+0000 7ff150074b80 1 -- 172.21.15.27:0/3739157518 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b6421b4780 con 0x55b6421a1f10 2024-05-21T03:23:07.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.214+0000 7ff122ff5700 1 -- 172.21.15.27:0/3739157518 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1380050d0 con 0x55b6421a1f10 2024-05-21T03:23:07.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.214+0000 7ff122ff5700 1 -- 172.21.15.27:0/3739157518 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff138036400 con 0x55b6421a1f10 2024-05-21T03:23:07.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.214+0000 7ff150074b80 10 client.15057 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T03:23:07.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.214+0000 7ff150074b80 10 client.15057 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-21T03:23:07.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.214+0000 7ff150074b80 20 client.15057 populate_metadata read hostname 'smithi027' 2024-05-21T03:23:07.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.214+0000 7ff150074b80 1 --2- 172.21.15.27:0/3739157518 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x55b6421b5ad0 0x55b6421b7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:07.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.214+0000 7ff150074b80 4 client.15057 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:23:07.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.214+0000 7ff150074b80 1 -- 172.21.15.27:0/3739157518 --> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b642169990 con 0x55b6421b5ad0 2024-05-21T03:23:07.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.215+0000 7ff13effd700 1 --2- 172.21.15.27:0/3739157518 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x55b6421b5ad0 0x55b6421b7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:07.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.215+0000 7ff13effd700 1 --2- 172.21.15.27:0/3739157518 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x55b6421b5ad0 0x55b6421b7c30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4568d595359a49dd server_cookie=c09642b30f9436c7 in_seq=0 out_seq=0 2024-05-21T03:23:07.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.215+0000 7ff122ff5700 10 client.15057 ms_handle_connect on v2:172.21.15.165:6838/4158170814 2024-05-21T03:23:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.216+0000 7ff122ff5700 1 -- 172.21.15.27:0/3739157518 <== mds.1 v2:172.21.15.165:6838/4158170814 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b642169990 con 0x55b6421b5ad0 2024-05-21T03:23:07.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.216+0000 7ff122ff5700 10 client.15057 handle_command_reply: tid=0 2024-05-21T03:23:07.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.216+0000 7ff150074b80 1 client.15057 shutdown 2024-05-21T03:23:07.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.217+0000 7ff150074b80 1 -- 172.21.15.27:0/3739157518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b6421a1f10 msgr2=0x55b6421a6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:07.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.217+0000 7ff150074b80 1 --2- 172.21.15.27:0/3739157518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b6421a1f10 0x55b6421a6bb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff1380098e0 tx=0x7ff138003860 comp rx=0 tx=0).stop 2024-05-21T03:23:07.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.217+0000 7ff150074b80 1 -- 172.21.15.27:0/3739157518 shutdown_connections 2024-05-21T03:23:07.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.217+0000 7ff150074b80 1 --2- 172.21.15.27:0/3739157518 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x55b6421b5ad0 0x55b6421b7c30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.217+0000 7ff150074b80 1 --2- 172.21.15.27:0/3739157518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b6421a1f10 0x55b6421a6bb0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.217+0000 7ff150074b80 1 --2- 172.21.15.27:0/3739157518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64219d810 0x55b6421a9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.217+0000 7ff122ff5700 0 client.15057 ms_handle_reset on v2:172.21.15.165:6838/4158170814 2024-05-21T03:23:07.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.217+0000 7ff150074b80 1 --2- 172.21.15.27:0/3739157518 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b64219e1c0 0x55b6421a6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.217+0000 7ff150074b80 1 -- 172.21.15.27:0/3739157518 >> 172.21.15.27:0/3739157518 conn(0x55b6420cb030 msgr2=0x55b6421a3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:07.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.217+0000 7ff150074b80 1 -- 172.21.15.27:0/3739157518 shutdown_connections 2024-05-21T03:23:07.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.217+0000 7ff150074b80 1 -- 172.21.15.27:0/3739157518 wait complete. 2024-05-21T03:23:07.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.217+0000 7ff150074b80 20 client.15057 trim_cache size 0 max 16384 2024-05-21T03:23:07.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.238+0000 7ff150074b80 1 Processor -- start 2024-05-21T03:23:07.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.239+0000 7ff150074b80 1 -- start start 2024-05-21T03:23:07.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.239+0000 7ff150074b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b64219d810 0x55b6421984e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:07.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.239+0000 7ff150074b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64219e1c0 0x55b64219c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:07.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.239+0000 7ff150074b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6421a1f10 0x55b64219cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:07.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.239+0000 7ff150074b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b6421b5f70 con 0x55b64219d810 2024-05-21T03:23:07.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.239+0000 7ff150074b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b6421b6640 con 0x55b64219e1c0 2024-05-21T03:23:07.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.239+0000 7ff150074b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b6421b6d10 con 0x55b6421a1f10 2024-05-21T03:23:07.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.239+0000 7ff13effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b64219d810 0x55b6421984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:07.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.239+0000 7ff13effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b64219d810 0x55b6421984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37840/0 (socket says 172.21.15.27:37840) 2024-05-21T03:23:07.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.239+0000 7ff13effd700 1 -- 172.21.15.27:0/1969671976 learned_addr learned my addr 172.21.15.27:0/1969671976 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:07.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.239+0000 7ff13effd700 1 -- 172.21.15.27:0/1969671976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6421a1f10 msgr2=0x55b64219cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:07.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.239+0000 7ff13effd700 1 --2- 172.21.15.27:0/1969671976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6421a1f10 0x55b64219cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.239+0000 7ff13effd700 1 -- 172.21.15.27:0/1969671976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64219e1c0 msgr2=0x55b64219c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:07.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.239+0000 7ff13e7fc700 1 --2- 172.21.15.27:0/1969671976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64219e1c0 0x55b64219c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:07.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.239+0000 7ff13effd700 1 --2- 172.21.15.27:0/1969671976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64219e1c0 0x55b64219c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.239+0000 7ff13effd700 1 -- 172.21.15.27:0/1969671976 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6421974c0 con 0x55b64219d810 2024-05-21T03:23:07.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.239+0000 7ff13e7fc700 1 --2- 172.21.15.27:0/1969671976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64219e1c0 0x55b64219c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:23:07.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.239+0000 7ff13effd700 1 --2- 172.21.15.27:0/1969671976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b64219d810 0x55b6421984e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff13000cdf0 tx=0x7ff130015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54861a3b8e3164ff server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:07.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.239+0000 7ff124ff9700 1 -- 172.21.15.27:0/1969671976 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff130032070 con 0x55b64219d810 2024-05-21T03:23:07.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.240+0000 7ff124ff9700 1 -- 172.21.15.27:0/1969671976 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff130017cb0 con 0x55b64219d810 2024-05-21T03:23:07.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.240+0000 7ff124ff9700 1 -- 172.21.15.27:0/1969671976 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13002d650 con 0x55b64219d810 2024-05-21T03:23:07.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.240+0000 7ff150074b80 1 -- 172.21.15.27:0/1969671976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b64219d810 msgr2=0x55b6421984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:07.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.240+0000 7ff150074b80 1 --2- 172.21.15.27:0/1969671976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b64219d810 0x55b6421984e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff13000cdf0 tx=0x7ff130015000 comp rx=0 tx=0).stop 2024-05-21T03:23:07.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.240+0000 7ff150074b80 1 -- 172.21.15.27:0/1969671976 shutdown_connections 2024-05-21T03:23:07.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.240+0000 7ff150074b80 1 --2- 172.21.15.27:0/1969671976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6421a1f10 0x55b64219cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.240+0000 7ff150074b80 1 --2- 172.21.15.27:0/1969671976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b64219d810 0x55b6421984e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.240+0000 7ff150074b80 1 --2- 172.21.15.27:0/1969671976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64219e1c0 0x55b64219c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.240+0000 7ff150074b80 1 -- 172.21.15.27:0/1969671976 >> 172.21.15.27:0/1969671976 conn(0x55b6420cb030 msgr2=0x55b642197dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:07.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.240+0000 7ff150074b80 1 -- 172.21.15.27:0/1969671976 shutdown_connections 2024-05-21T03:23:07.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.240+0000 7ff150074b80 1 -- 172.21.15.27:0/1969671976 wait complete. 2024-05-21T03:23:07.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.241+0000 7ff150074b80 1 -- start start 2024-05-21T03:23:07.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.241+0000 7ff150074b80 1 Processor -- start 2024-05-21T03:23:07.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.241+0000 7ff150074b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b64219d810 0x55b6421c1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:07.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.241+0000 7ff150074b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b64219e1c0 0x55b6421c17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:07.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.241+0000 7ff150074b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6421a1f10 0x55b6421bc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:07.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.241+0000 7ff150074b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b6421b7ab0 con 0x55b64219e1c0 2024-05-21T03:23:07.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.241+0000 7ff150074b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b6421b73e0 con 0x55b6421a1f10 2024-05-21T03:23:07.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.241+0000 7ff150074b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b6421b8180 con 0x55b64219d810 2024-05-21T03:23:07.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.241+0000 7ff150074b80 10 client.0 _ll_register_callbacks cb 0x55b6420878d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:23:07.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.241+0000 7ff13e7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b64219e1c0 0x55b6421c17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:07.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.241+0000 7ff13e7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b64219e1c0 0x55b6421c17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37842/0 (socket says 172.21.15.27:37842) 2024-05-21T03:23:07.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.241+0000 7ff13e7fc700 1 -- 172.21.15.27:0/3672042259 learned_addr learned my addr 172.21.15.27:0/3672042259 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:07.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.241+0000 7ff13effd700 1 --2- 172.21.15.27:0/3672042259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b64219d810 0x55b6421c1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:07.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.241+0000 7ff13f7fe700 1 --2- 172.21.15.27:0/3672042259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6421a1f10 0x55b6421bc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:07.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.241+0000 7ff13e7fc700 1 -- 172.21.15.27:0/3672042259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b64219d810 msgr2=0x55b6421c1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:07.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.241+0000 7ff13e7fc700 1 --2- 172.21.15.27:0/3672042259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b64219d810 0x55b6421c1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.241+0000 7ff13e7fc700 1 -- 172.21.15.27:0/3672042259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6421a1f10 msgr2=0x55b6421bc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:07.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.241+0000 7ff13e7fc700 1 --2- 172.21.15.27:0/3672042259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6421a1f10 0x55b6421bc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.241+0000 7ff13e7fc700 1 -- 172.21.15.27:0/3672042259 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6421bc760 con 0x55b64219e1c0 2024-05-21T03:23:07.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.242+0000 7ff13e7fc700 1 --2- 172.21.15.27:0/3672042259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b64219e1c0 0x55b6421c17d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff1340df9a0 tx=0x7ff134010b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d130407328f0e14e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:07.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.242+0000 7ff122ff5700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:23:07.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.242+0000 7ff122ff5700 1 -- 172.21.15.27:0/3672042259 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1340dfe60 con 0x55b64219e1c0 2024-05-21T03:23:07.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.242+0000 7ff122ff5700 1 -- 172.21.15.27:0/3672042259 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7ff1340aa830 con 0x55b64219e1c0 2024-05-21T03:23:07.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.242+0000 7ff150074b80 1 -- 172.21.15.27:0/3672042259 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff130017cb0 con 0x55b64219e1c0 2024-05-21T03:23:07.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.242+0000 7ff122ff5700 1 -- 172.21.15.27:0/3672042259 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff134020040 con 0x55b64219e1c0 2024-05-21T03:23:07.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.242+0000 7ff150074b80 10 client.15063 fetch_fsmap learned FSMap version 28 2024-05-21T03:23:07.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.242+0000 7ff150074b80 1 -- 172.21.15.27:0/3672042259 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b6422228a0 con 0x55b64219e1c0 2024-05-21T03:23:07.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.242+0000 7ff122ff5700 1 -- 172.21.15.27:0/3672042259 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff130017cb0 con 0x55b64219e1c0 2024-05-21T03:23:07.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.242+0000 7ff122ff5700 1 -- 172.21.15.27:0/3672042259 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff1340a9620 con 0x55b64219e1c0 2024-05-21T03:23:07.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.242+0000 7ff150074b80 10 client.15063 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T03:23:07.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.242+0000 7ff150074b80 10 client.15063 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-21T03:23:07.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.242+0000 7ff150074b80 20 client.15063 populate_metadata read hostname 'smithi027' 2024-05-21T03:23:07.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.242+0000 7ff150074b80 1 --2- 172.21.15.27:0/3672042259 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x55b642227410 0x55b642226120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:07.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.242+0000 7ff150074b80 4 client.15063 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T03:23:07.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.242+0000 7ff150074b80 1 -- 172.21.15.27:0/3672042259 --> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] -- command(tid 0: {"prefix": "status"}) -- 0x55b6420feb90 con 0x55b642227410 2024-05-21T03:23:07.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.243+0000 7ff13effd700 1 --2- 172.21.15.27:0/3672042259 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x55b642227410 0x55b642226120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:07.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.243+0000 7ff13effd700 1 --2- 172.21.15.27:0/3672042259 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x55b642227410 0x55b642226120 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=650fbb8843847d38 server_cookie=41d20a800217d31d in_seq=0 out_seq=0 2024-05-21T03:23:07.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.243+0000 7ff122ff5700 10 client.15063 ms_handle_connect on v2:172.21.15.165:6838/4158170814 2024-05-21T03:23:07.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.243+0000 7ff122ff5700 1 -- 172.21.15.27:0/3672042259 <== mds.1 v2:172.21.15.165:6838/4158170814 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b6420feb90 con 0x55b642227410 2024-05-21T03:23:07.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.243+0000 7ff122ff5700 10 client.15063 handle_command_reply: tid=0 2024-05-21T03:23:07.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.243+0000 7ff150074b80 1 client.15063 shutdown 2024-05-21T03:23:07.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.244+0000 7ff150074b80 1 -- 172.21.15.27:0/3672042259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b64219e1c0 msgr2=0x55b6421c17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:07.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.244+0000 7ff150074b80 1 --2- 172.21.15.27:0/3672042259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b64219e1c0 0x55b6421c17d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff1340df9a0 tx=0x7ff134010b60 comp rx=0 tx=0).stop 2024-05-21T03:23:07.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.244+0000 7ff150074b80 1 -- 172.21.15.27:0/3672042259 shutdown_connections 2024-05-21T03:23:07.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.244+0000 7ff150074b80 1 --2- 172.21.15.27:0/3672042259 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x55b642227410 0x55b642226120 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.244+0000 7ff150074b80 1 --2- 172.21.15.27:0/3672042259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6421a1f10 0x55b6421bc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.244+0000 7ff150074b80 1 --2- 172.21.15.27:0/3672042259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b64219d810 0x55b6421c1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.244+0000 7ff150074b80 1 --2- 172.21.15.27:0/3672042259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b64219e1c0 0x55b6421c17d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.244+0000 7ff150074b80 1 -- 172.21.15.27:0/3672042259 >> 172.21.15.27:0/3672042259 conn(0x55b6420cb030 msgr2=0x55b64219c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:07.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.244+0000 7ff122ff5700 0 client.15063 ms_handle_reset on v2:172.21.15.165:6838/4158170814 2024-05-21T03:23:07.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.244+0000 7ff150074b80 1 -- 172.21.15.27:0/3672042259 shutdown_connections 2024-05-21T03:23:07.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.244+0000 7ff150074b80 1 -- 172.21.15.27:0/3672042259 wait complete. 2024-05-21T03:23:07.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.244+0000 7ff150074b80 20 client.15063 trim_cache size 0 max 16384 2024-05-21T03:23:07.250 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:23:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "c9ca94be-171f-11ef-bc9a-c7b262605968", 2024-05-21T03:23:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 1, 2024-05-21T03:23:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24472, 2024-05-21T03:23:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-05-21T03:23:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-05-21T03:23:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-05-21T03:23:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 140.82215657399999, 2024-05-21T03:23:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 28, 2024-05-21T03:23:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 82, 2024-05-21T03:23:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 80, 2024-05-21T03:23:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 177.34381174200001 2024-05-21T03:23:07.251 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:23:07.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.244+0000 7ff1457de700 1 -- 172.21.15.27:0/2561230308 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7ff1280b2880 msgr2=0x7ff1280b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:07.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.244+0000 7ff1457de700 1 --2- 172.21.15.27:0/2561230308 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7ff1280b2880 0x7ff1280b4d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff130009f20 tx=0x7ff1300094f0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.244+0000 7ff1457de700 1 -- 172.21.15.27:0/2561230308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1401041a0 msgr2=0x7ff140075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:07.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.244+0000 7ff1457de700 1 --2- 172.21.15.27:0/2561230308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1401041a0 0x7ff140075d80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff13400d950 tx=0x7ff13400dd10 comp rx=0 tx=0).stop 2024-05-21T03:23:07.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.245+0000 7ff1457de700 1 -- 172.21.15.27:0/2561230308 shutdown_connections 2024-05-21T03:23:07.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.245+0000 7ff1457de700 1 --2- 172.21.15.27:0/2561230308 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7ff1280b2880 0x7ff1280b4d40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.245+0000 7ff1457de700 1 --2- 172.21.15.27:0/2561230308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1401088f0 0x7ff1400762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.245+0000 7ff1457de700 1 --2- 172.21.15.27:0/2561230308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1401037f0 0x7ff140075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.245+0000 7ff1457de700 1 --2- 172.21.15.27:0/2561230308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1401041a0 0x7ff140075d80 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:07.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.245+0000 7ff1457de700 1 -- 172.21.15.27:0/2561230308 >> 172.21.15.27:0/2561230308 conn(0x7ff14007a6a0 msgr2=0x7ff140101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:07.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.245+0000 7ff1457de700 1 -- 172.21.15.27:0/2561230308 shutdown_connections 2024-05-21T03:23:07.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:07.245+0000 7ff1457de700 1 -- 172.21.15.27:0/2561230308 wait complete. 2024-05-21T03:23:07.879 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell mds.b status 2024-05-21T03:23:08.129 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:23:08.538 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:08 smithi027 ceph-mon[32179]: pgmap v369: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:23:08.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:08 smithi039 ceph-mon[36429]: pgmap v369: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:23:08.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:08 smithi165 ceph-mon[36269]: pgmap v369: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:23:09.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.934+0000 7f5119b70700 1 -- 172.21.15.27:0/175825415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5114102440 msgr2=0x7f51141028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:09.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.934+0000 7f5119b70700 1 --2- 172.21.15.27:0/175825415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5114102440 0x7f51141028c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f50fc009990 tx=0x7f50fc009ca0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.934+0000 7f5119b70700 1 -- 172.21.15.27:0/175825415 shutdown_connections 2024-05-21T03:23:09.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.934+0000 7f5119b70700 1 --2- 172.21.15.27:0/175825415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5114102e00 0x7f511410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.934+0000 7f5119b70700 1 --2- 172.21.15.27:0/175825415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5114108520 0x7f5114108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.934+0000 7f5119b70700 1 --2- 172.21.15.27:0/175825415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5114102440 0x7f51141028c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.934+0000 7f5119b70700 1 -- 172.21.15.27:0/175825415 >> 172.21.15.27:0/175825415 conn(0x7f51140793f0 msgr2=0x7f5114079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:09.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.934+0000 7f5119b70700 1 -- 172.21.15.27:0/175825415 shutdown_connections 2024-05-21T03:23:09.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.934+0000 7f5119b70700 1 -- 172.21.15.27:0/175825415 wait complete. 2024-05-21T03:23:09.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.935+0000 7f5119b70700 1 Processor -- start 2024-05-21T03:23:09.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.935+0000 7f5119b70700 1 -- start start 2024-05-21T03:23:09.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.935+0000 7f5119b70700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5114102440 0x7f511419f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:09.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.935+0000 7f5119b70700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5114102e00 0x7f511419fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:09.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.935+0000 7f5119b70700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5114108520 0x7f5114199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:09.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.935+0000 7f5119b70700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f51141125d0 con 0x7f5114102440 2024-05-21T03:23:09.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.935+0000 7f5119b70700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5114112450 con 0x7f5114102e00 2024-05-21T03:23:09.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.935+0000 7f5119b70700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5114112750 con 0x7f5114108520 2024-05-21T03:23:09.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.936+0000 7f51137fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5114102440 0x7f511419f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:09.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.936+0000 7f51137fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5114102440 0x7f511419f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37870/0 (socket says 172.21.15.27:37870) 2024-05-21T03:23:09.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.936+0000 7f51137fe700 1 -- 172.21.15.27:0/3951240604 learned_addr learned my addr 172.21.15.27:0/3951240604 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:09.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.936+0000 7f5112ffd700 1 --2- 172.21.15.27:0/3951240604 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5114102e00 0x7f511419fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:09.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.936+0000 7f5113fff700 1 --2- 172.21.15.27:0/3951240604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5114108520 0x7f5114199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:09.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.936+0000 7f51137fe700 1 -- 172.21.15.27:0/3951240604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5114108520 msgr2=0x7f5114199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:09.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.936+0000 7f51137fe700 1 --2- 172.21.15.27:0/3951240604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5114108520 0x7f5114199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.936+0000 7f51137fe700 1 -- 172.21.15.27:0/3951240604 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5114102e00 msgr2=0x7f511419fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:09.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.936+0000 7f51137fe700 1 --2- 172.21.15.27:0/3951240604 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5114102e00 0x7f511419fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.936+0000 7f51137fe700 1 -- 172.21.15.27:0/3951240604 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f511419a220 con 0x7f5114102440 2024-05-21T03:23:09.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.936+0000 7f51137fe700 1 --2- 172.21.15.27:0/3951240604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5114102440 0x7f511419f900 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f51040060b0 tx=0x7f510400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:09.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.936+0000 7f5110ff9700 1 -- 172.21.15.27:0/3951240604 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510401a070 con 0x7f5114102440 2024-05-21T03:23:09.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.936+0000 7f5119b70700 1 -- 172.21.15.27:0/3951240604 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f511419a510 con 0x7f5114102440 2024-05-21T03:23:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.936+0000 7f5110ff9700 1 -- 172.21.15.27:0/3951240604 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f510400ddd0 con 0x7f5114102440 2024-05-21T03:23:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.936+0000 7f5110ff9700 1 -- 172.21.15.27:0/3951240604 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5104014720 con 0x7f5114102440 2024-05-21T03:23:09.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.937+0000 7f5119b70700 1 -- 172.21.15.27:0/3951240604 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51141a67d0 con 0x7f5114102440 2024-05-21T03:23:09.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.939+0000 7f5110ff9700 1 -- 172.21.15.27:0/3951240604 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f510401d070 con 0x7f5114102440 2024-05-21T03:23:09.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.940+0000 7f5110ff9700 1 --2- 172.21.15.27:0/3951240604 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f51000b2880 0x7f51000b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:09.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.940+0000 7f5112ffd700 1 --2- 172.21.15.27:0/3951240604 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f51000b2880 0x7f51000b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:09.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.940+0000 7f5110ff9700 1 -- 172.21.15.27:0/3951240604 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f51040e1a40 con 0x7f5114102440 2024-05-21T03:23:09.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.940+0000 7f5112ffd700 1 --2- 172.21.15.27:0/3951240604 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f51000b2880 0x7f51000b4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00b5c0 tx=0x7f50fc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:09.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.941+0000 7f5124406b80 1 Processor -- start 2024-05-21T03:23:09.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.941+0000 7f5124406b80 1 -- start start 2024-05-21T03:23:09.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.941+0000 7f5124406b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605db461810 0x5605db461bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:09.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.941+0000 7f5124406b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605db4621c0 0x5605db4658f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:09.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.941+0000 7f5124406b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605db465f10 0x5605db466350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:09.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.941+0000 7f5124406b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5605db3c0c40 con 0x5605db465f10 2024-05-21T03:23:09.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.941+0000 7f5124406b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5605db3b07e0 con 0x5605db4621c0 2024-05-21T03:23:09.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.941+0000 7f5124406b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5605db3b4820 con 0x5605db461810 2024-05-21T03:23:09.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.941+0000 7f5113fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605db465f10 0x5605db466350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:09.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.941+0000 7f5113fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605db465f10 0x5605db466350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37884/0 (socket says 172.21.15.27:37884) 2024-05-21T03:23:09.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.941+0000 7f5113fff700 1 -- 172.21.15.27:0/2116739898 learned_addr learned my addr 172.21.15.27:0/2116739898 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:09.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.941+0000 7f51137fe700 1 --2- 172.21.15.27:0/2116739898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605db461810 0x5605db461bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:09.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.941+0000 7f5112ffd700 1 --2- 172.21.15.27:0/2116739898 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605db4621c0 0x5605db4658f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:09.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.941+0000 7f5113fff700 1 -- 172.21.15.27:0/2116739898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605db461810 msgr2=0x5605db461bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:09.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.941+0000 7f5113fff700 1 --2- 172.21.15.27:0/2116739898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605db461810 0x5605db461bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.941+0000 7f5113fff700 1 -- 172.21.15.27:0/2116739898 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605db4621c0 msgr2=0x5605db4658f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:09.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.941+0000 7f5113fff700 1 --2- 172.21.15.27:0/2116739898 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605db4621c0 0x5605db4658f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.941+0000 7f5113fff700 1 -- 172.21.15.27:0/2116739898 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605db300880 con 0x5605db465f10 2024-05-21T03:23:09.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.941+0000 7f5112ffd700 1 --2- 172.21.15.27:0/2116739898 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605db4621c0 0x5605db4658f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:23:09.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.941+0000 7f51137fe700 1 --2- 172.21.15.27:0/2116739898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605db461810 0x5605db461bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:23:09.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.942+0000 7f5113fff700 1 --2- 172.21.15.27:0/2116739898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605db465f10 0x5605db466350 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f510c00cc70 tx=0x7f510c00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be793e811a3262c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:09.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.942+0000 7f5109ffb700 1 -- 172.21.15.27:0/2116739898 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c014070 con 0x5605db465f10 2024-05-21T03:23:09.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.942+0000 7f5109ffb700 1 -- 172.21.15.27:0/2116739898 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f510c00ee20 con 0x5605db465f10 2024-05-21T03:23:09.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.942+0000 7f5109ffb700 1 -- 172.21.15.27:0/2116739898 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c010430 con 0x5605db465f10 2024-05-21T03:23:09.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.942+0000 7f5124406b80 1 -- 172.21.15.27:0/2116739898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605db465f10 msgr2=0x5605db466350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:09.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.942+0000 7f5124406b80 1 --2- 172.21.15.27:0/2116739898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605db465f10 0x5605db466350 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f510c00cc70 tx=0x7f510c00cf80 comp rx=0 tx=0).stop 2024-05-21T03:23:09.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.942+0000 7f5124406b80 1 -- 172.21.15.27:0/2116739898 shutdown_connections 2024-05-21T03:23:09.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.942+0000 7f5124406b80 1 --2- 172.21.15.27:0/2116739898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605db465f10 0x5605db466350 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.943+0000 7f5124406b80 1 --2- 172.21.15.27:0/2116739898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605db461810 0x5605db461bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.943+0000 7f5124406b80 1 --2- 172.21.15.27:0/2116739898 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605db4621c0 0x5605db4658f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.943+0000 7f5124406b80 1 -- 172.21.15.27:0/2116739898 >> 172.21.15.27:0/2116739898 conn(0x5605db38f030 msgr2=0x5605db45be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:09.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.943+0000 7f5124406b80 1 -- 172.21.15.27:0/2116739898 shutdown_connections 2024-05-21T03:23:09.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.943+0000 7f5124406b80 1 -- 172.21.15.27:0/2116739898 wait complete. 2024-05-21T03:23:09.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.943+0000 7f5124406b80 1 -- start start 2024-05-21T03:23:09.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.943+0000 7f5124406b80 1 Processor -- start 2024-05-21T03:23:09.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.943+0000 7f5124406b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605db461810 0x5605db46d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:09.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.943+0000 7f5124406b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605db4621c0 0x5605db46a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:09.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.943+0000 7f5124406b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605db465f10 0x5605db46aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:09.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.943+0000 7f5124406b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5605db2b6f50 con 0x5605db461810 2024-05-21T03:23:09.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.943+0000 7f5124406b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5605db352770 con 0x5605db4621c0 2024-05-21T03:23:09.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.943+0000 7f5124406b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5605db46e0d0 con 0x5605db465f10 2024-05-21T03:23:09.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.944+0000 7f5124406b80 10 client.0 _ll_register_callbacks cb 0x5605db34b8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:23:09.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.944+0000 7f51137fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605db461810 0x5605db46d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:09.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.944+0000 7f51137fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605db461810 0x5605db46d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37886/0 (socket says 172.21.15.27:37886) 2024-05-21T03:23:09.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.944+0000 7f51137fe700 1 -- 172.21.15.27:0/734728276 learned_addr learned my addr 172.21.15.27:0/734728276 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:09.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.944+0000 7f5113fff700 1 --2- 172.21.15.27:0/734728276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605db465f10 0x5605db46aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:09.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.944+0000 7f5112ffd700 1 --2- 172.21.15.27:0/734728276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605db4621c0 0x5605db46a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:09.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.944+0000 7f51137fe700 1 -- 172.21.15.27:0/734728276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605db465f10 msgr2=0x5605db46aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:09.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.944+0000 7f51137fe700 1 --2- 172.21.15.27:0/734728276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605db465f10 0x5605db46aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.944+0000 7f51137fe700 1 -- 172.21.15.27:0/734728276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605db4621c0 msgr2=0x5605db46a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:09.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.944+0000 7f51137fe700 1 --2- 172.21.15.27:0/734728276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605db4621c0 0x5605db46a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.944+0000 7f51137fe700 1 -- 172.21.15.27:0/734728276 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605db34f640 con 0x5605db461810 2024-05-21T03:23:09.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.944+0000 7f51137fe700 1 --2- 172.21.15.27:0/734728276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605db461810 0x5605db46d360 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f51040df890 tx=0x7f5104015c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e32d127bbba5b61 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:09.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.944+0000 7f50f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:23:09.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.944+0000 7f50f2ffd700 1 -- 172.21.15.27:0/734728276 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51040a9040 con 0x5605db461810 2024-05-21T03:23:09.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.944+0000 7f50f2ffd700 1 -- 172.21.15.27:0/734728276 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f51040a5630 con 0x5605db461810 2024-05-21T03:23:09.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.944+0000 7f5124406b80 1 -- 172.21.15.27:0/734728276 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f510c00ee20 con 0x5605db461810 2024-05-21T03:23:09.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.944+0000 7f50f2ffd700 1 -- 172.21.15.27:0/734728276 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f51040ca050 con 0x5605db461810 2024-05-21T03:23:09.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.944+0000 7f5124406b80 10 client.15075 fetch_fsmap learned FSMap version 28 2024-05-21T03:23:09.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.944+0000 7f5124406b80 1 -- 172.21.15.27:0/734728276 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5605db478830 con 0x5605db461810 2024-05-21T03:23:09.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.945+0000 7f50f2ffd700 1 -- 172.21.15.27:0/734728276 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f510c00ee20 con 0x5605db461810 2024-05-21T03:23:09.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.945+0000 7f50f2ffd700 1 -- 172.21.15.27:0/734728276 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f51040cb050 con 0x5605db461810 2024-05-21T03:23:09.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.945+0000 7f5124406b80 10 client.15075 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T03:23:09.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.945+0000 7f5124406b80 10 client.15075 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-21T03:23:09.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.945+0000 7f5124406b80 20 client.15075 populate_metadata read hostname 'smithi027' 2024-05-21T03:23:09.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.945+0000 7f5124406b80 1 --2- 172.21.15.27:0/734728276 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x5605db47ad80 0x5605db47c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:09.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.945+0000 7f5124406b80 4 client.15075 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:23:09.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.945+0000 7f5124406b80 1 -- 172.21.15.27:0/734728276 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605db42d990 con 0x5605db47ad80 2024-05-21T03:23:09.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.945+0000 7f5112ffd700 1 --2- 172.21.15.27:0/734728276 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x5605db47ad80 0x5605db47c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:09.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.946+0000 7f5112ffd700 1 --2- 172.21.15.27:0/734728276 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x5605db47ad80 0x5605db47c1f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d98d2b2c1fee0c1 server_cookie=683008f0b2775a22 in_seq=0 out_seq=0 2024-05-21T03:23:09.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.946+0000 7f50f2ffd700 10 client.15075 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:23:09.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.946+0000 7f50f2ffd700 1 -- 172.21.15.27:0/734728276 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5605db42d990 con 0x5605db47ad80 2024-05-21T03:23:09.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.946+0000 7f50f2ffd700 10 client.15075 handle_command_reply: tid=0 2024-05-21T03:23:09.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.947+0000 7f5124406b80 1 client.15075 shutdown 2024-05-21T03:23:09.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.947+0000 7f5124406b80 1 -- 172.21.15.27:0/734728276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605db461810 msgr2=0x5605db46d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:09.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.947+0000 7f5124406b80 1 --2- 172.21.15.27:0/734728276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605db461810 0x5605db46d360 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f51040df890 tx=0x7f5104015c70 comp rx=0 tx=0).stop 2024-05-21T03:23:09.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.947+0000 7f5124406b80 1 -- 172.21.15.27:0/734728276 shutdown_connections 2024-05-21T03:23:09.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.947+0000 7f5124406b80 1 --2- 172.21.15.27:0/734728276 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x5605db47ad80 0x5605db47c1f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.947+0000 7f5124406b80 1 --2- 172.21.15.27:0/734728276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605db465f10 0x5605db46aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.947+0000 7f5124406b80 1 --2- 172.21.15.27:0/734728276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605db461810 0x5605db46d360 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.947+0000 7f5124406b80 1 --2- 172.21.15.27:0/734728276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605db4621c0 0x5605db46a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.947+0000 7f5124406b80 1 -- 172.21.15.27:0/734728276 >> 172.21.15.27:0/734728276 conn(0x5605db38f030 msgr2=0x5605db45b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:09.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.947+0000 7f50f2ffd700 0 client.15075 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:23:09.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.947+0000 7f5124406b80 1 -- 172.21.15.27:0/734728276 shutdown_connections 2024-05-21T03:23:09.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.947+0000 7f5124406b80 1 -- 172.21.15.27:0/734728276 wait complete. 2024-05-21T03:23:09.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.947+0000 7f5124406b80 20 client.15075 trim_cache size 0 max 16384 2024-05-21T03:23:09.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.968+0000 7f5124406b80 1 Processor -- start 2024-05-21T03:23:09.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.969+0000 7f5124406b80 1 -- start start 2024-05-21T03:23:09.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.969+0000 7f5124406b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605db461810 0x5605db458740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:09.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.969+0000 7f5124406b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605db4621c0 0x5605db464f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:09.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.969+0000 7f5124406b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605db465f10 0x5605db45bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:09.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.969+0000 7f5124406b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5605db47a530 con 0x5605db465f10 2024-05-21T03:23:09.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.969+0000 7f5124406b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5605db47ac00 con 0x5605db461810 2024-05-21T03:23:09.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.969+0000 7f5124406b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5605db47b2d0 con 0x5605db4621c0 2024-05-21T03:23:09.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.969+0000 7f5113fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605db465f10 0x5605db45bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:09.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.969+0000 7f5113fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605db465f10 0x5605db45bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37902/0 (socket says 172.21.15.27:37902) 2024-05-21T03:23:09.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.969+0000 7f5113fff700 1 -- 172.21.15.27:0/2580353343 learned_addr learned my addr 172.21.15.27:0/2580353343 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:09.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.969+0000 7f51137fe700 1 --2- 172.21.15.27:0/2580353343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605db461810 0x5605db458740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:09.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.969+0000 7f5112ffd700 1 --2- 172.21.15.27:0/2580353343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605db4621c0 0x5605db464f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:09.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.969+0000 7f5113fff700 1 -- 172.21.15.27:0/2580353343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605db4621c0 msgr2=0x5605db464f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:09.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.969+0000 7f5113fff700 1 --2- 172.21.15.27:0/2580353343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605db4621c0 0x5605db464f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.969+0000 7f5113fff700 1 -- 172.21.15.27:0/2580353343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605db461810 msgr2=0x5605db458740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:09.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.969+0000 7f5113fff700 1 --2- 172.21.15.27:0/2580353343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605db461810 0x5605db458740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.969+0000 7f5113fff700 1 -- 172.21.15.27:0/2580353343 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605db4695c0 con 0x5605db465f10 2024-05-21T03:23:09.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.969+0000 7f5113fff700 1 --2- 172.21.15.27:0/2580353343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605db465f10 0x5605db45bc40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f510c005910 tx=0x7f510c0054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f662068f61bcd45 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:09.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.970+0000 7f51097fa700 1 -- 172.21.15.27:0/2580353343 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c027070 con 0x5605db465f10 2024-05-21T03:23:09.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.970+0000 7f51097fa700 1 -- 172.21.15.27:0/2580353343 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f510c00ec20 con 0x5605db465f10 2024-05-21T03:23:09.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.970+0000 7f51097fa700 1 -- 172.21.15.27:0/2580353343 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c014040 con 0x5605db465f10 2024-05-21T03:23:09.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.970+0000 7f5124406b80 1 -- 172.21.15.27:0/2580353343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605db465f10 msgr2=0x5605db45bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:09.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.970+0000 7f5124406b80 1 --2- 172.21.15.27:0/2580353343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605db465f10 0x5605db45bc40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f510c005910 tx=0x7f510c0054a0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.970+0000 7f5124406b80 1 -- 172.21.15.27:0/2580353343 shutdown_connections 2024-05-21T03:23:09.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.970+0000 7f5124406b80 1 --2- 172.21.15.27:0/2580353343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605db465f10 0x5605db45bc40 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.970+0000 7f5124406b80 1 --2- 172.21.15.27:0/2580353343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605db461810 0x5605db458740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.970+0000 7f5124406b80 1 --2- 172.21.15.27:0/2580353343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605db4621c0 0x5605db464f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.970+0000 7f5124406b80 1 -- 172.21.15.27:0/2580353343 >> 172.21.15.27:0/2580353343 conn(0x5605db38f030 msgr2=0x5605db464860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:09.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.970+0000 7f5124406b80 1 -- 172.21.15.27:0/2580353343 shutdown_connections 2024-05-21T03:23:09.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.970+0000 7f5124406b80 1 -- 172.21.15.27:0/2580353343 wait complete. 2024-05-21T03:23:09.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.971+0000 7f5124406b80 1 -- start start 2024-05-21T03:23:09.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.971+0000 7f5124406b80 1 Processor -- start 2024-05-21T03:23:09.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.971+0000 7f5124406b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605db461810 0x5605db481bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:09.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.971+0000 7f5124406b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605db4621c0 0x5605db482130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:09.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.971+0000 7f5124406b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605db465f10 0x5605db4df550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:09.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.971+0000 7f5124406b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5605db47c070 con 0x5605db461810 2024-05-21T03:23:09.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.971+0000 7f5124406b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5605db47b9a0 con 0x5605db4621c0 2024-05-21T03:23:09.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.971+0000 7f5124406b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5605db47c740 con 0x5605db465f10 2024-05-21T03:23:09.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.971+0000 7f5124406b80 10 client.0 _ll_register_callbacks cb 0x5605db34b8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:23:09.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.971+0000 7f51137fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605db461810 0x5605db481bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:09.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.971+0000 7f51137fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605db461810 0x5605db481bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37918/0 (socket says 172.21.15.27:37918) 2024-05-21T03:23:09.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.971+0000 7f51137fe700 1 -- 172.21.15.27:0/332142466 learned_addr learned my addr 172.21.15.27:0/332142466 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:09.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.971+0000 7f5113fff700 1 --2- 172.21.15.27:0/332142466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605db465f10 0x5605db4df550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:09.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.972+0000 7f5112ffd700 1 --2- 172.21.15.27:0/332142466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605db4621c0 0x5605db482130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:09.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.972+0000 7f51137fe700 1 -- 172.21.15.27:0/332142466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605db465f10 msgr2=0x5605db4df550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:09.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.972+0000 7f51137fe700 1 --2- 172.21.15.27:0/332142466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605db465f10 0x5605db4df550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.972+0000 7f51137fe700 1 -- 172.21.15.27:0/332142466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605db4621c0 msgr2=0x5605db482130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:09.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.972+0000 7f51137fe700 1 --2- 172.21.15.27:0/332142466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605db4621c0 0x5605db482130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.972+0000 7f51137fe700 1 -- 172.21.15.27:0/332142466 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605db4dfb10 con 0x5605db461810 2024-05-21T03:23:09.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.972+0000 7f51137fe700 1 --2- 172.21.15.27:0/332142466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605db461810 0x5605db481bf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f51040a50c0 tx=0x7f51040a84d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9074ee4a97bf60c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:09.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.972+0000 7f50f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:23:09.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.972+0000 7f50f2ffd700 1 -- 172.21.15.27:0/332142466 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5104087060 con 0x5605db461810 2024-05-21T03:23:09.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.972+0000 7f50f2ffd700 1 -- 172.21.15.27:0/332142466 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f5104004ad0 con 0x5605db461810 2024-05-21T03:23:09.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.972+0000 7f5124406b80 1 -- 172.21.15.27:0/332142466 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f510c00ec20 con 0x5605db461810 2024-05-21T03:23:09.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.972+0000 7f50f2ffd700 1 -- 172.21.15.27:0/332142466 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f51040a9040 con 0x5605db461810 2024-05-21T03:23:09.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.972+0000 7f5124406b80 10 client.15081 fetch_fsmap learned FSMap version 28 2024-05-21T03:23:09.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.972+0000 7f5124406b80 1 -- 172.21.15.27:0/332142466 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5605db4e6ec0 con 0x5605db461810 2024-05-21T03:23:09.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.972+0000 7f50f2ffd700 1 -- 172.21.15.27:0/332142466 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f510c00ec20 con 0x5605db461810 2024-05-21T03:23:09.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.972+0000 7f50f2ffd700 1 -- 172.21.15.27:0/332142466 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f51040cb040 con 0x5605db461810 2024-05-21T03:23:09.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.972+0000 7f5124406b80 10 client.15081 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T03:23:09.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.973+0000 7f5124406b80 10 client.15081 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-21T03:23:09.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.973+0000 7f5124406b80 20 client.15081 populate_metadata read hostname 'smithi027' 2024-05-21T03:23:09.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.973+0000 7f5124406b80 1 --2- 172.21.15.27:0/332142466 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x5605db4eba30 0x5605db4ea740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:09.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.973+0000 7f5124406b80 4 client.15081 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T03:23:09.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.973+0000 7f5124406b80 1 -- 172.21.15.27:0/332142466 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "status"}) -- 0x5605db3c2b90 con 0x5605db4eba30 2024-05-21T03:23:09.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.973+0000 7f5112ffd700 1 --2- 172.21.15.27:0/332142466 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x5605db4eba30 0x5605db4ea740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:09.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.973+0000 7f5112ffd700 1 --2- 172.21.15.27:0/332142466 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x5605db4eba30 0x5605db4ea740 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b817b10b568aa60 server_cookie=eb296208a724734f in_seq=0 out_seq=0 2024-05-21T03:23:09.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.973+0000 7f50f2ffd700 10 client.15081 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:23:09.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.974+0000 7f50f2ffd700 1 -- 172.21.15.27:0/332142466 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5605db3c2b90 con 0x5605db4eba30 2024-05-21T03:23:09.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.974+0000 7f50f2ffd700 10 client.15081 handle_command_reply: tid=0 2024-05-21T03:23:09.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.974+0000 7f5124406b80 1 client.15081 shutdown 2024-05-21T03:23:09.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.974+0000 7f5124406b80 1 -- 172.21.15.27:0/332142466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605db461810 msgr2=0x5605db481bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:09.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.974+0000 7f5124406b80 1 --2- 172.21.15.27:0/332142466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605db461810 0x5605db481bf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f51040a50c0 tx=0x7f51040a84d0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.974+0000 7f5124406b80 1 -- 172.21.15.27:0/332142466 shutdown_connections 2024-05-21T03:23:09.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.974+0000 7f5124406b80 1 --2- 172.21.15.27:0/332142466 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x5605db4eba30 0x5605db4ea740 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.974+0000 7f5124406b80 1 --2- 172.21.15.27:0/332142466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605db465f10 0x5605db4df550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.974+0000 7f5124406b80 1 --2- 172.21.15.27:0/332142466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5605db461810 0x5605db481bf0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.974+0000 7f5124406b80 1 --2- 172.21.15.27:0/332142466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605db4621c0 0x5605db482130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.974+0000 7f5124406b80 1 -- 172.21.15.27:0/332142466 >> 172.21.15.27:0/332142466 conn(0x5605db38f030 msgr2=0x5605db4650f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:09.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.974+0000 7f50f2ffd700 0 client.15081 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:23:09.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.974+0000 7f5124406b80 1 -- 172.21.15.27:0/332142466 shutdown_connections 2024-05-21T03:23:09.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.974+0000 7f5124406b80 1 -- 172.21.15.27:0/332142466 wait complete. 2024-05-21T03:23:09.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.974+0000 7f5124406b80 20 client.15081 trim_cache size 0 max 16384 2024-05-21T03:23:09.983 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:23:09.983 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "c9ca94be-171f-11ef-bc9a-c7b262605968", 2024-05-21T03:23:09.983 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-05-21T03:23:09.983 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24476, 2024-05-21T03:23:09.983 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-05-21T03:23:09.983 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-05-21T03:23:09.984 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-05-21T03:23:09.984 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 155.71148744600001, 2024-05-21T03:23:09.984 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 28, 2024-05-21T03:23:09.984 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 82, 2024-05-21T03:23:09.984 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 80, 2024-05-21T03:23:09.984 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 169.739344536 2024-05-21T03:23:09.984 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:23:09.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.975+0000 7f5119b70700 1 -- 172.21.15.27:0/3951240604 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f51000b2880 msgr2=0x7f51000b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:09.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.975+0000 7f5119b70700 1 --2- 172.21.15.27:0/3951240604 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f51000b2880 0x7f51000b4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00b5c0 tx=0x7f50fc005960 comp rx=0 tx=0).stop 2024-05-21T03:23:09.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.975+0000 7f5119b70700 1 -- 172.21.15.27:0/3951240604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5114102440 msgr2=0x7f511419f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:09.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.975+0000 7f5119b70700 1 --2- 172.21.15.27:0/3951240604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5114102440 0x7f511419f900 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f51040060b0 tx=0x7f510400d3b0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.975+0000 7f5119b70700 1 -- 172.21.15.27:0/3951240604 shutdown_connections 2024-05-21T03:23:09.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.975+0000 7f5119b70700 1 --2- 172.21.15.27:0/3951240604 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f51000b2880 0x7f51000b4d40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.975+0000 7f5119b70700 1 --2- 172.21.15.27:0/3951240604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5114108520 0x7f5114199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.975+0000 7f5119b70700 1 --2- 172.21.15.27:0/3951240604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5114102440 0x7f511419f900 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.975+0000 7f5119b70700 1 --2- 172.21.15.27:0/3951240604 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5114102e00 0x7f511419fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:09.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.975+0000 7f5119b70700 1 -- 172.21.15.27:0/3951240604 >> 172.21.15.27:0/3951240604 conn(0x7f51140793f0 msgr2=0x7f511406b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:09.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.975+0000 7f5119b70700 1 -- 172.21.15.27:0/3951240604 shutdown_connections 2024-05-21T03:23:09.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:09.975+0000 7f5119b70700 1 -- 172.21.15.27:0/3951240604 wait complete. 2024-05-21T03:23:10.756 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:10 smithi027 ceph-mon[32179]: pgmap v370: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:23:10.758 INFO:tasks.cephadm:Setup complete, yielding 2024-05-21T03:23:10.758 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-21T03:23:10.769 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-05-21T03:23:10.769 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch status' 2024-05-21T03:23:10.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:10 smithi039 ceph-mon[36429]: pgmap v370: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:23:10.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:10 smithi165 ceph-mon[36269]: pgmap v370: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:23:11.021 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:23:12.744 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:12 smithi027 ceph-mon[32179]: pgmap v371: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:23:12.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.762+0000 7f012f7b8700 1 -- 172.21.15.27:0/138266132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0128077510 msgr2=0x7f0128075970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:12.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.762+0000 7f012f7b8700 1 --2- 172.21.15.27:0/138266132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0128077510 0x7f0128075970 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0124009990 tx=0x7f0124009ca0 comp rx=0 tx=0).stop 2024-05-21T03:23:12.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.762+0000 7f012f7b8700 1 -- 172.21.15.27:0/138266132 shutdown_connections 2024-05-21T03:23:12.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.762+0000 7f012f7b8700 1 --2- 172.21.15.27:0/138266132 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f012810b480 0x7f012810d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:12.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.762+0000 7f012f7b8700 1 --2- 172.21.15.27:0/138266132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0128077510 0x7f0128075970 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:12.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.762+0000 7f012f7b8700 1 --2- 172.21.15.27:0/138266132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0128075f40 0x7f01280763c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:12.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.762+0000 7f012f7b8700 1 -- 172.21.15.27:0/138266132 >> 172.21.15.27:0/138266132 conn(0x7f01280feb40 msgr2=0x7f0128100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:12.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.762+0000 7f012f7b8700 1 -- 172.21.15.27:0/138266132 shutdown_connections 2024-05-21T03:23:12.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.762+0000 7f012f7b8700 1 -- 172.21.15.27:0/138266132 wait complete. 2024-05-21T03:23:12.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.762+0000 7f012f7b8700 1 Processor -- start 2024-05-21T03:23:12.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.762+0000 7f012f7b8700 1 -- start start 2024-05-21T03:23:12.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.763+0000 7f012f7b8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0128075f40 0x7f012819b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:12.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.763+0000 7f012f7b8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0128077510 0x7f012819be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:12.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.763+0000 7f012f7b8700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f012810b480 0x7f01281a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:12.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.763+0000 7f012f7b8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0128111210 con 0x7f0128077510 2024-05-21T03:23:12.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.763+0000 7f012f7b8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0128111090 con 0x7f0128075f40 2024-05-21T03:23:12.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.763+0000 7f012f7b8700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0128111390 con 0x7f012810b480 2024-05-21T03:23:12.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.763+0000 7f012cd53700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0128077510 0x7f012819be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:12.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.763+0000 7f012cd53700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0128077510 0x7f012819be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35698/0 (socket says 172.21.15.27:35698) 2024-05-21T03:23:12.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.763+0000 7f012cd53700 1 -- 172.21.15.27:0/1422638710 learned_addr learned my addr 172.21.15.27:0/1422638710 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:12.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.763+0000 7f012d554700 1 --2- 172.21.15.27:0/1422638710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0128075f40 0x7f012819b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:12.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.763+0000 7f012dd55700 1 --2- 172.21.15.27:0/1422638710 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f012810b480 0x7f01281a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:12.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.763+0000 7f012cd53700 1 -- 172.21.15.27:0/1422638710 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f012810b480 msgr2=0x7f01281a2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:12.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.763+0000 7f012cd53700 1 --2- 172.21.15.27:0/1422638710 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f012810b480 0x7f01281a2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:12.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.763+0000 7f012cd53700 1 -- 172.21.15.27:0/1422638710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0128075f40 msgr2=0x7f012819b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:12.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.763+0000 7f012cd53700 1 --2- 172.21.15.27:0/1422638710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0128075f40 0x7f012819b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:12.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.763+0000 7f012cd53700 1 -- 172.21.15.27:0/1422638710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01281a3520 con 0x7f0128077510 2024-05-21T03:23:12.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.763+0000 7f012dd55700 1 --2- 172.21.15.27:0/1422638710 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f012810b480 0x7f01281a2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:23:12.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.763+0000 7f012cd53700 1 --2- 172.21.15.27:0/1422638710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0128077510 0x7f012819be50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f011800d980 tx=0x7f011800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:12.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.763+0000 7f011e7fc700 1 -- 172.21.15.27:0/1422638710 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0118016070 con 0x7f0128077510 2024-05-21T03:23:12.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.763+0000 7f012f7b8700 1 -- 172.21.15.27:0/1422638710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01281a3810 con 0x7f0128077510 2024-05-21T03:23:12.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.763+0000 7f011e7fc700 1 -- 172.21.15.27:0/1422638710 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f011800de70 con 0x7f0128077510 2024-05-21T03:23:12.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.763+0000 7f011e7fc700 1 -- 172.21.15.27:0/1422638710 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011800f3c0 con 0x7f0128077510 2024-05-21T03:23:12.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.764+0000 7f012f7b8700 1 -- 172.21.15.27:0/1422638710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01281a3d50 con 0x7f0128077510 2024-05-21T03:23:12.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.764+0000 7f012f7b8700 1 -- 172.21.15.27:0/1422638710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f012806a230 con 0x7f0128077510 2024-05-21T03:23:12.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.766+0000 7f011e7fc700 1 -- 172.21.15.27:0/1422638710 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0118019070 con 0x7f0128077510 2024-05-21T03:23:12.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.766+0000 7f011e7fc700 1 --2- 172.21.15.27:0/1422638710 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f01140b2880 0x7f01140b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:12.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.767+0000 7f012d554700 1 --2- 172.21.15.27:0/1422638710 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f01140b2880 0x7f01140b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:12.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.767+0000 7f011e7fc700 1 -- 172.21.15.27:0/1422638710 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f01180e1d60 con 0x7f0128077510 2024-05-21T03:23:12.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.767+0000 7f012d554700 1 --2- 172.21.15.27:0/1422638710 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f01140b2880 0x7f01140b4d40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f012400b5c0 tx=0x7f0124005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:12.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:12 smithi039 ceph-mon[36429]: pgmap v371: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:23:12.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:12.770+0000 7f011e7fc700 1 -- 172.21.15.27:0/1422638710 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f01180a9a60 con 0x7f0128077510 2024-05-21T03:23:12.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:12 smithi165 ceph-mon[36269]: pgmap v371: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:23:13.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:13.003+0000 7f012f7b8700 1 -- 172.21.15.27:0/1422638710 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f0128063980 con 0x7f01140b2880 2024-05-21T03:23:13.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:13.004+0000 7f011e7fc700 1 -- 172.21.15.27:0/1422638710 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f0128063980 con 0x7f01140b2880 2024-05-21T03:23:13.004 INFO:teuthology.orchestra.run.smithi027.stdout:Backend: cephadm 2024-05-21T03:23:13.004 INFO:teuthology.orchestra.run.smithi027.stdout:Available: Yes 2024-05-21T03:23:13.005 INFO:teuthology.orchestra.run.smithi027.stdout:Paused: No 2024-05-21T03:23:13.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:13.007+0000 7f012f7b8700 1 -- 172.21.15.27:0/1422638710 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f01140b2880 msgr2=0x7f01140b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:13.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:13.007+0000 7f012f7b8700 1 --2- 172.21.15.27:0/1422638710 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f01140b2880 0x7f01140b4d40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f012400b5c0 tx=0x7f0124005fb0 comp rx=0 tx=0).stop 2024-05-21T03:23:13.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:13.007+0000 7f012f7b8700 1 -- 172.21.15.27:0/1422638710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0128077510 msgr2=0x7f012819be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:13.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:13.007+0000 7f012f7b8700 1 --2- 172.21.15.27:0/1422638710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0128077510 0x7f012819be50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f011800d980 tx=0x7f011800dc90 comp rx=0 tx=0).stop 2024-05-21T03:23:13.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:13.007+0000 7f012f7b8700 1 -- 172.21.15.27:0/1422638710 shutdown_connections 2024-05-21T03:23:13.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:13.007+0000 7f012f7b8700 1 --2- 172.21.15.27:0/1422638710 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f01140b2880 0x7f01140b4d40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:13.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:13.007+0000 7f012f7b8700 1 --2- 172.21.15.27:0/1422638710 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f012810b480 0x7f01281a2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:13.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:13.007+0000 7f012f7b8700 1 --2- 172.21.15.27:0/1422638710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0128075f40 0x7f012819b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:13.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:13.007+0000 7f012f7b8700 1 --2- 172.21.15.27:0/1422638710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0128077510 0x7f012819be50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:13.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:13.007+0000 7f012f7b8700 1 -- 172.21.15.27:0/1422638710 >> 172.21.15.27:0/1422638710 conn(0x7f01280feb40 msgr2=0x7f01281073d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:13.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:13.007+0000 7f012f7b8700 1 -- 172.21.15.27:0/1422638710 shutdown_connections 2024-05-21T03:23:13.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:13.007+0000 7f012f7b8700 1 -- 172.21.15.27:0/1422638710 wait complete. 2024-05-21T03:23:13.610 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-21T03:23:13.860 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:23:14.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:14 smithi027 ceph-mon[32179]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:23:14.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:14 smithi027 ceph-mon[32179]: pgmap v372: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:23:14.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:14 smithi039 ceph-mon[36429]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:23:14.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:14 smithi039 ceph-mon[36429]: pgmap v372: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:23:14.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:14 smithi165 ceph-mon[36269]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:23:14.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:14 smithi165 ceph-mon[36269]: pgmap v372: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:23:15.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.599+0000 7efc8e4ae700 1 -- 172.21.15.27:0/331047462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc88104040 msgr2=0x7efc8810a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:15.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.599+0000 7efc8e4ae700 1 --2- 172.21.15.27:0/331047462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc88104040 0x7efc8810a8f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efc8000b0d0 tx=0x7efc8000b3e0 comp rx=0 tx=0).stop 2024-05-21T03:23:15.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.599+0000 7efc8e4ae700 1 -- 172.21.15.27:0/331047462 shutdown_connections 2024-05-21T03:23:15.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.599+0000 7efc8e4ae700 1 --2- 172.21.15.27:0/331047462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc88104040 0x7efc8810a8f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:15.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.599+0000 7efc8e4ae700 1 --2- 172.21.15.27:0/331047462 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc881024a0 0x7efc881028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:15.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.599+0000 7efc8e4ae700 1 --2- 172.21.15.27:0/331047462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc881036a0 0x7efc88103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:15.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.599+0000 7efc8e4ae700 1 -- 172.21.15.27:0/331047462 >> 172.21.15.27:0/331047462 conn(0x7efc880fda40 msgr2=0x7efc880ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:15.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.599+0000 7efc8e4ae700 1 -- 172.21.15.27:0/331047462 shutdown_connections 2024-05-21T03:23:15.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.599+0000 7efc8e4ae700 1 -- 172.21.15.27:0/331047462 wait complete. 2024-05-21T03:23:15.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.600+0000 7efc8e4ae700 1 Processor -- start 2024-05-21T03:23:15.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.600+0000 7efc8e4ae700 1 -- start start 2024-05-21T03:23:15.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.600+0000 7efc8e4ae700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc881024a0 0x7efc881996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:15.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.600+0000 7efc8e4ae700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc881036a0 0x7efc88199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:15.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.600+0000 7efc8e4ae700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc88104040 0x7efc881a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:15.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.601+0000 7efc8e4ae700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efc8810df40 con 0x7efc88104040 2024-05-21T03:23:15.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.601+0000 7efc8e4ae700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efc8810ddc0 con 0x7efc881036a0 2024-05-21T03:23:15.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.601+0000 7efc8e4ae700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efc8810e0c0 con 0x7efc881024a0 2024-05-21T03:23:15.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.601+0000 7efc8ca4b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc88104040 0x7efc881a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:15.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.601+0000 7efc8ca4b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc88104040 0x7efc881a0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35716/0 (socket says 172.21.15.27:35716) 2024-05-21T03:23:15.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.601+0000 7efc8ca4b700 1 -- 172.21.15.27:0/1401718597 learned_addr learned my addr 172.21.15.27:0/1401718597 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:15.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.601+0000 7efc87fff700 1 --2- 172.21.15.27:0/1401718597 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc881024a0 0x7efc881996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:15.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.601+0000 7efc877fe700 1 --2- 172.21.15.27:0/1401718597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc881036a0 0x7efc88199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:15.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.601+0000 7efc8ca4b700 1 -- 172.21.15.27:0/1401718597 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc881024a0 msgr2=0x7efc881996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:15.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.601+0000 7efc8ca4b700 1 --2- 172.21.15.27:0/1401718597 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc881024a0 0x7efc881996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:15.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.601+0000 7efc8ca4b700 1 -- 172.21.15.27:0/1401718597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc881036a0 msgr2=0x7efc88199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:15.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.601+0000 7efc8ca4b700 1 --2- 172.21.15.27:0/1401718597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc881036a0 0x7efc88199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:15.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.601+0000 7efc8ca4b700 1 -- 172.21.15.27:0/1401718597 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc881a1340 con 0x7efc88104040 2024-05-21T03:23:15.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.601+0000 7efc8ca4b700 1 --2- 172.21.15.27:0/1401718597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc88104040 0x7efc881a0ca0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efc80009850 tx=0x7efc80004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:15.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.601+0000 7efc857fa700 1 -- 172.21.15.27:0/1401718597 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8001f030 con 0x7efc88104040 2024-05-21T03:23:15.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.601+0000 7efc857fa700 1 -- 172.21.15.27:0/1401718597 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc80005040 con 0x7efc88104040 2024-05-21T03:23:15.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.601+0000 7efc8e4ae700 1 -- 172.21.15.27:0/1401718597 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc881a15d0 con 0x7efc88104040 2024-05-21T03:23:15.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.601+0000 7efc857fa700 1 -- 172.21.15.27:0/1401718597 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8001b690 con 0x7efc88104040 2024-05-21T03:23:15.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.601+0000 7efc8e4ae700 1 -- 172.21.15.27:0/1401718597 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc881a1a80 con 0x7efc88104040 2024-05-21T03:23:15.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.603+0000 7efc8e4ae700 1 -- 172.21.15.27:0/1401718597 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc8806a230 con 0x7efc88104040 2024-05-21T03:23:15.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.604+0000 7efc857fa700 1 -- 172.21.15.27:0/1401718597 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7efc80022030 con 0x7efc88104040 2024-05-21T03:23:15.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.604+0000 7efc857fa700 1 --2- 172.21.15.27:0/1401718597 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7efc740b2880 0x7efc740b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:15.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.605+0000 7efc87fff700 1 --2- 172.21.15.27:0/1401718597 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7efc740b2880 0x7efc740b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:15.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.605+0000 7efc857fa700 1 -- 172.21.15.27:0/1401718597 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7efc8000f020 con 0x7efc88104040 2024-05-21T03:23:15.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.605+0000 7efc87fff700 1 --2- 172.21.15.27:0/1401718597 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7efc740b2880 0x7efc740b4d40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efc78005fd0 tx=0x7efc7800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:15.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.609+0000 7efc857fa700 1 -- 172.21.15.27:0/1401718597 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efc80029020 con 0x7efc88104040 2024-05-21T03:23:15.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.835+0000 7efc8e4ae700 1 -- 172.21.15.27:0/1401718597 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7efc880ffc70 con 0x7efc740b2880 2024-05-21T03:23:15.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.849+0000 7efc857fa700 1 -- 172.21.15.27:0/1401718597 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7efc880ffc70 con 0x7efc740b2880 2024-05-21T03:23:15.850 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-21T03:23:15.850 INFO:teuthology.orchestra.run.smithi027.stdout:mds.a smithi027 running (2m) 36s ago 2m 19.7M - 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 8230c3edc162 2024-05-21T03:23:15.850 INFO:teuthology.orchestra.run.smithi027.stdout:mds.b smithi039 running (2m) 42s ago 2m 30.2M - 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 1f3a14b16cf9 2024-05-21T03:23:15.850 INFO:teuthology.orchestra.run.smithi027.stdout:mds.c smithi165 running (3m) 42s ago 3m 20.4M - 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 f306b4b0d890 2024-05-21T03:23:15.850 INFO:teuthology.orchestra.run.smithi027.stdout:mds.d smithi027 running (3m) 36s ago 3m 17.2M - 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 b69acbcdc74e 2024-05-21T03:23:15.850 INFO:teuthology.orchestra.run.smithi027.stdout:mds.e smithi039 running (3m) 42s ago 3m 20.8M - 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 67ae61586fc6 2024-05-21T03:23:15.850 INFO:teuthology.orchestra.run.smithi027.stdout:mds.f smithi165 running (3m) 42s ago 3m 18.9M - 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 5588f1b9598a 2024-05-21T03:23:15.850 INFO:teuthology.orchestra.run.smithi027.stdout:mds.g smithi027 running (3m) 36s ago 3m 21.1M - 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 1fe1fed96868 2024-05-21T03:23:15.850 INFO:teuthology.orchestra.run.smithi027.stdout:mds.h smithi039 running (3m) 42s ago 3m 22.2M - 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 4156b82453d0 2024-05-21T03:23:15.850 INFO:teuthology.orchestra.run.smithi027.stdout:mds.i smithi165 running (3m) 42s ago 3m 20.1M - 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 9929b3291d14 2024-05-21T03:23:15.850 INFO:teuthology.orchestra.run.smithi027.stdout:mds.j smithi027 running (3m) 36s ago 3m 14.1M - 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 f9ede6e050ed 2024-05-21T03:23:15.851 INFO:teuthology.orchestra.run.smithi027.stdout:mds.k smithi039 running (3m) 42s ago 3m 21.0M - 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 8ad1a58936f9 2024-05-21T03:23:15.851 INFO:teuthology.orchestra.run.smithi027.stdout:mds.l smithi165 running (3m) 42s ago 3m 21.0M - 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 3b6d506082ca 2024-05-21T03:23:15.851 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.x smithi027 *:9283,8765 running (10m) 36s ago 10m 533M - 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 41efc524711b 2024-05-21T03:23:15.851 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.y smithi039 *:8443,8765 running (8m) 42s ago 8m 469M - 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 fac9a0fe8ebc 2024-05-21T03:23:15.851 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.z smithi165 *:8443,8765 running (8m) 42s ago 8m 469M - 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 cd5326e5d395 2024-05-21T03:23:15.851 INFO:teuthology.orchestra.run.smithi027.stdout:mon.a smithi027 running (10m) 36s ago 10m 50.3M 2048M 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 eb3a7e6bf9c3 2024-05-21T03:23:15.851 INFO:teuthology.orchestra.run.smithi027.stdout:mon.b smithi039 running (9m) 42s ago 9m 39.2M 2048M 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 e7b68b78f66c 2024-05-21T03:23:15.851 INFO:teuthology.orchestra.run.smithi027.stdout:mon.c smithi165 running (9m) 42s ago 9m 41.5M 2048M 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 0f5237d01455 2024-05-21T03:23:15.851 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (8m) 36s ago 8m 49.6M 4272M 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 49e997ffe7ed 2024-05-21T03:23:15.851 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi039 running (8m) 42s ago 8m 59.8M 4272M 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 a2441f307a24 2024-05-21T03:23:15.851 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi165 running (7m) 42s ago 7m 53.1M 4272M 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 39b841c04834 2024-05-21T03:23:15.851 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi027 running (7m) 36s ago 7m 50.9M 4272M 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 d7ffb6f2a711 2024-05-21T03:23:15.851 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi039 running (6m) 42s ago 6m 52.7M 4272M 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 5bf104643617 2024-05-21T03:23:15.852 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi165 running (6m) 42s ago 6m 57.0M 4272M 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 57e27c2c1b1e 2024-05-21T03:23:15.852 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6 smithi027 running (6m) 36s ago 6m 45.5M 4272M 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 4a0b825924fd 2024-05-21T03:23:15.852 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7 smithi039 running (5m) 42s ago 5m 59.0M 4272M 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 ebc2f4cd30b6 2024-05-21T03:23:15.852 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8 smithi165 running (5m) 42s ago 5m 50.0M 4272M 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 2fc5bb0e1f58 2024-05-21T03:23:15.852 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9 smithi027 running (4m) 36s ago 4m 45.7M 4272M 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 0c30aa73ebb3 2024-05-21T03:23:15.852 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10 smithi039 running (4m) 42s ago 4m 49.3M 4272M 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 b10241f55e4a 2024-05-21T03:23:15.852 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11 smithi165 running (3m) 42s ago 3m 56.9M 4272M 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 a063c7f2b774 2024-05-21T03:23:15.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.852+0000 7efc8e4ae700 1 -- 172.21.15.27:0/1401718597 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7efc740b2880 msgr2=0x7efc740b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:15.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.852+0000 7efc8e4ae700 1 --2- 172.21.15.27:0/1401718597 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7efc740b2880 0x7efc740b4d40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efc78005fd0 tx=0x7efc7800a560 comp rx=0 tx=0).stop 2024-05-21T03:23:15.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.852+0000 7efc8e4ae700 1 -- 172.21.15.27:0/1401718597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc88104040 msgr2=0x7efc881a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:15.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.852+0000 7efc8e4ae700 1 --2- 172.21.15.27:0/1401718597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc88104040 0x7efc881a0ca0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efc80009850 tx=0x7efc80004e20 comp rx=0 tx=0).stop 2024-05-21T03:23:15.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.852+0000 7efc8e4ae700 1 -- 172.21.15.27:0/1401718597 shutdown_connections 2024-05-21T03:23:15.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.852+0000 7efc8e4ae700 1 --2- 172.21.15.27:0/1401718597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc88104040 0x7efc881a0ca0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:15.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.852+0000 7efc8e4ae700 1 --2- 172.21.15.27:0/1401718597 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7efc740b2880 0x7efc740b4d40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:15.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.852+0000 7efc8e4ae700 1 --2- 172.21.15.27:0/1401718597 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc881024a0 0x7efc881996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:15.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.852+0000 7efc8e4ae700 1 --2- 172.21.15.27:0/1401718597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc881036a0 0x7efc88199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:15.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.852+0000 7efc8e4ae700 1 -- 172.21.15.27:0/1401718597 >> 172.21.15.27:0/1401718597 conn(0x7efc880fda40 msgr2=0x7efc880ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:15.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.852+0000 7efc8e4ae700 1 -- 172.21.15.27:0/1401718597 shutdown_connections 2024-05-21T03:23:15.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:15.852+0000 7efc8e4ae700 1 -- 172.21.15.27:0/1401718597 wait complete. 2024-05-21T03:23:16.533 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-21T03:23:16.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:16 smithi039 ceph-mon[36429]: pgmap v373: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:23:16.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:16 smithi039 ceph-mon[36429]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:23:16.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:16 smithi165 ceph-mon[36269]: pgmap v373: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:23:16.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:16 smithi165 ceph-mon[36269]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:23:16.783 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:23:16.906 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:16 smithi027 ceph-mon[32179]: pgmap v373: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:23:16.906 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:16 smithi027 ceph-mon[32179]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:23:18.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.488+0000 7f19c3c06700 1 -- 172.21.15.27:0/2238659301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19bc1092d0 msgr2=0x7f19bc10b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:18.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.488+0000 7f19c3c06700 1 --2- 172.21.15.27:0/2238659301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19bc1092d0 0x7f19bc10b740 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f19b400b0a0 tx=0x7f19b400b3b0 comp rx=0 tx=0).stop 2024-05-21T03:23:18.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.488+0000 7f19c3c06700 1 -- 172.21.15.27:0/2238659301 shutdown_connections 2024-05-21T03:23:18.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.488+0000 7f19c3c06700 1 --2- 172.21.15.27:0/2238659301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19bc1092d0 0x7f19bc10b740 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:18.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.488+0000 7f19c3c06700 1 --2- 172.21.15.27:0/2238659301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19bc104070 0x7f19bc106460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:18.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.488+0000 7f19c3c06700 1 --2- 172.21.15.27:0/2238659301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19bc1069a0 0x7f19bc108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:18.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.488+0000 7f19c3c06700 1 -- 172.21.15.27:0/2238659301 >> 172.21.15.27:0/2238659301 conn(0x7f19bc0fdaa0 msgr2=0x7f19bc0ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:18.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.489+0000 7f19c3c06700 1 -- 172.21.15.27:0/2238659301 shutdown_connections 2024-05-21T03:23:18.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.489+0000 7f19c3c06700 1 -- 172.21.15.27:0/2238659301 wait complete. 2024-05-21T03:23:18.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.489+0000 7f19c3c06700 1 Processor -- start 2024-05-21T03:23:18.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.489+0000 7f19c3c06700 1 -- start start 2024-05-21T03:23:18.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.489+0000 7f19c3c06700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19bc104070 0x7f19bc1039d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:18.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.489+0000 7f19c3c06700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19bc1069a0 0x7f19bc102020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:18.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.489+0000 7f19c3c06700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19bc1092d0 0x7f19bc102560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:18.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.489+0000 7f19c3c06700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f19bc10f190 con 0x7f19bc104070 2024-05-21T03:23:18.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.489+0000 7f19c3c06700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f19bc10f010 con 0x7f19bc1069a0 2024-05-21T03:23:18.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.489+0000 7f19c3c06700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f19bc10f310 con 0x7f19bc1092d0 2024-05-21T03:23:18.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.490+0000 7f19c19a2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19bc104070 0x7f19bc1039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:18.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.490+0000 7f19c19a2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19bc104070 0x7f19bc1039d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35740/0 (socket says 172.21.15.27:35740) 2024-05-21T03:23:18.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.490+0000 7f19c19a2700 1 -- 172.21.15.27:0/4044529766 learned_addr learned my addr 172.21.15.27:0/4044529766 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:18.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.490+0000 7f19c21a3700 1 --2- 172.21.15.27:0/4044529766 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19bc1092d0 0x7f19bc102560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:18.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.490+0000 7f19c11a1700 1 --2- 172.21.15.27:0/4044529766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19bc1069a0 0x7f19bc102020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:18.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.490+0000 7f19c19a2700 1 -- 172.21.15.27:0/4044529766 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19bc1092d0 msgr2=0x7f19bc102560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:18.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.490+0000 7f19c19a2700 1 --2- 172.21.15.27:0/4044529766 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19bc1092d0 0x7f19bc102560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:18.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.490+0000 7f19c19a2700 1 -- 172.21.15.27:0/4044529766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19bc1069a0 msgr2=0x7f19bc102020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:18.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.490+0000 7f19c19a2700 1 --2- 172.21.15.27:0/4044529766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19bc1069a0 0x7f19bc102020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:18.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.490+0000 7f19c19a2700 1 -- 172.21.15.27:0/4044529766 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19bc102e10 con 0x7f19bc104070 2024-05-21T03:23:18.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.490+0000 7f19c19a2700 1 --2- 172.21.15.27:0/4044529766 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19bc104070 0x7f19bc1039d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f19b800b530 tx=0x7f19b800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:18.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.490+0000 7f19b2ffd700 1 -- 172.21.15.27:0/4044529766 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b8015020 con 0x7f19bc104070 2024-05-21T03:23:18.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.490+0000 7f19b2ffd700 1 -- 172.21.15.27:0/4044529766 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19b8011ce0 con 0x7f19bc104070 2024-05-21T03:23:18.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.490+0000 7f19b2ffd700 1 -- 172.21.15.27:0/4044529766 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b8010d20 con 0x7f19bc104070 2024-05-21T03:23:18.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.490+0000 7f19c3c06700 1 -- 172.21.15.27:0/4044529766 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19bc1a3a00 con 0x7f19bc104070 2024-05-21T03:23:18.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.490+0000 7f19c3c06700 1 -- 172.21.15.27:0/4044529766 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19bc1a3fc0 con 0x7f19bc104070 2024-05-21T03:23:18.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.491+0000 7f19c3c06700 1 -- 172.21.15.27:0/4044529766 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19bc06a230 con 0x7f19bc104070 2024-05-21T03:23:18.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.493+0000 7f19b2ffd700 1 -- 172.21.15.27:0/4044529766 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f19b8018020 con 0x7f19bc104070 2024-05-21T03:23:18.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.494+0000 7f19b2ffd700 1 --2- 172.21.15.27:0/4044529766 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f19a80b2900 0x7f19a80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:18.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.494+0000 7f19c11a1700 1 --2- 172.21.15.27:0/4044529766 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f19a80b2900 0x7f19a80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:18.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.494+0000 7f19c11a1700 1 --2- 172.21.15.27:0/4044529766 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f19a80b2900 0x7f19a80b4dc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f19ac006fd0 tx=0x7f19ac008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:18.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.494+0000 7f19b2ffd700 1 -- 172.21.15.27:0/4044529766 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f19b80a5b40 con 0x7f19bc104070 2024-05-21T03:23:18.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.499+0000 7f19b2ffd700 1 -- 172.21.15.27:0/4044529766 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f19b801f020 con 0x7f19bc104070 2024-05-21T03:23:18.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.722+0000 7f19c3c06700 1 -- 172.21.15.27:0/4044529766 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f19bc063980 con 0x7f19a80b2900 2024-05-21T03:23:18.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.728+0000 7f19b2ffd700 1 -- 172.21.15.27:0/4044529766 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f19bc063980 con 0x7f19a80b2900 2024-05-21T03:23:18.727 INFO:teuthology.orchestra.run.smithi027.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-21T03:23:18.727 INFO:teuthology.orchestra.run.smithi027.stdout:mds.all 12/12 45s ago 3m smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi039=b;smithi039=e;smithi039=h;smithi039=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l;count:12 2024-05-21T03:23:18.727 INFO:teuthology.orchestra.run.smithi027.stdout:mgr 3/3 45s ago 9m smithi027=x;smithi039=y;smithi165=z;count:3 2024-05-21T03:23:18.727 INFO:teuthology.orchestra.run.smithi027.stdout:mon 3/3 45s ago 9m smithi027:172.21.15.27=a;smithi039:172.21.15.39=b;smithi165:172.21.15.165=c;count:3 2024-05-21T03:23:18.728 INFO:teuthology.orchestra.run.smithi027.stdout:osd 12 45s ago - 2024-05-21T03:23:18.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.730+0000 7f19c3c06700 1 -- 172.21.15.27:0/4044529766 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f19a80b2900 msgr2=0x7f19a80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:18.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.730+0000 7f19c3c06700 1 --2- 172.21.15.27:0/4044529766 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f19a80b2900 0x7f19a80b4dc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f19ac006fd0 tx=0x7f19ac008040 comp rx=0 tx=0).stop 2024-05-21T03:23:18.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.730+0000 7f19c3c06700 1 -- 172.21.15.27:0/4044529766 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19bc104070 msgr2=0x7f19bc1039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:18.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.730+0000 7f19c3c06700 1 --2- 172.21.15.27:0/4044529766 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19bc104070 0x7f19bc1039d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f19b800b530 tx=0x7f19b800b8f0 comp rx=0 tx=0).stop 2024-05-21T03:23:18.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.730+0000 7f19c3c06700 1 -- 172.21.15.27:0/4044529766 shutdown_connections 2024-05-21T03:23:18.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.730+0000 7f19c3c06700 1 --2- 172.21.15.27:0/4044529766 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f19a80b2900 0x7f19a80b4dc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:18.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.730+0000 7f19c3c06700 1 --2- 172.21.15.27:0/4044529766 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19bc1092d0 0x7f19bc102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:18.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.730+0000 7f19c3c06700 1 --2- 172.21.15.27:0/4044529766 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19bc104070 0x7f19bc1039d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:18.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.730+0000 7f19c3c06700 1 --2- 172.21.15.27:0/4044529766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19bc1069a0 0x7f19bc102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:18.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.730+0000 7f19c3c06700 1 -- 172.21.15.27:0/4044529766 >> 172.21.15.27:0/4044529766 conn(0x7f19bc0fdaa0 msgr2=0x7f19bc1049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:18.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.731+0000 7f19c3c06700 1 -- 172.21.15.27:0/4044529766 shutdown_connections 2024-05-21T03:23:18.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:18.731+0000 7f19c3c06700 1 -- 172.21.15.27:0/4044529766 wait complete. 2024-05-21T03:23:18.732 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:18 smithi027 ceph-mon[32179]: pgmap v374: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:23:18.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:18 smithi039 ceph-mon[36429]: pgmap v374: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:23:18.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:18 smithi165 ceph-mon[36269]: pgmap v374: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:23:19.408 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-21T03:23:19.661 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:23:19.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:19 smithi027 ceph-mon[32179]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:23:19.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:19 smithi039 ceph-mon[36429]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:23:19.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:19 smithi165 ceph-mon[36269]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:23:20.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:20 smithi039 ceph-mon[36429]: pgmap v375: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:23:20.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:20 smithi165 ceph-mon[36269]: pgmap v375: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:23:20.852 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:20 smithi027 ceph-mon[32179]: pgmap v375: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:23:21.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.410+0000 7f4705057700 1 -- 172.21.15.27:0/520565763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4700105190 msgr2=0x7f470010ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:21.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.410+0000 7f4705057700 1 --2- 172.21.15.27:0/520565763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4700105190 0x7f470010ba40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f46f800b0d0 tx=0x7f46f800b3e0 comp rx=0 tx=0).stop 2024-05-21T03:23:21.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.410+0000 7f4705057700 1 -- 172.21.15.27:0/520565763 shutdown_connections 2024-05-21T03:23:21.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.410+0000 7f4705057700 1 --2- 172.21.15.27:0/520565763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4700105190 0x7f470010ba40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:21.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.410+0000 7f4705057700 1 --2- 172.21.15.27:0/520565763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47001035f0 0x7f4700103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:21.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.410+0000 7f4705057700 1 --2- 172.21.15.27:0/520565763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47001047f0 0x7f4700104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:21.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.410+0000 7f4705057700 1 -- 172.21.15.27:0/520565763 >> 172.21.15.27:0/520565763 conn(0x7f47000feb90 msgr2=0x7f4700100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:21.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.410+0000 7f4705057700 1 -- 172.21.15.27:0/520565763 shutdown_connections 2024-05-21T03:23:21.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.410+0000 7f4705057700 1 -- 172.21.15.27:0/520565763 wait complete. 2024-05-21T03:23:21.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.411+0000 7f4705057700 1 Processor -- start 2024-05-21T03:23:21.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.411+0000 7f4705057700 1 -- start start 2024-05-21T03:23:21.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.411+0000 7f4705057700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47001035f0 0x7f4700199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:21.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.411+0000 7f4705057700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47001047f0 0x7f4700199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:21.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.411+0000 7f4705057700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4700105190 0x7f47001a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:21.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.411+0000 7f4705057700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f470010f070 con 0x7f4700105190 2024-05-21T03:23:21.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.411+0000 7f4705057700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f470010eef0 con 0x7f47001035f0 2024-05-21T03:23:21.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.411+0000 7f4705057700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f470010f1f0 con 0x7f47001047f0 2024-05-21T03:23:21.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.412+0000 7f46ff59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4700105190 0x7f47001a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:21.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.412+0000 7f46ff59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4700105190 0x7f47001a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35750/0 (socket says 172.21.15.27:35750) 2024-05-21T03:23:21.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.412+0000 7f46ff59e700 1 -- 172.21.15.27:0/590279717 learned_addr learned my addr 172.21.15.27:0/590279717 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:21.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.412+0000 7f46fed9d700 1 --2- 172.21.15.27:0/590279717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47001035f0 0x7f4700199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:21.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.412+0000 7f46fe59c700 1 --2- 172.21.15.27:0/590279717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47001047f0 0x7f4700199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:21.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.412+0000 7f46ff59e700 1 -- 172.21.15.27:0/590279717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47001047f0 msgr2=0x7f4700199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:21.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.412+0000 7f46ff59e700 1 --2- 172.21.15.27:0/590279717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47001047f0 0x7f4700199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:21.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.412+0000 7f46ff59e700 1 -- 172.21.15.27:0/590279717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47001035f0 msgr2=0x7f4700199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:21.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.412+0000 7f46ff59e700 1 --2- 172.21.15.27:0/590279717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47001035f0 0x7f4700199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:21.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.412+0000 7f46ff59e700 1 -- 172.21.15.27:0/590279717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47001a12b0 con 0x7f4700105190 2024-05-21T03:23:21.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.412+0000 7f46ff59e700 1 --2- 172.21.15.27:0/590279717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4700105190 0x7f47001a0c10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009850 tx=0x7f46f8004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:21.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.412+0000 7f46e7fff700 1 -- 172.21.15.27:0/590279717 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f801f030 con 0x7f4700105190 2024-05-21T03:23:21.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.412+0000 7f46e7fff700 1 -- 172.21.15.27:0/590279717 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46f8007a80 con 0x7f4700105190 2024-05-21T03:23:21.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.412+0000 7f4705057700 1 -- 172.21.15.27:0/590279717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47001a1540 con 0x7f4700105190 2024-05-21T03:23:21.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.412+0000 7f46e7fff700 1 -- 172.21.15.27:0/590279717 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f801b690 con 0x7f4700105190 2024-05-21T03:23:21.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.412+0000 7f4705057700 1 -- 172.21.15.27:0/590279717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47001a19f0 con 0x7f4700105190 2024-05-21T03:23:21.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.414+0000 7f4705057700 1 -- 172.21.15.27:0/590279717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f470006a230 con 0x7f4700105190 2024-05-21T03:23:21.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.415+0000 7f46e7fff700 1 -- 172.21.15.27:0/590279717 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f46f8022030 con 0x7f4700105190 2024-05-21T03:23:21.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.416+0000 7f46e7fff700 1 --2- 172.21.15.27:0/590279717 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f46e80b2880 0x7f46e80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:21.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.416+0000 7f46fed9d700 1 --2- 172.21.15.27:0/590279717 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f46e80b2880 0x7f46e80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:21.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.416+0000 7f46e7fff700 1 -- 172.21.15.27:0/590279717 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f46f800f020 con 0x7f4700105190 2024-05-21T03:23:21.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.416+0000 7f46fed9d700 1 --2- 172.21.15.27:0/590279717 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f46e80b2880 0x7f46e80b4d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4700104650 tx=0x7f46f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:21.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.419+0000 7f46e7fff700 1 -- 172.21.15.27:0/590279717 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f46f8029020 con 0x7f4700105190 2024-05-21T03:23:21.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.642+0000 7f4705057700 1 -- 172.21.15.27:0/590279717 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f4700063a40 con 0x7f46e80b2880 2024-05-21T03:23:21.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.644+0000 7f46e7fff700 1 -- 172.21.15.27:0/590279717 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f4700063a40 con 0x7f46e80b2880 2024-05-21T03:23:21.643 INFO:teuthology.orchestra.run.smithi027.stdout:HOST ADDR LABELS STATUS 2024-05-21T03:23:21.643 INFO:teuthology.orchestra.run.smithi027.stdout:smithi027 172.21.15.27 2024-05-21T03:23:21.643 INFO:teuthology.orchestra.run.smithi027.stdout:smithi039 172.21.15.39 2024-05-21T03:23:21.643 INFO:teuthology.orchestra.run.smithi027.stdout:smithi165 172.21.15.165 2024-05-21T03:23:21.643 INFO:teuthology.orchestra.run.smithi027.stdout:3 hosts in cluster 2024-05-21T03:23:21.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.646+0000 7f4705057700 1 -- 172.21.15.27:0/590279717 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f46e80b2880 msgr2=0x7f46e80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:21.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.646+0000 7f4705057700 1 --2- 172.21.15.27:0/590279717 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f46e80b2880 0x7f46e80b4d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4700104650 tx=0x7f46f0009260 comp rx=0 tx=0).stop 2024-05-21T03:23:21.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.646+0000 7f4705057700 1 -- 172.21.15.27:0/590279717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4700105190 msgr2=0x7f47001a0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:21.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.646+0000 7f4705057700 1 --2- 172.21.15.27:0/590279717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4700105190 0x7f47001a0c10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009850 tx=0x7f46f8004e20 comp rx=0 tx=0).stop 2024-05-21T03:23:21.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.646+0000 7f4705057700 1 -- 172.21.15.27:0/590279717 shutdown_connections 2024-05-21T03:23:21.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.646+0000 7f4705057700 1 --2- 172.21.15.27:0/590279717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4700105190 0x7f47001a0c10 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:21.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.646+0000 7f4705057700 1 --2- 172.21.15.27:0/590279717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47001035f0 0x7f4700199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:21.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.646+0000 7f4705057700 1 --2- 172.21.15.27:0/590279717 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f46e80b2880 0x7f46e80b4d40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:21.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.646+0000 7f4705057700 1 --2- 172.21.15.27:0/590279717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47001047f0 0x7f4700199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:21.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.646+0000 7f4705057700 1 -- 172.21.15.27:0/590279717 >> 172.21.15.27:0/590279717 conn(0x7f47000feb90 msgr2=0x7f4700107af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:21.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.646+0000 7f4705057700 1 -- 172.21.15.27:0/590279717 shutdown_connections 2024-05-21T03:23:21.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:21.646+0000 7f4705057700 1 -- 172.21.15.27:0/590279717 wait complete. 2024-05-21T03:23:22.460 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-21T03:23:22.714 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:23:22.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:22 smithi039 ceph-mon[36429]: pgmap v376: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:23:22.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:22 smithi039 ceph-mon[36429]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:23:22.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:22 smithi165 ceph-mon[36269]: pgmap v376: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:23:22.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:22 smithi165 ceph-mon[36269]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:23:22.837 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:22 smithi027 ceph-mon[32179]: pgmap v376: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:23:22.837 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:22 smithi027 ceph-mon[32179]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:23:24.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.422+0000 7f30a6a12700 1 -- 172.21.15.27:0/942559153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30a01024c0 msgr2=0x7f30a01028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:24.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.422+0000 7f30a6a12700 1 --2- 172.21.15.27:0/942559153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30a01024c0 0x7f30a01028e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f30900099c0 tx=0x7f3090009cd0 comp rx=0 tx=0).stop 2024-05-21T03:23:24.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.422+0000 7f30a6a12700 1 -- 172.21.15.27:0/942559153 shutdown_connections 2024-05-21T03:23:24.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.422+0000 7f30a6a12700 1 --2- 172.21.15.27:0/942559153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30a0104060 0x7f30a010a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:24.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.422+0000 7f30a6a12700 1 --2- 172.21.15.27:0/942559153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30a01024c0 0x7f30a01028e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:24.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.422+0000 7f30a6a12700 1 --2- 172.21.15.27:0/942559153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30a01036c0 0x7f30a0103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:24.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.422+0000 7f30a6a12700 1 -- 172.21.15.27:0/942559153 >> 172.21.15.27:0/942559153 conn(0x7f30a00fda60 msgr2=0x7f30a00ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:24.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.422+0000 7f30a6a12700 1 -- 172.21.15.27:0/942559153 shutdown_connections 2024-05-21T03:23:24.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.422+0000 7f30a6a12700 1 -- 172.21.15.27:0/942559153 wait complete. 2024-05-21T03:23:24.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.423+0000 7f30a6a12700 1 Processor -- start 2024-05-21T03:23:24.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.423+0000 7f30a6a12700 1 -- start start 2024-05-21T03:23:24.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.423+0000 7f30a6a12700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30a01024c0 0x7f30a0197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:24.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.423+0000 7f30a6a12700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30a01036c0 0x7f30a0197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:24.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.423+0000 7f30a6a12700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30a0104060 0x7f30a0193bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:24.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.423+0000 7f30a6a12700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f30a010dfe0 con 0x7f30a01024c0 2024-05-21T03:23:24.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.424+0000 7f30a6a12700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f30a010de60 con 0x7f30a0104060 2024-05-21T03:23:24.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.424+0000 7f30a6a12700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f30a010e160 con 0x7f30a01036c0 2024-05-21T03:23:24.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.424+0000 7f309ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30a01024c0 0x7f30a0197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:24.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.424+0000 7f309ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30a01024c0 0x7f30a0197030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56312/0 (socket says 172.21.15.27:56312) 2024-05-21T03:23:24.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.424+0000 7f309ffff700 1 -- 172.21.15.27:0/2102884664 learned_addr learned my addr 172.21.15.27:0/2102884664 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:24.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.424+0000 7f30a4faf700 1 --2- 172.21.15.27:0/2102884664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30a0104060 0x7f30a0193bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:24.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.424+0000 7f309f7fe700 1 --2- 172.21.15.27:0/2102884664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30a01036c0 0x7f30a0197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:24.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.424+0000 7f309ffff700 1 -- 172.21.15.27:0/2102884664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30a01036c0 msgr2=0x7f30a0197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:24.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.424+0000 7f309ffff700 1 --2- 172.21.15.27:0/2102884664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30a01036c0 0x7f30a0197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:24.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.424+0000 7f309ffff700 1 -- 172.21.15.27:0/2102884664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30a0104060 msgr2=0x7f30a0193bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:24.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.424+0000 7f309ffff700 1 --2- 172.21.15.27:0/2102884664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30a0104060 0x7f30a0193bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:24.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.424+0000 7f309ffff700 1 -- 172.21.15.27:0/2102884664 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a0194480 con 0x7f30a01024c0 2024-05-21T03:23:24.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.424+0000 7f309ffff700 1 --2- 172.21.15.27:0/2102884664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30a01024c0 0x7f30a0197030 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f30900038c0 tx=0x7f30900047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:24.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.424+0000 7f309d7fa700 1 -- 172.21.15.27:0/2102884664 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3090027070 con 0x7f30a01024c0 2024-05-21T03:23:24.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.424+0000 7f309d7fa700 1 -- 172.21.15.27:0/2102884664 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30900040e0 con 0x7f30a01024c0 2024-05-21T03:23:24.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.424+0000 7f309d7fa700 1 -- 172.21.15.27:0/2102884664 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3090022df0 con 0x7f30a01024c0 2024-05-21T03:23:24.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.424+0000 7f30a6a12700 1 -- 172.21.15.27:0/2102884664 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30a01a7de0 con 0x7f30a01024c0 2024-05-21T03:23:24.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.424+0000 7f30a6a12700 1 -- 172.21.15.27:0/2102884664 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30a01a8180 con 0x7f30a01024c0 2024-05-21T03:23:24.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.425+0000 7f30a6a12700 1 -- 172.21.15.27:0/2102884664 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30a006a230 con 0x7f30a01024c0 2024-05-21T03:23:24.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.428+0000 7f309d7fa700 1 -- 172.21.15.27:0/2102884664 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f309002a070 con 0x7f30a01024c0 2024-05-21T03:23:24.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.428+0000 7f309d7fa700 1 --2- 172.21.15.27:0/2102884664 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f30880b2900 0x7f30880b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:24.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.429+0000 7f309f7fe700 1 --2- 172.21.15.27:0/2102884664 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f30880b2900 0x7f30880b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:24.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.429+0000 7f309d7fa700 1 -- 172.21.15.27:0/2102884664 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f30900e2de0 con 0x7f30a01024c0 2024-05-21T03:23:24.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.429+0000 7f309f7fe700 1 --2- 172.21.15.27:0/2102884664 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f30880b2900 0x7f30880b4dc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3094009a10 tx=0x7f3094009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:24.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.432+0000 7f309d7fa700 1 -- 172.21.15.27:0/2102884664 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3090013070 con 0x7f30a01024c0 2024-05-21T03:23:24.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.654+0000 7f30a6a12700 1 -- 172.21.15.27:0/2102884664 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f30a00ffc60 con 0x7f30880b2900 2024-05-21T03:23:24.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.656+0000 7f309d7fa700 1 -- 172.21.15.27:0/2102884664 <== mgr.14152 v2:172.21.15.27:6800/3683408243 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f30a00ffc60 con 0x7f30880b2900 2024-05-21T03:23:24.655 INFO:teuthology.orchestra.run.smithi027.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-21T03:23:24.655 INFO:teuthology.orchestra.run.smithi027.stdout:smithi027 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331001C400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-21T03:23:24.655 INFO:teuthology.orchestra.run.smithi027.stdout:smithi039 /dev/nvme0n1 ssd INTEL_SSDPEDKE040T7_BTLE736102QX4P0KGN 3726G No 4m ago Has a FileSystem, LVM detected 2024-05-21T03:23:24.655 INFO:teuthology.orchestra.run.smithi027.stdout:smithi165 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CE400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-05-21T03:23:24.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.658+0000 7f30a6a12700 1 -- 172.21.15.27:0/2102884664 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f30880b2900 msgr2=0x7f30880b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:24.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.658+0000 7f30a6a12700 1 --2- 172.21.15.27:0/2102884664 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f30880b2900 0x7f30880b4dc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3094009a10 tx=0x7f3094009190 comp rx=0 tx=0).stop 2024-05-21T03:23:24.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.658+0000 7f30a6a12700 1 -- 172.21.15.27:0/2102884664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30a01024c0 msgr2=0x7f30a0197030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:24.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.658+0000 7f30a6a12700 1 --2- 172.21.15.27:0/2102884664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30a01024c0 0x7f30a0197030 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f30900038c0 tx=0x7f30900047b0 comp rx=0 tx=0).stop 2024-05-21T03:23:24.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.658+0000 7f30a6a12700 1 -- 172.21.15.27:0/2102884664 shutdown_connections 2024-05-21T03:23:24.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.658+0000 7f30a6a12700 1 --2- 172.21.15.27:0/2102884664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30a0104060 0x7f30a0193bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:24.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.658+0000 7f30a6a12700 1 --2- 172.21.15.27:0/2102884664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30a01024c0 0x7f30a0197030 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:24.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.658+0000 7f30a6a12700 1 --2- 172.21.15.27:0/2102884664 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f30880b2900 0x7f30880b4dc0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:24.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.658+0000 7f30a6a12700 1 --2- 172.21.15.27:0/2102884664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30a01036c0 0x7f30a0197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:24.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.658+0000 7f30a6a12700 1 -- 172.21.15.27:0/2102884664 >> 172.21.15.27:0/2102884664 conn(0x7f30a00fda60 msgr2=0x7f30a00ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:24.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.658+0000 7f30a6a12700 1 -- 172.21.15.27:0/2102884664 shutdown_connections 2024-05-21T03:23:24.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:24.658+0000 7f30a6a12700 1 -- 172.21.15.27:0/2102884664 wait complete. 2024-05-21T03:23:24.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:24 smithi039 ceph-mon[36429]: pgmap v377: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:23:24.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:24 smithi165 ceph-mon[36269]: pgmap v377: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:23:24.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:24 smithi027 ceph-mon[32179]: pgmap v377: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:23:25.372 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-21T03:23:25.382 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-05-21T03:23:25.382 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-21T03:23:25.639 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:23:25.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:25 smithi027 ceph-mon[32179]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:23:25.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:25 smithi039 ceph-mon[36429]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:23:25.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:25 smithi165 ceph-mon[36269]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:23:26.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:26 smithi039 ceph-mon[36429]: pgmap v378: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:23:26.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:26 smithi165 ceph-mon[36269]: pgmap v378: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:23:26.875 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:26 smithi027 ceph-mon[32179]: pgmap v378: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:23:27.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.395+0000 7f0848555700 1 -- 172.21.15.27:0/251241260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0840104030 msgr2=0x7f0840106420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:27.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.395+0000 7f0848555700 1 --2- 172.21.15.27:0/251241260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0840104030 0x7f0840106420 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f083c009990 tx=0x7f083c009ca0 comp rx=0 tx=0).stop 2024-05-21T03:23:27.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.395+0000 7f0848555700 1 -- 172.21.15.27:0/251241260 shutdown_connections 2024-05-21T03:23:27.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.395+0000 7f0848555700 1 --2- 172.21.15.27:0/251241260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0840109290 0x7f084010b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:27.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.395+0000 7f0848555700 1 --2- 172.21.15.27:0/251241260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0840104030 0x7f0840106420 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:27.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.395+0000 7f0848555700 1 --2- 172.21.15.27:0/251241260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0840106960 0x7f0840108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:27.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.395+0000 7f0848555700 1 -- 172.21.15.27:0/251241260 >> 172.21.15.27:0/251241260 conn(0x7f08400fda40 msgr2=0x7f08400ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:27.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.395+0000 7f0848555700 1 -- 172.21.15.27:0/251241260 shutdown_connections 2024-05-21T03:23:27.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.395+0000 7f0848555700 1 -- 172.21.15.27:0/251241260 wait complete. 2024-05-21T03:23:27.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.396+0000 7f0848555700 1 Processor -- start 2024-05-21T03:23:27.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.396+0000 7f0848555700 1 -- start start 2024-05-21T03:23:27.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.396+0000 7f0848555700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0840104030 0x7f0840199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:27.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.396+0000 7f0848555700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0840106960 0x7f0840199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:27.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.396+0000 7f0848555700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0840109290 0x7f08401a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:27.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.396+0000 7f0848555700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f084010f060 con 0x7f0840104030 2024-05-21T03:23:27.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.396+0000 7f0848555700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f084010eee0 con 0x7f0840106960 2024-05-21T03:23:27.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.396+0000 7f0848555700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f084010f1e0 con 0x7f0840109290 2024-05-21T03:23:27.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.396+0000 7f08462f1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0840104030 0x7f0840199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:27.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.396+0000 7f08462f1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0840104030 0x7f0840199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56326/0 (socket says 172.21.15.27:56326) 2024-05-21T03:23:27.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.396+0000 7f08462f1700 1 -- 172.21.15.27:0/1979440606 learned_addr learned my addr 172.21.15.27:0/1979440606 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:27.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.397+0000 7f0846af2700 1 --2- 172.21.15.27:0/1979440606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0840109290 0x7f08401a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:27.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.397+0000 7f0845af0700 1 --2- 172.21.15.27:0/1979440606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0840106960 0x7f0840199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:27.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.397+0000 7f08462f1700 1 -- 172.21.15.27:0/1979440606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0840109290 msgr2=0x7f08401a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:27.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.397+0000 7f08462f1700 1 --2- 172.21.15.27:0/1979440606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0840109290 0x7f08401a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:27.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.397+0000 7f08462f1700 1 -- 172.21.15.27:0/1979440606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0840106960 msgr2=0x7f0840199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:27.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.397+0000 7f08462f1700 1 --2- 172.21.15.27:0/1979440606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0840106960 0x7f0840199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:27.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.397+0000 7f08462f1700 1 -- 172.21.15.27:0/1979440606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08401a1380 con 0x7f0840104030 2024-05-21T03:23:27.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.397+0000 7f0846af2700 1 --2- 172.21.15.27:0/1979440606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0840109290 0x7f08401a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:23:27.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.397+0000 7f08462f1700 1 --2- 172.21.15.27:0/1979440606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0840104030 0x7f0840199720 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f083c0059d0 tx=0x7f083c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:27.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.397+0000 7f08337fe700 1 -- 172.21.15.27:0/1979440606 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c027070 con 0x7f0840104030 2024-05-21T03:23:27.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.397+0000 7f08337fe700 1 -- 172.21.15.27:0/1979440606 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f083c004080 con 0x7f0840104030 2024-05-21T03:23:27.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.397+0000 7f0848555700 1 -- 172.21.15.27:0/1979440606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08401a1610 con 0x7f0840104030 2024-05-21T03:23:27.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.397+0000 7f08337fe700 1 -- 172.21.15.27:0/1979440606 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c022cf0 con 0x7f0840104030 2024-05-21T03:23:27.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.397+0000 7f0848555700 1 -- 172.21.15.27:0/1979440606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08401a1a60 con 0x7f0840104030 2024-05-21T03:23:27.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.398+0000 7f0848555700 1 -- 172.21.15.27:0/1979440606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f084006a230 con 0x7f0840104030 2024-05-21T03:23:27.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.399+0000 7f08337fe700 1 -- 172.21.15.27:0/1979440606 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f083c02a070 con 0x7f0840104030 2024-05-21T03:23:27.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.400+0000 7f08337fe700 1 --2- 172.21.15.27:0/1979440606 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f082c0b2880 0x7f082c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:27.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.400+0000 7f0845af0700 1 --2- 172.21.15.27:0/1979440606 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f082c0b2880 0x7f082c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:27.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.400+0000 7f08337fe700 1 -- 172.21.15.27:0/1979440606 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f083c0e3920 con 0x7f0840104030 2024-05-21T03:23:27.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.401+0000 7f0845af0700 1 --2- 172.21.15.27:0/1979440606 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f082c0b2880 0x7f082c0b4d40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f083400bbd0 tx=0x7f083400b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:27.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.404+0000 7f08337fe700 1 -- 172.21.15.27:0/1979440606 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f083c031070 con 0x7f0840104030 2024-05-21T03:23:27.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.668+0000 7f0848555700 1 -- 172.21.15.27:0/1979440606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f0840064150 con 0x7f0840104030 2024-05-21T03:23:27.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.669+0000 7f08337fe700 1 -- 172.21.15.27:0/1979440606 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3086 (secure 0 0 0) 0x7f083c013070 con 0x7f0840104030 2024-05-21T03:23:27.668 INFO:teuthology.orchestra.run.smithi027.stdout:e28 2024-05-21T03:23:27.668 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-21T03:23:27.669 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-21T03:23:27.669 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2024-05-21T03:23:27.669 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:23:27.669 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2024-05-21T03:23:27.669 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2024-05-21T03:23:27.669 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 28 2024-05-21T03:23:27.669 INFO:teuthology.orchestra.run.smithi027.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-21T03:23:27.669 INFO:teuthology.orchestra.run.smithi027.stdout:created 2024-05-21T03:20:34.055985+0000 2024-05-21T03:23:27.669 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2024-05-21T03:20:52.467919+0000 2024-05-21T03:23:27.669 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2024-05-21T03:23:27.669 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2024-05-21T03:23:27.669 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 300 2024-05-21T03:23:27.669 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2024-05-21T03:23:27.669 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2024-05-21T03:23:27.670 INFO:teuthology.orchestra.run.smithi027.stdout:max_xattr_size 65536 2024-05-21T03:23:27.670 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2024-05-21T03:23:27.670 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2024-05-21T03:23:27.670 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 0 2024-05-21T03:23:27.670 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-21T03:23:27.670 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 5 2024-05-21T03:23:27.670 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1,2,3,4 2024-05-21T03:23:27.670 INFO:teuthology.orchestra.run.smithi027.stdout:up {0=24476,1=24472,2=24457,3=24452,4=24451} 2024-05-21T03:23:27.670 INFO:teuthology.orchestra.run.smithi027.stdout:failed 2024-05-21T03:23:27.670 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2024-05-21T03:23:27.670 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2024-05-21T03:23:27.670 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2024-05-21T03:23:27.670 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2024-05-21T03:23:27.671 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2024-05-21T03:23:27.671 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2024-05-21T03:23:27.671 INFO:teuthology.orchestra.run.smithi027.stdout:bal_rank_mask -1 2024-05-21T03:23:27.671 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 0 2024-05-21T03:23:27.671 INFO:teuthology.orchestra.run.smithi027.stdout:qdb_cluster leader: 24476 members: 24451,24472,24452,24457,24476 2024-05-21T03:23:27.671 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.b{0:24476} state up:active seq 5 addr [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T03:23:27.671 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.e{0:24422} state up:standby-replay seq 1 addr [v2:172.21.15.39:6834/3497521608,v1:172.21.15.39:6835/3497521608] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T03:23:27.671 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.i{1:24472} state up:active seq 11 addr [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T03:23:27.671 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.l{1:24418} state up:standby-replay seq 1 addr [v2:172.21.15.165:6832/4176762898,v1:172.21.15.165:6833/4176762898] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T03:23:27.671 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.c{2:24457} state up:active seq 12 addr [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T03:23:27.671 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.k{2:24401} state up:standby-replay seq 1 addr [v2:172.21.15.39:6832/1102873763,v1:172.21.15.39:6833/1102873763] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T03:23:27.671 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.h{3:24452} state up:active seq 12 addr [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T03:23:27.671 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.a{3:14625} state up:standby-replay seq 1 addr [v2:172.21.15.27:6840/3982752351,v1:172.21.15.27:6841/3982752351] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T03:23:27.671 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.f{4:24451} state up:active seq 13 addr [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T03:23:27.671 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.g{4:14613} state up:standby-replay seq 1 addr [v2:172.21.15.27:6838/745486764,v1:172.21.15.27:6839/745486764] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T03:23:27.672 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:23:27.672 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:23:27.672 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2024-05-21T03:23:27.672 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:23:27.672 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.j{-1:14550} state up:standby seq 1 addr [v2:172.21.15.27:6834/148854869,v1:172.21.15.27:6835/148854869] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T03:23:27.672 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.d{-1:14562} state up:standby seq 1 addr [v2:172.21.15.27:6836/4145544730,v1:172.21.15.27:6837/4145544730] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T03:23:27.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.671+0000 7f0848555700 1 -- 172.21.15.27:0/1979440606 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f082c0b2880 msgr2=0x7f082c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:27.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.671+0000 7f0848555700 1 --2- 172.21.15.27:0/1979440606 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f082c0b2880 0x7f082c0b4d40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f083400bbd0 tx=0x7f083400b5b0 comp rx=0 tx=0).stop 2024-05-21T03:23:27.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.671+0000 7f0848555700 1 -- 172.21.15.27:0/1979440606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0840104030 msgr2=0x7f0840199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:27.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.671+0000 7f0848555700 1 --2- 172.21.15.27:0/1979440606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0840104030 0x7f0840199720 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f083c0059d0 tx=0x7f083c009f90 comp rx=0 tx=0).stop 2024-05-21T03:23:27.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.671+0000 7f0848555700 1 -- 172.21.15.27:0/1979440606 shutdown_connections 2024-05-21T03:23:27.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.671+0000 7f0848555700 1 --2- 172.21.15.27:0/1979440606 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f082c0b2880 0x7f082c0b4d40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:27.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.671+0000 7f0848555700 1 --2- 172.21.15.27:0/1979440606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0840109290 0x7f08401a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:27.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.671+0000 7f0848555700 1 --2- 172.21.15.27:0/1979440606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0840104030 0x7f0840199720 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:27.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.671+0000 7f0848555700 1 --2- 172.21.15.27:0/1979440606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0840106960 0x7f0840199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:27.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.671+0000 7f0848555700 1 -- 172.21.15.27:0/1979440606 >> 172.21.15.27:0/1979440606 conn(0x7f08400fda40 msgr2=0x7f0840104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:27.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.671+0000 7f0848555700 1 -- 172.21.15.27:0/1979440606 shutdown_connections 2024-05-21T03:23:27.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:27.671+0000 7f0848555700 1 -- 172.21.15.27:0/1979440606 wait complete. 2024-05-21T03:23:27.674 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 28 2024-05-21T03:23:28.345 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-21T03:23:28.598 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:23:28.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:28 smithi027 ceph-mon[32179]: pgmap v379: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:23:28.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:28 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/1979440606' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-21T03:23:28.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:28 smithi039 ceph-mon[36429]: pgmap v379: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:23:28.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:28 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/1979440606' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-21T03:23:28.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:28 smithi165 ceph-mon[36269]: pgmap v379: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:23:28.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:28 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/1979440606' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-21T03:23:30.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.416+0000 7f3fadb93700 1 -- 172.21.15.27:0/18793231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fa81069e0 msgr2=0x7f3fa8108dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:30.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.416+0000 7f3fadb93700 1 --2- 172.21.15.27:0/18793231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fa81069e0 0x7f3fa8108dd0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009990 tx=0x7f3f98009ca0 comp rx=0 tx=0).stop 2024-05-21T03:23:30.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.416+0000 7f3fadb93700 1 -- 172.21.15.27:0/18793231 shutdown_connections 2024-05-21T03:23:30.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.416+0000 7f3fadb93700 1 --2- 172.21.15.27:0/18793231 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fa8109310 0x7f3fa810b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:30.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.416+0000 7f3fadb93700 1 --2- 172.21.15.27:0/18793231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fa81040b0 0x7f3fa81064a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:30.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.416+0000 7f3fadb93700 1 --2- 172.21.15.27:0/18793231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fa81069e0 0x7f3fa8108dd0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:30.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.416+0000 7f3fadb93700 1 -- 172.21.15.27:0/18793231 >> 172.21.15.27:0/18793231 conn(0x7f3fa80fdaa0 msgr2=0x7f3fa80ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:30.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.416+0000 7f3fadb93700 1 -- 172.21.15.27:0/18793231 shutdown_connections 2024-05-21T03:23:30.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.416+0000 7f3fadb93700 1 -- 172.21.15.27:0/18793231 wait complete. 2024-05-21T03:23:30.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.416+0000 7f3fadb93700 1 Processor -- start 2024-05-21T03:23:30.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.417+0000 7f3fadb93700 1 -- start start 2024-05-21T03:23:30.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.417+0000 7f3fadb93700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fa81040b0 0x7f3fa81996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:30.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.417+0000 7f3fadb93700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fa81069e0 0x7f3fa8199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:30.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.417+0000 7f3fadb93700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fa8109310 0x7f3fa81a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:30.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.417+0000 7f3fadb93700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa810f080 con 0x7f3fa81040b0 2024-05-21T03:23:30.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.417+0000 7f3fadb93700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa810ef00 con 0x7f3fa81069e0 2024-05-21T03:23:30.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.417+0000 7f3fadb93700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa810f200 con 0x7f3fa8109310 2024-05-21T03:23:30.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.417+0000 7f3fa77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fa81040b0 0x7f3fa81996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:30.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.417+0000 7f3fa77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fa81040b0 0x7f3fa81996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56342/0 (socket says 172.21.15.27:56342) 2024-05-21T03:23:30.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.417+0000 7f3fa77fe700 1 -- 172.21.15.27:0/1785007220 learned_addr learned my addr 172.21.15.27:0/1785007220 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:30.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.417+0000 7f3fa6ffd700 1 --2- 172.21.15.27:0/1785007220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fa81069e0 0x7f3fa8199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:30.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.417+0000 7f3fa7fff700 1 --2- 172.21.15.27:0/1785007220 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fa8109310 0x7f3fa81a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:30.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.417+0000 7f3fa77fe700 1 -- 172.21.15.27:0/1785007220 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fa8109310 msgr2=0x7f3fa81a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:30.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.417+0000 7f3fa77fe700 1 --2- 172.21.15.27:0/1785007220 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fa8109310 0x7f3fa81a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:30.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.417+0000 7f3fa77fe700 1 -- 172.21.15.27:0/1785007220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fa81069e0 msgr2=0x7f3fa8199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:30.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.417+0000 7f3fa77fe700 1 --2- 172.21.15.27:0/1785007220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fa81069e0 0x7f3fa8199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:30.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.417+0000 7f3fa77fe700 1 -- 172.21.15.27:0/1785007220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa81a12d0 con 0x7f3fa81040b0 2024-05-21T03:23:30.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.417+0000 7f3fa77fe700 1 --2- 172.21.15.27:0/1785007220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fa81040b0 0x7f3fa81996e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c00c950 tx=0x7f3f9c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:30.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.418+0000 7f3fa4ff9700 1 -- 172.21.15.27:0/1785007220 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c007ac0 con 0x7f3fa81040b0 2024-05-21T03:23:30.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.418+0000 7f3fa4ff9700 1 -- 172.21.15.27:0/1785007220 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f9c004ce0 con 0x7f3fa81040b0 2024-05-21T03:23:30.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.418+0000 7f3fadb93700 1 -- 172.21.15.27:0/1785007220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fa81a15c0 con 0x7f3fa81040b0 2024-05-21T03:23:30.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.418+0000 7f3fa4ff9700 1 -- 172.21.15.27:0/1785007220 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c005580 con 0x7f3fa81040b0 2024-05-21T03:23:30.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.418+0000 7f3fadb93700 1 -- 172.21.15.27:0/1785007220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fa81a1b00 con 0x7f3fa81040b0 2024-05-21T03:23:30.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.419+0000 7f3fadb93700 1 -- 172.21.15.27:0/1785007220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fa806a230 con 0x7f3fa81040b0 2024-05-21T03:23:30.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.420+0000 7f3fa4ff9700 1 -- 172.21.15.27:0/1785007220 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3f9c007c60 con 0x7f3fa81040b0 2024-05-21T03:23:30.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.421+0000 7f3fa4ff9700 1 --2- 172.21.15.27:0/1785007220 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f3f900b2880 0x7f3f900b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:30.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.421+0000 7f3fa6ffd700 1 --2- 172.21.15.27:0/1785007220 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f3f900b2880 0x7f3f900b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:30.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.421+0000 7f3fa4ff9700 1 -- 172.21.15.27:0/1785007220 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f3f9c0e1d90 con 0x7f3fa81040b0 2024-05-21T03:23:30.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.422+0000 7f3fa6ffd700 1 --2- 172.21.15.27:0/1785007220 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f3f900b2880 0x7f3f900b4d40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800b5c0 tx=0x7f3f98005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:30.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.426+0000 7f3fa4ff9700 1 -- 172.21.15.27:0/1785007220 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3f9c014030 con 0x7f3fa81040b0 2024-05-21T03:23:30.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.641+0000 7f3fadb93700 1 -- 172.21.15.27:0/1785007220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f3fa8064150 con 0x7f3fa81040b0 2024-05-21T03:23:30.641 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:30 smithi027 ceph-mon[32179]: pgmap v380: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:23:30.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.642+0000 7f3fa4ff9700 1 -- 172.21.15.27:0/1785007220 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v82) ==== 56+0+5312 (secure 0 0 0) 0x7f3f9c0a9a90 con 0x7f3fa81040b0 2024-05-21T03:23:30.641 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 82 2024-05-21T03:23:30.641 INFO:teuthology.orchestra.run.smithi027.stdout:fsid c9ca94be-171f-11ef-bc9a-c7b262605968 2024-05-21T03:23:30.641 INFO:teuthology.orchestra.run.smithi027.stdout:created 2024-05-21T03:12:18.217236+0000 2024-05-21T03:23:30.642 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2024-05-21T03:21:14.540989+0000 2024-05-21T03:23:30.642 INFO:teuthology.orchestra.run.smithi027.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-21T03:23:30.642 INFO:teuthology.orchestra.run.smithi027.stdout:crush_version 26 2024-05-21T03:23:30.642 INFO:teuthology.orchestra.run.smithi027.stdout:full_ratio 0.95 2024-05-21T03:23:30.642 INFO:teuthology.orchestra.run.smithi027.stdout:backfillfull_ratio 0.9 2024-05-21T03:23:30.642 INFO:teuthology.orchestra.run.smithi027.stdout:nearfull_ratio 0.85 2024-05-21T03:23:30.642 INFO:teuthology.orchestra.run.smithi027.stdout:require_min_compat_client luminous 2024-05-21T03:23:30.642 INFO:teuthology.orchestra.run.smithi027.stdout:min_compat_client luminous 2024-05-21T03:23:30.642 INFO:teuthology.orchestra.run.smithi027.stdout:require_osd_release squid 2024-05-21T03:23:30.642 INFO:teuthology.orchestra.run.smithi027.stdout:stretch_mode_enabled false 2024-05-21T03:23:30.642 INFO:teuthology.orchestra.run.smithi027.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-21T03:23:30.642 INFO:teuthology.orchestra.run.smithi027.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-21T03:23:30.643 INFO:teuthology.orchestra.run.smithi027.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-21T03:23:30.643 INFO:teuthology.orchestra.run.smithi027.stdout:max_osd 12 2024-05-21T03:23:30.643 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 up in weight 1 up_from 8 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6802/1399996449,v1:172.21.15.27:6803/1399996449] [v2:172.21.15.27:6804/1399996449,v1:172.21.15.27:6805/1399996449] exists,up c745c09f-9a76-4d05-af75-c7554f6d5e6e 2024-05-21T03:23:30.643 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 up in weight 1 up_from 13 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6800/1776318847,v1:172.21.15.39:6801/1776318847] [v2:172.21.15.39:6802/1776318847,v1:172.21.15.39:6803/1776318847] exists,up 8b73aa00-4364-438f-9722-53caa4e9bec5 2024-05-21T03:23:30.643 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 up in weight 1 up_from 18 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6800/703322625,v1:172.21.15.165:6801/703322625] [v2:172.21.15.165:6802/703322625,v1:172.21.15.165:6803/703322625] exists,up 0822043e-7d3c-4e98-9dc0-94b98b4d562a 2024-05-21T03:23:30.643 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 up in weight 1 up_from 26 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6810/1933403135,v1:172.21.15.27:6811/1933403135] [v2:172.21.15.27:6812/1933403135,v1:172.21.15.27:6813/1933403135] exists,up c563b1b2-efcb-4f8c-8121-c1938f83131e 2024-05-21T03:23:30.643 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 up in weight 1 up_from 32 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6808/155579331,v1:172.21.15.39:6809/155579331] [v2:172.21.15.39:6810/155579331,v1:172.21.15.39:6811/155579331] exists,up 0c793b87-b045-4b4a-89e7-87ec36deeb0e 2024-05-21T03:23:30.643 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 up in weight 1 up_from 38 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6808/386850244,v1:172.21.15.165:6809/386850244] [v2:172.21.15.165:6810/386850244,v1:172.21.15.165:6811/386850244] exists,up 367eeb5a-9d75-4644-bf46-317c03f33d8a 2024-05-21T03:23:30.643 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6 up in weight 1 up_from 44 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6818/3531376679,v1:172.21.15.27:6819/3531376679] [v2:172.21.15.27:6820/3531376679,v1:172.21.15.27:6821/3531376679] exists,up e000167f-24fd-4115-83f0-8afbfa8d8991 2024-05-21T03:23:30.643 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7 up in weight 1 up_from 49 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6816/1550035852,v1:172.21.15.39:6817/1550035852] [v2:172.21.15.39:6818/1550035852,v1:172.21.15.39:6819/1550035852] exists,up 4b8d486b-868e-4f33-98a1-b99686adeaaf 2024-05-21T03:23:30.643 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8 up in weight 1 up_from 55 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6816/3184704971,v1:172.21.15.165:6817/3184704971] [v2:172.21.15.165:6818/3184704971,v1:172.21.15.165:6819/3184704971] exists,up 2259f35c-2edd-4331-9a63-8c0c1acbdc71 2024-05-21T03:23:30.643 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9 up in weight 1 up_from 61 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6826/2800048793,v1:172.21.15.27:6827/2800048793] [v2:172.21.15.27:6828/2800048793,v1:172.21.15.27:6829/2800048793] exists,up 10307ecc-8270-4be2-b62b-9708e09bb207 2024-05-21T03:23:30.644 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10 up in weight 1 up_from 66 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6824/363500811,v1:172.21.15.39:6825/363500811] [v2:172.21.15.39:6826/363500811,v1:172.21.15.39:6827/363500811] exists,up dfe1b8ca-ff8a-4b7f-ade5-ad4f00cf60cf 2024-05-21T03:23:30.644 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11 up in weight 1 up_from 72 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6824/2274332243,v1:172.21.15.165:6825/2274332243] [v2:172.21.15.165:6826/2274332243,v1:172.21.15.165:6827/2274332243] exists,up f9531145-5d09-4dbd-a050-3159ee09db24 2024-05-21T03:23:30.644 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.4 [2,11] 2024-05-21T03:23:30.644 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.c [2,4] 2024-05-21T03:23:30.644 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.1d [8,11] 2024-05-21T03:23:30.644 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.6 [3,8] 2024-05-21T03:23:30.644 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.c [3,9] 2024-05-21T03:23:30.644 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.14 [4,7] 2024-05-21T03:23:30.644 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.17 [4,8] 2024-05-21T03:23:30.644 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.1b [5,9] 2024-05-21T03:23:30.644 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.20 [5,9] 2024-05-21T03:23:30.644 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.31 [4,6] 2024-05-21T03:23:30.644 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6800/1185518061 expires 2024-05-22T03:13:13.249220+0000 2024-05-21T03:23:30.644 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6801/1185518061 expires 2024-05-22T03:13:13.249220+0000 2024-05-21T03:23:30.645 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/4194431212 expires 2024-05-22T03:13:13.249220+0000 2024-05-21T03:23:30.645 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/1890032101 expires 2024-05-22T03:13:13.249220+0000 2024-05-21T03:23:30.645 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6800/175229478 expires 2024-05-22T03:12:45.739195+0000 2024-05-21T03:23:30.645 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/999325706 expires 2024-05-22T03:12:45.739195+0000 2024-05-21T03:23:30.645 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6801/175229478 expires 2024-05-22T03:12:45.739195+0000 2024-05-21T03:23:30.645 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/4184128736 expires 2024-05-22T03:13:13.249220+0000 2024-05-21T03:23:30.645 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/3000197506 expires 2024-05-22T03:12:45.739195+0000 2024-05-21T03:23:30.645 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/972677319 expires 2024-05-22T03:12:45.739195+0000 2024-05-21T03:23:30.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.644+0000 7f3fadb93700 1 -- 172.21.15.27:0/1785007220 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f3f900b2880 msgr2=0x7f3f900b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:30.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.644+0000 7f3fadb93700 1 --2- 172.21.15.27:0/1785007220 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f3f900b2880 0x7f3f900b4d40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800b5c0 tx=0x7f3f98005fb0 comp rx=0 tx=0).stop 2024-05-21T03:23:30.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.644+0000 7f3fadb93700 1 -- 172.21.15.27:0/1785007220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fa81040b0 msgr2=0x7f3fa81996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:30.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.644+0000 7f3fadb93700 1 --2- 172.21.15.27:0/1785007220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fa81040b0 0x7f3fa81996e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c00c950 tx=0x7f3f9c00cc60 comp rx=0 tx=0).stop 2024-05-21T03:23:30.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.644+0000 7f3fadb93700 1 -- 172.21.15.27:0/1785007220 shutdown_connections 2024-05-21T03:23:30.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.644+0000 7f3fadb93700 1 --2- 172.21.15.27:0/1785007220 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f3f900b2880 0x7f3f900b4d40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:30.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.644+0000 7f3fadb93700 1 --2- 172.21.15.27:0/1785007220 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fa8109310 0x7f3fa81a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:30.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.644+0000 7f3fadb93700 1 --2- 172.21.15.27:0/1785007220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fa81040b0 0x7f3fa81996e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:30.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.644+0000 7f3fadb93700 1 --2- 172.21.15.27:0/1785007220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fa81069e0 0x7f3fa8199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:30.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.644+0000 7f3fadb93700 1 -- 172.21.15.27:0/1785007220 >> 172.21.15.27:0/1785007220 conn(0x7f3fa80fdaa0 msgr2=0x7f3fa8104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:30.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.644+0000 7f3fadb93700 1 -- 172.21.15.27:0/1785007220 shutdown_connections 2024-05-21T03:23:30.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:30.644+0000 7f3fadb93700 1 -- 172.21.15.27:0/1785007220 wait complete. 2024-05-21T03:23:30.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:30 smithi165 ceph-mon[36269]: pgmap v380: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:23:31.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:30 smithi039 ceph-mon[36429]: pgmap v380: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:23:31.408 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-21T03:23:31.420 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:23:31.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:31 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/1785007220' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-21T03:23:31.679 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:23:31.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:31 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/1785007220' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-21T03:23:31.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:31 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/1785007220' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-21T03:23:32.892 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:32 smithi027 ceph-mon[32179]: pgmap v381: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:23:33.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:32 smithi039 ceph-mon[36429]: pgmap v381: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:23:33.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:32 smithi165 ceph-mon[36269]: pgmap v381: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:23:33.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.247+0000 7fe763820700 1 -- 172.21.15.27:0/2260330631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe75c108520 msgr2=0x7fe75c108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:33.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.247+0000 7fe763820700 1 --2- 172.21.15.27:0/2260330631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe75c108520 0x7fe75c108900 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00b0a0 tx=0x7fe74c00b3b0 comp rx=0 tx=0).stop 2024-05-21T03:23:33.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.247+0000 7fe763820700 1 -- 172.21.15.27:0/2260330631 shutdown_connections 2024-05-21T03:23:33.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.247+0000 7fe763820700 1 --2- 172.21.15.27:0/2260330631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe75c102e00 0x7fe75c10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:33.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.247+0000 7fe763820700 1 --2- 172.21.15.27:0/2260330631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe75c108520 0x7fe75c108900 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:33.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.247+0000 7fe763820700 1 --2- 172.21.15.27:0/2260330631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75c102440 0x7fe75c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:33.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.247+0000 7fe763820700 1 -- 172.21.15.27:0/2260330631 >> 172.21.15.27:0/2260330631 conn(0x7fe75c0793f0 msgr2=0x7fe75c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:33.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.247+0000 7fe763820700 1 -- 172.21.15.27:0/2260330631 shutdown_connections 2024-05-21T03:23:33.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.247+0000 7fe763820700 1 -- 172.21.15.27:0/2260330631 wait complete. 2024-05-21T03:23:33.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.248+0000 7fe763820700 1 Processor -- start 2024-05-21T03:23:33.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.248+0000 7fe763820700 1 -- start start 2024-05-21T03:23:33.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.248+0000 7fe763820700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe75c102440 0x7fe75c19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:33.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.249+0000 7fe763820700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe75c102e00 0x7fe75c19ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:33.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.249+0000 7fe763820700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75c108520 0x7fe75c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:33.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.249+0000 7fe763820700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c1125f0 con 0x7fe75c102e00 2024-05-21T03:23:33.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.249+0000 7fe763820700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c112470 con 0x7fe75c108520 2024-05-21T03:23:33.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.249+0000 7fe763820700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c112770 con 0x7fe75c102440 2024-05-21T03:23:33.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.249+0000 7fe760dbb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe75c102e00 0x7fe75c19ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:33.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.249+0000 7fe760dbb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe75c102e00 0x7fe75c19ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33380/0 (socket says 172.21.15.27:33380) 2024-05-21T03:23:33.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.249+0000 7fe760dbb700 1 -- 172.21.15.27:0/365986685 learned_addr learned my addr 172.21.15.27:0/365986685 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:33.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.249+0000 7fe761dbd700 1 --2- 172.21.15.27:0/365986685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75c108520 0x7fe75c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:33.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.249+0000 7fe7615bc700 1 --2- 172.21.15.27:0/365986685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe75c102440 0x7fe75c19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:33.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.249+0000 7fe760dbb700 1 -- 172.21.15.27:0/365986685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe75c102440 msgr2=0x7fe75c19e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:33.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.249+0000 7fe760dbb700 1 --2- 172.21.15.27:0/365986685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe75c102440 0x7fe75c19e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:33.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.249+0000 7fe760dbb700 1 -- 172.21.15.27:0/365986685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75c108520 msgr2=0x7fe75c198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:33.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.249+0000 7fe760dbb700 1 --2- 172.21.15.27:0/365986685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75c108520 0x7fe75c198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:33.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.249+0000 7fe760dbb700 1 -- 172.21.15.27:0/365986685 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe75c199070 con 0x7fe75c102e00 2024-05-21T03:23:33.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.249+0000 7fe760dbb700 1 --2- 172.21.15.27:0/365986685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe75c102e00 0x7fe75c19ec90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe75000bfa0 tx=0x7fe75000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:33.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.249+0000 7fe75a7fc700 1 -- 172.21.15.27:0/365986685 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75001a070 con 0x7fe75c102e00 2024-05-21T03:23:33.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.249+0000 7fe75a7fc700 1 -- 172.21.15.27:0/365986685 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe750009bb0 con 0x7fe75c102e00 2024-05-21T03:23:33.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.249+0000 7fe763820700 1 -- 172.21.15.27:0/365986685 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe75c199360 con 0x7fe75c102e00 2024-05-21T03:23:33.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.249+0000 7fe75a7fc700 1 -- 172.21.15.27:0/365986685 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75000f8f0 con 0x7fe75c102e00 2024-05-21T03:23:33.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.249+0000 7fe763820700 1 -- 172.21.15.27:0/365986685 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe75c06b220 con 0x7fe75c102e00 2024-05-21T03:23:33.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.251+0000 7fe763820700 1 -- 172.21.15.27:0/365986685 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe75c06a230 con 0x7fe75c102e00 2024-05-21T03:23:33.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.252+0000 7fe75a7fc700 1 -- 172.21.15.27:0/365986685 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe75001d070 con 0x7fe75c102e00 2024-05-21T03:23:33.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.253+0000 7fe75a7fc700 1 --2- 172.21.15.27:0/365986685 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe7440b2900 0x7fe7440b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:33.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.253+0000 7fe7615bc700 1 --2- 172.21.15.27:0/365986685 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe7440b2900 0x7fe7440b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:33.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.253+0000 7fe75a7fc700 1 -- 172.21.15.27:0/365986685 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fe7500e61c0 con 0x7fe75c102e00 2024-05-21T03:23:33.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.253+0000 7fe7615bc700 1 --2- 172.21.15.27:0/365986685 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe7440b2900 0x7fe7440b4dc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00b830 tx=0x7fe74c009be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:33.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.256+0000 7fe75a7fc700 1 -- 172.21.15.27:0/365986685 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe7500adec0 con 0x7fe75c102e00 2024-05-21T03:23:33.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.517+0000 7fe763820700 1 -- 172.21.15.27:0/365986685 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe75c199e40 con 0x7fe75c102e00 2024-05-21T03:23:33.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.518+0000 7fe75a7fc700 1 -- 172.21.15.27:0/365986685 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11924 (secure 0 0 0) 0x7fe7500ad7c0 con 0x7fe75c102e00 2024-05-21T03:23:33.519 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:23:33.519 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/148854869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":148854869},{"type":"v1","addr":"172.21.15.27:6835","nonce":148854869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/4145544730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":4145544730},{"type":"v1","addr":"172.21.15.27:6837","nonce":4145544730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T03:20:34.055985+0000","modified":"2024-05-21T03:20:52.467919+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24472,"mds_2":24457,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/745486764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":745486764},{"type":"v1","addr":"172.21.15.27:6839","nonce":745486764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6841/3982752351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3982752351},{"type":"v1","addr":"172.21.15.27:6841","nonce":3982752351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/3497521608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3497521608},{"type":"v1","addr":"172.21.15.39:6835","nonce":3497521608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.165:6835/3576910218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3576910218},{"type":"v1","addr":"172.21.15.165:6835","nonce":3576910218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/3673846852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3673846852},{"type":"v1","addr":"172.21.15.39:6837","nonce":3673846852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6837/691356139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":691356139},{"type":"v1","addr":"172.21.15.165:6837","nonce":691356139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6839/4158170814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4158170814},{"type":"v1","addr":"172.21.15.165:6839","nonce":4158170814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6839/287607819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":287607819},{"type":"v1","addr":"172.21.15.39:6839","nonce":287607819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24451,24472,24452,24457,24476]},"id":1}]} 2024-05-21T03:23:33.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.520+0000 7fe763820700 1 -- 172.21.15.27:0/365986685 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe7440b2900 msgr2=0x7fe7440b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:33.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.520+0000 7fe763820700 1 --2- 172.21.15.27:0/365986685 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe7440b2900 0x7fe7440b4dc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00b830 tx=0x7fe74c009be0 comp rx=0 tx=0).stop 2024-05-21T03:23:33.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.520+0000 7fe763820700 1 -- 172.21.15.27:0/365986685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe75c102e00 msgr2=0x7fe75c19ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:33.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.520+0000 7fe763820700 1 --2- 172.21.15.27:0/365986685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe75c102e00 0x7fe75c19ec90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe75000bfa0 tx=0x7fe75000dc60 comp rx=0 tx=0).stop 2024-05-21T03:23:33.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.520+0000 7fe763820700 1 -- 172.21.15.27:0/365986685 shutdown_connections 2024-05-21T03:23:33.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.520+0000 7fe763820700 1 --2- 172.21.15.27:0/365986685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75c108520 0x7fe75c198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:33.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.520+0000 7fe763820700 1 --2- 172.21.15.27:0/365986685 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe7440b2900 0x7fe7440b4dc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:33.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.520+0000 7fe763820700 1 --2- 172.21.15.27:0/365986685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe75c102440 0x7fe75c19e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:33.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.520+0000 7fe763820700 1 --2- 172.21.15.27:0/365986685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe75c102e00 0x7fe75c19ec90 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:33.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.520+0000 7fe763820700 1 -- 172.21.15.27:0/365986685 >> 172.21.15.27:0/365986685 conn(0x7fe75c0793f0 msgr2=0x7fe75c101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:33.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.521+0000 7fe763820700 1 -- 172.21.15.27:0/365986685 shutdown_connections 2024-05-21T03:23:33.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:33.521+0000 7fe763820700 1 -- 172.21.15.27:0/365986685 wait complete. 2024-05-21T03:23:33.522 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 28 2024-05-21T03:23:33.901 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:33 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/365986685' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:23:34.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:33 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/365986685' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:23:34.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:33 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/365986685' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:23:34.304 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:23:34.558 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:23:34.815 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:34 smithi027 ceph-mon[32179]: pgmap v382: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:23:35.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:34 smithi039 ceph-mon[36429]: pgmap v382: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:23:35.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:34 smithi165 ceph-mon[36269]: pgmap v382: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:23:36.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.249+0000 7fdcaf2d1700 1 -- 172.21.15.27:0/571395156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdca8108470 msgr2=0x7fdca806b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:36.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.249+0000 7fdcaf2d1700 1 --2- 172.21.15.27:0/571395156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdca8108470 0x7fdca806b250 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdca4009990 tx=0x7fdca4009ca0 comp rx=0 tx=0).stop 2024-05-21T03:23:36.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.249+0000 7fdcaf2d1700 1 -- 172.21.15.27:0/571395156 shutdown_connections 2024-05-21T03:23:36.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.249+0000 7fdcaf2d1700 1 --2- 172.21.15.27:0/571395156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdca8105f80 0x7fdca8106410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:36.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.249+0000 7fdcaf2d1700 1 --2- 172.21.15.27:0/571395156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdca8108470 0x7fdca806b250 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:36.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.249+0000 7fdcaf2d1700 1 --2- 172.21.15.27:0/571395156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdca806b790 0x7fdca806bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:36.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.249+0000 7fdcaf2d1700 1 -- 172.21.15.27:0/571395156 >> 172.21.15.27:0/571395156 conn(0x7fdca80793f0 msgr2=0x7fdca8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:36.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.249+0000 7fdcaf2d1700 1 -- 172.21.15.27:0/571395156 shutdown_connections 2024-05-21T03:23:36.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.249+0000 7fdcaf2d1700 1 -- 172.21.15.27:0/571395156 wait complete. 2024-05-21T03:23:36.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.249+0000 7fdcaf2d1700 1 Processor -- start 2024-05-21T03:23:36.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.250+0000 7fdcaf2d1700 1 -- start start 2024-05-21T03:23:36.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.250+0000 7fdcaf2d1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdca806b790 0x7fdca81a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:36.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.250+0000 7fdcaf2d1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdca8105f80 0x7fdca81a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:36.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.250+0000 7fdcaf2d1700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdca8108470 0x7fdca81a4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:36.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.250+0000 7fdcaf2d1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdca8119cb0 con 0x7fdca806b790 2024-05-21T03:23:36.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.250+0000 7fdcaf2d1700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdca8119b30 con 0x7fdca8105f80 2024-05-21T03:23:36.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.250+0000 7fdcaf2d1700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdca8119e30 con 0x7fdca8108470 2024-05-21T03:23:36.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.250+0000 7fdcad06d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdca806b790 0x7fdca81a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:36.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.250+0000 7fdcad06d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdca806b790 0x7fdca81a0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33406/0 (socket says 172.21.15.27:33406) 2024-05-21T03:23:36.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.250+0000 7fdcad06d700 1 -- 172.21.15.27:0/1467258126 learned_addr learned my addr 172.21.15.27:0/1467258126 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:36.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.250+0000 7fdcad86e700 1 --2- 172.21.15.27:0/1467258126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdca8108470 0x7fdca81a4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:36.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.250+0000 7fdcad06d700 1 -- 172.21.15.27:0/1467258126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdca8108470 msgr2=0x7fdca81a4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:36.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.250+0000 7fdcac86c700 1 --2- 172.21.15.27:0/1467258126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdca8105f80 0x7fdca81a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:36.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.250+0000 7fdcad06d700 1 --2- 172.21.15.27:0/1467258126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdca8108470 0x7fdca81a4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:36.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.250+0000 7fdcad06d700 1 -- 172.21.15.27:0/1467258126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdca8105f80 msgr2=0x7fdca81a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:36.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.250+0000 7fdcad06d700 1 --2- 172.21.15.27:0/1467258126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdca8105f80 0x7fdca81a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:36.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.250+0000 7fdcad06d700 1 -- 172.21.15.27:0/1467258126 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca81a5170 con 0x7fdca806b790 2024-05-21T03:23:36.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.250+0000 7fdcad86e700 1 --2- 172.21.15.27:0/1467258126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdca8108470 0x7fdca81a4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:23:36.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.250+0000 7fdcad06d700 1 --2- 172.21.15.27:0/1467258126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdca806b790 0x7fdca81a0160 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdca40057c0 tx=0x7fdca40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:36.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.251+0000 7fdc9e7fc700 1 -- 172.21.15.27:0/1467258126 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca4027070 con 0x7fdca806b790 2024-05-21T03:23:36.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.251+0000 7fdc9e7fc700 1 -- 172.21.15.27:0/1467258126 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdca4004020 con 0x7fdca806b790 2024-05-21T03:23:36.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.251+0000 7fdcaf2d1700 1 -- 172.21.15.27:0/1467258126 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca81a5400 con 0x7fdca806b790 2024-05-21T03:23:36.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.251+0000 7fdc9e7fc700 1 -- 172.21.15.27:0/1467258126 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca4022bf0 con 0x7fdca806b790 2024-05-21T03:23:36.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.251+0000 7fdcaf2d1700 1 -- 172.21.15.27:0/1467258126 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca81acce0 con 0x7fdca806b790 2024-05-21T03:23:36.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.252+0000 7fdcaf2d1700 1 -- 172.21.15.27:0/1467258126 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdca8102a60 con 0x7fdca806b790 2024-05-21T03:23:36.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.253+0000 7fdc9e7fc700 1 -- 172.21.15.27:0/1467258126 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdca402a070 con 0x7fdca806b790 2024-05-21T03:23:36.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.254+0000 7fdc9e7fc700 1 --2- 172.21.15.27:0/1467258126 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fdc940b2880 0x7fdc940b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:36.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.254+0000 7fdcac86c700 1 --2- 172.21.15.27:0/1467258126 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fdc940b2880 0x7fdc940b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:36.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.254+0000 7fdc9e7fc700 1 -- 172.21.15.27:0/1467258126 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fdca40e3140 con 0x7fdca806b790 2024-05-21T03:23:36.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.254+0000 7fdcac86c700 1 --2- 172.21.15.27:0/1467258126 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fdc940b2880 0x7fdc940b4d40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdc98005950 tx=0x7fdc980058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:36.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.259+0000 7fdc9e7fc700 1 -- 172.21.15.27:0/1467258126 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdca4013070 con 0x7fdca806b790 2024-05-21T03:23:36.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.522+0000 7fdcaf2d1700 1 -- 172.21.15.27:0/1467258126 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdca81aceb0 con 0x7fdca806b790 2024-05-21T03:23:36.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.522+0000 7fdc9e7fc700 1 -- 172.21.15.27:0/1467258126 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11924 (secure 0 0 0) 0x7fdca40041d0 con 0x7fdca806b790 2024-05-21T03:23:36.523 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:23:36.523 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/148854869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":148854869},{"type":"v1","addr":"172.21.15.27:6835","nonce":148854869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/4145544730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":4145544730},{"type":"v1","addr":"172.21.15.27:6837","nonce":4145544730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T03:20:34.055985+0000","modified":"2024-05-21T03:20:52.467919+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24472,"mds_2":24457,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/745486764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":745486764},{"type":"v1","addr":"172.21.15.27:6839","nonce":745486764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6841/3982752351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3982752351},{"type":"v1","addr":"172.21.15.27:6841","nonce":3982752351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/3497521608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3497521608},{"type":"v1","addr":"172.21.15.39:6835","nonce":3497521608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.165:6835/3576910218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3576910218},{"type":"v1","addr":"172.21.15.165:6835","nonce":3576910218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/3673846852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3673846852},{"type":"v1","addr":"172.21.15.39:6837","nonce":3673846852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6837/691356139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":691356139},{"type":"v1","addr":"172.21.15.165:6837","nonce":691356139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6839/4158170814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4158170814},{"type":"v1","addr":"172.21.15.165:6839","nonce":4158170814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6839/287607819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":287607819},{"type":"v1","addr":"172.21.15.39:6839","nonce":287607819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24451,24472,24452,24457,24476]},"id":1}]} 2024-05-21T03:23:36.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.525+0000 7fdcaf2d1700 1 -- 172.21.15.27:0/1467258126 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fdc940b2880 msgr2=0x7fdc940b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:36.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.525+0000 7fdcaf2d1700 1 --2- 172.21.15.27:0/1467258126 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fdc940b2880 0x7fdc940b4d40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdc98005950 tx=0x7fdc980058e0 comp rx=0 tx=0).stop 2024-05-21T03:23:36.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.525+0000 7fdcaf2d1700 1 -- 172.21.15.27:0/1467258126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdca806b790 msgr2=0x7fdca81a0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:36.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.525+0000 7fdcaf2d1700 1 --2- 172.21.15.27:0/1467258126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdca806b790 0x7fdca81a0160 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdca40057c0 tx=0x7fdca40057f0 comp rx=0 tx=0).stop 2024-05-21T03:23:36.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.525+0000 7fdcaf2d1700 1 -- 172.21.15.27:0/1467258126 shutdown_connections 2024-05-21T03:23:36.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.525+0000 7fdcaf2d1700 1 --2- 172.21.15.27:0/1467258126 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fdc940b2880 0x7fdc940b4d40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:36.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.525+0000 7fdcaf2d1700 1 --2- 172.21.15.27:0/1467258126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdca8108470 0x7fdca81a4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:36.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.525+0000 7fdcaf2d1700 1 --2- 172.21.15.27:0/1467258126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdca806b790 0x7fdca81a0160 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:36.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.525+0000 7fdcaf2d1700 1 --2- 172.21.15.27:0/1467258126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdca8105f80 0x7fdca81a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:36.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.525+0000 7fdcaf2d1700 1 -- 172.21.15.27:0/1467258126 >> 172.21.15.27:0/1467258126 conn(0x7fdca80793f0 msgr2=0x7fdca8101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:36.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.525+0000 7fdcaf2d1700 1 -- 172.21.15.27:0/1467258126 shutdown_connections 2024-05-21T03:23:36.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:36.525+0000 7fdcaf2d1700 1 -- 172.21.15.27:0/1467258126 wait complete. 2024-05-21T03:23:36.527 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 28 2024-05-21T03:23:36.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:36 smithi027 ceph-mon[32179]: pgmap v383: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:23:36.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:36 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/1467258126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:23:37.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:36 smithi039 ceph-mon[36429]: pgmap v383: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:23:37.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:36 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/1467258126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:23:37.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:36 smithi165 ceph-mon[36269]: pgmap v383: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:23:37.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:36 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/1467258126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:23:37.107 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-21T03:23:37.361 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:23:38.883 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:38 smithi027 ceph-mon[32179]: pgmap v384: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:23:39.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:38 smithi039 ceph-mon[36429]: pgmap v384: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:23:39.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:38 smithi165 ceph-mon[36269]: pgmap v384: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:23:39.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.132+0000 7f495da87700 1 -- 172.21.15.27:0/1133445407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49581096a0 msgr2=0x7f4958109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:39.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.132+0000 7f495da87700 1 --2- 172.21.15.27:0/1133445407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49581096a0 0x7f4958109a80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4948009990 tx=0x7f4948009ca0 comp rx=0 tx=0).stop 2024-05-21T03:23:39.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.132+0000 7f495da87700 1 -- 172.21.15.27:0/1133445407 shutdown_connections 2024-05-21T03:23:39.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.132+0000 7f495da87700 1 --2- 172.21.15.27:0/1133445407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4958103f50 0x7f4958110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:39.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.132+0000 7f495da87700 1 --2- 172.21.15.27:0/1133445407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49581096a0 0x7f4958109a80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:39.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.132+0000 7f495da87700 1 --2- 172.21.15.27:0/1133445407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4958103590 0x7f4958103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:39.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.132+0000 7f495da87700 1 -- 172.21.15.27:0/1133445407 >> 172.21.15.27:0/1133445407 conn(0x7f495807a6f0 msgr2=0x7f495807ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:39.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.132+0000 7f495da87700 1 -- 172.21.15.27:0/1133445407 shutdown_connections 2024-05-21T03:23:39.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.132+0000 7f495da87700 1 -- 172.21.15.27:0/1133445407 wait complete. 2024-05-21T03:23:39.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.133+0000 7f495da87700 1 Processor -- start 2024-05-21T03:23:39.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.133+0000 7f495da87700 1 -- start start 2024-05-21T03:23:39.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.133+0000 7f495da87700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4958103590 0x7f495819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:39.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.134+0000 7f495da87700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4958103f50 0x7f495819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:39.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.134+0000 7f495da87700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49581096a0 0x7f4958199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:39.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.134+0000 7f495da87700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4958113710 con 0x7f4958103590 2024-05-21T03:23:39.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.134+0000 7f495da87700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4958113590 con 0x7f4958103f50 2024-05-21T03:23:39.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.134+0000 7f495da87700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4958113890 con 0x7f49581096a0 2024-05-21T03:23:39.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.134+0000 7f49577fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4958103590 0x7f495819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:39.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.134+0000 7f49577fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4958103590 0x7f495819f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33412/0 (socket says 172.21.15.27:33412) 2024-05-21T03:23:39.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.134+0000 7f49577fe700 1 -- 172.21.15.27:0/838941006 learned_addr learned my addr 172.21.15.27:0/838941006 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:39.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.134+0000 7f4956ffd700 1 --2- 172.21.15.27:0/838941006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4958103f50 0x7f495819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:39.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.134+0000 7f4957fff700 1 --2- 172.21.15.27:0/838941006 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49581096a0 0x7f4958199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:39.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.134+0000 7f49577fe700 1 -- 172.21.15.27:0/838941006 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49581096a0 msgr2=0x7f4958199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:39.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.134+0000 7f49577fe700 1 --2- 172.21.15.27:0/838941006 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49581096a0 0x7f4958199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:39.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.134+0000 7f49577fe700 1 -- 172.21.15.27:0/838941006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4958103f50 msgr2=0x7f495819fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:39.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.134+0000 7f49577fe700 1 --2- 172.21.15.27:0/838941006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4958103f50 0x7f495819fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:39.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.134+0000 7f49577fe700 1 -- 172.21.15.27:0/838941006 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495819a170 con 0x7f4958103590 2024-05-21T03:23:39.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.134+0000 7f49577fe700 1 --2- 172.21.15.27:0/838941006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4958103590 0x7f495819f800 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f49480057f0 tx=0x7f4948003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:39.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.134+0000 7f4954ff9700 1 -- 172.21.15.27:0/838941006 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4948027070 con 0x7f4958103590 2024-05-21T03:23:39.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.135+0000 7f495da87700 1 -- 172.21.15.27:0/838941006 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495819a400 con 0x7f4958103590 2024-05-21T03:23:39.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.135+0000 7f4954ff9700 1 -- 172.21.15.27:0/838941006 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49480040a0 con 0x7f4958103590 2024-05-21T03:23:39.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.135+0000 7f4954ff9700 1 -- 172.21.15.27:0/838941006 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4948022dc0 con 0x7f4958103590 2024-05-21T03:23:39.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.135+0000 7f495da87700 1 -- 172.21.15.27:0/838941006 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4958075770 con 0x7f4958103590 2024-05-21T03:23:39.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.136+0000 7f495da87700 1 -- 172.21.15.27:0/838941006 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f495810baf0 con 0x7f4958103590 2024-05-21T03:23:39.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.137+0000 7f4954ff9700 1 -- 172.21.15.27:0/838941006 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f494802a070 con 0x7f4958103590 2024-05-21T03:23:39.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.138+0000 7f4954ff9700 1 --2- 172.21.15.27:0/838941006 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f49440b2880 0x7f49440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:39.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.138+0000 7f4956ffd700 1 --2- 172.21.15.27:0/838941006 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f49440b2880 0x7f49440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:39.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.138+0000 7f4954ff9700 1 -- 172.21.15.27:0/838941006 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f49480e3740 con 0x7f4958103590 2024-05-21T03:23:39.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.139+0000 7f4956ffd700 1 --2- 172.21.15.27:0/838941006 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f49440b2880 0x7f49440b4d40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f495807acf0 tx=0x7f494000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:39.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.142+0000 7f4954ff9700 1 -- 172.21.15.27:0/838941006 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4948031020 con 0x7f4958103590 2024-05-21T03:23:39.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.361+0000 7f495da87700 1 -- 172.21.15.27:0/838941006 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f495819aef0 con 0x7f4958103590 2024-05-21T03:23:39.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.362+0000 7f4954ff9700 1 -- 172.21.15.27:0/838941006 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22683 (secure 0 0 0) 0x7f4948013070 con 0x7f4958103590 2024-05-21T03:23:39.363 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:23:39.363 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":82,"fsid":"c9ca94be-171f-11ef-bc9a-c7b262605968","created":"2024-05-21T03:12:18.217236+0000","modified":"2024-05-21T03:21:14.540989+0000","last_up_change":"2024-05-21T03:19:30.382150+0000","last_in_change":"2024-05-21T03:19:09.058249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T03:15:41.340194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T03:20:27.507346+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T03:20:30.830169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c745c09f-9a76-4d05-af75-c7554f6d5e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6803","nonce":1399996449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6805","nonce":1399996449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6809","nonce":1399996449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6807","nonce":1399996449}]},"public_addr":"172.21.15.27:6803/1399996449","cluster_addr":"172.21.15.27:6805/1399996449","heartbeat_back_addr":"172.21.15.27:6809/1399996449","heartbeat_front_addr":"172.21.15.27:6807/1399996449","state":["exists","up"]},{"osd":1,"uuid":"8b73aa00-4364-438f-9722-53caa4e9bec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6801","nonce":1776318847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6803","nonce":1776318847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6807","nonce":1776318847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6805","nonce":1776318847}]},"public_addr":"172.21.15.39:6801/1776318847","cluster_addr":"172.21.15.39:6803/1776318847","heartbeat_back_addr":"172.21.15.39:6807/1776318847","heartbeat_front_addr":"172.21.15.39:6805/1776318847","state":["exists","up"]},{"osd":2,"uuid":"0822043e-7d3c-4e98-9dc0-94b98b4d562a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6801","nonce":703322625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6803","nonce":703322625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6807","nonce":703322625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6805","nonce":703322625}]},"public_addr":"172.21.15.165:6801/703322625","cluster_addr":"172.21.15.165:6803/703322625","heartbeat_back_addr":"172.21.15.165:6807/703322625","heartbeat_front_addr":"172.21.15.165:6805/703322625","state":["exists","up"]},{"osd":3,"uuid":"c563b1b2-efcb-4f8c-8121-c1938f83131e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6811","nonce":1933403135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6813","nonce":1933403135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6817","nonce":1933403135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6815","nonce":1933403135}]},"public_addr":"172.21.15.27:6811/1933403135","cluster_addr":"172.21.15.27:6813/1933403135","heartbeat_back_addr":"172.21.15.27:6817/1933403135","heartbeat_front_addr":"172.21.15.27:6815/1933403135","state":["exists","up"]},{"osd":4,"uuid":"0c793b87-b045-4b4a-89e7-87ec36deeb0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6809","nonce":155579331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6811","nonce":155579331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6815","nonce":155579331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6813","nonce":155579331}]},"public_addr":"172.21.15.39:6809/155579331","cluster_addr":"172.21.15.39:6811/155579331","heartbeat_back_addr":"172.21.15.39:6815/155579331","heartbeat_front_addr":"172.21.15.39:6813/155579331","state":["exists","up"]},{"osd":5,"uuid":"367eeb5a-9d75-4644-bf46-317c03f33d8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6809","nonce":386850244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6811","nonce":386850244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6815","nonce":386850244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6813","nonce":386850244}]},"public_addr":"172.21.15.165:6809/386850244","cluster_addr":"172.21.15.165:6811/386850244","heartbeat_back_addr":"172.21.15.165:6815/386850244","heartbeat_front_addr":"172.21.15.165:6813/386850244","state":["exists","up"]},{"osd":6,"uuid":"e000167f-24fd-4115-83f0-8afbfa8d8991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6819","nonce":3531376679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6821","nonce":3531376679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6825","nonce":3531376679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6823","nonce":3531376679}]},"public_addr":"172.21.15.27:6819/3531376679","cluster_addr":"172.21.15.27:6821/3531376679","heartbeat_back_addr":"172.21.15.27:6825/3531376679","heartbeat_front_addr":"172.21.15.27:6823/3531376679","state":["exists","up"]},{"osd":7,"uuid":"4b8d486b-868e-4f33-98a1-b99686adeaaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6817","nonce":1550035852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6819","nonce":1550035852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6823","nonce":1550035852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6821","nonce":1550035852}]},"public_addr":"172.21.15.39:6817/1550035852","cluster_addr":"172.21.15.39:6819/1550035852","heartbeat_back_addr":"172.21.15.39:6823/1550035852","heartbeat_front_addr":"172.21.15.39:6821/1550035852","state":["exists","up"]},{"osd":8,"uuid":"2259f35c-2edd-4331-9a63-8c0c1acbdc71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6817","nonce":3184704971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6819","nonce":3184704971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6823","nonce":3184704971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6821","nonce":3184704971}]},"public_addr":"172.21.15.165:6817/3184704971","cluster_addr":"172.21.15.165:6819/3184704971","heartbeat_back_addr":"172.21.15.165:6823/3184704971","heartbeat_front_addr":"172.21.15.165:6821/3184704971","state":["exists","up"]},{"osd":9,"uuid":"10307ecc-8270-4be2-b62b-9708e09bb207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6827","nonce":2800048793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6829","nonce":2800048793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6833","nonce":2800048793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6831","nonce":2800048793}]},"public_addr":"172.21.15.27:6827/2800048793","cluster_addr":"172.21.15.27:6829/2800048793","heartbeat_back_addr":"172.21.15.27:6833/2800048793","heartbeat_front_addr":"172.21.15.27:6831/2800048793","state":["exists","up"]},{"osd":10,"uuid":"dfe1b8ca-ff8a-4b7f-ade5-ad4f00cf60cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6825","nonce":363500811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6827","nonce":363500811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6831","nonce":363500811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6829","nonce":363500811}]},"public_addr":"172.21.15.39:6825/363500811","cluster_addr":"172.21.15.39:6827/363500811","heartbeat_back_addr":"172.21.15.39:6831/363500811","heartbeat_front_addr":"172.21.15.39:6829/363500811","state":["exists","up"]},{"osd":11,"uuid":"f9531145-5d09-4dbd-a050-3159ee09db24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6825","nonce":2274332243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6827","nonce":2274332243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6831","nonce":2274332243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6829","nonce":2274332243}]},"public_addr":"172.21.15.165:6825/2274332243","cluster_addr":"172.21.15.165:6827/2274332243","heartbeat_back_addr":"172.21.15.165:6831/2274332243","heartbeat_front_addr":"172.21.15.165:6829/2274332243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:14:47.077860+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:15:13.272879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:15:38.046651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:03.376483+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:29.179280+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:53.391261+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:17:18.472829+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:17:44.349925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:18:08.652193+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:18:35.282669+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:19:01.637151+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:19:27.461566+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6800/1185518061":"2024-05-22T03:13:13.249220+0000","172.21.15.27:6801/1185518061":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/4194431212":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/1890032101":"2024-05-22T03:13:13.249220+0000","172.21.15.27:6800/175229478":"2024-05-22T03:12:45.739195+0000","172.21.15.27:0/999325706":"2024-05-22T03:12:45.739195+0000","172.21.15.27:6801/175229478":"2024-05-22T03:12:45.739195+0000","172.21.15.27:0/4184128736":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/3000197506":"2024-05-22T03:12:45.739195+0000","172.21.15.27:0/972677319":"2024-05-22T03:12:45.739195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T03:23:39.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.364+0000 7f495da87700 1 -- 172.21.15.27:0/838941006 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f49440b2880 msgr2=0x7f49440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:39.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.364+0000 7f495da87700 1 --2- 172.21.15.27:0/838941006 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f49440b2880 0x7f49440b4d40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f495807acf0 tx=0x7f494000a380 comp rx=0 tx=0).stop 2024-05-21T03:23:39.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.364+0000 7f495da87700 1 -- 172.21.15.27:0/838941006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4958103590 msgr2=0x7f495819f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:39.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.364+0000 7f495da87700 1 --2- 172.21.15.27:0/838941006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4958103590 0x7f495819f800 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f49480057f0 tx=0x7f4948003680 comp rx=0 tx=0).stop 2024-05-21T03:23:39.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.365+0000 7f495da87700 1 -- 172.21.15.27:0/838941006 shutdown_connections 2024-05-21T03:23:39.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.365+0000 7f495da87700 1 --2- 172.21.15.27:0/838941006 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f49440b2880 0x7f49440b4d40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:39.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.365+0000 7f495da87700 1 --2- 172.21.15.27:0/838941006 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49581096a0 0x7f4958199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:39.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.365+0000 7f495da87700 1 --2- 172.21.15.27:0/838941006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4958103590 0x7f495819f800 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:39.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.365+0000 7f495da87700 1 --2- 172.21.15.27:0/838941006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4958103f50 0x7f495819fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:39.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.365+0000 7f495da87700 1 -- 172.21.15.27:0/838941006 >> 172.21.15.27:0/838941006 conn(0x7f495807a6f0 msgr2=0x7f49581021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:39.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.365+0000 7f495da87700 1 -- 172.21.15.27:0/838941006 shutdown_connections 2024-05-21T03:23:39.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:39.365+0000 7f495da87700 1 -- 172.21.15.27:0/838941006 wait complete. 2024-05-21T03:23:39.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:39 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/838941006' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:23:40.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:39 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/838941006' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:23:40.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:39 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/838941006' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:23:40.056 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-21T03:20:34.055985+0000', 'modified': '2024-05-21T03:20:52.467919+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24476, 'mds_1': 24472, 'mds_2': 24457, 'mds_3': 24452, 'mds_4': 24451}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14613': {'gid': 14613, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.27:6839/745486764', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6838', 'nonce': 745486764}, {'type': 'v1', 'addr': '172.21.15.27:6839', 'nonce': 745486764}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14625': {'gid': 14625, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.27:6841/3982752351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6840', 'nonce': 3982752351}, {'type': 'v1', 'addr': '172.21.15.27:6841', 'nonce': 3982752351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24401': {'gid': 24401, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.39:6833/1102873763', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 1102873763}, {'type': 'v1', 'addr': '172.21.15.39:6833', 'nonce': 1102873763}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24418': {'gid': 24418, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6833/4176762898', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6832', 'nonce': 4176762898}, {'type': 'v1', 'addr': '172.21.15.165:6833', 'nonce': 4176762898}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.39:6835/3497521608', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6834', 'nonce': 3497521608}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 3497521608}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.165:6835/3576910218', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6834', 'nonce': 3576910218}, {'type': 'v1', 'addr': '172.21.15.165:6835', 'nonce': 3576910218}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'h', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.39:6837/3673846852', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6836', 'nonce': 3673846852}, {'type': 'v1', 'addr': '172.21.15.39:6837', 'nonce': 3673846852}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.165:6837/691356139', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6836', 'nonce': 691356139}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 691356139}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.165:6839/4158170814', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6838', 'nonce': 4158170814}, {'type': 'v1', 'addr': '172.21.15.165:6839', 'nonce': 4158170814}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6839/287607819', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6838', 'nonce': 287607819}, {'type': 'v1', 'addr': '172.21.15.39:6839', 'nonce': 287607819}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24476, 'qdb_cluster': [24451, 24472, 24452, 24457, 24476]} 2024-05-21T03:23:40.057 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-21T03:23:40.057 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell mds.f status 2024-05-21T03:23:40.310 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:23:40.715 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:40 smithi027 ceph-mon[32179]: pgmap v385: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:23:40.715 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:40 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:23:40.715 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:40 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:23:40.715 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:40 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:23:41.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:40 smithi039 ceph-mon[36429]: pgmap v385: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:23:41.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:40 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:23:41.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:40 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:23:41.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:40 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:23:41.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:40 smithi165 ceph-mon[36269]: pgmap v385: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:23:41.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:40 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:23:41.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:40 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:23:41.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:40 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:23:41.671 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:41 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:23:41.672 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:41 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:23:41.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.904+0000 7f962d787700 1 -- 172.21.15.27:0/3284328815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f962806bab0 msgr2=0x7f962806bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:41.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.904+0000 7f962d787700 1 --2- 172.21.15.27:0/3284328815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f962806bab0 0x7f962806bf30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9610009990 tx=0x7f9610009ca0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.904+0000 7f962d787700 1 -- 172.21.15.27:0/3284328815 shutdown_connections 2024-05-21T03:23:41.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.904+0000 7f962d787700 1 --2- 172.21.15.27:0/3284328815 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f962806c470 0x7f96281103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.904+0000 7f962d787700 1 --2- 172.21.15.27:0/3284328815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96281095a0 0x7f9628109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.904+0000 7f962d787700 1 --2- 172.21.15.27:0/3284328815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f962806bab0 0x7f962806bf30 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.904+0000 7f962d787700 1 -- 172.21.15.27:0/3284328815 >> 172.21.15.27:0/3284328815 conn(0x7f96280793b0 msgr2=0x7f96280797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:41.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.904+0000 7f962d787700 1 -- 172.21.15.27:0/3284328815 shutdown_connections 2024-05-21T03:23:41.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.904+0000 7f962d787700 1 -- 172.21.15.27:0/3284328815 wait complete. 2024-05-21T03:23:41.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.905+0000 7f962d787700 1 Processor -- start 2024-05-21T03:23:41.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.905+0000 7f962d787700 1 -- start start 2024-05-21T03:23:41.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.905+0000 7f962d787700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f962806bab0 0x7f962819f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:41.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.905+0000 7f962d787700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f962806c470 0x7f962819fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:41.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.905+0000 7f962d787700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96281095a0 0x7f9628199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:41.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.905+0000 7f962d787700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9628113750 con 0x7f96281095a0 2024-05-21T03:23:41.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.905+0000 7f962d787700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f96281135d0 con 0x7f962806bab0 2024-05-21T03:23:41.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.905+0000 7f962d787700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f96281138d0 con 0x7f962806c470 2024-05-21T03:23:41.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.905+0000 7f96277fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96281095a0 0x7f9628199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:41.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.905+0000 7f96277fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96281095a0 0x7f9628199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33442/0 (socket says 172.21.15.27:33442) 2024-05-21T03:23:41.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.905+0000 7f96277fe700 1 -- 172.21.15.27:0/875451995 learned_addr learned my addr 172.21.15.27:0/875451995 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:41.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.905+0000 7f96267fc700 1 --2- 172.21.15.27:0/875451995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f962806c470 0x7f962819fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:41.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.905+0000 7f9626ffd700 1 --2- 172.21.15.27:0/875451995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f962806bab0 0x7f962819f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:41.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.906+0000 7f96277fe700 1 -- 172.21.15.27:0/875451995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f962806c470 msgr2=0x7f962819fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:41.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.906+0000 7f96277fe700 1 --2- 172.21.15.27:0/875451995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f962806c470 0x7f962819fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.906+0000 7f96277fe700 1 -- 172.21.15.27:0/875451995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f962806bab0 msgr2=0x7f962819f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:41.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.906+0000 7f96277fe700 1 --2- 172.21.15.27:0/875451995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f962806bab0 0x7f962819f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.906+0000 7f96277fe700 1 -- 172.21.15.27:0/875451995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f962819a230 con 0x7f96281095a0 2024-05-21T03:23:41.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.906+0000 7f96277fe700 1 --2- 172.21.15.27:0/875451995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96281095a0 0x7f9628199ad0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f961800c970 tx=0x7f961800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:41.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.906+0000 7f961ffff700 1 -- 172.21.15.27:0/875451995 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9618004a30 con 0x7f96281095a0 2024-05-21T03:23:41.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.906+0000 7f962d787700 1 -- 172.21.15.27:0/875451995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f962819a520 con 0x7f96281095a0 2024-05-21T03:23:41.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.906+0000 7f961ffff700 1 -- 172.21.15.27:0/875451995 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96180054d0 con 0x7f96281095a0 2024-05-21T03:23:41.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.906+0000 7f961ffff700 1 -- 172.21.15.27:0/875451995 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961800ec20 con 0x7f96281095a0 2024-05-21T03:23:41.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.906+0000 7f962d787700 1 -- 172.21.15.27:0/875451995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96281a67e0 con 0x7f96281095a0 2024-05-21T03:23:41.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.909+0000 7f961ffff700 1 -- 172.21.15.27:0/875451995 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9618004fa0 con 0x7f96281095a0 2024-05-21T03:23:41.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.909+0000 7f961ffff700 1 --2- 172.21.15.27:0/875451995 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f96140b2880 0x7f96140b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:41.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.910+0000 7f9626ffd700 1 --2- 172.21.15.27:0/875451995 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f96140b2880 0x7f96140b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:41.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.910+0000 7f961ffff700 1 -- 172.21.15.27:0/875451995 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f96180e1fe0 con 0x7f96281095a0 2024-05-21T03:23:41.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.910+0000 7f9626ffd700 1 --2- 172.21.15.27:0/875451995 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f96140b2880 0x7f96140b4d40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f962000a400 tx=0x7f962000d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:41.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.910+0000 7f963801db80 1 Processor -- start 2024-05-21T03:23:41.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.910+0000 7f963801db80 1 -- start start 2024-05-21T03:23:41.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.911+0000 7f963801db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ab1f9b72d0 0x55ab1f9b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:41.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.911+0000 7f963801db80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab1f9b1850 0x55ab1f9b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:41.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.911+0000 7f963801db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab1f9b2210 0x55ab1f9be030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:41.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.911+0000 7f963801db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ab1f90fd20 con 0x55ab1f9b2210 2024-05-21T03:23:41.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.911+0000 7f963801db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ab1f8ff8c0 con 0x55ab1f9b72d0 2024-05-21T03:23:41.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.911+0000 7f963801db80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ab1f903900 con 0x55ab1f9b1850 2024-05-21T03:23:41.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.911+0000 7f96277fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab1f9b2210 0x55ab1f9be030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:41.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.911+0000 7f96267fc700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab1f9b1850 0x55ab1f9b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:41.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.911+0000 7f96277fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab1f9b2210 0x55ab1f9be030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33444/0 (socket says 172.21.15.27:33444) 2024-05-21T03:23:41.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.911+0000 7f96277fe700 1 -- 172.21.15.27:0/3707297021 learned_addr learned my addr 172.21.15.27:0/3707297021 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:41.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.911+0000 7f9626ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ab1f9b72d0 0x55ab1f9b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:41.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.911+0000 7f96277fe700 1 -- 172.21.15.27:0/3707297021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab1f9b1850 msgr2=0x55ab1f9b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:41.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.911+0000 7f96277fe700 1 --2- 172.21.15.27:0/3707297021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab1f9b1850 0x55ab1f9b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.911+0000 7f96277fe700 1 -- 172.21.15.27:0/3707297021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ab1f9b72d0 msgr2=0x55ab1f9b76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:41.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.911+0000 7f96277fe700 1 --2- 172.21.15.27:0/3707297021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ab1f9b72d0 0x55ab1f9b76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.911+0000 7f96277fe700 1 -- 172.21.15.27:0/3707297021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab1f8a8f60 con 0x55ab1f9b2210 2024-05-21T03:23:41.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.911+0000 7f9626ffd700 1 --2- 172.21.15.27:0/3707297021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ab1f9b72d0 0x55ab1f9b76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:23:41.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.912+0000 7f96277fe700 1 --2- 172.21.15.27:0/3707297021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab1f9b2210 0x55ab1f9be030 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f96180a6850 tx=0x7f96180a9ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beccf64e894c70be server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:41.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.912+0000 7f961d7fa700 1 -- 172.21.15.27:0/3707297021 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961801d040 con 0x55ab1f9b2210 2024-05-21T03:23:41.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.912+0000 7f961d7fa700 1 -- 172.21.15.27:0/3707297021 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96180b2440 con 0x55ab1f9b2210 2024-05-21T03:23:41.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.912+0000 7f961d7fa700 1 -- 172.21.15.27:0/3707297021 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96180910e0 con 0x55ab1f9b2210 2024-05-21T03:23:41.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.913+0000 7f963801db80 1 -- 172.21.15.27:0/3707297021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab1f9b2210 msgr2=0x55ab1f9be030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:41.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.913+0000 7f963801db80 1 --2- 172.21.15.27:0/3707297021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab1f9b2210 0x55ab1f9be030 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f96180a6850 tx=0x7f96180a9ce0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.913+0000 7f963801db80 1 -- 172.21.15.27:0/3707297021 shutdown_connections 2024-05-21T03:23:41.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.913+0000 7f963801db80 1 --2- 172.21.15.27:0/3707297021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab1f9b2210 0x55ab1f9be030 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.913+0000 7f963801db80 1 --2- 172.21.15.27:0/3707297021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ab1f9b72d0 0x55ab1f9b76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.913+0000 7f963801db80 1 --2- 172.21.15.27:0/3707297021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab1f9b1850 0x55ab1f9b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.913+0000 7f963801db80 1 -- 172.21.15.27:0/3707297021 >> 172.21.15.27:0/3707297021 conn(0x55ab1f8dd1f0 msgr2=0x55ab1f9ac390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:41.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.913+0000 7f963801db80 1 -- 172.21.15.27:0/3707297021 shutdown_connections 2024-05-21T03:23:41.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.913+0000 7f963801db80 1 -- 172.21.15.27:0/3707297021 wait complete. 2024-05-21T03:23:41.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.914+0000 7f963801db80 1 -- start start 2024-05-21T03:23:41.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.914+0000 7f963801db80 1 Processor -- start 2024-05-21T03:23:41.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.914+0000 7f963801db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ab1f9b1850 0x55ab1f9bdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:41.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.914+0000 7f963801db80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab1f9b2210 0x55ab1f9bb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:41.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.914+0000 7f963801db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab1f9b72d0 0x55ab1f9bb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:41.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.914+0000 7f963801db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ab1f805f50 con 0x55ab1f9b72d0 2024-05-21T03:23:41.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.914+0000 7f963801db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ab1f867d50 con 0x55ab1f9b1850 2024-05-21T03:23:41.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.914+0000 7f963801db80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ab1f9be690 con 0x55ab1f9b2210 2024-05-21T03:23:41.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.914+0000 7f963801db80 10 client.0 _ll_register_callbacks cb 0x55ab1f89a900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:23:41.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.914+0000 7f96277fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab1f9b72d0 0x55ab1f9bb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:41.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.914+0000 7f96277fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab1f9b72d0 0x55ab1f9bb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33460/0 (socket says 172.21.15.27:33460) 2024-05-21T03:23:41.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.914+0000 7f96277fe700 1 -- 172.21.15.27:0/1613464290 learned_addr learned my addr 172.21.15.27:0/1613464290 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:41.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.914+0000 7f9626ffd700 1 --2- 172.21.15.27:0/1613464290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ab1f9b1850 0x55ab1f9bdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:41.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.914+0000 7f96267fc700 1 --2- 172.21.15.27:0/1613464290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab1f9b2210 0x55ab1f9bb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:41.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.914+0000 7f96277fe700 1 -- 172.21.15.27:0/1613464290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab1f9b2210 msgr2=0x55ab1f9bb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:41.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.915+0000 7f96277fe700 1 --2- 172.21.15.27:0/1613464290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab1f9b2210 0x55ab1f9bb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.915+0000 7f96277fe700 1 -- 172.21.15.27:0/1613464290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ab1f9b1850 msgr2=0x55ab1f9bdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:41.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.915+0000 7f96277fe700 1 --2- 172.21.15.27:0/1613464290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ab1f9b1850 0x55ab1f9bdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.915+0000 7f96277fe700 1 -- 172.21.15.27:0/1613464290 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab1f84f880 con 0x55ab1f9b72d0 2024-05-21T03:23:41.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.915+0000 7f9626ffd700 1 --2- 172.21.15.27:0/1613464290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ab1f9b1850 0x55ab1f9bdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:23:41.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.915+0000 7f96277fe700 1 --2- 172.21.15.27:0/1613464290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab1f9b72d0 0x55ab1f9bb550 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f96180a9520 tx=0x7f9618004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c026cfcf679a893 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:41.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.915+0000 7f960e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:23:41.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.915+0000 7f960e7fc700 1 -- 172.21.15.27:0/1613464290 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961801d070 con 0x55ab1f9b72d0 2024-05-21T03:23:41.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.915+0000 7f960e7fc700 1 -- 172.21.15.27:0/1613464290 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f96180910c0 con 0x55ab1f9b72d0 2024-05-21T03:23:41.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.915+0000 7f963801db80 1 -- 172.21.15.27:0/1613464290 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96180b2440 con 0x55ab1f9b72d0 2024-05-21T03:23:41.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.915+0000 7f960e7fc700 1 -- 172.21.15.27:0/1613464290 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9618005680 con 0x55ab1f9b72d0 2024-05-21T03:23:41.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.915+0000 7f963801db80 10 client.15153 fetch_fsmap learned FSMap version 28 2024-05-21T03:23:41.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.915+0000 7f963801db80 1 -- 172.21.15.27:0/1613464290 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ab1f8b9710 con 0x55ab1f9b72d0 2024-05-21T03:23:41.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.915+0000 7f960e7fc700 1 -- 172.21.15.27:0/1613464290 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96180b2440 con 0x55ab1f9b72d0 2024-05-21T03:23:41.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.916+0000 7f960e7fc700 1 -- 172.21.15.27:0/1613464290 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f96180a8840 con 0x55ab1f9b72d0 2024-05-21T03:23:41.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.916+0000 7f963801db80 10 client.15153 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T03:23:41.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.916+0000 7f963801db80 10 client.15153 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-21T03:23:41.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.916+0000 7f963801db80 20 client.15153 populate_metadata read hostname 'smithi027' 2024-05-21T03:23:41.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.916+0000 7f963801db80 1 --2- 172.21.15.27:0/1613464290 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x55ab1f9c9f20 0x55ab1f9cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:41.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.916+0000 7f963801db80 4 client.15153 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:23:41.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.916+0000 7f963801db80 1 -- 172.21.15.27:0/1613464290 --> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab1f911c70 con 0x55ab1f9c9f20 2024-05-21T03:23:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.916+0000 7f9626ffd700 1 --2- 172.21.15.27:0/1613464290 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x55ab1f9c9f20 0x55ab1f9cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.917+0000 7f9626ffd700 1 --2- 172.21.15.27:0/1613464290 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x55ab1f9c9f20 0x55ab1f9cc0c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f7c049a9834ad7c3 server_cookie=e874c38471d11261 in_seq=0 out_seq=0 2024-05-21T03:23:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.917+0000 7f960e7fc700 10 client.15153 ms_handle_connect on v2:172.21.15.165:6834/3576910218 2024-05-21T03:23:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.918+0000 7f960e7fc700 1 -- 172.21.15.27:0/1613464290 <== mds.4 v2:172.21.15.165:6834/3576910218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ab1f911c70 con 0x55ab1f9c9f20 2024-05-21T03:23:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.918+0000 7f960e7fc700 10 client.15153 handle_command_reply: tid=0 2024-05-21T03:23:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.918+0000 7f963801db80 1 client.15153 shutdown 2024-05-21T03:23:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.918+0000 7f963801db80 1 -- 172.21.15.27:0/1613464290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab1f9b72d0 msgr2=0x55ab1f9bb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.918+0000 7f963801db80 1 --2- 172.21.15.27:0/1613464290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab1f9b72d0 0x55ab1f9bb550 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f96180a9520 tx=0x7f9618004e00 comp rx=0 tx=0).stop 2024-05-21T03:23:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.918+0000 7f963801db80 1 -- 172.21.15.27:0/1613464290 shutdown_connections 2024-05-21T03:23:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.918+0000 7f963801db80 1 --2- 172.21.15.27:0/1613464290 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x55ab1f9c9f20 0x55ab1f9cc0c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.918+0000 7f963801db80 1 --2- 172.21.15.27:0/1613464290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab1f9b72d0 0x55ab1f9bb550 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.918+0000 7f963801db80 1 --2- 172.21.15.27:0/1613464290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ab1f9b1850 0x55ab1f9bdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.918+0000 7f963801db80 1 --2- 172.21.15.27:0/1613464290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab1f9b2210 0x55ab1f9bb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.918+0000 7f963801db80 1 -- 172.21.15.27:0/1613464290 >> 172.21.15.27:0/1613464290 conn(0x55ab1f8dd1f0 msgr2=0x55ab1f9abaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:41.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.918+0000 7f960e7fc700 0 client.15153 ms_handle_reset on v2:172.21.15.165:6834/3576910218 2024-05-21T03:23:41.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.918+0000 7f963801db80 1 -- 172.21.15.27:0/1613464290 shutdown_connections 2024-05-21T03:23:41.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.918+0000 7f963801db80 1 -- 172.21.15.27:0/1613464290 wait complete. 2024-05-21T03:23:41.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.918+0000 7f963801db80 20 client.15153 trim_cache size 0 max 16384 2024-05-21T03:23:41.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.943+0000 7f963801db80 1 Processor -- start 2024-05-21T03:23:41.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.943+0000 7f963801db80 1 -- start start 2024-05-21T03:23:41.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.943+0000 7f963801db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab1f9b1850 0x55ab1f9ac220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:41.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.943+0000 7f963801db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ab1f9b2210 0x55ab1f9b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:41.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.943+0000 7f963801db80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab1f9b72d0 0x55ab1f9b4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:41.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.943+0000 7f963801db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ab1f9ca400 con 0x55ab1f9b1850 2024-05-21T03:23:41.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.943+0000 7f963801db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ab1f9caad0 con 0x55ab1f9b2210 2024-05-21T03:23:41.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.943+0000 7f963801db80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ab1f9cb1a0 con 0x55ab1f9b72d0 2024-05-21T03:23:41.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.943+0000 7f9626ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab1f9b1850 0x55ab1f9ac220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:41.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.943+0000 7f9626ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab1f9b1850 0x55ab1f9ac220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33472/0 (socket says 172.21.15.27:33472) 2024-05-21T03:23:41.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.943+0000 7f9626ffd700 1 -- 172.21.15.27:0/2923907865 learned_addr learned my addr 172.21.15.27:0/2923907865 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:41.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.944+0000 7f96267fc700 1 --2- 172.21.15.27:0/2923907865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ab1f9b2210 0x55ab1f9b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:41.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.944+0000 7f9626ffd700 1 -- 172.21.15.27:0/2923907865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab1f9b72d0 msgr2=0x55ab1f9b4240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T03:23:41.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.944+0000 7f9626ffd700 1 --2- 172.21.15.27:0/2923907865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab1f9b72d0 0x55ab1f9b4240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.944+0000 7f9626ffd700 1 -- 172.21.15.27:0/2923907865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ab1f9b2210 msgr2=0x55ab1f9b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:41.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.944+0000 7f9626ffd700 1 --2- 172.21.15.27:0/2923907865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ab1f9b2210 0x55ab1f9b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.944+0000 7f9626ffd700 1 -- 172.21.15.27:0/2923907865 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab1f9ca8f0 con 0x55ab1f9b1850 2024-05-21T03:23:41.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.944+0000 7f96267fc700 1 --2- 172.21.15.27:0/2923907865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ab1f9b2210 0x55ab1f9b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:23:41.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.944+0000 7f9626ffd700 1 --2- 172.21.15.27:0/2923907865 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab1f9b1850 0x55ab1f9ac220 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f962000afd0 tx=0x7f9620011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1bc57e541785b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:41.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.944+0000 7f961cff9700 1 -- 172.21.15.27:0/2923907865 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962002a330 con 0x55ab1f9b1850 2024-05-21T03:23:41.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.944+0000 7f961cff9700 1 -- 172.21.15.27:0/2923907865 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f962002a980 con 0x55ab1f9b1850 2024-05-21T03:23:41.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.944+0000 7f961cff9700 1 -- 172.21.15.27:0/2923907865 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9620023bb0 con 0x55ab1f9b1850 2024-05-21T03:23:41.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.945+0000 7f963801db80 1 -- 172.21.15.27:0/2923907865 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab1f9b1850 msgr2=0x55ab1f9ac220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:41.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.945+0000 7f963801db80 1 --2- 172.21.15.27:0/2923907865 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab1f9b1850 0x55ab1f9ac220 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f962000afd0 tx=0x7f9620011000 comp rx=0 tx=0).stop 2024-05-21T03:23:41.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.945+0000 7f963801db80 1 -- 172.21.15.27:0/2923907865 shutdown_connections 2024-05-21T03:23:41.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.945+0000 7f963801db80 1 --2- 172.21.15.27:0/2923907865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab1f9b72d0 0x55ab1f9b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.945+0000 7f963801db80 1 --2- 172.21.15.27:0/2923907865 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab1f9b1850 0x55ab1f9ac220 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.945+0000 7f963801db80 1 --2- 172.21.15.27:0/2923907865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ab1f9b2210 0x55ab1f9b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.945+0000 7f963801db80 1 -- 172.21.15.27:0/2923907865 >> 172.21.15.27:0/2923907865 conn(0x55ab1f8dd1f0 msgr2=0x55ab1f9add40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:41.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.945+0000 7f963801db80 1 -- 172.21.15.27:0/2923907865 shutdown_connections 2024-05-21T03:23:41.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.945+0000 7f963801db80 1 -- 172.21.15.27:0/2923907865 wait complete. 2024-05-21T03:23:41.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.946+0000 7f963801db80 1 -- start start 2024-05-21T03:23:41.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.946+0000 7f963801db80 1 Processor -- start 2024-05-21T03:23:41.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.946+0000 7f963801db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ab1f9b1850 0x55ab1f9cf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:41.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.946+0000 7f963801db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab1f9b2210 0x55ab1f9cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:41.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.946+0000 7f963801db80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab1f9b72d0 0x55ab1f9cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:41.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.946+0000 7f963801db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ab1f9cbf40 con 0x55ab1f9b2210 2024-05-21T03:23:41.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.946+0000 7f963801db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ab1f9cb870 con 0x55ab1f9b1850 2024-05-21T03:23:41.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.946+0000 7f963801db80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ab1f9cc610 con 0x55ab1f9b72d0 2024-05-21T03:23:41.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.946+0000 7f963801db80 10 client.0 _ll_register_callbacks cb 0x55ab1f89a900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:23:41.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.946+0000 7f96267fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab1f9b2210 0x55ab1f9cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:41.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.946+0000 7f96267fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab1f9b2210 0x55ab1f9cf760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33478/0 (socket says 172.21.15.27:33478) 2024-05-21T03:23:41.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.946+0000 7f96267fc700 1 -- 172.21.15.27:0/1768328677 learned_addr learned my addr 172.21.15.27:0/1768328677 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:41.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.946+0000 7f9626ffd700 1 --2- 172.21.15.27:0/1768328677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ab1f9b1850 0x55ab1f9cf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:41.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.946+0000 7f96277fe700 1 --2- 172.21.15.27:0/1768328677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab1f9b72d0 0x55ab1f9cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:41.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.946+0000 7f96267fc700 1 -- 172.21.15.27:0/1768328677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab1f9b72d0 msgr2=0x55ab1f9cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:41.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.946+0000 7f96267fc700 1 --2- 172.21.15.27:0/1768328677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab1f9b72d0 0x55ab1f9cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.946+0000 7f96267fc700 1 -- 172.21.15.27:0/1768328677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ab1f9b1850 msgr2=0x55ab1f9cf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:41.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.946+0000 7f96267fc700 1 --2- 172.21.15.27:0/1768328677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ab1f9b1850 0x55ab1f9cf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.946+0000 7f96267fc700 1 -- 172.21.15.27:0/1768328677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab1f9d3990 con 0x55ab1f9b2210 2024-05-21T03:23:41.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.947+0000 7f96267fc700 1 --2- 172.21.15.27:0/1768328677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab1f9b2210 0x55ab1f9cf760 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9610009990 tx=0x7f9610003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ccc0ebf5e93fa0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:41.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.947+0000 7f960e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:23:41.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.947+0000 7f960e7fc700 1 -- 172.21.15.27:0/1768328677 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9610005610 con 0x55ab1f9b2210 2024-05-21T03:23:41.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.947+0000 7f960e7fc700 1 -- 172.21.15.27:0/1768328677 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f961000b7d0 con 0x55ab1f9b2210 2024-05-21T03:23:41.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.947+0000 7f963801db80 1 -- 172.21.15.27:0/1768328677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f962002a980 con 0x55ab1f9b2210 2024-05-21T03:23:41.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.947+0000 7f960e7fc700 1 -- 172.21.15.27:0/1768328677 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f961000ba10 con 0x55ab1f9b2210 2024-05-21T03:23:41.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.947+0000 7f963801db80 10 client.15159 fetch_fsmap learned FSMap version 28 2024-05-21T03:23:41.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.947+0000 7f963801db80 1 -- 172.21.15.27:0/1768328677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ab1fa36680 con 0x55ab1f9b2210 2024-05-21T03:23:41.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.947+0000 7f960e7fc700 1 -- 172.21.15.27:0/1768328677 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f962002a980 con 0x55ab1f9b2210 2024-05-21T03:23:41.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.948+0000 7f960e7fc700 1 -- 172.21.15.27:0/1768328677 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f961000f460 con 0x55ab1f9b2210 2024-05-21T03:23:41.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.948+0000 7f963801db80 10 client.15159 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T03:23:41.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.948+0000 7f963801db80 10 client.15159 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-21T03:23:41.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.948+0000 7f963801db80 20 client.15159 populate_metadata read hostname 'smithi027' 2024-05-21T03:23:41.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.948+0000 7f963801db80 1 --2- 172.21.15.27:0/1768328677 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x55ab1fa3b2c0 0x55ab1fa39fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:41.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.948+0000 7f963801db80 4 client.15159 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T03:23:41.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.948+0000 7f963801db80 1 -- 172.21.15.27:0/1768328677 --> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] -- command(tid 0: {"prefix": "status"}) -- 0x55ab1f90b450 con 0x55ab1fa3b2c0 2024-05-21T03:23:41.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.948+0000 7f9626ffd700 1 --2- 172.21.15.27:0/1768328677 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x55ab1fa3b2c0 0x55ab1fa39fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:41.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.948+0000 7f9626ffd700 1 --2- 172.21.15.27:0/1768328677 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x55ab1fa3b2c0 0x55ab1fa39fd0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1787e40ed6fbf320 server_cookie=145543eb82677b15 in_seq=0 out_seq=0 2024-05-21T03:23:41.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.948+0000 7f960e7fc700 10 client.15159 ms_handle_connect on v2:172.21.15.165:6834/3576910218 2024-05-21T03:23:41.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.949+0000 7f960e7fc700 1 -- 172.21.15.27:0/1768328677 <== mds.4 v2:172.21.15.165:6834/3576910218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ab1f90b450 con 0x55ab1fa3b2c0 2024-05-21T03:23:41.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.949+0000 7f960e7fc700 10 client.15159 handle_command_reply: tid=0 2024-05-21T03:23:41.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.949+0000 7f963801db80 1 client.15159 shutdown 2024-05-21T03:23:41.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.949+0000 7f963801db80 1 -- 172.21.15.27:0/1768328677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab1f9b2210 msgr2=0x55ab1f9cf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:41.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.949+0000 7f963801db80 1 --2- 172.21.15.27:0/1768328677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab1f9b2210 0x55ab1f9cf760 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9610009990 tx=0x7f9610003f00 comp rx=0 tx=0).stop 2024-05-21T03:23:41.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.949+0000 7f963801db80 1 -- 172.21.15.27:0/1768328677 shutdown_connections 2024-05-21T03:23:41.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.949+0000 7f963801db80 1 --2- 172.21.15.27:0/1768328677 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x55ab1fa3b2c0 0x55ab1fa39fd0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.949+0000 7f963801db80 1 --2- 172.21.15.27:0/1768328677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab1f9b72d0 0x55ab1f9cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.949+0000 7f963801db80 1 --2- 172.21.15.27:0/1768328677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ab1f9b1850 0x55ab1f9cf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.949+0000 7f963801db80 1 --2- 172.21.15.27:0/1768328677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ab1f9b2210 0x55ab1f9cf760 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.949+0000 7f963801db80 1 -- 172.21.15.27:0/1768328677 >> 172.21.15.27:0/1768328677 conn(0x55ab1f8dd1f0 msgr2=0x55ab1f9b6140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:41.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.949+0000 7f960e7fc700 0 client.15159 ms_handle_reset on v2:172.21.15.165:6834/3576910218 2024-05-21T03:23:41.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.949+0000 7f963801db80 1 -- 172.21.15.27:0/1768328677 shutdown_connections 2024-05-21T03:23:41.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.949+0000 7f963801db80 1 -- 172.21.15.27:0/1768328677 wait complete. 2024-05-21T03:23:41.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.949+0000 7f963801db80 20 client.15159 trim_cache size 0 max 16384 2024-05-21T03:23:41.953 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:23:41.953 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "c9ca94be-171f-11ef-bc9a-c7b262605968", 2024-05-21T03:23:41.953 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 4, 2024-05-21T03:23:41.954 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24451, 2024-05-21T03:23:41.954 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-05-21T03:23:41.954 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-05-21T03:23:41.954 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-05-21T03:23:41.954 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 172.48840659800001, 2024-05-21T03:23:41.954 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 28, 2024-05-21T03:23:41.954 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 82, 2024-05-21T03:23:41.954 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 80, 2024-05-21T03:23:41.954 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 220.35591325600001 2024-05-21T03:23:41.954 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:23:41.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.950+0000 7f962d787700 1 -- 172.21.15.27:0/875451995 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f96140b2880 msgr2=0x7f96140b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:41.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.950+0000 7f962d787700 1 --2- 172.21.15.27:0/875451995 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f96140b2880 0x7f96140b4d40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f962000a400 tx=0x7f962000d040 comp rx=0 tx=0).stop 2024-05-21T03:23:41.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.950+0000 7f962d787700 1 -- 172.21.15.27:0/875451995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96281095a0 msgr2=0x7f9628199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:41.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.950+0000 7f962d787700 1 --2- 172.21.15.27:0/875451995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96281095a0 0x7f9628199ad0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f961800c970 tx=0x7f961800cc80 comp rx=0 tx=0).stop 2024-05-21T03:23:41.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.950+0000 7f962d787700 1 -- 172.21.15.27:0/875451995 shutdown_connections 2024-05-21T03:23:41.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.950+0000 7f962d787700 1 --2- 172.21.15.27:0/875451995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96281095a0 0x7f9628199ad0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.950+0000 7f962d787700 1 --2- 172.21.15.27:0/875451995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f962806bab0 0x7f962819f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.950+0000 7f962d787700 1 --2- 172.21.15.27:0/875451995 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f96140b2880 0x7f96140b4d40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.950+0000 7f962d787700 1 --2- 172.21.15.27:0/875451995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f962806c470 0x7f962819fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:41.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.950+0000 7f962d787700 1 -- 172.21.15.27:0/875451995 >> 172.21.15.27:0/875451995 conn(0x7f96280793b0 msgr2=0x7f9628101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:41.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.950+0000 7f962d787700 1 -- 172.21.15.27:0/875451995 shutdown_connections 2024-05-21T03:23:41.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:41.950+0000 7f962d787700 1 -- 172.21.15.27:0/875451995 wait complete. 2024-05-21T03:23:42.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:41 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:23:42.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:41 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:23:42.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:41 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:23:42.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:41 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:23:42.588 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell mds.h status 2024-05-21T03:23:42.837 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:23:42.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:42 smithi027 ceph-mon[32179]: pgmap v386: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:23:42.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:42 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:23:42.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:42 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:23:42.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:42 smithi027 ceph-mon[32179]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:23:43.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:42 smithi039 ceph-mon[36429]: pgmap v386: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:23:43.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:42 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:23:43.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:42 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:23:43.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:42 smithi039 ceph-mon[36429]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:23:43.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:42 smithi165 ceph-mon[36269]: pgmap v386: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:23:43.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:42 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:23:43.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:42 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:23:43.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:42 smithi165 ceph-mon[36269]: from='mgr.14152 172.21.15.27:0/275946653' entity='mgr.x' 2024-05-21T03:23:44.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.287+0000 7fed8eab9700 1 -- 172.21.15.27:0/2695479080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed88103820 msgr2=0x7fed88103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:44.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.287+0000 7fed8eab9700 1 --2- 172.21.15.27:0/2695479080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed88103820 0x7fed88103c00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0099c0 tx=0x7fed7c009cd0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.287+0000 7fed8eab9700 1 -- 172.21.15.27:0/2695479080 shutdown_connections 2024-05-21T03:23:44.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.287+0000 7fed8eab9700 1 --2- 172.21.15.27:0/2695479080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed88108910 0x7fed88110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.287+0000 7fed8eab9700 1 --2- 172.21.15.27:0/2695479080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed88103820 0x7fed88103c00 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.287+0000 7fed8eab9700 1 --2- 172.21.15.27:0/2695479080 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed881041d0 0x7fed881081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.287+0000 7fed8eab9700 1 -- 172.21.15.27:0/2695479080 >> 172.21.15.27:0/2695479080 conn(0x7fed8807a6b0 msgr2=0x7fed8807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:44.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.287+0000 7fed8eab9700 1 -- 172.21.15.27:0/2695479080 shutdown_connections 2024-05-21T03:23:44.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.287+0000 7fed8eab9700 1 -- 172.21.15.27:0/2695479080 wait complete. 2024-05-21T03:23:44.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.287+0000 7fed8eab9700 1 Processor -- start 2024-05-21T03:23:44.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.287+0000 7fed8eab9700 1 -- start start 2024-05-21T03:23:44.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.288+0000 7fed8eab9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed88103820 0x7fed8819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:44.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.288+0000 7fed8eab9700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed881041d0 0x7fed8819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:44.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.288+0000 7fed8eab9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed88108910 0x7fed88199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:44.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.288+0000 7fed8eab9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fed88113710 con 0x7fed88108910 2024-05-21T03:23:44.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.288+0000 7fed8eab9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fed88113590 con 0x7fed88103820 2024-05-21T03:23:44.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.288+0000 7fed8eab9700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fed88113890 con 0x7fed881041d0 2024-05-21T03:23:44.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.288+0000 7fed8d056700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed88108910 0x7fed88199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:44.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.288+0000 7fed8d056700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed88108910 0x7fed88199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35244/0 (socket says 172.21.15.27:35244) 2024-05-21T03:23:44.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.288+0000 7fed8d056700 1 -- 172.21.15.27:0/3058305533 learned_addr learned my addr 172.21.15.27:0/3058305533 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:44.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.288+0000 7fed8c855700 1 --2- 172.21.15.27:0/3058305533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed88103820 0x7fed8819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:44.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.288+0000 7fed87fff700 1 --2- 172.21.15.27:0/3058305533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed881041d0 0x7fed8819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:44.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.288+0000 7fed8d056700 1 -- 172.21.15.27:0/3058305533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed881041d0 msgr2=0x7fed8819fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:44.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.288+0000 7fed8d056700 1 --2- 172.21.15.27:0/3058305533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed881041d0 0x7fed8819fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.288+0000 7fed8d056700 1 -- 172.21.15.27:0/3058305533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed88103820 msgr2=0x7fed8819f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:44.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.288+0000 7fed8d056700 1 --2- 172.21.15.27:0/3058305533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed88103820 0x7fed8819f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.288+0000 7fed8d056700 1 -- 172.21.15.27:0/3058305533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8819a170 con 0x7fed88108910 2024-05-21T03:23:44.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.288+0000 7fed87fff700 1 --2- 172.21.15.27:0/3058305533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed881041d0 0x7fed8819fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:23:44.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.288+0000 7fed8d056700 1 --2- 172.21.15.27:0/3058305533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed88108910 0x7fed88199a10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fed80007ee0 tx=0x7fed8000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:44.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.288+0000 7fed85ffb700 1 -- 172.21.15.27:0/3058305533 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8000dbd0 con 0x7fed88108910 2024-05-21T03:23:44.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.288+0000 7fed85ffb700 1 -- 172.21.15.27:0/3058305533 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed8000dd70 con 0x7fed88108910 2024-05-21T03:23:44.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.289+0000 7fed8eab9700 1 -- 172.21.15.27:0/3058305533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed8819a460 con 0x7fed88108910 2024-05-21T03:23:44.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.289+0000 7fed85ffb700 1 -- 172.21.15.27:0/3058305533 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed80005990 con 0x7fed88108910 2024-05-21T03:23:44.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.289+0000 7fed8eab9700 1 -- 172.21.15.27:0/3058305533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed881a66d0 con 0x7fed88108910 2024-05-21T03:23:44.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.291+0000 7fed85ffb700 1 -- 172.21.15.27:0/3058305533 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fed8001a020 con 0x7fed88108910 2024-05-21T03:23:44.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.292+0000 7fed85ffb700 1 --2- 172.21.15.27:0/3058305533 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fed780b2880 0x7fed780b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:44.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.292+0000 7fed8c855700 1 --2- 172.21.15.27:0/3058305533 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fed780b2880 0x7fed780b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:44.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.292+0000 7fed85ffb700 1 -- 172.21.15.27:0/3058305533 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fed800e19d0 con 0x7fed88108910 2024-05-21T03:23:44.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.292+0000 7fed8c855700 1 --2- 172.21.15.27:0/3058305533 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fed780b2880 0x7fed780b4d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fed88108470 tx=0x7fed7c005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:44.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.292+0000 7fed9934fb80 1 Processor -- start 2024-05-21T03:23:44.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.292+0000 7fed9934fb80 1 -- start start 2024-05-21T03:23:44.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.292+0000 7fed9934fb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b7d77f2d0 0x561b7d77f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:44.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.292+0000 7fed9934fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b7d779850 0x561b7d779cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:44.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.293+0000 7fed9934fb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561b7d77a210 0x561b7d786030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:44.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.293+0000 7fed9934fb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561b7d6d7d20 con 0x561b7d77f2d0 2024-05-21T03:23:44.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.293+0000 7fed9934fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561b7d6c78c0 con 0x561b7d779850 2024-05-21T03:23:44.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.293+0000 7fed9934fb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561b7d6cb900 con 0x561b7d77a210 2024-05-21T03:23:44.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.293+0000 7fed8c855700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b7d77f2d0 0x561b7d77f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:44.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.293+0000 7fed8c855700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b7d77f2d0 0x561b7d77f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35254/0 (socket says 172.21.15.27:35254) 2024-05-21T03:23:44.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.293+0000 7fed8c855700 1 -- 172.21.15.27:0/3531584850 learned_addr learned my addr 172.21.15.27:0/3531584850 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:44.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.293+0000 7fed87fff700 1 --2- 172.21.15.27:0/3531584850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b7d779850 0x561b7d779cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:44.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.293+0000 7fed8c855700 1 -- 172.21.15.27:0/3531584850 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561b7d77a210 msgr2=0x561b7d786030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:44.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.293+0000 7fed8c855700 1 --2- 172.21.15.27:0/3531584850 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561b7d77a210 0x561b7d786030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.293+0000 7fed8c855700 1 -- 172.21.15.27:0/3531584850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b7d779850 msgr2=0x561b7d779cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:44.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.293+0000 7fed8c855700 1 --2- 172.21.15.27:0/3531584850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b7d779850 0x561b7d779cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.293+0000 7fed8c855700 1 -- 172.21.15.27:0/3531584850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b7d670f60 con 0x561b7d77f2d0 2024-05-21T03:23:44.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.293+0000 7fed87fff700 1 --2- 172.21.15.27:0/3531584850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b7d779850 0x561b7d779cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:23:44.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.293+0000 7fed8c855700 1 --2- 172.21.15.27:0/3531584850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b7d77f2d0 0x561b7d77f6b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0099c0 tx=0x7fed7c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49748690dc8c6adc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:44.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.293+0000 7fed72ffd700 1 -- 172.21.15.27:0/3531584850 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c00be70 con 0x561b7d77f2d0 2024-05-21T03:23:44.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.293+0000 7fed72ffd700 1 -- 172.21.15.27:0/3531584850 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed7c00fcf0 con 0x561b7d77f2d0 2024-05-21T03:23:44.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.293+0000 7fed72ffd700 1 -- 172.21.15.27:0/3531584850 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c024cf0 con 0x561b7d77f2d0 2024-05-21T03:23:44.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.294+0000 7fed9934fb80 1 -- 172.21.15.27:0/3531584850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b7d77f2d0 msgr2=0x561b7d77f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:44.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.294+0000 7fed9934fb80 1 --2- 172.21.15.27:0/3531584850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b7d77f2d0 0x561b7d77f6b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0099c0 tx=0x7fed7c020000 comp rx=0 tx=0).stop 2024-05-21T03:23:44.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.294+0000 7fed9934fb80 1 -- 172.21.15.27:0/3531584850 shutdown_connections 2024-05-21T03:23:44.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.294+0000 7fed9934fb80 1 --2- 172.21.15.27:0/3531584850 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561b7d77a210 0x561b7d786030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.294+0000 7fed9934fb80 1 --2- 172.21.15.27:0/3531584850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b7d77f2d0 0x561b7d77f6b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.294+0000 7fed9934fb80 1 --2- 172.21.15.27:0/3531584850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b7d779850 0x561b7d779cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.294+0000 7fed9934fb80 1 -- 172.21.15.27:0/3531584850 >> 172.21.15.27:0/3531584850 conn(0x561b7d6a51f0 msgr2=0x561b7d774390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:44.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.294+0000 7fed9934fb80 1 -- 172.21.15.27:0/3531584850 shutdown_connections 2024-05-21T03:23:44.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.294+0000 7fed9934fb80 1 -- 172.21.15.27:0/3531584850 wait complete. 2024-05-21T03:23:44.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.294+0000 7fed9934fb80 1 -- start start 2024-05-21T03:23:44.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.294+0000 7fed9934fb80 1 Processor -- start 2024-05-21T03:23:44.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.295+0000 7fed9934fb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561b7d779850 0x561b7d785d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:44.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.295+0000 7fed9934fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b7d77a210 0x561b7d783030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:44.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.295+0000 7fed9934fb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b7d77f2d0 0x561b7d783570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:44.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.295+0000 7fed9934fb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561b7d5cdf50 con 0x561b7d77f2d0 2024-05-21T03:23:44.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.295+0000 7fed9934fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561b7d62fd50 con 0x561b7d77a210 2024-05-21T03:23:44.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.295+0000 7fed9934fb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561b7d7865d0 con 0x561b7d779850 2024-05-21T03:23:44.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.295+0000 7fed9934fb80 10 client.0 _ll_register_callbacks cb 0x561b7d662900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:23:44.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.295+0000 7fed8d056700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b7d77f2d0 0x561b7d783570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:44.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.295+0000 7fed8d056700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b7d77f2d0 0x561b7d783570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35264/0 (socket says 172.21.15.27:35264) 2024-05-21T03:23:44.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.295+0000 7fed87fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b7d77a210 0x561b7d783030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:44.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.295+0000 7fed8d056700 1 -- 172.21.15.27:0/2430284804 learned_addr learned my addr 172.21.15.27:0/2430284804 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:44.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.295+0000 7fed8c855700 1 --2- 172.21.15.27:0/2430284804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561b7d779850 0x561b7d785d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:44.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.295+0000 7fed8d056700 1 -- 172.21.15.27:0/2430284804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561b7d779850 msgr2=0x561b7d785d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:44.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.295+0000 7fed8d056700 1 --2- 172.21.15.27:0/2430284804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561b7d779850 0x561b7d785d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.295+0000 7fed8d056700 1 -- 172.21.15.27:0/2430284804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b7d77a210 msgr2=0x561b7d783030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:44.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.295+0000 7fed8d056700 1 --2- 172.21.15.27:0/2430284804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b7d77a210 0x561b7d783030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.295+0000 7fed8d056700 1 -- 172.21.15.27:0/2430284804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b7d617880 con 0x561b7d77f2d0 2024-05-21T03:23:44.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.295+0000 7fed8d056700 1 --2- 172.21.15.27:0/2430284804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b7d77f2d0 0x561b7d783570 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fed800a8cd0 tx=0x7fed800a8ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f53f842865174c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:44.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.295+0000 7fed67fff700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:23:44.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.296+0000 7fed67fff700 1 -- 172.21.15.27:0/2430284804 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed800c3060 con 0x561b7d77f2d0 2024-05-21T03:23:44.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.296+0000 7fed67fff700 1 -- 172.21.15.27:0/2430284804 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fed800a9440 con 0x561b7d77f2d0 2024-05-21T03:23:44.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.296+0000 7fed9934fb80 1 -- 172.21.15.27:0/2430284804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed7c00fcf0 con 0x561b7d77f2d0 2024-05-21T03:23:44.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.296+0000 7fed67fff700 1 -- 172.21.15.27:0/2430284804 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fed800ad380 con 0x561b7d77f2d0 2024-05-21T03:23:44.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.296+0000 7fed9934fb80 10 client.15171 fetch_fsmap learned FSMap version 28 2024-05-21T03:23:44.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.296+0000 7fed9934fb80 1 -- 172.21.15.27:0/2430284804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561b7d681710 con 0x561b7d77f2d0 2024-05-21T03:23:44.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.296+0000 7fed67fff700 1 -- 172.21.15.27:0/2430284804 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed7c00fcf0 con 0x561b7d77f2d0 2024-05-21T03:23:44.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.296+0000 7fed67fff700 1 -- 172.21.15.27:0/2430284804 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fed800051b0 con 0x561b7d77f2d0 2024-05-21T03:23:44.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.296+0000 7fed9934fb80 10 client.15171 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T03:23:44.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.296+0000 7fed9934fb80 10 client.15171 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-21T03:23:44.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.296+0000 7fed9934fb80 20 client.15171 populate_metadata read hostname 'smithi027' 2024-05-21T03:23:44.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.296+0000 7fed9934fb80 1 --2- 172.21.15.27:0/2430284804 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x561b7d793150 0x561b7d7945c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:44.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.296+0000 7fed9934fb80 4 client.15171 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:23:44.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.296+0000 7fed9934fb80 1 -- 172.21.15.27:0/2430284804 --> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b7d6d9c70 con 0x561b7d793150 2024-05-21T03:23:44.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.297+0000 7fed8c855700 1 --2- 172.21.15.27:0/2430284804 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x561b7d793150 0x561b7d7945c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:44.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.297+0000 7fed8c855700 1 --2- 172.21.15.27:0/2430284804 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x561b7d793150 0x561b7d7945c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=18c6a3524efec86a server_cookie=67d8dbbefd23f4fa in_seq=0 out_seq=0 2024-05-21T03:23:44.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.297+0000 7fed67fff700 10 client.15171 ms_handle_connect on v2:172.21.15.39:6836/3673846852 2024-05-21T03:23:44.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.298+0000 7fed67fff700 1 -- 172.21.15.27:0/2430284804 <== mds.3 v2:172.21.15.39:6836/3673846852 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561b7d6d9c70 con 0x561b7d793150 2024-05-21T03:23:44.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.298+0000 7fed67fff700 10 client.15171 handle_command_reply: tid=0 2024-05-21T03:23:44.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.298+0000 7fed9934fb80 1 client.15171 shutdown 2024-05-21T03:23:44.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.298+0000 7fed9934fb80 1 -- 172.21.15.27:0/2430284804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b7d77f2d0 msgr2=0x561b7d783570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:44.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.298+0000 7fed9934fb80 1 --2- 172.21.15.27:0/2430284804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b7d77f2d0 0x561b7d783570 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fed800a8cd0 tx=0x7fed800a8ae0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.298+0000 7fed9934fb80 1 -- 172.21.15.27:0/2430284804 shutdown_connections 2024-05-21T03:23:44.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.298+0000 7fed9934fb80 1 --2- 172.21.15.27:0/2430284804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b7d77f2d0 0x561b7d783570 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.298+0000 7fed9934fb80 1 --2- 172.21.15.27:0/2430284804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561b7d779850 0x561b7d785d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.298+0000 7fed9934fb80 1 --2- 172.21.15.27:0/2430284804 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x561b7d793150 0x561b7d7945c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.298+0000 7fed67fff700 0 client.15171 ms_handle_reset on v2:172.21.15.39:6836/3673846852 2024-05-21T03:23:44.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.298+0000 7fed9934fb80 1 --2- 172.21.15.27:0/2430284804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b7d77a210 0x561b7d783030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.298+0000 7fed9934fb80 1 -- 172.21.15.27:0/2430284804 >> 172.21.15.27:0/2430284804 conn(0x561b7d6a51f0 msgr2=0x561b7d773aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:44.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.298+0000 7fed9934fb80 1 -- 172.21.15.27:0/2430284804 shutdown_connections 2024-05-21T03:23:44.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.298+0000 7fed9934fb80 1 -- 172.21.15.27:0/2430284804 wait complete. 2024-05-21T03:23:44.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.298+0000 7fed9934fb80 20 client.15171 trim_cache size 0 max 16384 2024-05-21T03:23:44.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.320+0000 7fed9934fb80 1 Processor -- start 2024-05-21T03:23:44.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.320+0000 7fed9934fb80 1 -- start start 2024-05-21T03:23:44.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.320+0000 7fed9934fb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b7d779850 0x561b7d774220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:44.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.320+0000 7fed9934fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b7d77a210 0x561b7d77e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:44.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.320+0000 7fed9934fb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561b7d77f2d0 0x561b7d77e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:44.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.320+0000 7fed9934fb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561b7d792900 con 0x561b7d779850 2024-05-21T03:23:44.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.320+0000 7fed9934fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561b7d792fd0 con 0x561b7d77a210 2024-05-21T03:23:44.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.320+0000 7fed9934fb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561b7d7936a0 con 0x561b7d77f2d0 2024-05-21T03:23:44.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.321+0000 7fed8c855700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b7d779850 0x561b7d774220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:44.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.321+0000 7fed8c855700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b7d779850 0x561b7d774220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35270/0 (socket says 172.21.15.27:35270) 2024-05-21T03:23:44.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.321+0000 7fed8c855700 1 -- 172.21.15.27:0/1709525749 learned_addr learned my addr 172.21.15.27:0/1709525749 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:44.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.321+0000 7fed8c855700 1 -- 172.21.15.27:0/1709525749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561b7d77f2d0 msgr2=0x561b7d77e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:44.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.321+0000 7fed8c855700 1 --2- 172.21.15.27:0/1709525749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561b7d77f2d0 0x561b7d77e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.321+0000 7fed8c855700 1 -- 172.21.15.27:0/1709525749 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b7d77a210 msgr2=0x561b7d77e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:44.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.321+0000 7fed8c855700 1 --2- 172.21.15.27:0/1709525749 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b7d77a210 0x561b7d77e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.321+0000 7fed8c855700 1 -- 172.21.15.27:0/1709525749 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b7d794930 con 0x561b7d779850 2024-05-21T03:23:44.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.321+0000 7fed8c855700 1 --2- 172.21.15.27:0/1709525749 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b7d779850 0x561b7d774220 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fed7c005b00 tx=0x7fed7c01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa30e33b56a3d54 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:44.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.321+0000 7fed727fc700 1 -- 172.21.15.27:0/1709525749 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c043440 con 0x561b7d779850 2024-05-21T03:23:44.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.321+0000 7fed727fc700 1 -- 172.21.15.27:0/1709525749 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed7c043a90 con 0x561b7d779850 2024-05-21T03:23:44.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.321+0000 7fed727fc700 1 -- 172.21.15.27:0/1709525749 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c024900 con 0x561b7d779850 2024-05-21T03:23:44.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.322+0000 7fed9934fb80 1 -- 172.21.15.27:0/1709525749 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b7d779850 msgr2=0x561b7d774220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:44.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.322+0000 7fed9934fb80 1 --2- 172.21.15.27:0/1709525749 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b7d779850 0x561b7d774220 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fed7c005b00 tx=0x7fed7c01a040 comp rx=0 tx=0).stop 2024-05-21T03:23:44.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.322+0000 7fed9934fb80 1 -- 172.21.15.27:0/1709525749 shutdown_connections 2024-05-21T03:23:44.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.322+0000 7fed9934fb80 1 --2- 172.21.15.27:0/1709525749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561b7d77f2d0 0x561b7d77e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.322+0000 7fed9934fb80 1 --2- 172.21.15.27:0/1709525749 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b7d779850 0x561b7d774220 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.322+0000 7fed9934fb80 1 --2- 172.21.15.27:0/1709525749 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b7d77a210 0x561b7d77e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.322+0000 7fed9934fb80 1 -- 172.21.15.27:0/1709525749 >> 172.21.15.27:0/1709525749 conn(0x561b7d6a51f0 msgr2=0x561b7d792080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:44.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.322+0000 7fed9934fb80 1 -- 172.21.15.27:0/1709525749 shutdown_connections 2024-05-21T03:23:44.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.322+0000 7fed9934fb80 1 -- 172.21.15.27:0/1709525749 wait complete. 2024-05-21T03:23:44.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.322+0000 7fed9934fb80 1 -- start start 2024-05-21T03:23:44.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.322+0000 7fed9934fb80 1 Processor -- start 2024-05-21T03:23:44.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.323+0000 7fed9934fb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b7d779850 0x561b7d7737f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:44.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.323+0000 7fed9934fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b7d77a210 0x561b7d773d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:44.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.323+0000 7fed9934fb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561b7d77f2d0 0x561b7d774270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:44.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.323+0000 7fed9934fb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561b7d794440 con 0x561b7d779850 2024-05-21T03:23:44.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.323+0000 7fed9934fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561b7d793d70 con 0x561b7d77a210 2024-05-21T03:23:44.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.323+0000 7fed9934fb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561b7d794b10 con 0x561b7d77f2d0 2024-05-21T03:23:44.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.323+0000 7fed9934fb80 10 client.0 _ll_register_callbacks cb 0x7fed800a8dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:23:44.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.323+0000 7fed8c855700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b7d779850 0x561b7d7737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:44.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.323+0000 7fed8c855700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b7d779850 0x561b7d7737f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35280/0 (socket says 172.21.15.27:35280) 2024-05-21T03:23:44.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.323+0000 7fed8c855700 1 -- 172.21.15.27:0/2232890514 learned_addr learned my addr 172.21.15.27:0/2232890514 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:44.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.323+0000 7fed87fff700 1 --2- 172.21.15.27:0/2232890514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b7d77a210 0x561b7d773d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:44.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.323+0000 7fed8d056700 1 --2- 172.21.15.27:0/2232890514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561b7d77f2d0 0x561b7d774270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:44.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.323+0000 7fed8c855700 1 -- 172.21.15.27:0/2232890514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561b7d77f2d0 msgr2=0x561b7d774270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:44.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.323+0000 7fed8c855700 1 --2- 172.21.15.27:0/2232890514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561b7d77f2d0 0x561b7d774270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.323+0000 7fed8c855700 1 -- 172.21.15.27:0/2232890514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b7d77a210 msgr2=0x561b7d773d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:44.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.323+0000 7fed8c855700 1 --2- 172.21.15.27:0/2232890514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b7d77a210 0x561b7d773d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.323+0000 7fed8c855700 1 -- 172.21.15.27:0/2232890514 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b7d79be70 con 0x561b7d779850 2024-05-21T03:23:44.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.323+0000 7fed87fff700 1 --2- 172.21.15.27:0/2232890514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b7d77a210 0x561b7d773d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:23:44.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.323+0000 7fed8c855700 1 --2- 172.21.15.27:0/2232890514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b7d779850 0x561b7d7737f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fed7c006010 tx=0x7fed7c00fb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974e26dcba7cb9a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:44.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.323+0000 7fed67fff700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:23:44.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.324+0000 7fed67fff700 1 -- 172.21.15.27:0/2232890514 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c047070 con 0x561b7d779850 2024-05-21T03:23:44.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.324+0000 7fed67fff700 1 -- 172.21.15.27:0/2232890514 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fed7c043790 con 0x561b7d779850 2024-05-21T03:23:44.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.324+0000 7fed9934fb80 1 -- 172.21.15.27:0/2232890514 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed7c043a90 con 0x561b7d779850 2024-05-21T03:23:44.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.324+0000 7fed67fff700 1 -- 172.21.15.27:0/2232890514 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fed7c024660 con 0x561b7d779850 2024-05-21T03:23:44.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.324+0000 7fed9934fb80 10 client.15177 fetch_fsmap learned FSMap version 28 2024-05-21T03:23:44.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.324+0000 7fed9934fb80 1 -- 172.21.15.27:0/2232890514 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561b7d7feb70 con 0x561b7d779850 2024-05-21T03:23:44.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.324+0000 7fed67fff700 1 -- 172.21.15.27:0/2232890514 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed7c043a90 con 0x561b7d779850 2024-05-21T03:23:44.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.324+0000 7fed67fff700 1 -- 172.21.15.27:0/2232890514 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fed7c041020 con 0x561b7d779850 2024-05-21T03:23:44.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.324+0000 7fed9934fb80 10 client.15177 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T03:23:44.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.324+0000 7fed9934fb80 10 client.15177 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-21T03:23:44.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.324+0000 7fed9934fb80 20 client.15177 populate_metadata read hostname 'smithi027' 2024-05-21T03:23:44.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.324+0000 7fed9934fb80 1 --2- 172.21.15.27:0/2232890514 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x561b7d802fd0 0x561b7d801ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:44.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.324+0000 7fed9934fb80 4 client.15177 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T03:23:44.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.324+0000 7fed9934fb80 1 -- 172.21.15.27:0/2232890514 --> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] -- command(tid 0: {"prefix": "status"}) -- 0x561b7d6d3450 con 0x561b7d802fd0 2024-05-21T03:23:44.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.325+0000 7fed87fff700 1 --2- 172.21.15.27:0/2232890514 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x561b7d802fd0 0x561b7d801ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:44.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.325+0000 7fed87fff700 1 --2- 172.21.15.27:0/2232890514 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x561b7d802fd0 0x561b7d801ce0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=326774678279536b server_cookie=ae93f562ad59163c in_seq=0 out_seq=0 2024-05-21T03:23:44.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.325+0000 7fed67fff700 10 client.15177 ms_handle_connect on v2:172.21.15.39:6836/3673846852 2024-05-21T03:23:44.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.325+0000 7fed67fff700 1 -- 172.21.15.27:0/2232890514 <== mds.3 v2:172.21.15.39:6836/3673846852 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561b7d6d3450 con 0x561b7d802fd0 2024-05-21T03:23:44.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.325+0000 7fed67fff700 10 client.15177 handle_command_reply: tid=0 2024-05-21T03:23:44.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.325+0000 7fed9934fb80 1 client.15177 shutdown 2024-05-21T03:23:44.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.325+0000 7fed9934fb80 1 -- 172.21.15.27:0/2232890514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b7d779850 msgr2=0x561b7d7737f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:44.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.325+0000 7fed9934fb80 1 --2- 172.21.15.27:0/2232890514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b7d779850 0x561b7d7737f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fed7c006010 tx=0x7fed7c00fb10 comp rx=0 tx=0).stop 2024-05-21T03:23:44.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.325+0000 7fed9934fb80 1 -- 172.21.15.27:0/2232890514 shutdown_connections 2024-05-21T03:23:44.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.325+0000 7fed9934fb80 1 --2- 172.21.15.27:0/2232890514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561b7d77f2d0 0x561b7d774270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.325+0000 7fed9934fb80 1 --2- 172.21.15.27:0/2232890514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561b7d779850 0x561b7d7737f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.325+0000 7fed9934fb80 1 --2- 172.21.15.27:0/2232890514 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x561b7d802fd0 0x561b7d801ce0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.325+0000 7fed67fff700 0 client.15177 ms_handle_reset on v2:172.21.15.39:6836/3673846852 2024-05-21T03:23:44.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.325+0000 7fed9934fb80 1 --2- 172.21.15.27:0/2232890514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b7d77a210 0x561b7d773d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.325+0000 7fed9934fb80 1 -- 172.21.15.27:0/2232890514 >> 172.21.15.27:0/2232890514 conn(0x561b7d6a51f0 msgr2=0x561b7d77cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:44.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.326+0000 7fed9934fb80 1 -- 172.21.15.27:0/2232890514 shutdown_connections 2024-05-21T03:23:44.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.326+0000 7fed9934fb80 1 -- 172.21.15.27:0/2232890514 wait complete. 2024-05-21T03:23:44.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.326+0000 7fed9934fb80 20 client.15177 trim_cache size 0 max 16384 2024-05-21T03:23:44.331 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:23:44.331 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "c9ca94be-171f-11ef-bc9a-c7b262605968", 2024-05-21T03:23:44.331 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 3, 2024-05-21T03:23:44.331 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24452, 2024-05-21T03:23:44.331 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-05-21T03:23:44.331 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-05-21T03:23:44.332 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-05-21T03:23:44.332 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 175.88156276500001, 2024-05-21T03:23:44.332 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 28, 2024-05-21T03:23:44.332 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 82, 2024-05-21T03:23:44.332 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 80, 2024-05-21T03:23:44.332 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 217.079078192 2024-05-21T03:23:44.332 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:23:44.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.326+0000 7fed8eab9700 1 -- 172.21.15.27:0/3058305533 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fed780b2880 msgr2=0x7fed780b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:44.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.326+0000 7fed8eab9700 1 --2- 172.21.15.27:0/3058305533 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fed780b2880 0x7fed780b4d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fed88108470 tx=0x7fed7c005c40 comp rx=0 tx=0).stop 2024-05-21T03:23:44.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.326+0000 7fed8eab9700 1 -- 172.21.15.27:0/3058305533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed88108910 msgr2=0x7fed88199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:44.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.326+0000 7fed8eab9700 1 --2- 172.21.15.27:0/3058305533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed88108910 0x7fed88199a10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fed80007ee0 tx=0x7fed8000d3b0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.326+0000 7fed8eab9700 1 -- 172.21.15.27:0/3058305533 shutdown_connections 2024-05-21T03:23:44.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.326+0000 7fed8eab9700 1 --2- 172.21.15.27:0/3058305533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed88108910 0x7fed88199a10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.326+0000 7fed8eab9700 1 --2- 172.21.15.27:0/3058305533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed88103820 0x7fed8819f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.326+0000 7fed8eab9700 1 --2- 172.21.15.27:0/3058305533 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fed780b2880 0x7fed780b4d40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.326+0000 7fed8eab9700 1 --2- 172.21.15.27:0/3058305533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed881041d0 0x7fed8819fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:44.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.326+0000 7fed8eab9700 1 -- 172.21.15.27:0/3058305533 >> 172.21.15.27:0/3058305533 conn(0x7fed8807a6b0 msgr2=0x7fed88102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:44.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.326+0000 7fed8eab9700 1 -- 172.21.15.27:0/3058305533 shutdown_connections 2024-05-21T03:23:44.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:44.326+0000 7fed8eab9700 1 -- 172.21.15.27:0/3058305533 wait complete. 2024-05-21T03:23:44.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:44 smithi027 ceph-mon[32179]: pgmap v387: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:23:45.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:44 smithi039 ceph-mon[36429]: pgmap v387: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:23:45.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:44 smithi165 ceph-mon[36269]: pgmap v387: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:23:45.056 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell mds.c status 2024-05-21T03:23:45.308 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:23:46.744 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:46 smithi027 ceph-mon[32179]: pgmap v388: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:23:46.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.850+0000 7f73d3ab4700 1 -- 172.21.15.27:0/46167708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73cc103150 msgr2=0x7f73cc10f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:46.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.850+0000 7f73d3ab4700 1 --2- 172.21.15.27:0/46167708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73cc103150 0x7f73cc10f2a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f73c40099c0 tx=0x7f73c4009cd0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.851+0000 7f73d3ab4700 1 -- 172.21.15.27:0/46167708 shutdown_connections 2024-05-21T03:23:46.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.851+0000 7f73d3ab4700 1 --2- 172.21.15.27:0/46167708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73cc103150 0x7f73cc10f2a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.851+0000 7f73d3ab4700 1 --2- 172.21.15.27:0/46167708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73cc108630 0x7f73cc108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.851+0000 7f73d3ab4700 1 --2- 172.21.15.27:0/46167708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73cc102790 0x7f73cc102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.851+0000 7f73d3ab4700 1 -- 172.21.15.27:0/46167708 >> 172.21.15.27:0/46167708 conn(0x7f73cc0782b0 msgr2=0x7f73cc0786c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:46.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.851+0000 7f73d3ab4700 1 -- 172.21.15.27:0/46167708 shutdown_connections 2024-05-21T03:23:46.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.851+0000 7f73d3ab4700 1 -- 172.21.15.27:0/46167708 wait complete. 2024-05-21T03:23:46.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.851+0000 7f73d3ab4700 1 Processor -- start 2024-05-21T03:23:46.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.851+0000 7f73d3ab4700 1 -- start start 2024-05-21T03:23:46.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.852+0000 7f73d3ab4700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73cc102790 0x7f73cc104100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:46.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.852+0000 7f73d3ab4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73cc103150 0x7f73cc104640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:46.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.852+0000 7f73d3ab4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73cc108630 0x7f73cc104b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:46.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.852+0000 7f73d3ab4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc1126c0 con 0x7f73cc103150 2024-05-21T03:23:46.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.852+0000 7f73d3ab4700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc112540 con 0x7f73cc108630 2024-05-21T03:23:46.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.852+0000 7f73d3ab4700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc112840 con 0x7f73cc102790 2024-05-21T03:23:46.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.852+0000 7f73d104f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73cc103150 0x7f73cc104640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:46.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.852+0000 7f73d104f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73cc103150 0x7f73cc104640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35294/0 (socket says 172.21.15.27:35294) 2024-05-21T03:23:46.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.852+0000 7f73d104f700 1 -- 172.21.15.27:0/788417636 learned_addr learned my addr 172.21.15.27:0/788417636 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:46.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.852+0000 7f73d2051700 1 --2- 172.21.15.27:0/788417636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73cc108630 0x7f73cc104b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:46.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.852+0000 7f73d1850700 1 --2- 172.21.15.27:0/788417636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73cc102790 0x7f73cc104100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:46.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.852+0000 7f73d104f700 1 -- 172.21.15.27:0/788417636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73cc102790 msgr2=0x7f73cc104100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:46.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.852+0000 7f73d104f700 1 --2- 172.21.15.27:0/788417636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73cc102790 0x7f73cc104100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.852+0000 7f73d104f700 1 -- 172.21.15.27:0/788417636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73cc108630 msgr2=0x7f73cc104b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:46.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.852+0000 7f73d104f700 1 --2- 172.21.15.27:0/788417636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73cc108630 0x7f73cc104b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.852+0000 7f73d104f700 1 -- 172.21.15.27:0/788417636 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73cc1a8020 con 0x7f73cc103150 2024-05-21T03:23:46.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.852+0000 7f73d104f700 1 --2- 172.21.15.27:0/788417636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73cc103150 0x7f73cc104640 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f73c000d980 tx=0x7f73c000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:46.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.852+0000 7f73caffd700 1 -- 172.21.15.27:0/788417636 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c0017070 con 0x7f73cc103150 2024-05-21T03:23:46.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.852+0000 7f73caffd700 1 -- 172.21.15.27:0/788417636 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73c0004830 con 0x7f73cc103150 2024-05-21T03:23:46.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.852+0000 7f73caffd700 1 -- 172.21.15.27:0/788417636 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c0005440 con 0x7f73cc103150 2024-05-21T03:23:46.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.852+0000 7f73d3ab4700 1 -- 172.21.15.27:0/788417636 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73cc1a8310 con 0x7f73cc103150 2024-05-21T03:23:46.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.853+0000 7f73d3ab4700 1 -- 172.21.15.27:0/788417636 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73cc1a8800 con 0x7f73cc103150 2024-05-21T03:23:46.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.855+0000 7f73caffd700 1 -- 172.21.15.27:0/788417636 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f73c00055e0 con 0x7f73cc103150 2024-05-21T03:23:46.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.856+0000 7f73caffd700 1 --2- 172.21.15.27:0/788417636 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f73b40b2880 0x7f73b40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:46.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.856+0000 7f73d1850700 1 --2- 172.21.15.27:0/788417636 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f73b40b2880 0x7f73b40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:46.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.856+0000 7f73caffd700 1 -- 172.21.15.27:0/788417636 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f73c00e2240 con 0x7f73cc103150 2024-05-21T03:23:46.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.856+0000 7f73de34ab80 1 Processor -- start 2024-05-21T03:23:46.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.857+0000 7f73d1850700 1 --2- 172.21.15.27:0/788417636 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f73b40b2880 0x7f73b40b4d40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00d150 tx=0x7f73bc009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:46.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.857+0000 7f73de34ab80 1 -- start start 2024-05-21T03:23:46.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.857+0000 7f73de34ab80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b42065810 0x559b42065bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:46.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.857+0000 7f73de34ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b420661c0 0x559b420698f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:46.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.857+0000 7f73de34ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b42069f10 0x559b4206a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:46.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.857+0000 7f73de34ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559b41fc4c40 con 0x559b42069f10 2024-05-21T03:23:46.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.857+0000 7f73de34ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559b41fb47e0 con 0x559b420661c0 2024-05-21T03:23:46.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.857+0000 7f73de34ab80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559b41fb8820 con 0x559b42065810 2024-05-21T03:23:46.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.857+0000 7f73d2051700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b42069f10 0x559b4206a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:46.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.857+0000 7f73d2051700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b42069f10 0x559b4206a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35302/0 (socket says 172.21.15.27:35302) 2024-05-21T03:23:46.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.857+0000 7f73d2051700 1 -- 172.21.15.27:0/1711686033 learned_addr learned my addr 172.21.15.27:0/1711686033 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:46.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.857+0000 7f73d104f700 1 --2- 172.21.15.27:0/1711686033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b420661c0 0x559b420698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:46.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.857+0000 7f73d1850700 1 --2- 172.21.15.27:0/1711686033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b42065810 0x559b42065bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:46.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.857+0000 7f73d2051700 1 -- 172.21.15.27:0/1711686033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b42065810 msgr2=0x559b42065bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:46.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.857+0000 7f73d2051700 1 --2- 172.21.15.27:0/1711686033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b42065810 0x559b42065bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.857+0000 7f73d2051700 1 -- 172.21.15.27:0/1711686033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b420661c0 msgr2=0x559b420698f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:46.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.857+0000 7f73d2051700 1 --2- 172.21.15.27:0/1711686033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b420661c0 0x559b420698f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.857+0000 7f73d2051700 1 -- 172.21.15.27:0/1711686033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b41f04880 con 0x559b42069f10 2024-05-21T03:23:46.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.858+0000 7f73d2051700 1 --2- 172.21.15.27:0/1711686033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b42069f10 0x559b4206a350 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f73c40099c0 tx=0x7f73c4005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e9b60619b3cc5d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:46.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.858+0000 7f73b3fff700 1 -- 172.21.15.27:0/1711686033 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c4027070 con 0x559b42069f10 2024-05-21T03:23:46.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.858+0000 7f73b3fff700 1 -- 172.21.15.27:0/1711686033 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73c4005070 con 0x559b42069f10 2024-05-21T03:23:46.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.858+0000 7f73b3fff700 1 -- 172.21.15.27:0/1711686033 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c400b9b0 con 0x559b42069f10 2024-05-21T03:23:46.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.858+0000 7f73de34ab80 1 -- 172.21.15.27:0/1711686033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b42069f10 msgr2=0x559b4206a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:46.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.858+0000 7f73de34ab80 1 --2- 172.21.15.27:0/1711686033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b42069f10 0x559b4206a350 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f73c40099c0 tx=0x7f73c4005c80 comp rx=0 tx=0).stop 2024-05-21T03:23:46.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.859+0000 7f73de34ab80 1 -- 172.21.15.27:0/1711686033 shutdown_connections 2024-05-21T03:23:46.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.859+0000 7f73de34ab80 1 --2- 172.21.15.27:0/1711686033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b42069f10 0x559b4206a350 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.859+0000 7f73de34ab80 1 --2- 172.21.15.27:0/1711686033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b42065810 0x559b42065bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.859+0000 7f73de34ab80 1 --2- 172.21.15.27:0/1711686033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b420661c0 0x559b420698f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.859+0000 7f73de34ab80 1 -- 172.21.15.27:0/1711686033 >> 172.21.15.27:0/1711686033 conn(0x559b41f93030 msgr2=0x559b4205fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:46.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.859+0000 7f73de34ab80 1 -- 172.21.15.27:0/1711686033 shutdown_connections 2024-05-21T03:23:46.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.859+0000 7f73de34ab80 1 -- 172.21.15.27:0/1711686033 wait complete. 2024-05-21T03:23:46.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.859+0000 7f73de34ab80 1 -- start start 2024-05-21T03:23:46.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.859+0000 7f73de34ab80 1 Processor -- start 2024-05-21T03:23:46.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.860+0000 7f73de34ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b42065810 0x559b42071360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:46.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.860+0000 7f73de34ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b420661c0 0x559b4206e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:46.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.860+0000 7f73de34ab80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b42069f10 0x559b4206eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:46.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.860+0000 7f73de34ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559b41ebaf50 con 0x559b42065810 2024-05-21T03:23:46.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.860+0000 7f73de34ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559b41f56770 con 0x559b420661c0 2024-05-21T03:23:46.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.860+0000 7f73de34ab80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559b42072070 con 0x559b42069f10 2024-05-21T03:23:46.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.860+0000 7f73de34ab80 10 client.0 _ll_register_callbacks cb 0x559b41f4f8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:23:46.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.860+0000 7f73d1850700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b42065810 0x559b42071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:46.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.860+0000 7f73d1850700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b42065810 0x559b42071360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35306/0 (socket says 172.21.15.27:35306) 2024-05-21T03:23:46.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.860+0000 7f73d1850700 1 -- 172.21.15.27:0/370367967 learned_addr learned my addr 172.21.15.27:0/370367967 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:46.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.860+0000 7f73d104f700 1 --2- 172.21.15.27:0/370367967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b420661c0 0x559b4206e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:46.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.860+0000 7f73d2051700 1 --2- 172.21.15.27:0/370367967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b42069f10 0x559b4206eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:46.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.860+0000 7f73d1850700 1 -- 172.21.15.27:0/370367967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b42069f10 msgr2=0x559b4206eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:46.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.860+0000 7f73d1850700 1 --2- 172.21.15.27:0/370367967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b42069f10 0x559b4206eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.860+0000 7f73d1850700 1 -- 172.21.15.27:0/370367967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b420661c0 msgr2=0x559b4206e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:46.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.860+0000 7f73d1850700 1 --2- 172.21.15.27:0/370367967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b420661c0 0x559b4206e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.860+0000 7f73d1850700 1 -- 172.21.15.27:0/370367967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b41f53640 con 0x559b42065810 2024-05-21T03:23:46.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.860+0000 7f73d2051700 1 --2- 172.21.15.27:0/370367967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b42069f10 0x559b4206eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:23:46.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.860+0000 7f73d1850700 1 --2- 172.21.15.27:0/370367967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b42065810 0x559b42071360 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f73bc0149d0 tx=0x7f73bc004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba25a1e710bccde7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:46.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.860+0000 7f73b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:23:46.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.860+0000 7f73b17fa700 1 -- 172.21.15.27:0/370367967 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73bc014e40 con 0x559b42065810 2024-05-21T03:23:46.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.860+0000 7f73b17fa700 1 -- 172.21.15.27:0/370367967 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f73bc012a70 con 0x559b42065810 2024-05-21T03:23:46.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.860+0000 7f73de34ab80 1 -- 172.21.15.27:0/370367967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73c4005070 con 0x559b42065810 2024-05-21T03:23:46.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.861+0000 7f73b17fa700 1 -- 172.21.15.27:0/370367967 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f73bc012c70 con 0x559b42065810 2024-05-21T03:23:46.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.861+0000 7f73de34ab80 10 client.15189 fetch_fsmap learned FSMap version 28 2024-05-21T03:23:46.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.861+0000 7f73de34ab80 1 -- 172.21.15.27:0/370367967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559b4207c770 con 0x559b42065810 2024-05-21T03:23:46.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.861+0000 7f73b17fa700 1 -- 172.21.15.27:0/370367967 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73c4005070 con 0x559b42065810 2024-05-21T03:23:46.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.861+0000 7f73b17fa700 1 -- 172.21.15.27:0/370367967 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f73bc020a90 con 0x559b42065810 2024-05-21T03:23:46.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.861+0000 7f73de34ab80 10 client.15189 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T03:23:46.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.861+0000 7f73de34ab80 10 client.15189 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-21T03:23:46.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.861+0000 7f73de34ab80 20 client.15189 populate_metadata read hostname 'smithi027' 2024-05-21T03:23:46.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.861+0000 7f73de34ab80 1 --2- 172.21.15.27:0/370367967 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x559b4207ee10 0x559b42080280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:46.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.861+0000 7f73de34ab80 4 client.15189 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:23:46.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.861+0000 7f73de34ab80 1 -- 172.21.15.27:0/370367967 --> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b42031990 con 0x559b4207ee10 2024-05-21T03:23:46.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.861+0000 7f73d104f700 1 --2- 172.21.15.27:0/370367967 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x559b4207ee10 0x559b42080280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:46.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.862+0000 7f73d104f700 1 --2- 172.21.15.27:0/370367967 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x559b4207ee10 0x559b42080280 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a89181ce2f7ea9a5 server_cookie=d5058cfaeac29c8d in_seq=0 out_seq=0 2024-05-21T03:23:46.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.862+0000 7f73b17fa700 10 client.15189 ms_handle_connect on v2:172.21.15.165:6836/691356139 2024-05-21T03:23:46.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.863+0000 7f73b17fa700 1 -- 172.21.15.27:0/370367967 <== mds.2 v2:172.21.15.165:6836/691356139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559b42031990 con 0x559b4207ee10 2024-05-21T03:23:46.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.863+0000 7f73b17fa700 10 client.15189 handle_command_reply: tid=0 2024-05-21T03:23:46.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.863+0000 7f73de34ab80 1 client.15189 shutdown 2024-05-21T03:23:46.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.863+0000 7f73de34ab80 1 -- 172.21.15.27:0/370367967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b42065810 msgr2=0x559b42071360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:46.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.863+0000 7f73de34ab80 1 --2- 172.21.15.27:0/370367967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b42065810 0x559b42071360 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f73bc0149d0 tx=0x7f73bc004b40 comp rx=0 tx=0).stop 2024-05-21T03:23:46.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.863+0000 7f73de34ab80 1 -- 172.21.15.27:0/370367967 shutdown_connections 2024-05-21T03:23:46.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.863+0000 7f73de34ab80 1 --2- 172.21.15.27:0/370367967 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x559b4207ee10 0x559b42080280 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.863+0000 7f73de34ab80 1 --2- 172.21.15.27:0/370367967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b42069f10 0x559b4206eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.863+0000 7f73de34ab80 1 --2- 172.21.15.27:0/370367967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b42065810 0x559b42071360 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.863+0000 7f73de34ab80 1 --2- 172.21.15.27:0/370367967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b420661c0 0x559b4206e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.863+0000 7f73b17fa700 0 client.15189 ms_handle_reset on v2:172.21.15.165:6836/691356139 2024-05-21T03:23:46.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.863+0000 7f73de34ab80 1 -- 172.21.15.27:0/370367967 >> 172.21.15.27:0/370367967 conn(0x559b41f93030 msgr2=0x559b4205f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:46.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.863+0000 7f73de34ab80 1 -- 172.21.15.27:0/370367967 shutdown_connections 2024-05-21T03:23:46.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.863+0000 7f73de34ab80 1 -- 172.21.15.27:0/370367967 wait complete. 2024-05-21T03:23:46.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.863+0000 7f73de34ab80 20 client.15189 trim_cache size 0 max 16384 2024-05-21T03:23:46.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.884+0000 7f73de34ab80 1 Processor -- start 2024-05-21T03:23:46.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.885+0000 7f73de34ab80 1 -- start start 2024-05-21T03:23:46.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.885+0000 7f73de34ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b42065810 0x559b4206ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:46.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.885+0000 7f73de34ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b420661c0 0x559b4205fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:46.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.885+0000 7f73de34ab80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b42069f10 0x559b420896e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:46.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.885+0000 7f73de34ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559b4207e5c0 con 0x559b420661c0 2024-05-21T03:23:46.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.885+0000 7f73de34ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559b4207ec90 con 0x559b42065810 2024-05-21T03:23:46.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.885+0000 7f73de34ab80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559b4207f360 con 0x559b42069f10 2024-05-21T03:23:46.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.885+0000 7f73d104f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b420661c0 0x559b4205fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:46.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.885+0000 7f73d104f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b420661c0 0x559b4205fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35314/0 (socket says 172.21.15.27:35314) 2024-05-21T03:23:46.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.885+0000 7f73d104f700 1 -- 172.21.15.27:0/3225783522 learned_addr learned my addr 172.21.15.27:0/3225783522 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:46.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.885+0000 7f73d1850700 1 --2- 172.21.15.27:0/3225783522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b42065810 0x559b4206ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:46.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.885+0000 7f73d104f700 1 -- 172.21.15.27:0/3225783522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b42069f10 msgr2=0x559b420896e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:46.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.885+0000 7f73d104f700 1 --2- 172.21.15.27:0/3225783522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b42069f10 0x559b420896e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.885+0000 7f73d104f700 1 -- 172.21.15.27:0/3225783522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b42065810 msgr2=0x559b4206ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:46.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.885+0000 7f73d104f700 1 --2- 172.21.15.27:0/3225783522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b42065810 0x559b4206ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.885+0000 7f73d104f700 1 -- 172.21.15.27:0/3225783522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b4206d930 con 0x559b420661c0 2024-05-21T03:23:46.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.885+0000 7f73d104f700 1 --2- 172.21.15.27:0/3225783522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b420661c0 0x559b4205fc40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f73c00a6a50 tx=0x7f73c00b2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420f1b655363e48d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:46.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.885+0000 7f73b37fe700 1 -- 172.21.15.27:0/3225783522 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c00dfdd0 con 0x559b420661c0 2024-05-21T03:23:46.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.885+0000 7f73b37fe700 1 -- 172.21.15.27:0/3225783522 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73c00e1040 con 0x559b420661c0 2024-05-21T03:23:46.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.885+0000 7f73b37fe700 1 -- 172.21.15.27:0/3225783522 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c00b21b0 con 0x559b420661c0 2024-05-21T03:23:46.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.886+0000 7f73de34ab80 1 -- 172.21.15.27:0/3225783522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b420661c0 msgr2=0x559b4205fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:46.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.886+0000 7f73de34ab80 1 --2- 172.21.15.27:0/3225783522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b420661c0 0x559b4205fc40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f73c00a6a50 tx=0x7f73c00b2040 comp rx=0 tx=0).stop 2024-05-21T03:23:46.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.886+0000 7f73de34ab80 1 -- 172.21.15.27:0/3225783522 shutdown_connections 2024-05-21T03:23:46.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.886+0000 7f73de34ab80 1 --2- 172.21.15.27:0/3225783522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b42069f10 0x559b420896e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.886+0000 7f73de34ab80 1 --2- 172.21.15.27:0/3225783522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b42065810 0x559b4206ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.886+0000 7f73de34ab80 1 --2- 172.21.15.27:0/3225783522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b420661c0 0x559b4205fc40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.886+0000 7f73de34ab80 1 -- 172.21.15.27:0/3225783522 >> 172.21.15.27:0/3225783522 conn(0x559b41f93030 msgr2=0x559b420690f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:46.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.886+0000 7f73de34ab80 1 -- 172.21.15.27:0/3225783522 shutdown_connections 2024-05-21T03:23:46.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.886+0000 7f73de34ab80 1 -- 172.21.15.27:0/3225783522 wait complete. 2024-05-21T03:23:46.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.886+0000 7f73de34ab80 1 -- start start 2024-05-21T03:23:46.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.887+0000 7f73de34ab80 1 Processor -- start 2024-05-21T03:23:46.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.887+0000 7f73de34ab80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b42065810 0x559b420899f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:46.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.887+0000 7f73de34ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b420661c0 0x559b42089f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:46.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.887+0000 7f73de34ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b42069f10 0x559b42085f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:46.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.887+0000 7f73de34ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559b42080100 con 0x559b420661c0 2024-05-21T03:23:46.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.887+0000 7f73de34ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559b4207fa30 con 0x559b42069f10 2024-05-21T03:23:46.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.887+0000 7f73de34ab80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559b420807d0 con 0x559b42065810 2024-05-21T03:23:46.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.887+0000 7f73de34ab80 10 client.0 _ll_register_callbacks cb 0x559b41f4f8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:23:46.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.887+0000 7f73d104f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b420661c0 0x559b42089f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:46.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.887+0000 7f73d104f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b420661c0 0x559b42089f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35328/0 (socket says 172.21.15.27:35328) 2024-05-21T03:23:46.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.887+0000 7f73d104f700 1 -- 172.21.15.27:0/821528253 learned_addr learned my addr 172.21.15.27:0/821528253 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:46.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.887+0000 7f73d2051700 1 --2- 172.21.15.27:0/821528253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b42069f10 0x559b42085f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:46.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.887+0000 7f73d1850700 1 --2- 172.21.15.27:0/821528253 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b42065810 0x559b420899f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:46.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.888+0000 7f73d104f700 1 -- 172.21.15.27:0/821528253 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b42065810 msgr2=0x559b420899f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:46.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.888+0000 7f73d104f700 1 --2- 172.21.15.27:0/821528253 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b42065810 0x559b420899f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.888+0000 7f73d104f700 1 -- 172.21.15.27:0/821528253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b42069f10 msgr2=0x559b42085f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:46.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.888+0000 7f73d104f700 1 --2- 172.21.15.27:0/821528253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b42069f10 0x559b42085f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.888+0000 7f73d104f700 1 -- 172.21.15.27:0/821528253 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b42086780 con 0x559b420661c0 2024-05-21T03:23:46.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.888+0000 7f73d104f700 1 --2- 172.21.15.27:0/821528253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b420661c0 0x559b42089f30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f73c00a9770 tx=0x7f73c000def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cef29c7bf84d1bd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:46.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.888+0000 7f73b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:23:46.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.888+0000 7f73b17fa700 1 -- 172.21.15.27:0/821528253 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c006f070 con 0x559b420661c0 2024-05-21T03:23:46.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.888+0000 7f73b17fa700 1 -- 172.21.15.27:0/821528253 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f73c006f210 con 0x559b420661c0 2024-05-21T03:23:46.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.888+0000 7f73de34ab80 1 -- 172.21.15.27:0/821528253 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73c00e1040 con 0x559b420661c0 2024-05-21T03:23:46.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.888+0000 7f73b17fa700 1 -- 172.21.15.27:0/821528253 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f73c00a9a40 con 0x559b420661c0 2024-05-21T03:23:46.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.888+0000 7f73de34ab80 10 client.15195 fetch_fsmap learned FSMap version 28 2024-05-21T03:23:46.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.888+0000 7f73de34ab80 1 -- 172.21.15.27:0/821528253 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559b420eb240 con 0x559b420661c0 2024-05-21T03:23:46.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.888+0000 7f73b17fa700 1 -- 172.21.15.27:0/821528253 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73c00e1040 con 0x559b420661c0 2024-05-21T03:23:46.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.888+0000 7f73b17fa700 1 -- 172.21.15.27:0/821528253 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f73c00aab10 con 0x559b420661c0 2024-05-21T03:23:46.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.888+0000 7f73de34ab80 10 client.15195 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T03:23:46.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.888+0000 7f73de34ab80 10 client.15195 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-21T03:23:46.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.888+0000 7f73de34ab80 20 client.15195 populate_metadata read hostname 'smithi027' 2024-05-21T03:23:46.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.889+0000 7f73de34ab80 1 --2- 172.21.15.27:0/821528253 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x559b420ef890 0x559b420ee5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:46.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.889+0000 7f73de34ab80 4 client.15195 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T03:23:46.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.889+0000 7f73de34ab80 1 -- 172.21.15.27:0/821528253 --> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] -- command(tid 0: {"prefix": "status"}) -- 0x559b41fc6b90 con 0x559b420ef890 2024-05-21T03:23:46.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.889+0000 7f73d1850700 1 --2- 172.21.15.27:0/821528253 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x559b420ef890 0x559b420ee5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:46.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.889+0000 7f73d1850700 1 --2- 172.21.15.27:0/821528253 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x559b420ef890 0x559b420ee5a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fe547ac83f43ee79 server_cookie=38665c57aed87697 in_seq=0 out_seq=0 2024-05-21T03:23:46.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.889+0000 7f73b17fa700 10 client.15195 ms_handle_connect on v2:172.21.15.165:6836/691356139 2024-05-21T03:23:46.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.890+0000 7f73b17fa700 1 -- 172.21.15.27:0/821528253 <== mds.2 v2:172.21.15.165:6836/691356139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x559b41fc6b90 con 0x559b420ef890 2024-05-21T03:23:46.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.890+0000 7f73b17fa700 10 client.15195 handle_command_reply: tid=0 2024-05-21T03:23:46.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.890+0000 7f73de34ab80 1 client.15195 shutdown 2024-05-21T03:23:46.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.890+0000 7f73de34ab80 1 -- 172.21.15.27:0/821528253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b420661c0 msgr2=0x559b42089f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:46.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.890+0000 7f73de34ab80 1 --2- 172.21.15.27:0/821528253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b420661c0 0x559b42089f30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f73c00a9770 tx=0x7f73c000def0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.890+0000 7f73de34ab80 1 -- 172.21.15.27:0/821528253 shutdown_connections 2024-05-21T03:23:46.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.890+0000 7f73de34ab80 1 --2- 172.21.15.27:0/821528253 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x559b420ef890 0x559b420ee5a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.890+0000 7f73de34ab80 1 --2- 172.21.15.27:0/821528253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b42069f10 0x559b42085f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.890+0000 7f73b17fa700 0 client.15195 ms_handle_reset on v2:172.21.15.165:6836/691356139 2024-05-21T03:23:46.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.890+0000 7f73de34ab80 1 --2- 172.21.15.27:0/821528253 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b42065810 0x559b420899f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.890+0000 7f73de34ab80 1 --2- 172.21.15.27:0/821528253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559b420661c0 0x559b42089f30 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.890+0000 7f73de34ab80 1 -- 172.21.15.27:0/821528253 >> 172.21.15.27:0/821528253 conn(0x559b41f93030 msgr2=0x559b42064620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:46.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.890+0000 7f73de34ab80 1 -- 172.21.15.27:0/821528253 shutdown_connections 2024-05-21T03:23:46.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.890+0000 7f73de34ab80 1 -- 172.21.15.27:0/821528253 wait complete. 2024-05-21T03:23:46.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.890+0000 7f73de34ab80 20 client.15195 trim_cache size 0 max 16384 2024-05-21T03:23:46.897 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:23:46.898 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "c9ca94be-171f-11ef-bc9a-c7b262605968", 2024-05-21T03:23:46.898 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 2, 2024-05-21T03:23:46.898 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24457, 2024-05-21T03:23:46.898 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-05-21T03:23:46.898 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-05-21T03:23:46.898 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-05-21T03:23:46.898 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 179.45367294499999, 2024-05-21T03:23:46.898 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 28, 2024-05-21T03:23:46.898 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 82, 2024-05-21T03:23:46.898 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 80, 2024-05-21T03:23:46.898 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 222.55302266000001 2024-05-21T03:23:46.899 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:23:46.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.890+0000 7f73d3ab4700 1 -- 172.21.15.27:0/788417636 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f73b40b2880 msgr2=0x7f73b40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:46.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.890+0000 7f73d3ab4700 1 --2- 172.21.15.27:0/788417636 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f73b40b2880 0x7f73b40b4d40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00d150 tx=0x7f73bc009040 comp rx=0 tx=0).stop 2024-05-21T03:23:46.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.890+0000 7f73d3ab4700 1 -- 172.21.15.27:0/788417636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73cc103150 msgr2=0x7f73cc104640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:46.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.890+0000 7f73d3ab4700 1 --2- 172.21.15.27:0/788417636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73cc103150 0x7f73cc104640 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f73c000d980 tx=0x7f73c000dd40 comp rx=0 tx=0).stop 2024-05-21T03:23:46.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.891+0000 7f73d3ab4700 1 -- 172.21.15.27:0/788417636 shutdown_connections 2024-05-21T03:23:46.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.891+0000 7f73d3ab4700 1 --2- 172.21.15.27:0/788417636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73cc108630 0x7f73cc104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.891+0000 7f73d3ab4700 1 --2- 172.21.15.27:0/788417636 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f73b40b2880 0x7f73b40b4d40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.891+0000 7f73d3ab4700 1 --2- 172.21.15.27:0/788417636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73cc102790 0x7f73cc104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.891+0000 7f73d3ab4700 1 --2- 172.21.15.27:0/788417636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73cc103150 0x7f73cc104640 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:46.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.891+0000 7f73d3ab4700 1 -- 172.21.15.27:0/788417636 >> 172.21.15.27:0/788417636 conn(0x7f73cc0782b0 msgr2=0x7f73cc1019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:46.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.891+0000 7f73d3ab4700 1 -- 172.21.15.27:0/788417636 shutdown_connections 2024-05-21T03:23:46.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:46.891+0000 7f73d3ab4700 1 -- 172.21.15.27:0/788417636 wait complete. 2024-05-21T03:23:47.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:46 smithi039 ceph-mon[36429]: pgmap v388: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:23:47.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:46 smithi165 ceph-mon[36269]: pgmap v388: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:23:47.610 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell mds.i status 2024-05-21T03:23:47.858 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:23:48.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:48 smithi027 ceph-mon[32179]: pgmap v389: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:23:49.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:48 smithi039 ceph-mon[36429]: pgmap v389: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:23:49.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:48 smithi165 ceph-mon[36269]: pgmap v389: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-21T03:23:49.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.517+0000 7fd65fe72700 1 -- 172.21.15.27:0/4237605138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd658107c30 msgr2=0x7fd658101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:49.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.517+0000 7fd65fe72700 1 --2- 172.21.15.27:0/4237605138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd658107c30 0x7fd658101ee0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd648009990 tx=0x7fd648009ca0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.517+0000 7fd65fe72700 1 -- 172.21.15.27:0/4237605138 shutdown_connections 2024-05-21T03:23:49.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.517+0000 7fd65fe72700 1 --2- 172.21.15.27:0/4237605138 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6581024b0 0x7fd658102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.517+0000 7fd65fe72700 1 --2- 172.21.15.27:0/4237605138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd658107280 0x7fd658107660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.517+0000 7fd65fe72700 1 --2- 172.21.15.27:0/4237605138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd658107c30 0x7fd658101ee0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.517+0000 7fd65fe72700 1 -- 172.21.15.27:0/4237605138 >> 172.21.15.27:0/4237605138 conn(0x7fd658077760 msgr2=0x7fd658077b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:49.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.517+0000 7fd65fe72700 1 -- 172.21.15.27:0/4237605138 shutdown_connections 2024-05-21T03:23:49.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.517+0000 7fd65fe72700 1 -- 172.21.15.27:0/4237605138 wait complete. 2024-05-21T03:23:49.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.517+0000 7fd65fe72700 1 Processor -- start 2024-05-21T03:23:49.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.518+0000 7fd65fe72700 1 -- start start 2024-05-21T03:23:49.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.518+0000 7fd65fe72700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6581024b0 0x7fd6581a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:49.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.518+0000 7fd65fe72700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd658107280 0x7fd6581a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:49.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.518+0000 7fd65fe72700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd658107c30 0x7fd6581a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:49.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.518+0000 7fd65fe72700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd658119e90 con 0x7fd658107280 2024-05-21T03:23:49.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.518+0000 7fd65fe72700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd658119d10 con 0x7fd658107c30 2024-05-21T03:23:49.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.518+0000 7fd65fe72700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd65811a010 con 0x7fd6581024b0 2024-05-21T03:23:49.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.518+0000 7fd65d40d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd658107280 0x7fd6581a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:49.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.518+0000 7fd65d40d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd658107280 0x7fd6581a08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35350/0 (socket says 172.21.15.27:35350) 2024-05-21T03:23:49.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.518+0000 7fd65d40d700 1 -- 172.21.15.27:0/643683344 learned_addr learned my addr 172.21.15.27:0/643683344 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:49.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.518+0000 7fd65dc0e700 1 --2- 172.21.15.27:0/643683344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6581024b0 0x7fd6581a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:49.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.518+0000 7fd65e40f700 1 --2- 172.21.15.27:0/643683344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd658107c30 0x7fd6581a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:49.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.518+0000 7fd65d40d700 1 -- 172.21.15.27:0/643683344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6581024b0 msgr2=0x7fd6581a0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:49.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.518+0000 7fd65d40d700 1 --2- 172.21.15.27:0/643683344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6581024b0 0x7fd6581a0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.518+0000 7fd65d40d700 1 -- 172.21.15.27:0/643683344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd658107c30 msgr2=0x7fd6581a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:49.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.518+0000 7fd65d40d700 1 --2- 172.21.15.27:0/643683344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd658107c30 0x7fd6581a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.518+0000 7fd65d40d700 1 -- 172.21.15.27:0/643683344 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6581a5370 con 0x7fd658107280 2024-05-21T03:23:49.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.518+0000 7fd65dc0e700 1 --2- 172.21.15.27:0/643683344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6581024b0 0x7fd6581a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:23:49.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.518+0000 7fd65d40d700 1 --2- 172.21.15.27:0/643683344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd658107280 0x7fd6581a08a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd6480057f0 tx=0x7fd648003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:49.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.519+0000 7fd64effd700 1 -- 172.21.15.27:0/643683344 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648027070 con 0x7fd658107280 2024-05-21T03:23:49.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.519+0000 7fd64effd700 1 -- 172.21.15.27:0/643683344 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6480041c0 con 0x7fd658107280 2024-05-21T03:23:49.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.519+0000 7fd65fe72700 1 -- 172.21.15.27:0/643683344 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6581a5600 con 0x7fd658107280 2024-05-21T03:23:49.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.519+0000 7fd64effd700 1 -- 172.21.15.27:0/643683344 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648022e60 con 0x7fd658107280 2024-05-21T03:23:49.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.519+0000 7fd65fe72700 1 -- 172.21.15.27:0/643683344 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6581acee0 con 0x7fd658107280 2024-05-21T03:23:49.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.521+0000 7fd64effd700 1 -- 172.21.15.27:0/643683344 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd64802a070 con 0x7fd658107280 2024-05-21T03:23:49.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.522+0000 7fd64effd700 1 --2- 172.21.15.27:0/643683344 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd6440b2880 0x7fd6440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:49.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.523+0000 7fd65dc0e700 1 --2- 172.21.15.27:0/643683344 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd6440b2880 0x7fd6440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:49.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.523+0000 7fd64effd700 1 -- 172.21.15.27:0/643683344 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fd6480e2f60 con 0x7fd658107280 2024-05-21T03:23:49.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.523+0000 7fd66a708b80 1 Processor -- start 2024-05-21T03:23:49.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.523+0000 7fd65dc0e700 1 --2- 172.21.15.27:0/643683344 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd6440b2880 0x7fd6440b4d40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd658078da0 tx=0x7fd654005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:49.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.523+0000 7fd66a708b80 1 -- start start 2024-05-21T03:23:49.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.523+0000 7fd66a708b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adfbf73810 0x55adfbf73bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:49.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.523+0000 7fd66a708b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55adfbf741c0 0x55adfbf778f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:49.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.523+0000 7fd66a708b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55adfbf77f10 0x55adfbf78350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:49.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.523+0000 7fd66a708b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55adfbed2c40 con 0x55adfbf73810 2024-05-21T03:23:49.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.523+0000 7fd66a708b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55adfbec27e0 con 0x55adfbf77f10 2024-05-21T03:23:49.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.523+0000 7fd66a708b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55adfbec6820 con 0x55adfbf741c0 2024-05-21T03:23:49.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.523+0000 7fd65dc0e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adfbf73810 0x55adfbf73bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:49.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.523+0000 7fd65dc0e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adfbf73810 0x55adfbf73bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35358/0 (socket says 172.21.15.27:35358) 2024-05-21T03:23:49.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.523+0000 7fd65dc0e700 1 -- 172.21.15.27:0/2504239756 learned_addr learned my addr 172.21.15.27:0/2504239756 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:49.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.524+0000 7fd65d40d700 1 --2- 172.21.15.27:0/2504239756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55adfbf741c0 0x55adfbf778f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:49.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.524+0000 7fd65dc0e700 1 -- 172.21.15.27:0/2504239756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55adfbf741c0 msgr2=0x55adfbf778f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:49.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.524+0000 7fd65e40f700 1 --2- 172.21.15.27:0/2504239756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55adfbf77f10 0x55adfbf78350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:49.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.524+0000 7fd65dc0e700 1 --2- 172.21.15.27:0/2504239756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55adfbf741c0 0x55adfbf778f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.524+0000 7fd65dc0e700 1 -- 172.21.15.27:0/2504239756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55adfbf77f10 msgr2=0x55adfbf78350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:49.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.524+0000 7fd65dc0e700 1 --2- 172.21.15.27:0/2504239756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55adfbf77f10 0x55adfbf78350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.524+0000 7fd65dc0e700 1 -- 172.21.15.27:0/2504239756 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adfbe12880 con 0x55adfbf73810 2024-05-21T03:23:49.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.524+0000 7fd65dc0e700 1 --2- 172.21.15.27:0/2504239756 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adfbf73810 0x55adfbf73bf0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd65400ecd0 tx=0x7fd654011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1072f7bd9601e7cd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:49.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.524+0000 7fd643fff700 1 -- 172.21.15.27:0/2504239756 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd654021070 con 0x55adfbf73810 2024-05-21T03:23:49.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.524+0000 7fd643fff700 1 -- 172.21.15.27:0/2504239756 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd654004ce0 con 0x55adfbf73810 2024-05-21T03:23:49.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.524+0000 7fd643fff700 1 -- 172.21.15.27:0/2504239756 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65401d450 con 0x55adfbf73810 2024-05-21T03:23:49.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.525+0000 7fd66a708b80 1 -- 172.21.15.27:0/2504239756 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adfbf73810 msgr2=0x55adfbf73bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:49.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.525+0000 7fd66a708b80 1 --2- 172.21.15.27:0/2504239756 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adfbf73810 0x55adfbf73bf0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd65400ecd0 tx=0x7fd654011000 comp rx=0 tx=0).stop 2024-05-21T03:23:49.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.525+0000 7fd66a708b80 1 -- 172.21.15.27:0/2504239756 shutdown_connections 2024-05-21T03:23:49.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.525+0000 7fd66a708b80 1 --2- 172.21.15.27:0/2504239756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55adfbf77f10 0x55adfbf78350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.525+0000 7fd66a708b80 1 --2- 172.21.15.27:0/2504239756 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adfbf73810 0x55adfbf73bf0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.525+0000 7fd66a708b80 1 --2- 172.21.15.27:0/2504239756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55adfbf741c0 0x55adfbf778f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.525+0000 7fd66a708b80 1 -- 172.21.15.27:0/2504239756 >> 172.21.15.27:0/2504239756 conn(0x55adfbea1030 msgr2=0x55adfbf6de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:49.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.525+0000 7fd66a708b80 1 -- 172.21.15.27:0/2504239756 shutdown_connections 2024-05-21T03:23:49.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.525+0000 7fd66a708b80 1 -- 172.21.15.27:0/2504239756 wait complete. 2024-05-21T03:23:49.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.525+0000 7fd66a708b80 1 -- start start 2024-05-21T03:23:49.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.525+0000 7fd66a708b80 1 Processor -- start 2024-05-21T03:23:49.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.526+0000 7fd66a708b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55adfbf73810 0x55adfbf7f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:49.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.526+0000 7fd66a708b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55adfbf741c0 0x55adfbf7c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:49.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.526+0000 7fd66a708b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adfbf77f10 0x55adfbf7cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:49.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.526+0000 7fd66a708b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55adfbdc8f50 con 0x55adfbf77f10 2024-05-21T03:23:49.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.526+0000 7fd66a708b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55adfbe64770 con 0x55adfbf741c0 2024-05-21T03:23:49.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.526+0000 7fd66a708b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55adfbf800d0 con 0x55adfbf73810 2024-05-21T03:23:49.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.526+0000 7fd66a708b80 10 client.0 _ll_register_callbacks cb 0x55adfbe5d8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:23:49.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.526+0000 7fd65e40f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adfbf77f10 0x55adfbf7cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:49.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.526+0000 7fd65e40f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adfbf77f10 0x55adfbf7cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35360/0 (socket says 172.21.15.27:35360) 2024-05-21T03:23:49.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.526+0000 7fd65e40f700 1 -- 172.21.15.27:0/2999506684 learned_addr learned my addr 172.21.15.27:0/2999506684 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:49.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.526+0000 7fd65d40d700 1 --2- 172.21.15.27:0/2999506684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55adfbf741c0 0x55adfbf7c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:49.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.526+0000 7fd65e40f700 1 -- 172.21.15.27:0/2999506684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55adfbf73810 msgr2=0x55adfbf7f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:49.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.526+0000 7fd65e40f700 1 --2- 172.21.15.27:0/2999506684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55adfbf73810 0x55adfbf7f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.526+0000 7fd65e40f700 1 -- 172.21.15.27:0/2999506684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55adfbf741c0 msgr2=0x55adfbf7c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:49.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.526+0000 7fd65e40f700 1 --2- 172.21.15.27:0/2999506684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55adfbf741c0 0x55adfbf7c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.526+0000 7fd65e40f700 1 -- 172.21.15.27:0/2999506684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adfbe61640 con 0x55adfbf77f10 2024-05-21T03:23:49.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.526+0000 7fd65e40f700 1 --2- 172.21.15.27:0/2999506684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adfbf77f10 0x55adfbf7cba0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd650009a90 tx=0x7fd650009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3515451b3d2364 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:49.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.526+0000 7fd6417fa700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:23:49.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.527+0000 7fd6417fa700 1 -- 172.21.15.27:0/2999506684 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6500078c0 con 0x55adfbf77f10 2024-05-21T03:23:49.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.527+0000 7fd6417fa700 1 -- 172.21.15.27:0/2999506684 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fd650007a60 con 0x55adfbf77f10 2024-05-21T03:23:49.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.527+0000 7fd66a708b80 1 -- 172.21.15.27:0/2999506684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd654004ce0 con 0x55adfbf77f10 2024-05-21T03:23:49.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.527+0000 7fd6417fa700 1 -- 172.21.15.27:0/2999506684 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd650007ca0 con 0x55adfbf77f10 2024-05-21T03:23:49.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.527+0000 7fd66a708b80 10 client.15207 fetch_fsmap learned FSMap version 28 2024-05-21T03:23:49.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.527+0000 7fd66a708b80 1 -- 172.21.15.27:0/2999506684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55adfbf8a7f0 con 0x55adfbf77f10 2024-05-21T03:23:49.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.527+0000 7fd6417fa700 1 -- 172.21.15.27:0/2999506684 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd654004ce0 con 0x55adfbf77f10 2024-05-21T03:23:49.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.527+0000 7fd6417fa700 1 -- 172.21.15.27:0/2999506684 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd6500143f0 con 0x55adfbf77f10 2024-05-21T03:23:49.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.527+0000 7fd66a708b80 10 client.15207 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T03:23:49.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.527+0000 7fd66a708b80 10 client.15207 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-21T03:23:49.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.527+0000 7fd66a708b80 20 client.15207 populate_metadata read hostname 'smithi027' 2024-05-21T03:23:49.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.527+0000 7fd66a708b80 1 --2- 172.21.15.27:0/2999506684 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x55adfbf8ceb0 0x55adfbf8e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:49.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.527+0000 7fd66a708b80 4 client.15207 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:23:49.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.527+0000 7fd66a708b80 1 -- 172.21.15.27:0/2999506684 --> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adfbf3f990 con 0x55adfbf8ceb0 2024-05-21T03:23:49.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.528+0000 7fd65dc0e700 1 --2- 172.21.15.27:0/2999506684 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x55adfbf8ceb0 0x55adfbf8e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:49.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.528+0000 7fd6417fa700 10 client.15207 ms_handle_connect on v2:172.21.15.165:6838/4158170814 2024-05-21T03:23:49.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.528+0000 7fd65dc0e700 1 --2- 172.21.15.27:0/2999506684 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x55adfbf8ceb0 0x55adfbf8e320 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=15277c705217e15f server_cookie=801f6a5a1c831afb in_seq=0 out_seq=0 2024-05-21T03:23:49.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.529+0000 7fd6417fa700 1 -- 172.21.15.27:0/2999506684 <== mds.1 v2:172.21.15.165:6838/4158170814 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55adfbf3f990 con 0x55adfbf8ceb0 2024-05-21T03:23:49.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.529+0000 7fd6417fa700 10 client.15207 handle_command_reply: tid=0 2024-05-21T03:23:49.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.529+0000 7fd66a708b80 1 client.15207 shutdown 2024-05-21T03:23:49.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.529+0000 7fd66a708b80 1 -- 172.21.15.27:0/2999506684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adfbf77f10 msgr2=0x55adfbf7cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:49.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.529+0000 7fd66a708b80 1 --2- 172.21.15.27:0/2999506684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adfbf77f10 0x55adfbf7cba0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd650009a90 tx=0x7fd650009ac0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.529+0000 7fd66a708b80 1 -- 172.21.15.27:0/2999506684 shutdown_connections 2024-05-21T03:23:49.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.529+0000 7fd66a708b80 1 --2- 172.21.15.27:0/2999506684 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x55adfbf8ceb0 0x55adfbf8e320 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.529+0000 7fd66a708b80 1 --2- 172.21.15.27:0/2999506684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adfbf77f10 0x55adfbf7cba0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.529+0000 7fd66a708b80 1 --2- 172.21.15.27:0/2999506684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55adfbf73810 0x55adfbf7f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.529+0000 7fd66a708b80 1 --2- 172.21.15.27:0/2999506684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55adfbf741c0 0x55adfbf7c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.529+0000 7fd6417fa700 0 client.15207 ms_handle_reset on v2:172.21.15.165:6838/4158170814 2024-05-21T03:23:49.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.529+0000 7fd66a708b80 1 -- 172.21.15.27:0/2999506684 >> 172.21.15.27:0/2999506684 conn(0x55adfbea1030 msgr2=0x55adfbf6d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:49.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.530+0000 7fd66a708b80 1 -- 172.21.15.27:0/2999506684 shutdown_connections 2024-05-21T03:23:49.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.530+0000 7fd66a708b80 1 -- 172.21.15.27:0/2999506684 wait complete. 2024-05-21T03:23:49.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.530+0000 7fd66a708b80 20 client.15207 trim_cache size 0 max 16384 2024-05-21T03:23:49.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.552+0000 7fd66a708b80 1 Processor -- start 2024-05-21T03:23:49.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.552+0000 7fd66a708b80 1 -- start start 2024-05-21T03:23:49.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.552+0000 7fd66a708b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55adfbf73810 0x55adfbf770f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:49.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.552+0000 7fd66a708b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adfbf741c0 0x55adfbf7e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:49.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.552+0000 7fd66a708b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55adfbf77f10 0x55adfbf6dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:49.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.552+0000 7fd66a708b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55adfbf8c660 con 0x55adfbf741c0 2024-05-21T03:23:49.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.552+0000 7fd66a708b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55adfbf8cd30 con 0x55adfbf77f10 2024-05-21T03:23:49.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.552+0000 7fd66a708b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55adfbf8d400 con 0x55adfbf73810 2024-05-21T03:23:49.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.552+0000 7fd65d40d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adfbf741c0 0x55adfbf7e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:49.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.552+0000 7fd65d40d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adfbf741c0 0x55adfbf7e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35368/0 (socket says 172.21.15.27:35368) 2024-05-21T03:23:49.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.552+0000 7fd65d40d700 1 -- 172.21.15.27:0/3890169223 learned_addr learned my addr 172.21.15.27:0/3890169223 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:49.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.552+0000 7fd65e40f700 1 --2- 172.21.15.27:0/3890169223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55adfbf77f10 0x55adfbf6dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:49.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.552+0000 7fd65dc0e700 1 --2- 172.21.15.27:0/3890169223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55adfbf73810 0x55adfbf770f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:49.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.553+0000 7fd65d40d700 1 -- 172.21.15.27:0/3890169223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55adfbf73810 msgr2=0x55adfbf770f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:49.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.553+0000 7fd65d40d700 1 --2- 172.21.15.27:0/3890169223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55adfbf73810 0x55adfbf770f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.553+0000 7fd65d40d700 1 -- 172.21.15.27:0/3890169223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55adfbf77f10 msgr2=0x55adfbf6dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:49.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.553+0000 7fd65d40d700 1 --2- 172.21.15.27:0/3890169223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55adfbf77f10 0x55adfbf6dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.553+0000 7fd65d40d700 1 -- 172.21.15.27:0/3890169223 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adfbf77c60 con 0x55adfbf741c0 2024-05-21T03:23:49.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.553+0000 7fd65e40f700 1 --2- 172.21.15.27:0/3890169223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55adfbf77f10 0x55adfbf6dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:23:49.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.553+0000 7fd65d40d700 1 --2- 172.21.15.27:0/3890169223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adfbf741c0 0x55adfbf7e550 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd6480a6670 tx=0x7fd648092040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29a6be6a5dce885 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:49.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.553+0000 7fd6437fe700 1 -- 172.21.15.27:0/3890169223 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6480aa5f0 con 0x55adfbf741c0 2024-05-21T03:23:49.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.553+0000 7fd6437fe700 1 -- 172.21.15.27:0/3890169223 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6480ae9a0 con 0x55adfbf741c0 2024-05-21T03:23:49.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.553+0000 7fd6437fe700 1 -- 172.21.15.27:0/3890169223 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6480b2f30 con 0x55adfbf741c0 2024-05-21T03:23:49.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.554+0000 7fd66a708b80 1 -- 172.21.15.27:0/3890169223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adfbf741c0 msgr2=0x55adfbf7e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:49.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.554+0000 7fd66a708b80 1 --2- 172.21.15.27:0/3890169223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adfbf741c0 0x55adfbf7e550 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd6480a6670 tx=0x7fd648092040 comp rx=0 tx=0).stop 2024-05-21T03:23:49.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.554+0000 7fd66a708b80 1 -- 172.21.15.27:0/3890169223 shutdown_connections 2024-05-21T03:23:49.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.554+0000 7fd66a708b80 1 --2- 172.21.15.27:0/3890169223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55adfbf77f10 0x55adfbf6dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.554+0000 7fd66a708b80 1 --2- 172.21.15.27:0/3890169223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55adfbf73810 0x55adfbf770f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.554+0000 7fd66a708b80 1 --2- 172.21.15.27:0/3890169223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adfbf741c0 0x55adfbf7e550 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.554+0000 7fd66a708b80 1 -- 172.21.15.27:0/3890169223 >> 172.21.15.27:0/3890169223 conn(0x55adfbea1030 msgr2=0x55adfbf7de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:49.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.554+0000 7fd66a708b80 1 -- 172.21.15.27:0/3890169223 shutdown_connections 2024-05-21T03:23:49.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.554+0000 7fd66a708b80 1 -- 172.21.15.27:0/3890169223 wait complete. 2024-05-21T03:23:49.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.554+0000 7fd66a708b80 1 -- start start 2024-05-21T03:23:49.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.554+0000 7fd66a708b80 1 Processor -- start 2024-05-21T03:23:49.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.554+0000 7fd66a708b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55adfbf73810 0x55adfbf97780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:49.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.554+0000 7fd66a708b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55adfbf741c0 0x55adfbf97cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:49.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.554+0000 7fd66a708b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adfbf77f10 0x55adfbf93c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:49.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.554+0000 7fd66a708b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55adfbf8e1a0 con 0x55adfbf77f10 2024-05-21T03:23:49.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.555+0000 7fd66a708b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55adfbf8dad0 con 0x55adfbf73810 2024-05-21T03:23:49.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.555+0000 7fd66a708b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55adfbf8e870 con 0x55adfbf741c0 2024-05-21T03:23:49.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.555+0000 7fd66a708b80 10 client.0 _ll_register_callbacks cb 0x55adfbe5d8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:23:49.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.555+0000 7fd65e40f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adfbf77f10 0x55adfbf93c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:49.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.555+0000 7fd65e40f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adfbf77f10 0x55adfbf93c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35378/0 (socket says 172.21.15.27:35378) 2024-05-21T03:23:49.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.555+0000 7fd65e40f700 1 -- 172.21.15.27:0/3970439162 learned_addr learned my addr 172.21.15.27:0/3970439162 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:49.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.555+0000 7fd65dc0e700 1 --2- 172.21.15.27:0/3970439162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55adfbf73810 0x55adfbf97780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:49.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.555+0000 7fd65d40d700 1 --2- 172.21.15.27:0/3970439162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55adfbf741c0 0x55adfbf97cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:49.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.555+0000 7fd65e40f700 1 -- 172.21.15.27:0/3970439162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55adfbf741c0 msgr2=0x55adfbf97cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:49.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.555+0000 7fd65e40f700 1 --2- 172.21.15.27:0/3970439162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55adfbf741c0 0x55adfbf97cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.555+0000 7fd65e40f700 1 -- 172.21.15.27:0/3970439162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55adfbf73810 msgr2=0x55adfbf97780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:49.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.555+0000 7fd65e40f700 1 --2- 172.21.15.27:0/3970439162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55adfbf73810 0x55adfbf97780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.555+0000 7fd65e40f700 1 -- 172.21.15.27:0/3970439162 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adfbf943e0 con 0x55adfbf77f10 2024-05-21T03:23:49.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.555+0000 7fd65e40f700 1 --2- 172.21.15.27:0/3970439162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adfbf77f10 0x55adfbf93c10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd650009780 tx=0x7fd65000a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201560b436ac3e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:49.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.555+0000 7fd6417fa700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:23:49.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.555+0000 7fd6417fa700 1 -- 172.21.15.27:0/3970439162 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65002f020 con 0x55adfbf77f10 2024-05-21T03:23:49.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.555+0000 7fd6417fa700 1 -- 172.21.15.27:0/3970439162 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fd65002b480 con 0x55adfbf77f10 2024-05-21T03:23:49.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.555+0000 7fd66a708b80 1 -- 172.21.15.27:0/3970439162 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6480ae9a0 con 0x55adfbf77f10 2024-05-21T03:23:49.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.555+0000 7fd6417fa700 1 -- 172.21.15.27:0/3970439162 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd65003a050 con 0x55adfbf77f10 2024-05-21T03:23:49.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.555+0000 7fd66a708b80 10 client.15213 fetch_fsmap learned FSMap version 28 2024-05-21T03:23:49.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.555+0000 7fd66a708b80 1 -- 172.21.15.27:0/3970439162 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55adfbff9110 con 0x55adfbf77f10 2024-05-21T03:23:49.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.556+0000 7fd6417fa700 1 -- 172.21.15.27:0/3970439162 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6480ae9a0 con 0x55adfbf77f10 2024-05-21T03:23:49.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.556+0000 7fd6417fa700 1 -- 172.21.15.27:0/3970439162 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd650029430 con 0x55adfbf77f10 2024-05-21T03:23:49.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.556+0000 7fd66a708b80 10 client.15213 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T03:23:49.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.556+0000 7fd66a708b80 10 client.15213 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-21T03:23:49.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.556+0000 7fd66a708b80 20 client.15213 populate_metadata read hostname 'smithi027' 2024-05-21T03:23:49.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.556+0000 7fd66a708b80 1 --2- 172.21.15.27:0/3970439162 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x55adfbffdda0 0x55adfbffcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:49.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.556+0000 7fd66a708b80 4 client.15213 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T03:23:49.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.556+0000 7fd66a708b80 1 -- 172.21.15.27:0/3970439162 --> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] -- command(tid 0: {"prefix": "status"}) -- 0x55adfbed4b90 con 0x55adfbffdda0 2024-05-21T03:23:49.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.556+0000 7fd65dc0e700 1 --2- 172.21.15.27:0/3970439162 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x55adfbffdda0 0x55adfbffcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:49.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.556+0000 7fd65dc0e700 1 --2- 172.21.15.27:0/3970439162 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x55adfbffdda0 0x55adfbffcab0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=75fed30201a4c5db server_cookie=6fbde4f2790e0076 in_seq=0 out_seq=0 2024-05-21T03:23:49.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.556+0000 7fd6417fa700 10 client.15213 ms_handle_connect on v2:172.21.15.165:6838/4158170814 2024-05-21T03:23:49.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.557+0000 7fd6417fa700 1 -- 172.21.15.27:0/3970439162 <== mds.1 v2:172.21.15.165:6838/4158170814 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55adfbed4b90 con 0x55adfbffdda0 2024-05-21T03:23:49.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.557+0000 7fd6417fa700 10 client.15213 handle_command_reply: tid=0 2024-05-21T03:23:49.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.557+0000 7fd66a708b80 1 client.15213 shutdown 2024-05-21T03:23:49.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.557+0000 7fd66a708b80 1 -- 172.21.15.27:0/3970439162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adfbf77f10 msgr2=0x55adfbf93c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:49.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.557+0000 7fd66a708b80 1 --2- 172.21.15.27:0/3970439162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adfbf77f10 0x55adfbf93c10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd650009780 tx=0x7fd65000a540 comp rx=0 tx=0).stop 2024-05-21T03:23:49.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.557+0000 7fd66a708b80 1 -- 172.21.15.27:0/3970439162 shutdown_connections 2024-05-21T03:23:49.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.557+0000 7fd66a708b80 1 --2- 172.21.15.27:0/3970439162 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x55adfbffdda0 0x55adfbffcab0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.557+0000 7fd66a708b80 1 --2- 172.21.15.27:0/3970439162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adfbf77f10 0x55adfbf93c10 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.557+0000 7fd66a708b80 1 --2- 172.21.15.27:0/3970439162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55adfbf73810 0x55adfbf97780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.557+0000 7fd66a708b80 1 --2- 172.21.15.27:0/3970439162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55adfbf741c0 0x55adfbf97cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.557+0000 7fd6417fa700 0 client.15213 ms_handle_reset on v2:172.21.15.165:6838/4158170814 2024-05-21T03:23:49.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.557+0000 7fd66a708b80 1 -- 172.21.15.27:0/3970439162 >> 172.21.15.27:0/3970439162 conn(0x55adfbea1030 msgr2=0x55adfbf72eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:49.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.557+0000 7fd66a708b80 1 -- 172.21.15.27:0/3970439162 shutdown_connections 2024-05-21T03:23:49.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.557+0000 7fd66a708b80 1 -- 172.21.15.27:0/3970439162 wait complete. 2024-05-21T03:23:49.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.557+0000 7fd66a708b80 20 client.15213 trim_cache size 0 max 16384 2024-05-21T03:23:49.579 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:23:49.579 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "c9ca94be-171f-11ef-bc9a-c7b262605968", 2024-05-21T03:23:49.579 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 1, 2024-05-21T03:23:49.579 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24472, 2024-05-21T03:23:49.580 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-05-21T03:23:49.580 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-05-21T03:23:49.580 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-05-21T03:23:49.580 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 183.13542799199999, 2024-05-21T03:23:49.580 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 28, 2024-05-21T03:23:49.580 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 82, 2024-05-21T03:23:49.580 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 80, 2024-05-21T03:23:49.580 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 219.65708547200001 2024-05-21T03:23:49.580 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:23:49.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.558+0000 7fd65fe72700 1 -- 172.21.15.27:0/643683344 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd6440b2880 msgr2=0x7fd6440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:49.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.558+0000 7fd65fe72700 1 --2- 172.21.15.27:0/643683344 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd6440b2880 0x7fd6440b4d40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd658078da0 tx=0x7fd654005f90 comp rx=0 tx=0).stop 2024-05-21T03:23:49.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.558+0000 7fd65fe72700 1 -- 172.21.15.27:0/643683344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd658107280 msgr2=0x7fd6581a08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:49.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.558+0000 7fd65fe72700 1 --2- 172.21.15.27:0/643683344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd658107280 0x7fd6581a08a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd6480057f0 tx=0x7fd648003730 comp rx=0 tx=0).stop 2024-05-21T03:23:49.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.558+0000 7fd65fe72700 1 -- 172.21.15.27:0/643683344 shutdown_connections 2024-05-21T03:23:49.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.558+0000 7fd65fe72700 1 --2- 172.21.15.27:0/643683344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd658107c30 0x7fd6581a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.558+0000 7fd65fe72700 1 --2- 172.21.15.27:0/643683344 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd6440b2880 0x7fd6440b4d40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.558+0000 7fd65fe72700 1 --2- 172.21.15.27:0/643683344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6581024b0 0x7fd6581a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.558+0000 7fd65fe72700 1 --2- 172.21.15.27:0/643683344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd658107280 0x7fd6581a08a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:49.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.558+0000 7fd65fe72700 1 -- 172.21.15.27:0/643683344 >> 172.21.15.27:0/643683344 conn(0x7fd658077760 msgr2=0x7fd658102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:49.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.558+0000 7fd65fe72700 1 -- 172.21.15.27:0/643683344 shutdown_connections 2024-05-21T03:23:49.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:49.558+0000 7fd65fe72700 1 -- 172.21.15.27:0/643683344 wait complete. 2024-05-21T03:23:50.225 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell mds.b status 2024-05-21T03:23:50.475 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:23:50.893 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:50 smithi027 ceph-mon[32179]: pgmap v390: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:23:51.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:50 smithi039 ceph-mon[36429]: pgmap v390: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:23:51.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:50 smithi165 ceph-mon[36269]: pgmap v390: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:23:52.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.017+0000 7fbc78969700 1 -- 172.21.15.27:0/3526926362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc74102a80 msgr2=0x7fbc74102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:52.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.017+0000 7fbc78969700 1 --2- 172.21.15.27:0/3526926362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc74102a80 0x7fbc74102e60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbc640099c0 tx=0x7fbc64009cd0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.017+0000 7fbc78969700 1 -- 172.21.15.27:0/3526926362 shutdown_connections 2024-05-21T03:23:52.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.017+0000 7fbc78969700 1 --2- 172.21.15.27:0/3526926362 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc741077d0 0x7fbc7410f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.017+0000 7fbc78969700 1 --2- 172.21.15.27:0/3526926362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc74102a80 0x7fbc74102e60 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.017+0000 7fbc78969700 1 --2- 172.21.15.27:0/3526926362 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc74103430 0x7fbc741070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.017+0000 7fbc78969700 1 -- 172.21.15.27:0/3526926362 >> 172.21.15.27:0/3526926362 conn(0x7fbc740793f0 msgr2=0x7fbc74079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:52.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.017+0000 7fbc78969700 1 -- 172.21.15.27:0/3526926362 shutdown_connections 2024-05-21T03:23:52.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.017+0000 7fbc78969700 1 -- 172.21.15.27:0/3526926362 wait complete. 2024-05-21T03:23:52.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.018+0000 7fbc78969700 1 Processor -- start 2024-05-21T03:23:52.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.018+0000 7fbc78969700 1 -- start start 2024-05-21T03:23:52.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.018+0000 7fbc78969700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc74102a80 0x7fbc7419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:52.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.019+0000 7fbc78969700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc74103430 0x7fbc7419b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:52.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.019+0000 7fbc78969700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc741077d0 0x7fbc741a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:52.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.019+0000 7fbc78969700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbc74112650 con 0x7fbc74103430 2024-05-21T03:23:52.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.019+0000 7fbc78969700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbc741124d0 con 0x7fbc741077d0 2024-05-21T03:23:52.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.019+0000 7fbc78969700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbc741127d0 con 0x7fbc74102a80 2024-05-21T03:23:52.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.019+0000 7fbc71d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc74103430 0x7fbc7419b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:52.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.019+0000 7fbc71d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc74103430 0x7fbc7419b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35402/0 (socket says 172.21.15.27:35402) 2024-05-21T03:23:52.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.019+0000 7fbc71d9b700 1 -- 172.21.15.27:0/1777964301 learned_addr learned my addr 172.21.15.27:0/1777964301 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:52.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.019+0000 7fbc72d9d700 1 --2- 172.21.15.27:0/1777964301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc741077d0 0x7fbc741a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:52.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.019+0000 7fbc7259c700 1 --2- 172.21.15.27:0/1777964301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc74102a80 0x7fbc7419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:52.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.019+0000 7fbc71d9b700 1 -- 172.21.15.27:0/1777964301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc74102a80 msgr2=0x7fbc7419ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:52.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.019+0000 7fbc71d9b700 1 --2- 172.21.15.27:0/1777964301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc74102a80 0x7fbc7419ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.019+0000 7fbc71d9b700 1 -- 172.21.15.27:0/1777964301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc741077d0 msgr2=0x7fbc741a5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:52.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.019+0000 7fbc71d9b700 1 --2- 172.21.15.27:0/1777964301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc741077d0 0x7fbc741a5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.019+0000 7fbc71d9b700 1 -- 172.21.15.27:0/1777964301 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc741a6110 con 0x7fbc74103430 2024-05-21T03:23:52.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.019+0000 7fbc71d9b700 1 --2- 172.21.15.27:0/1777964301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc74103430 0x7fbc7419b090 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbc6800d980 tx=0x7fbc6800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:52.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.019+0000 7fbc637fe700 1 -- 172.21.15.27:0/1777964301 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc68017070 con 0x7fbc74103430 2024-05-21T03:23:52.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.019+0000 7fbc637fe700 1 -- 172.21.15.27:0/1777964301 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc68005020 con 0x7fbc74103430 2024-05-21T03:23:52.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.019+0000 7fbc78969700 1 -- 172.21.15.27:0/1777964301 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc741a6340 con 0x7fbc74103430 2024-05-21T03:23:52.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.019+0000 7fbc637fe700 1 -- 172.21.15.27:0/1777964301 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc68009990 con 0x7fbc74103430 2024-05-21T03:23:52.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.020+0000 7fbc78969700 1 -- 172.21.15.27:0/1777964301 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc741a6880 con 0x7fbc74103430 2024-05-21T03:23:52.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.022+0000 7fbc637fe700 1 -- 172.21.15.27:0/1777964301 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbc680054f0 con 0x7fbc74103430 2024-05-21T03:23:52.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.023+0000 7fbc637fe700 1 --2- 172.21.15.27:0/1777964301 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fbc5c0b2900 0x7fbc5c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:52.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.023+0000 7fbc7259c700 1 --2- 172.21.15.27:0/1777964301 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fbc5c0b2900 0x7fbc5c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:52.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.023+0000 7fbc637fe700 1 -- 172.21.15.27:0/1777964301 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fbc680e1ac0 con 0x7fbc74103430 2024-05-21T03:23:52.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.024+0000 7fbc7259c700 1 --2- 172.21.15.27:0/1777964301 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fbc5c0b2900 0x7fbc5c0b4dc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbc64003800 tx=0x7fbc6401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:52.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.024+0000 7fbc831ffb80 1 Processor -- start 2024-05-21T03:23:52.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.024+0000 7fbc831ffb80 1 -- start start 2024-05-21T03:23:52.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.024+0000 7fbc831ffb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c46aeec20 0x562c46aef000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:52.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.024+0000 7fbc831ffb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c46aef5d0 0x562c46af2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:52.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.024+0000 7fbc831ffb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c46af3320 0x562c46af3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:52.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.024+0000 7fbc831ffb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562c46a49c40 con 0x562c46aeec20 2024-05-21T03:23:52.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.024+0000 7fbc831ffb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562c46a397e0 con 0x562c46aef5d0 2024-05-21T03:23:52.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.024+0000 7fbc831ffb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562c46a3d820 con 0x562c46af3320 2024-05-21T03:23:52.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.024+0000 7fbc7259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c46aeec20 0x562c46aef000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:52.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.024+0000 7fbc7259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c46aeec20 0x562c46aef000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35410/0 (socket says 172.21.15.27:35410) 2024-05-21T03:23:52.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.024+0000 7fbc7259c700 1 -- 172.21.15.27:0/2315973771 learned_addr learned my addr 172.21.15.27:0/2315973771 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:52.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.024+0000 7fbc71d9b700 1 --2- 172.21.15.27:0/2315973771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c46aef5d0 0x562c46af2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:52.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.025+0000 7fbc72d9d700 1 --2- 172.21.15.27:0/2315973771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c46af3320 0x562c46af3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:52.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.025+0000 7fbc7259c700 1 -- 172.21.15.27:0/2315973771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c46af3320 msgr2=0x562c46af3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:52.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.025+0000 7fbc7259c700 1 --2- 172.21.15.27:0/2315973771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c46af3320 0x562c46af3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.025+0000 7fbc7259c700 1 -- 172.21.15.27:0/2315973771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c46aef5d0 msgr2=0x562c46af2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:52.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.025+0000 7fbc7259c700 1 --2- 172.21.15.27:0/2315973771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c46aef5d0 0x562c46af2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.025+0000 7fbc7259c700 1 -- 172.21.15.27:0/2315973771 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c46989880 con 0x562c46aeec20 2024-05-21T03:23:52.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.025+0000 7fbc72d9d700 1 --2- 172.21.15.27:0/2315973771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c46af3320 0x562c46af3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:23:52.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.025+0000 7fbc7259c700 1 --2- 172.21.15.27:0/2315973771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c46aeec20 0x562c46aef000 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbc64015000 tx=0x7fbc6401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c81eecc1ac2c00b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:52.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.025+0000 7fbc60ff9700 1 -- 172.21.15.27:0/2315973771 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6400f760 con 0x562c46aeec20 2024-05-21T03:23:52.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.025+0000 7fbc60ff9700 1 -- 172.21.15.27:0/2315973771 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc64032070 con 0x562c46aeec20 2024-05-21T03:23:52.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.025+0000 7fbc60ff9700 1 -- 172.21.15.27:0/2315973771 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc64023b80 con 0x562c46aeec20 2024-05-21T03:23:52.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.026+0000 7fbc831ffb80 1 -- 172.21.15.27:0/2315973771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c46aeec20 msgr2=0x562c46aef000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:52.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.026+0000 7fbc831ffb80 1 --2- 172.21.15.27:0/2315973771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c46aeec20 0x562c46aef000 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbc64015000 tx=0x7fbc6401e000 comp rx=0 tx=0).stop 2024-05-21T03:23:52.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.026+0000 7fbc831ffb80 1 -- 172.21.15.27:0/2315973771 shutdown_connections 2024-05-21T03:23:52.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.026+0000 7fbc831ffb80 1 --2- 172.21.15.27:0/2315973771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c46af3320 0x562c46af3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.026+0000 7fbc831ffb80 1 --2- 172.21.15.27:0/2315973771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c46aeec20 0x562c46aef000 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.026+0000 7fbc831ffb80 1 --2- 172.21.15.27:0/2315973771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c46aef5d0 0x562c46af2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.026+0000 7fbc831ffb80 1 -- 172.21.15.27:0/2315973771 >> 172.21.15.27:0/2315973771 conn(0x562c46a18030 msgr2=0x562c46ae4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:52.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.026+0000 7fbc831ffb80 1 -- 172.21.15.27:0/2315973771 shutdown_connections 2024-05-21T03:23:52.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.026+0000 7fbc831ffb80 1 -- 172.21.15.27:0/2315973771 wait complete. 2024-05-21T03:23:52.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.026+0000 7fbc831ffb80 1 -- start start 2024-05-21T03:23:52.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.026+0000 7fbc831ffb80 1 Processor -- start 2024-05-21T03:23:52.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.027+0000 7fbc831ffb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c46aeec20 0x562c46afa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:52.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.027+0000 7fbc831ffb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c46aef5d0 0x562c46af7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:52.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.027+0000 7fbc831ffb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c46af3320 0x562c46af7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:52.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.027+0000 7fbc831ffb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562c4693ff50 con 0x562c46aeec20 2024-05-21T03:23:52.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.027+0000 7fbc831ffb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562c469db770 con 0x562c46af3320 2024-05-21T03:23:52.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.027+0000 7fbc831ffb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562c46afb480 con 0x562c46aef5d0 2024-05-21T03:23:52.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.027+0000 7fbc831ffb80 10 client.0 _ll_register_callbacks cb 0x562c469d48d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:23:52.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.027+0000 7fbc7259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c46aeec20 0x562c46afa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:52.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.027+0000 7fbc7259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c46aeec20 0x562c46afa7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35426/0 (socket says 172.21.15.27:35426) 2024-05-21T03:23:52.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.027+0000 7fbc7259c700 1 -- 172.21.15.27:0/1928588858 learned_addr learned my addr 172.21.15.27:0/1928588858 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:52.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.027+0000 7fbc72d9d700 1 --2- 172.21.15.27:0/1928588858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c46af3320 0x562c46af7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:52.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.027+0000 7fbc71d9b700 1 --2- 172.21.15.27:0/1928588858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c46aef5d0 0x562c46af7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:52.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.027+0000 7fbc7259c700 1 -- 172.21.15.27:0/1928588858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c46aef5d0 msgr2=0x562c46af7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:52.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.027+0000 7fbc7259c700 1 --2- 172.21.15.27:0/1928588858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c46aef5d0 0x562c46af7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.027+0000 7fbc7259c700 1 -- 172.21.15.27:0/1928588858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c46af3320 msgr2=0x562c46af7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:52.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.027+0000 7fbc7259c700 1 --2- 172.21.15.27:0/1928588858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c46af3320 0x562c46af7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.027+0000 7fbc7259c700 1 -- 172.21.15.27:0/1928588858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c469d8640 con 0x562c46aeec20 2024-05-21T03:23:52.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.027+0000 7fbc7259c700 1 --2- 172.21.15.27:0/1928588858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c46aeec20 0x562c46afa7a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbc64009d10 tx=0x7fbc64023640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c72820651a44ece server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:52.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.027+0000 7fbc59ffb700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:23:52.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.027+0000 7fbc59ffb700 1 -- 172.21.15.27:0/1928588858 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6400fa30 con 0x562c46aeec20 2024-05-21T03:23:52.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.028+0000 7fbc59ffb700 1 -- 172.21.15.27:0/1928588858 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fbc6400fbd0 con 0x562c46aeec20 2024-05-21T03:23:52.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.028+0000 7fbc831ffb80 1 -- 172.21.15.27:0/1928588858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc64032070 con 0x562c46aeec20 2024-05-21T03:23:52.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.028+0000 7fbc59ffb700 1 -- 172.21.15.27:0/1928588858 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbc64048050 con 0x562c46aeec20 2024-05-21T03:23:52.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.028+0000 7fbc831ffb80 10 client.15225 fetch_fsmap learned FSMap version 28 2024-05-21T03:23:52.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.028+0000 7fbc831ffb80 1 -- 172.21.15.27:0/1928588858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562c46b05b20 con 0x562c46aeec20 2024-05-21T03:23:52.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.028+0000 7fbc59ffb700 1 -- 172.21.15.27:0/1928588858 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc64032070 con 0x562c46aeec20 2024-05-21T03:23:52.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.028+0000 7fbc59ffb700 1 -- 172.21.15.27:0/1928588858 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbc6402d3f0 con 0x562c46aeec20 2024-05-21T03:23:52.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.028+0000 7fbc831ffb80 10 client.15225 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T03:23:52.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.028+0000 7fbc831ffb80 10 client.15225 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-21T03:23:52.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.028+0000 7fbc831ffb80 20 client.15225 populate_metadata read hostname 'smithi027' 2024-05-21T03:23:52.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.028+0000 7fbc831ffb80 1 --2- 172.21.15.27:0/1928588858 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x562c46b07c20 0x562c46b09090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:52.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.028+0000 7fbc831ffb80 4 client.15225 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:23:52.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.028+0000 7fbc831ffb80 1 -- 172.21.15.27:0/1928588858 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c46ab6990 con 0x562c46b07c20 2024-05-21T03:23:52.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.029+0000 7fbc71d9b700 1 --2- 172.21.15.27:0/1928588858 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x562c46b07c20 0x562c46b09090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:52.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.029+0000 7fbc71d9b700 1 --2- 172.21.15.27:0/1928588858 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x562c46b07c20 0x562c46b09090 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1580282acbc33e9 server_cookie=fa13202892ffd7db in_seq=0 out_seq=0 2024-05-21T03:23:52.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.029+0000 7fbc59ffb700 10 client.15225 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:23:52.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.030+0000 7fbc59ffb700 1 -- 172.21.15.27:0/1928588858 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562c46ab6990 con 0x562c46b07c20 2024-05-21T03:23:52.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.030+0000 7fbc59ffb700 10 client.15225 handle_command_reply: tid=0 2024-05-21T03:23:52.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.030+0000 7fbc831ffb80 1 client.15225 shutdown 2024-05-21T03:23:52.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.030+0000 7fbc831ffb80 1 -- 172.21.15.27:0/1928588858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c46aeec20 msgr2=0x562c46afa7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:52.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.030+0000 7fbc831ffb80 1 --2- 172.21.15.27:0/1928588858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c46aeec20 0x562c46afa7a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbc64009d10 tx=0x7fbc64023640 comp rx=0 tx=0).stop 2024-05-21T03:23:52.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.030+0000 7fbc831ffb80 1 -- 172.21.15.27:0/1928588858 shutdown_connections 2024-05-21T03:23:52.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.030+0000 7fbc831ffb80 1 --2- 172.21.15.27:0/1928588858 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x562c46b07c20 0x562c46b09090 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.030+0000 7fbc831ffb80 1 --2- 172.21.15.27:0/1928588858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c46af3320 0x562c46af7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.030+0000 7fbc831ffb80 1 --2- 172.21.15.27:0/1928588858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c46aeec20 0x562c46afa7a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.030+0000 7fbc831ffb80 1 --2- 172.21.15.27:0/1928588858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c46aef5d0 0x562c46af7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.030+0000 7fbc831ffb80 1 -- 172.21.15.27:0/1928588858 >> 172.21.15.27:0/1928588858 conn(0x562c46a18030 msgr2=0x562c46ae44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:52.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.030+0000 7fbc59ffb700 0 client.15225 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:23:52.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.030+0000 7fbc831ffb80 1 -- 172.21.15.27:0/1928588858 shutdown_connections 2024-05-21T03:23:52.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.030+0000 7fbc831ffb80 1 -- 172.21.15.27:0/1928588858 wait complete. 2024-05-21T03:23:52.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.030+0000 7fbc831ffb80 20 client.15225 trim_cache size 0 max 16384 2024-05-21T03:23:52.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.051+0000 7fbc831ffb80 1 Processor -- start 2024-05-21T03:23:52.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.051+0000 7fbc831ffb80 1 -- start start 2024-05-21T03:23:52.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.052+0000 7fbc831ffb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c46aeec20 0x562c46af0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:52.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.052+0000 7fbc831ffb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c46aef5d0 0x562c46af11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:52.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.052+0000 7fbc831ffb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c46af3320 0x562c46ae4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:52.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.052+0000 7fbc831ffb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562c46b073d0 con 0x562c46aef5d0 2024-05-21T03:23:52.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.052+0000 7fbc831ffb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562c46b07aa0 con 0x562c46aeec20 2024-05-21T03:23:52.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.052+0000 7fbc831ffb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562c46b08170 con 0x562c46af3320 2024-05-21T03:23:52.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.052+0000 7fbc71d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c46aef5d0 0x562c46af11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:52.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.052+0000 7fbc71d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c46aef5d0 0x562c46af11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35434/0 (socket says 172.21.15.27:35434) 2024-05-21T03:23:52.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.052+0000 7fbc71d9b700 1 -- 172.21.15.27:0/304092128 learned_addr learned my addr 172.21.15.27:0/304092128 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:52.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.052+0000 7fbc7259c700 1 --2- 172.21.15.27:0/304092128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c46aeec20 0x562c46af0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:52.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.052+0000 7fbc72d9d700 1 --2- 172.21.15.27:0/304092128 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c46af3320 0x562c46ae4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:52.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.052+0000 7fbc71d9b700 1 -- 172.21.15.27:0/304092128 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c46af3320 msgr2=0x562c46ae4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:52.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.052+0000 7fbc71d9b700 1 --2- 172.21.15.27:0/304092128 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c46af3320 0x562c46ae4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.052+0000 7fbc71d9b700 1 -- 172.21.15.27:0/304092128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c46aeec20 msgr2=0x562c46af0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:52.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.052+0000 7fbc71d9b700 1 --2- 172.21.15.27:0/304092128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c46aeec20 0x562c46af0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.052+0000 7fbc71d9b700 1 -- 172.21.15.27:0/304092128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c46af16f0 con 0x562c46aef5d0 2024-05-21T03:23:52.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.052+0000 7fbc7259c700 1 --2- 172.21.15.27:0/304092128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c46aeec20 0x562c46af0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:23:52.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.052+0000 7fbc71d9b700 1 --2- 172.21.15.27:0/304092128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c46aef5d0 0x562c46af11b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbc680df540 tx=0x7fbc680df410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7636682c033221c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:52.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.052+0000 7fbc5bfff700 1 -- 172.21.15.27:0/304092128 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc680c8070 con 0x562c46aef5d0 2024-05-21T03:23:52.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.052+0000 7fbc5bfff700 1 -- 172.21.15.27:0/304092128 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc680ad410 con 0x562c46aef5d0 2024-05-21T03:23:52.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.052+0000 7fbc5bfff700 1 -- 172.21.15.27:0/304092128 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc680c2030 con 0x562c46aef5d0 2024-05-21T03:23:52.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.053+0000 7fbc831ffb80 1 -- 172.21.15.27:0/304092128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c46aef5d0 msgr2=0x562c46af11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:52.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.053+0000 7fbc831ffb80 1 --2- 172.21.15.27:0/304092128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c46aef5d0 0x562c46af11b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbc680df540 tx=0x7fbc680df410 comp rx=0 tx=0).stop 2024-05-21T03:23:52.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.053+0000 7fbc831ffb80 1 -- 172.21.15.27:0/304092128 shutdown_connections 2024-05-21T03:23:52.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.053+0000 7fbc831ffb80 1 --2- 172.21.15.27:0/304092128 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c46af3320 0x562c46ae4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.053+0000 7fbc831ffb80 1 --2- 172.21.15.27:0/304092128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c46aeec20 0x562c46af0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.053+0000 7fbc831ffb80 1 --2- 172.21.15.27:0/304092128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c46aef5d0 0x562c46af11b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.053+0000 7fbc831ffb80 1 -- 172.21.15.27:0/304092128 >> 172.21.15.27:0/304092128 conn(0x562c46a18030 msgr2=0x562c46af9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:52.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.053+0000 7fbc831ffb80 1 -- 172.21.15.27:0/304092128 shutdown_connections 2024-05-21T03:23:52.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.053+0000 7fbc831ffb80 1 -- 172.21.15.27:0/304092128 wait complete. 2024-05-21T03:23:52.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.053+0000 7fbc831ffb80 1 -- start start 2024-05-21T03:23:52.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.053+0000 7fbc831ffb80 1 Processor -- start 2024-05-21T03:23:52.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.054+0000 7fbc831ffb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c46aeec20 0x562c46b12a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:52.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.054+0000 7fbc831ffb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c46aef5d0 0x562c46b0cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:52.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.054+0000 7fbc831ffb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c46af3320 0x562c46b0d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:52.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.054+0000 7fbc831ffb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562c46b08f10 con 0x562c46aeec20 2024-05-21T03:23:52.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.054+0000 7fbc831ffb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562c46b08840 con 0x562c46af3320 2024-05-21T03:23:52.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.054+0000 7fbc831ffb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562c46b095e0 con 0x562c46aef5d0 2024-05-21T03:23:52.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.054+0000 7fbc831ffb80 10 client.0 _ll_register_callbacks cb 0x562c469d48d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:23:52.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.054+0000 7fbc7259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c46aeec20 0x562c46b12a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:52.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.054+0000 7fbc7259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c46aeec20 0x562c46b12a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35444/0 (socket says 172.21.15.27:35444) 2024-05-21T03:23:52.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.054+0000 7fbc7259c700 1 -- 172.21.15.27:0/3385264615 learned_addr learned my addr 172.21.15.27:0/3385264615 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:52.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.054+0000 7fbc72d9d700 1 --2- 172.21.15.27:0/3385264615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c46af3320 0x562c46b0d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:52.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.054+0000 7fbc71d9b700 1 --2- 172.21.15.27:0/3385264615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c46aef5d0 0x562c46b0cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:52.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.054+0000 7fbc7259c700 1 -- 172.21.15.27:0/3385264615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c46aef5d0 msgr2=0x562c46b0cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:52.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.054+0000 7fbc7259c700 1 --2- 172.21.15.27:0/3385264615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c46aef5d0 0x562c46b0cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.054+0000 7fbc7259c700 1 -- 172.21.15.27:0/3385264615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c46af3320 msgr2=0x562c46b0d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:23:52.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.054+0000 7fbc7259c700 1 --2- 172.21.15.27:0/3385264615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c46af3320 0x562c46b0d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.054+0000 7fbc7259c700 1 -- 172.21.15.27:0/3385264615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c46b0d990 con 0x562c46aeec20 2024-05-21T03:23:52.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.054+0000 7fbc7259c700 1 --2- 172.21.15.27:0/3385264615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c46aeec20 0x562c46b12a10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbc640098c0 tx=0x7fbc6402dd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb5aa9560d1a825 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:52.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.054+0000 7fbc59ffb700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:23:52.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.055+0000 7fbc59ffb700 1 -- 172.21.15.27:0/3385264615 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6400fa30 con 0x562c46aeec20 2024-05-21T03:23:52.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.055+0000 7fbc59ffb700 1 -- 172.21.15.27:0/3385264615 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fbc6400fbd0 con 0x562c46aeec20 2024-05-21T03:23:52.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.055+0000 7fbc831ffb80 1 -- 172.21.15.27:0/3385264615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc680ad410 con 0x562c46aeec20 2024-05-21T03:23:52.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.055+0000 7fbc59ffb700 1 -- 172.21.15.27:0/3385264615 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbc6402d600 con 0x562c46aeec20 2024-05-21T03:23:52.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.055+0000 7fbc831ffb80 10 client.15231 fetch_fsmap learned FSMap version 28 2024-05-21T03:23:52.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.055+0000 7fbc831ffb80 1 -- 172.21.15.27:0/3385264615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562c46b742c0 con 0x562c46aeec20 2024-05-21T03:23:52.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.055+0000 7fbc59ffb700 1 -- 172.21.15.27:0/3385264615 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc680ad410 con 0x562c46aeec20 2024-05-21T03:23:52.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.055+0000 7fbc59ffb700 1 -- 172.21.15.27:0/3385264615 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbc64042430 con 0x562c46aeec20 2024-05-21T03:23:52.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.055+0000 7fbc831ffb80 10 client.15231 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T03:23:52.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.055+0000 7fbc831ffb80 10 client.15231 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-21T03:23:52.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.055+0000 7fbc831ffb80 20 client.15231 populate_metadata read hostname 'smithi027' 2024-05-21T03:23:52.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.055+0000 7fbc831ffb80 1 --2- 172.21.15.27:0/3385264615 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x562c46b78e10 0x562c46b77b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:52.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.055+0000 7fbc831ffb80 4 client.15231 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T03:23:52.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.055+0000 7fbc831ffb80 1 -- 172.21.15.27:0/3385264615 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "status"}) -- 0x562c46a4bb90 con 0x562c46b78e10 2024-05-21T03:23:52.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.055+0000 7fbc71d9b700 1 --2- 172.21.15.27:0/3385264615 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x562c46b78e10 0x562c46b77b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:52.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.056+0000 7fbc71d9b700 1 --2- 172.21.15.27:0/3385264615 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x562c46b78e10 0x562c46b77b20 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24c4850d30e4c5c5 server_cookie=d1ac6024d10a2142 in_seq=0 out_seq=0 2024-05-21T03:23:52.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.056+0000 7fbc59ffb700 10 client.15231 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:23:52.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.056+0000 7fbc59ffb700 1 -- 172.21.15.27:0/3385264615 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x562c46a4bb90 con 0x562c46b78e10 2024-05-21T03:23:52.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.056+0000 7fbc59ffb700 10 client.15231 handle_command_reply: tid=0 2024-05-21T03:23:52.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.056+0000 7fbc831ffb80 1 client.15231 shutdown 2024-05-21T03:23:52.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.056+0000 7fbc831ffb80 1 -- 172.21.15.27:0/3385264615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c46aeec20 msgr2=0x562c46b12a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:52.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.056+0000 7fbc831ffb80 1 --2- 172.21.15.27:0/3385264615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c46aeec20 0x562c46b12a10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbc640098c0 tx=0x7fbc6402dd30 comp rx=0 tx=0).stop 2024-05-21T03:23:52.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.056+0000 7fbc831ffb80 1 -- 172.21.15.27:0/3385264615 shutdown_connections 2024-05-21T03:23:52.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.056+0000 7fbc831ffb80 1 --2- 172.21.15.27:0/3385264615 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x562c46b78e10 0x562c46b77b20 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.056+0000 7fbc831ffb80 1 --2- 172.21.15.27:0/3385264615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c46af3320 0x562c46b0d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.056+0000 7fbc831ffb80 1 --2- 172.21.15.27:0/3385264615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c46aeec20 0x562c46b12a10 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.056+0000 7fbc831ffb80 1 --2- 172.21.15.27:0/3385264615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c46aef5d0 0x562c46b0cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.056+0000 7fbc59ffb700 0 client.15231 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:23:52.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.056+0000 7fbc831ffb80 1 -- 172.21.15.27:0/3385264615 >> 172.21.15.27:0/3385264615 conn(0x562c46a18030 msgr2=0x562c46af18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:52.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.056+0000 7fbc831ffb80 1 -- 172.21.15.27:0/3385264615 shutdown_connections 2024-05-21T03:23:52.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.056+0000 7fbc831ffb80 1 -- 172.21.15.27:0/3385264615 wait complete. 2024-05-21T03:23:52.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.056+0000 7fbc831ffb80 20 client.15231 trim_cache size 0 max 16384 2024-05-21T03:23:52.063 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:23:52.063 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "c9ca94be-171f-11ef-bc9a-c7b262605968", 2024-05-21T03:23:52.063 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-05-21T03:23:52.063 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24476, 2024-05-21T03:23:52.063 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-05-21T03:23:52.063 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-05-21T03:23:52.063 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-05-21T03:23:52.063 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 197.79390570800001, 2024-05-21T03:23:52.063 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 28, 2024-05-21T03:23:52.064 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 82, 2024-05-21T03:23:52.064 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 80, 2024-05-21T03:23:52.064 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 211.82176217 2024-05-21T03:23:52.064 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:23:52.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.057+0000 7fbc78969700 1 -- 172.21.15.27:0/1777964301 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fbc5c0b2900 msgr2=0x7fbc5c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:52.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.057+0000 7fbc78969700 1 --2- 172.21.15.27:0/1777964301 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fbc5c0b2900 0x7fbc5c0b4dc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbc64003800 tx=0x7fbc6401a040 comp rx=0 tx=0).stop 2024-05-21T03:23:52.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.057+0000 7fbc78969700 1 -- 172.21.15.27:0/1777964301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc74103430 msgr2=0x7fbc7419b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:52.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.057+0000 7fbc78969700 1 --2- 172.21.15.27:0/1777964301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc74103430 0x7fbc7419b090 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbc6800d980 tx=0x7fbc6800dc90 comp rx=0 tx=0).stop 2024-05-21T03:23:52.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.057+0000 7fbc78969700 1 -- 172.21.15.27:0/1777964301 shutdown_connections 2024-05-21T03:23:52.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.057+0000 7fbc78969700 1 --2- 172.21.15.27:0/1777964301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc741077d0 0x7fbc741a5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.057+0000 7fbc78969700 1 --2- 172.21.15.27:0/1777964301 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fbc5c0b2900 0x7fbc5c0b4dc0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.057+0000 7fbc78969700 1 --2- 172.21.15.27:0/1777964301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc74102a80 0x7fbc7419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.057+0000 7fbc78969700 1 --2- 172.21.15.27:0/1777964301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc74103430 0x7fbc7419b090 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.057+0000 7fbc78969700 1 -- 172.21.15.27:0/1777964301 >> 172.21.15.27:0/1777964301 conn(0x7fbc740793f0 msgr2=0x7fbc74100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:52.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.057+0000 7fbc78969700 1 -- 172.21.15.27:0/1777964301 shutdown_connections 2024-05-21T03:23:52.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.057+0000 7fbc78969700 1 -- 172.21.15.27:0/1777964301 wait complete. 2024-05-21T03:23:52.717 INFO:teuthology.run_tasks:Running task sequential... 2024-05-21T03:23:52.726 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-21T03:23:52.727 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-21T03:23:52.727 INFO:teuthology.task.print:Enabling mgr modules 2024-05-21T03:23:52.727 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-05-21T03:23:52.728 INFO:teuthology.task.exec:Executing custom commands... 2024-05-21T03:23:52.728 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-05-21T03:23:52.728 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-05-21T03:23:52.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.836+0000 7fd8a5978640 1 -- 172.21.15.27:0/3220520421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8a0101510 msgr2=0x7fd8a010b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:52.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.836+0000 7fd8a5978640 1 --2- 172.21.15.27:0/3220520421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8a0101510 0x7fd8a010b0c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd898009ca0 tx=0x7fd89802a8c0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.836+0000 7fd8a5978640 1 -- 172.21.15.27:0/3220520421 shutdown_connections 2024-05-21T03:23:52.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.836+0000 7fd8a5978640 1 --2- 172.21.15.27:0/3220520421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8a0101510 0x7fd8a010b0c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.837+0000 7fd8a5978640 1 --2- 172.21.15.27:0/3220520421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8a0100b50 0x7fd8a0100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.837+0000 7fd8a5978640 1 --2- 172.21.15.27:0/3220520421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8a01040e0 0x7fd8a0104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.837+0000 7fd8a5978640 1 -- 172.21.15.27:0/3220520421 >> 172.21.15.27:0/3220520421 conn(0x7fd8a00fc6d0 msgr2=0x7fd8a00feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:52.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.837+0000 7fd8a5978640 1 -- 172.21.15.27:0/3220520421 shutdown_connections 2024-05-21T03:23:52.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.837+0000 7fd8a5978640 1 -- 172.21.15.27:0/3220520421 wait complete. 2024-05-21T03:23:52.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.837+0000 7fd8a5978640 1 Processor -- start 2024-05-21T03:23:52.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.837+0000 7fd8a5978640 1 -- start start 2024-05-21T03:23:52.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.837+0000 7fd8a5978640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8a0100b50 0x7fd8a019afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:52.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.838+0000 7fd8a5978640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8a0101510 0x7fd8a019b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:52.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.838+0000 7fd8a5978640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8a01040e0 0x7fd8a01a2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:52.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.838+0000 7fd8a5978640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a010e760 con 0x7fd8a0100b50 2024-05-21T03:23:52.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.838+0000 7fd8a5978640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a010e5e0 con 0x7fd8a0101510 2024-05-21T03:23:52.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.838+0000 7fd8a5978640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a010e8e0 con 0x7fd8a01040e0 2024-05-21T03:23:52.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.838+0000 7fd89effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8a0100b50 0x7fd8a019afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:52.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.838+0000 7fd89effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8a0100b50 0x7fd8a019afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54364/0 (socket says 172.21.15.27:54364) 2024-05-21T03:23:52.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.838+0000 7fd89effd640 1 -- 172.21.15.27:0/72435730 learned_addr learned my addr 172.21.15.27:0/72435730 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:52.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.838+0000 7fd89effd640 1 -- 172.21.15.27:0/72435730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8a01040e0 msgr2=0x7fd8a01a2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:52.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.838+0000 7fd89effd640 1 --2- 172.21.15.27:0/72435730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8a01040e0 0x7fd8a01a2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.838+0000 7fd89effd640 1 -- 172.21.15.27:0/72435730 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8a0101510 msgr2=0x7fd8a019b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:52.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.838+0000 7fd89e7fc640 1 --2- 172.21.15.27:0/72435730 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8a0101510 0x7fd8a019b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:52.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.838+0000 7fd89effd640 1 --2- 172.21.15.27:0/72435730 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8a0101510 0x7fd8a019b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:52.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.838+0000 7fd89effd640 1 -- 172.21.15.27:0/72435730 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a01a2be0 con 0x7fd8a0100b50 2024-05-21T03:23:52.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.838+0000 7fd89e7fc640 1 --2- 172.21.15.27:0/72435730 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8a0101510 0x7fd8a019b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:23:52.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.838+0000 7fd89effd640 1 --2- 172.21.15.27:0/72435730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8a0100b50 0x7fd8a019afc0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd89000b450 tx=0x7fd89000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:52.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.838+0000 7fd8a4976640 1 -- 172.21.15.27:0/72435730 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd890015020 con 0x7fd8a0100b50 2024-05-21T03:23:52.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.838+0000 7fd8a4976640 1 -- 172.21.15.27:0/72435730 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd890011d40 con 0x7fd8a0100b50 2024-05-21T03:23:52.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.838+0000 7fd8a5978640 1 -- 172.21.15.27:0/72435730 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8a01a2ed0 con 0x7fd8a0100b50 2024-05-21T03:23:52.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.838+0000 7fd8a4976640 1 -- 172.21.15.27:0/72435730 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89000f340 con 0x7fd8a0100b50 2024-05-21T03:23:52.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.838+0000 7fd8a5978640 1 -- 172.21.15.27:0/72435730 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8a01a33c0 con 0x7fd8a0100b50 2024-05-21T03:23:52.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.839+0000 7fd8a5978640 1 -- 172.21.15.27:0/72435730 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8a01a36e0 con 0x7fd8a0100b50 2024-05-21T03:23:52.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.841+0000 7fd8a4976640 1 -- 172.21.15.27:0/72435730 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd890018020 con 0x7fd8a0100b50 2024-05-21T03:23:52.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.842+0000 7fd8a4976640 1 --2- 172.21.15.27:0/72435730 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd8880b28a0 0x7fd8880b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:52.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.842+0000 7fd89e7fc640 1 --2- 172.21.15.27:0/72435730 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd8880b28a0 0x7fd8880b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:52.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.842+0000 7fd8a4976640 1 -- 172.21.15.27:0/72435730 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fd8900e23a0 con 0x7fd8a0100b50 2024-05-21T03:23:52.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.842+0000 7fd89e7fc640 1 --2- 172.21.15.27:0/72435730 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd8880b28a0 0x7fd8880b4d60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd894006fd0 tx=0x7fd894008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:52.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:52.846+0000 7fd8a4976640 1 -- 172.21.15.27:0/72435730 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd89001f020 con 0x7fd8a0100b50 2024-05-21T03:23:52.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:52 smithi027 ceph-mon[32179]: pgmap v391: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:23:53.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:52 smithi039 ceph-mon[36429]: pgmap v391: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:23:53.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:52 smithi165 ceph-mon[36269]: pgmap v391: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-21T03:23:53.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:53.088+0000 7fd8a5978640 1 -- 172.21.15.27:0/72435730 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fd8a01a38b0 con 0x7fd8a0100b50 2024-05-21T03:23:53.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:53.654+0000 7fd8a4976640 1 -- 172.21.15.27:0/72435730 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fd89000d040 con 0x7fd8a0100b50 2024-05-21T03:23:53.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:53.656+0000 7fd8a5978640 1 -- 172.21.15.27:0/72435730 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd8880b28a0 msgr2=0x7fd8880b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:53.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:53.656+0000 7fd8a5978640 1 --2- 172.21.15.27:0/72435730 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd8880b28a0 0x7fd8880b4d60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd894006fd0 tx=0x7fd894008040 comp rx=0 tx=0).stop 2024-05-21T03:23:53.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:53.656+0000 7fd8a5978640 1 -- 172.21.15.27:0/72435730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8a0100b50 msgr2=0x7fd8a019afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:53.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:53.656+0000 7fd8a5978640 1 --2- 172.21.15.27:0/72435730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8a0100b50 0x7fd8a019afc0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd89000b450 tx=0x7fd89000b910 comp rx=0 tx=0).stop 2024-05-21T03:23:53.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:53.657+0000 7fd8a5978640 1 -- 172.21.15.27:0/72435730 shutdown_connections 2024-05-21T03:23:53.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:53.657+0000 7fd8a5978640 1 --2- 172.21.15.27:0/72435730 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fd8880b28a0 0x7fd8880b4d60 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:53.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:53.657+0000 7fd8a5978640 1 --2- 172.21.15.27:0/72435730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8a01040e0 0x7fd8a01a2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:53.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:53.657+0000 7fd8a5978640 1 --2- 172.21.15.27:0/72435730 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8a0101510 0x7fd8a019b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:53.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:53.657+0000 7fd8a5978640 1 --2- 172.21.15.27:0/72435730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8a0100b50 0x7fd8a019afc0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:53.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:53.657+0000 7fd8a5978640 1 -- 172.21.15.27:0/72435730 >> 172.21.15.27:0/72435730 conn(0x7fd8a00fc6d0 msgr2=0x7fd8a0101c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:53.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:53.658+0000 7fd8a5978640 1 -- 172.21.15.27:0/72435730 shutdown_connections 2024-05-21T03:23:53.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:53.658+0000 7fd8a5978640 1 -- 172.21.15.27:0/72435730 wait complete. 2024-05-21T03:23:53.675 INFO:teuthology.run_tasks:Running task kclient... 2024-05-21T03:23:53.703 INFO:tasks.kclient:Mounting kernel clients... 2024-05-21T03:23:53.703 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-21T03:23:53.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T03:23:53.703 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-05-21T03:23:53.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T03:23:53.721 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2024-05-21T03:23:53.791 INFO:teuthology.orchestra.run.smithi027.stderr:Cannot find device "ceph-brx" 2024-05-21T03:23:53.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T03:23:53.793 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-05-21T03:23:53.793 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-21T03:23:53.793 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-21T03:23:53.793 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs ls 2024-05-21T03:23:53.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:53 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/72435730' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-21T03:23:53.927 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:23:53 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: ignoring --setuser ceph since I am not root 2024-05-21T03:23:53.927 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:23:53 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: ignoring --setgroup ceph since I am not root 2024-05-21T03:23:53.927 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:23:53 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:23:53.846+0000 7f04ba2c3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T03:23:54.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:53 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/72435730' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-21T03:23:54.019 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:23:53 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: ignoring --setuser ceph since I am not root 2024-05-21T03:23:54.019 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:23:53 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: ignoring --setgroup ceph since I am not root 2024-05-21T03:23:54.019 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:23:53 smithi039 ceph-mgr[37811]: -- 172.21.15.39:0/3863913024 <== mon.2 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559e54d7c5a0 con 0x559e54d50800 2024-05-21T03:23:54.019 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:23:53 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:23:53.840+0000 7ff69aaf4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T03:23:54.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:53 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/72435730' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-21T03:23:54.021 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:23:53 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: ignoring --setuser ceph since I am not root 2024-05-21T03:23:54.021 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:23:53 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: ignoring --setgroup ceph since I am not root 2024-05-21T03:23:54.021 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:23:53 smithi165 ceph-mgr[38117]: -- 172.21.15.165:0/3702287748 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56223b50e5a0 con 0x56223b4e2c00 2024-05-21T03:23:54.021 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:23:53 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:23:53.838+0000 7fb35ad12200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T03:23:54.090 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:23:54.327 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:23:54 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:23:54.048+0000 7f04ba2c3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T03:23:54.327 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:23:54 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:23:54.164+0000 7f04ba2c3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T03:23:54.433 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:23:54 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:23:54.023+0000 7fb35ad12200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T03:23:54.433 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:23:54 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:23:54.124+0000 7fb35ad12200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T03:23:54.462 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:23:54 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:23:54.034+0000 7ff69aaf4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T03:23:54.462 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:23:54 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:23:54.140+0000 7ff69aaf4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T03:23:54.594 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:23:54 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:23:54.502+0000 7f04ba2c3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T03:23:54.594 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:23:54 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:23:54.593+0000 7f04ba2c3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T03:23:54.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:54 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/72435730' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-21T03:23:54.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:54 smithi039 ceph-mon[36429]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-21T03:23:54.769 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:23:54 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:23:54.461+0000 7ff69aaf4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T03:23:54.769 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:23:54 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:23:54.551+0000 7ff69aaf4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T03:23:54.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:54 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/72435730' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-21T03:23:54.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:54 smithi165 ceph-mon[36269]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-21T03:23:54.771 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:23:54 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:23:54.431+0000 7fb35ad12200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T03:23:54.771 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:23:54 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:23:54.517+0000 7fb35ad12200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T03:23:54.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:54 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/72435730' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-21T03:23:54.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:54 smithi027 ceph-mon[32179]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-21T03:23:55.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.545+0000 7f4954b81700 1 -- 172.21.15.27:0/287846244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4950102e00 msgr2=0x7f495010f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:55.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.545+0000 7f4954b81700 1 --2- 172.21.15.27:0/287846244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4950102e00 0x7f495010f2f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f494800b0a0 tx=0x7f494800b3b0 comp rx=0 tx=0).stop 2024-05-21T03:23:55.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.545+0000 7f4954b81700 1 -- 172.21.15.27:0/287846244 shutdown_connections 2024-05-21T03:23:55.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.545+0000 7f4954b81700 1 --2- 172.21.15.27:0/287846244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4950102e00 0x7f495010f2f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:55.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.545+0000 7f4954b81700 1 --2- 172.21.15.27:0/287846244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4950108520 0x7f4950108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:55.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.545+0000 7f4954b81700 1 --2- 172.21.15.27:0/287846244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4950102440 0x7f49501028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:55.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.545+0000 7f4954b81700 1 -- 172.21.15.27:0/287846244 >> 172.21.15.27:0/287846244 conn(0x7f49500793f0 msgr2=0x7f4950079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:55.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.545+0000 7f4954b81700 1 -- 172.21.15.27:0/287846244 shutdown_connections 2024-05-21T03:23:55.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.545+0000 7f4954b81700 1 -- 172.21.15.27:0/287846244 wait complete. 2024-05-21T03:23:55.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.546+0000 7f4954b81700 1 Processor -- start 2024-05-21T03:23:55.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.546+0000 7f4954b81700 1 -- start start 2024-05-21T03:23:55.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.546+0000 7f4954b81700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4950102440 0x7f495019f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:55.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.546+0000 7f4954b81700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4950102e00 0x7f495019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:55.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.547+0000 7f4954b81700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4950108520 0x7f4950199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:55.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.547+0000 7f4954b81700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f49501125d0 con 0x7f4950108520 2024-05-21T03:23:55.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.547+0000 7f4954b81700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4950112450 con 0x7f4950102440 2024-05-21T03:23:55.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.547+0000 7f4954b81700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4950112750 con 0x7f4950102e00 2024-05-21T03:23:55.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.547+0000 7f494ed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4950108520 0x7f4950199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:55.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.547+0000 7f494ed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4950108520 0x7f4950199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54394/0 (socket says 172.21.15.27:54394) 2024-05-21T03:23:55.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.547+0000 7f494ed9d700 1 -- 172.21.15.27:0/3451683045 learned_addr learned my addr 172.21.15.27:0/3451683045 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:55.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.547+0000 7f494e59c700 1 --2- 172.21.15.27:0/3451683045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4950102440 0x7f495019f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:55.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.547+0000 7f494dd9b700 1 --2- 172.21.15.27:0/3451683045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4950102e00 0x7f495019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:55.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.547+0000 7f494ed9d700 1 -- 172.21.15.27:0/3451683045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4950102e00 msgr2=0x7f495019fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:55.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.547+0000 7f494ed9d700 1 --2- 172.21.15.27:0/3451683045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4950102e00 0x7f495019fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:55.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.547+0000 7f494ed9d700 1 -- 172.21.15.27:0/3451683045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4950102440 msgr2=0x7f495019f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:55.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.547+0000 7f494ed9d700 1 --2- 172.21.15.27:0/3451683045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4950102440 0x7f495019f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:55.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.547+0000 7f494ed9d700 1 -- 172.21.15.27:0/3451683045 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495019a1d0 con 0x7f4950108520 2024-05-21T03:23:55.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.547+0000 7f494ed9d700 1 --2- 172.21.15.27:0/3451683045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4950108520 0x7f4950199a70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f494800b830 tx=0x7f4948004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:55.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.547+0000 7f493f7fe700 1 -- 172.21.15.27:0/3451683045 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4948021070 con 0x7f4950108520 2024-05-21T03:23:55.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.547+0000 7f4954b81700 1 -- 172.21.15.27:0/3451683045 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495019a460 con 0x7f4950108520 2024-05-21T03:23:55.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.547+0000 7f493f7fe700 1 -- 172.21.15.27:0/3451683045 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4948007e30 con 0x7f4950108520 2024-05-21T03:23:55.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.547+0000 7f4954b81700 1 -- 172.21.15.27:0/3451683045 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49501a6730 con 0x7f4950108520 2024-05-21T03:23:55.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.547+0000 7f493f7fe700 1 -- 172.21.15.27:0/3451683045 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4948004700 con 0x7f4950108520 2024-05-21T03:23:55.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.548+0000 7f4954b81700 1 -- 172.21.15.27:0/3451683045 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f495006a230 con 0x7f4950108520 2024-05-21T03:23:55.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.550+0000 7f493f7fe700 1 -- 172.21.15.27:0/3451683045 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f4948003d70 con 0x7f4950108520 2024-05-21T03:23:55.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.551+0000 7f493f7fe700 1 --2- 172.21.15.27:0/3451683045 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f49380b28d0 0x7f49380b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:55.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.551+0000 7f494e59c700 1 -- 172.21.15.27:0/3451683045 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f49380b28d0 msgr2=0x7f49380b4d90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3683408243 2024-05-21T03:23:55.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.551+0000 7f493f7fe700 1 -- 172.21.15.27:0/3451683045 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f49480e3290 con 0x7f4950108520 2024-05-21T03:23:55.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.551+0000 7f494e59c700 1 --2- 172.21.15.27:0/3451683045 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f49380b28d0 0x7f49380b4d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T03:23:55.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.555+0000 7f493f7fe700 1 -- 172.21.15.27:0/3451683045 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f49480ab020 con 0x7f4950108520 2024-05-21T03:23:55.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.752+0000 7f494e59c700 1 -- 172.21.15.27:0/3451683045 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f49380b28d0 msgr2=0x7f49380b4d90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3683408243 2024-05-21T03:23:55.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.752+0000 7f494e59c700 1 --2- 172.21.15.27:0/3451683045 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f49380b28d0 0x7f49380b4d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T03:23:55.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.820+0000 7f4954b81700 1 -- 172.21.15.27:0/3451683045 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f495002e560 con 0x7f4950108520 2024-05-21T03:23:55.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.821+0000 7f493f7fe700 1 -- 172.21.15.27:0/3451683045 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+73 (secure 0 0 0) 0x7f49480aa920 con 0x7f4950108520 2024-05-21T03:23:55.821 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-21T03:23:55.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.823+0000 7f4954b81700 1 -- 172.21.15.27:0/3451683045 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f49380b28d0 msgr2=0x7f49380b4d90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T03:23:55.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.823+0000 7f4954b81700 1 --2- 172.21.15.27:0/3451683045 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f49380b28d0 0x7f49380b4d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:55.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.823+0000 7f4954b81700 1 -- 172.21.15.27:0/3451683045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4950108520 msgr2=0x7f4950199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:55.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.823+0000 7f4954b81700 1 --2- 172.21.15.27:0/3451683045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4950108520 0x7f4950199a70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f494800b830 tx=0x7f4948004ce0 comp rx=0 tx=0).stop 2024-05-21T03:23:55.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.823+0000 7f4954b81700 1 -- 172.21.15.27:0/3451683045 shutdown_connections 2024-05-21T03:23:55.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.823+0000 7f4954b81700 1 --2- 172.21.15.27:0/3451683045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4950108520 0x7f4950199a70 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:55.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.823+0000 7f4954b81700 1 --2- 172.21.15.27:0/3451683045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4950102440 0x7f495019f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:55.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.823+0000 7f4954b81700 1 --2- 172.21.15.27:0/3451683045 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f49380b28d0 0x7f49380b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:55.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.823+0000 7f4954b81700 1 --2- 172.21.15.27:0/3451683045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4950102e00 0x7f495019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:55.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.823+0000 7f4954b81700 1 -- 172.21.15.27:0/3451683045 >> 172.21.15.27:0/3451683045 conn(0x7f49500793f0 msgr2=0x7f495006b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:55.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.823+0000 7f4954b81700 1 -- 172.21.15.27:0/3451683045 shutdown_connections 2024-05-21T03:23:55.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:55.823+0000 7f4954b81700 1 -- 172.21.15.27:0/3451683045 wait complete. 2024-05-21T03:23:56.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:55 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/3451683045' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-21T03:23:56.021 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:23:55 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:23:55.701+0000 7fb35ad12200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T03:23:56.021 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:23:55 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:23:55.871+0000 7fb35ad12200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T03:23:56.070 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:55 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3451683045' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-21T03:23:56.070 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:23:55 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:23:55.886+0000 7f04ba2c3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T03:23:56.070 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:23:56 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:23:56.067+0000 7f04ba2c3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T03:23:56.154 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:55 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/3451683045' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-21T03:23:56.154 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:23:55 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:23:55.800+0000 7ff69aaf4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T03:23:56.155 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:23:55 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:23:55.978+0000 7ff69aaf4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T03:23:56.155 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:23:56 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:23:56.153+0000 7ff69aaf4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T03:23:56.330 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:23:56 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:23:56.042+0000 7fb35ad12200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T03:23:56.426 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:23:56 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:23:56.248+0000 7f04ba2c3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T03:23:56.471 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-21T03:23:56.471 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-21T03:23:56.472 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2024-05-21T03:23:56.472 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-21T03:23:56.472 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-21T03:23:56.472 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-21T03:23:56.472 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-21T03:23:56.472 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-21T03:23:56.472 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-21T03:23:56.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T03:23:56.472 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2024-05-21T03:23:56.490 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-21T03:23:56.490 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-21T03:23:56.490 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2024-05-21T03:23:56.490 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-05-21T03:23:56.490 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2024-05-21T03:23:56.490 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-05-21T03:23:56.491 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-21T03:23:56.491 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2024-05-21T03:23:56.491 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2024-05-21T03:23:56.491 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-21T03:23:56.491 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2024-05-21T03:23:56.491 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2024-05-21T03:23:56.491 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-21T03:23:56.491 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2024-05-21T03:23:56.491 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2024-05-21T03:23:56.491 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-21T03:23:56.491 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 41952sec preferred_lft 41952sec 2024-05-21T03:23:56.491 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link noprefixroute 2024-05-21T03:23:56.491 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-05-21T03:23:56.491 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-21T03:23:56.491 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2024-05-21T03:23:56.492 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2024-05-21T03:23:56.492 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-21T03:23:56.492 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-21T03:23:56.492 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-05-21T03:23:56.492 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add name ceph-brx type bridge 2024-05-21T03:23:56.492 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr flush dev ceph-brx 2024-05-21T03:23:56.492 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set ceph-brx up 2024-05-21T03:23:56.493 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-21T03:23:56.493 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-05-21T03:23:56.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-21T03:23:56.628 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:23:56 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:23:56.330+0000 7fb35ad12200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T03:23:56.629 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:23:56 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:23:56.423+0000 7fb35ad12200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T03:23:56.629 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:23:56 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:23:56.629+0000 7fb35ad12200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T03:23:56.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T03:23:56.693 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:23:56 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:23:56.564+0000 7f04ba2c3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T03:23:56.693 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:23:56 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:23:56.668+0000 7f04ba2c3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T03:23:56.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T03:23:56.700 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-21T03:23:56.726 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-05-21T03:23:56.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T03:23:56.727 DEBUG:teuthology.orchestra.run.smithi027:> route 2024-05-21T03:23:56.761 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:23:56 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:23:56.454+0000 7ff69aaf4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T03:23:56.762 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:23:56 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:23:56.550+0000 7ff69aaf4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T03:23:56.800 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2024-05-21T03:23:56.800 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-21T03:23:56.800 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-21T03:23:56.800 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-21T03:23:56.800 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-21T03:23:56.801 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-21T03:23:56.801 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-05-21T03:23:56.801 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-21T03:23:56.801 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-21T03:23:56.801 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-21T03:23:56.801 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-05-21T03:23:56.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-21T03:23:56.996 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:23:56 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:23:56.889+0000 7f04ba2c3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T03:23:57.019 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:23:56 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:23:56.761+0000 7ff69aaf4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T03:23:57.019 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:23:56 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:23:56.962+0000 7ff69aaf4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T03:23:57.021 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:23:56 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:23:56.824+0000 7fb35ad12200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T03:23:57.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T03:23:57.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T03:23:57.109 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-05-21T03:23:57.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T03:23:57.126 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list-id 2024-05-21T03:23:57.182 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-21T03:23:57.182 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-05-21T03:23:57.182 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-21T03:23:57.182 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-21T03:23:57.182 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-05-21T03:23:57.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-21T03:23:57.282 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:23:57 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:23:57.102+0000 7f04ba2c3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T03:23:57.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T03:23:57.304 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-21T03:23:57.305 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-21T03:23:57.305 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-05-21T03:23:57.305 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-21T03:23:57.305 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-21T03:23:57.305 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-21T03:23:57.305 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-21T03:23:57.305 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-21T03:23:57.305 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-05-21T03:23:57.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-21T03:23:57.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T03:23:57.443 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-21T03:23:57.443 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-05-21T03:23:57.443 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2024-05-21T03:23:57.443 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set dev brx.0 master ceph-brx 2024-05-21T03:23:57.444 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-05-21T03:23:57.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-21T03:23:57.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T03:23:57.528 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-21T03:23:57.529 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-21T03:23:57.544 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-21T03:23:57.545 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-21T03:23:57.545 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-21T03:23:57.602 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-21T03:23:57.602 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-21T03:23:57.602 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/3bb4c492-edff-4f68-974a-5fef1e70bd24 2024-05-21T03:23:57.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T03:23:57.602 DEBUG:teuthology.orchestra.run.smithi027:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/3bb4c492-edff-4f68-974a-5fef1e70bd24 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-05-21T03:23:57.715 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-05-21T03:23:57.715 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-05-21T03:23:57.715 INFO:teuthology.orchestra.run.smithi027.stdout:invalid new device string format 2024-05-21T03:23:57.771 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:23:57 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:23:57.429+0000 7fb35ad12200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T03:23:57.771 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:23:57 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:23:57.650+0000 7fb35ad12200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-21T03:23:57.833 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-05-21T03:23:57.834 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-05-21T03:23:57.834 INFO:teuthology.orchestra.run.smithi027.stdout:invalid new device string format 2024-05-21T03:23:57.834 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.39:3300,172.21.15.165:3300" 2024-05-21T03:23:57.834 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.27:3300,172.21.15.39:3300,172.21.15.165:3300:/volumes/qa/sv_1/3bb4c492-edff-4f68-974a-5fef1e70bd24 2024-05-21T03:23:57.834 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync,key=0,fsid=c9ca94be-171f-11ef-bc9a-c7b262605968" will pass to kernel 2024-05-21T03:23:57.835 INFO:teuthology.orchestra.run.smithi027.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-21T03:23:57.835 INFO:teuthology.orchestra.run.smithi027.stdout: You just mounted a file system that supports labels which does not 2024-05-21T03:23:57.835 INFO:teuthology.orchestra.run.smithi027.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-21T03:23:57.835 INFO:teuthology.orchestra.run.smithi027.stdout: applications will generate AVC messages and not be allowed access to 2024-05-21T03:23:57.835 INFO:teuthology.orchestra.run.smithi027.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-21T03:23:57.837 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-21T03:23:57.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T03:23:57.837 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-21T03:23:57.865 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-21T03:23:57.926 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:23:57 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:23:57.770+0000 7f04ba2c3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T03:23:57.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-21T03:23:57.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T03:23:57.970 INFO:teuthology.orchestra.run.smithi027.stdout:client24922 2024-05-21T03:23:57.970 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-05-21T03:23:57.970 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-21T03:23:58.018 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:23:57 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:23:57.592+0000 7ff69aaf4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T03:23:58.019 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:23:57 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:23:57.823+0000 7ff69aaf4200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-21T03:23:58.027 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-21T03:23:58.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-21T03:23:58.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T03:23:58.127 INFO:teuthology.orchestra.run.smithi027.stdout:client24922 2024-05-21T03:23:58.128 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-21T03:23:58.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-21T03:23:58.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T03:23:58.186 INFO:teuthology.orchestra.run.smithi027.stdout:client24922 2024-05-21T03:23:58.187 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968.client24922/status') 2024-05-21T03:23:58.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-21T03:23:58.248 INFO:teuthology.orchestra.run.smithi027.stdout:instance: client.24922 (3)172.21.15.27:0/3276112055 2024-05-21T03:23:58.248 INFO:teuthology.orchestra.run.smithi027.stdout:blocklisted: false 2024-05-21T03:23:58.248 INFO:teuthology.orchestra.run.smithi027.stderr:0+1 records in 2024-05-21T03:23:58.248 INFO:teuthology.orchestra.run.smithi027.stderr:0+1 records out 2024-05-21T03:23:58.248 INFO:teuthology.orchestra.run.smithi027.stderr:71 bytes copied, 2.7341e-05 s, 2.6 MB/s 2024-05-21T03:23:58.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T03:23:58.256 INFO:teuthology.run_tasks:Running task exec... 2024-05-21T03:23:58.265 INFO:teuthology.task.exec:Executing custom commands... 2024-05-21T03:23:58.265 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-05-21T03:23:58.266 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-21T03:23:58.288 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:23:58 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:23:58.015+0000 7f04ba2c3200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-21T03:23:58.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.374+0000 7f9e97558640 1 -- 172.21.15.27:0/2810078485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e90102ba0 msgr2=0x7f9e90103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:58.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.374+0000 7f9e97558640 1 --2- 172.21.15.27:0/2810078485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e90102ba0 0x7f9e90103020 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9e80009900 tx=0x7f9e8002a410 comp rx=0 tx=0).stop 2024-05-21T03:23:58.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.374+0000 7f9e97558640 1 -- 172.21.15.27:0/2810078485 shutdown_connections 2024-05-21T03:23:58.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.374+0000 7f9e97558640 1 --2- 172.21.15.27:0/2810078485 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e90103560 0x7f9e90109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:58.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.374+0000 7f9e97558640 1 --2- 172.21.15.27:0/2810078485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e90102ba0 0x7f9e90103020 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:58.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.374+0000 7f9e97558640 1 --2- 172.21.15.27:0/2810078485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e901019b0 0x7f9e90101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:58.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.374+0000 7f9e97558640 1 -- 172.21.15.27:0/2810078485 >> 172.21.15.27:0/2810078485 conn(0x7f9e900fcf20 msgr2=0x7f9e900ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:58.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.374+0000 7f9e97558640 1 -- 172.21.15.27:0/2810078485 shutdown_connections 2024-05-21T03:23:58.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.374+0000 7f9e97558640 1 -- 172.21.15.27:0/2810078485 wait complete. 2024-05-21T03:23:58.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.375+0000 7f9e97558640 1 Processor -- start 2024-05-21T03:23:58.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.375+0000 7f9e97558640 1 -- start start 2024-05-21T03:23:58.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.375+0000 7f9e97558640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e901019b0 0x7f9e9006ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:58.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.375+0000 7f9e97558640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e90102ba0 0x7f9e9019d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:58.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.375+0000 7f9e97558640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e90103560 0x7f9e9019fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:58.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.375+0000 7f9e97558640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9010d510 con 0x7f9e901019b0 2024-05-21T03:23:58.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.375+0000 7f9e97558640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9010d390 con 0x7f9e90102ba0 2024-05-21T03:23:58.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.375+0000 7f9e97558640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9010d690 con 0x7f9e90103560 2024-05-21T03:23:58.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.375+0000 7f9e952f4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e901019b0 0x7f9e9006ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:58.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.375+0000 7f9e952f4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e901019b0 0x7f9e9006ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54410/0 (socket says 172.21.15.27:54410) 2024-05-21T03:23:58.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.375+0000 7f9e952f4640 1 -- 172.21.15.27:0/1016363937 learned_addr learned my addr 172.21.15.27:0/1016363937 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:58.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.375+0000 7f9e94af3640 1 --2- 172.21.15.27:0/1016363937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e90102ba0 0x7f9e9019d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:58.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.375+0000 7f9e95af5640 1 --2- 172.21.15.27:0/1016363937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e90103560 0x7f9e9019fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:58.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.375+0000 7f9e952f4640 1 -- 172.21.15.27:0/1016363937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e90103560 msgr2=0x7f9e9019fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:58.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.375+0000 7f9e952f4640 1 --2- 172.21.15.27:0/1016363937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e90103560 0x7f9e9019fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:58.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.375+0000 7f9e952f4640 1 -- 172.21.15.27:0/1016363937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e90102ba0 msgr2=0x7f9e9019d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:58.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.375+0000 7f9e952f4640 1 --2- 172.21.15.27:0/1016363937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e90102ba0 0x7f9e9019d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:58.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.375+0000 7f9e952f4640 1 -- 172.21.15.27:0/1016363937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e901a0900 con 0x7f9e901019b0 2024-05-21T03:23:58.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.376+0000 7f9e952f4640 1 --2- 172.21.15.27:0/1016363937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e901019b0 0x7f9e9006ac70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c00e9c0 tx=0x7f9e8c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:58.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.376+0000 7f9e867fc640 1 -- 172.21.15.27:0/1016363937 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c00cd80 con 0x7f9e901019b0 2024-05-21T03:23:58.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.376+0000 7f9e97558640 1 -- 172.21.15.27:0/1016363937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e901a0b90 con 0x7f9e901019b0 2024-05-21T03:23:58.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.376+0000 7f9e867fc640 1 -- 172.21.15.27:0/1016363937 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e8c0035f0 con 0x7f9e901019b0 2024-05-21T03:23:58.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.376+0000 7f9e867fc640 1 -- 172.21.15.27:0/1016363937 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c010690 con 0x7f9e901019b0 2024-05-21T03:23:58.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.376+0000 7f9e97558640 1 -- 172.21.15.27:0/1016363937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e901a10b0 con 0x7f9e901019b0 2024-05-21T03:23:58.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.377+0000 7f9e97558640 1 -- 172.21.15.27:0/1016363937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e90108d50 con 0x7f9e901019b0 2024-05-21T03:23:58.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.378+0000 7f9e867fc640 1 -- 172.21.15.27:0/1016363937 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f9e8c0037a0 con 0x7f9e901019b0 2024-05-21T03:23:58.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.379+0000 7f9e867fc640 1 --2- 172.21.15.27:0/1016363937 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f9e7c0b28f0 0x7f9e7c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:58.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.380+0000 7f9e94af3640 1 -- 172.21.15.27:0/1016363937 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f9e7c0b28f0 msgr2=0x7f9e7c0b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3683408243 2024-05-21T03:23:58.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.380+0000 7f9e867fc640 1 -- 172.21.15.27:0/1016363937 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f9e8c0e2720 con 0x7f9e901019b0 2024-05-21T03:23:58.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.380+0000 7f9e94af3640 1 --2- 172.21.15.27:0/1016363937 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f9e7c0b28f0 0x7f9e7c0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T03:23:58.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.384+0000 7f9e867fc640 1 -- 172.21.15.27:0/1016363937 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9e8c0aa420 con 0x7f9e901019b0 2024-05-21T03:23:58.521 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:23:58 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:23:58.266+0000 7ff69aaf4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T03:23:58.521 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:23:58 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:23:58.367+0000 7ff69aaf4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T03:23:58.522 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:23:58 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:23:58.076+0000 7fb35ad12200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T03:23:58.522 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:23:58 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:23:58.172+0000 7fb35ad12200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T03:23:58.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.581+0000 7f9e94af3640 1 -- 172.21.15.27:0/1016363937 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f9e7c0b28f0 msgr2=0x7f9e7c0b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3683408243 2024-05-21T03:23:58.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.581+0000 7f9e94af3640 1 --2- 172.21.15.27:0/1016363937 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f9e7c0b28f0 0x7f9e7c0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T03:23:58.617 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:23:58 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:23:58.513+0000 7f04ba2c3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T03:23:58.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.657+0000 7f9e97558640 1 -- 172.21.15.27:0/1016363937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f9e900636f0 con 0x7f9e901019b0 2024-05-21T03:23:58.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.709+0000 7f9e867fc640 1 -- 172.21.15.27:0/1016363937 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v29) ==== 117+0+0 (secure 0 0 0) 0x7f9e8c0a9d20 con 0x7f9e901019b0 2024-05-21T03:23:58.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.711+0000 7f9e97558640 1 -- 172.21.15.27:0/1016363937 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f9e7c0b28f0 msgr2=0x7f9e7c0b4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T03:23:58.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.711+0000 7f9e97558640 1 --2- 172.21.15.27:0/1016363937 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f9e7c0b28f0 0x7f9e7c0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:58.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.711+0000 7f9e97558640 1 -- 172.21.15.27:0/1016363937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e901019b0 msgr2=0x7f9e9006ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:58.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.711+0000 7f9e97558640 1 --2- 172.21.15.27:0/1016363937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e901019b0 0x7f9e9006ac70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c00e9c0 tx=0x7f9e8c00ee80 comp rx=0 tx=0).stop 2024-05-21T03:23:58.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.711+0000 7f9e97558640 1 -- 172.21.15.27:0/1016363937 shutdown_connections 2024-05-21T03:23:58.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.711+0000 7f9e97558640 1 --2- 172.21.15.27:0/1016363937 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f9e7c0b28f0 0x7f9e7c0b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:58.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.711+0000 7f9e97558640 1 --2- 172.21.15.27:0/1016363937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e90103560 0x7f9e9019fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:58.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.711+0000 7f9e97558640 1 --2- 172.21.15.27:0/1016363937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e90102ba0 0x7f9e9019d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:58.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.711+0000 7f9e97558640 1 --2- 172.21.15.27:0/1016363937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e901019b0 0x7f9e9006ac70 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:58.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.711+0000 7f9e97558640 1 -- 172.21.15.27:0/1016363937 >> 172.21.15.27:0/1016363937 conn(0x7f9e900fcf20 msgr2=0x7f9e90105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:58.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.711+0000 7f9e97558640 1 -- 172.21.15.27:0/1016363937 shutdown_connections 2024-05-21T03:23:58.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.711+0000 7f9e97558640 1 -- 172.21.15.27:0/1016363937 wait complete. 2024-05-21T03:23:58.722 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-21T03:23:58.737 INFO:teuthology.run_tasks:Running task exec... 2024-05-21T03:23:58.745 INFO:teuthology.task.exec:Executing custom commands... 2024-05-21T03:23:58.745 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-05-21T03:23:58.745 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-05-21T03:23:58.805 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:58 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/1016363937' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-21T03:23:58.806 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:23:58 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:23:58.756+0000 7ff69aaf4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T03:23:58.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.874+0000 7fccfd269640 1 -- 172.21.15.27:0/279294199 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccf8103560 msgr2=0x7fccf8109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:58.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.874+0000 7fccfd269640 1 --2- 172.21.15.27:0/279294199 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccf8103560 0x7fccf8109df0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fccf0009ca0 tx=0x7fccf002a8c0 comp rx=0 tx=0).stop 2024-05-21T03:23:58.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.874+0000 7fccfd269640 1 -- 172.21.15.27:0/279294199 shutdown_connections 2024-05-21T03:23:58.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.874+0000 7fccfd269640 1 --2- 172.21.15.27:0/279294199 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccf8103560 0x7fccf8109df0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:58.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.874+0000 7fccfd269640 1 --2- 172.21.15.27:0/279294199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccf8102ba0 0x7fccf8103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:58.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.874+0000 7fccfd269640 1 --2- 172.21.15.27:0/279294199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fccf81019b0 0x7fccf8101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:58.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.874+0000 7fccfd269640 1 -- 172.21.15.27:0/279294199 >> 172.21.15.27:0/279294199 conn(0x7fccf80fcf20 msgr2=0x7fccf80ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:58.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.874+0000 7fccfd269640 1 -- 172.21.15.27:0/279294199 shutdown_connections 2024-05-21T03:23:58.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.874+0000 7fccfd269640 1 -- 172.21.15.27:0/279294199 wait complete. 2024-05-21T03:23:58.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.875+0000 7fccfd269640 1 Processor -- start 2024-05-21T03:23:58.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.875+0000 7fccfd269640 1 -- start start 2024-05-21T03:23:58.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.875+0000 7fccfd269640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccf81019b0 0x7fccf8105940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:58.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.875+0000 7fccfd269640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccf8102ba0 0x7fccf8103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:58.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.875+0000 7fccfd269640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fccf8103560 0x7fccf81044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:58.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.875+0000 7fccfd269640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fccf810d4e0 con 0x7fccf81019b0 2024-05-21T03:23:58.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.875+0000 7fccfd269640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fccf810d360 con 0x7fccf8102ba0 2024-05-21T03:23:58.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.875+0000 7fccfd269640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fccf810d660 con 0x7fccf8103560 2024-05-21T03:23:58.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.875+0000 7fccf6ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccf81019b0 0x7fccf8105940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:58.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.875+0000 7fccf6ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccf81019b0 0x7fccf8105940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54432/0 (socket says 172.21.15.27:54432) 2024-05-21T03:23:58.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.875+0000 7fccf6ffd640 1 -- 172.21.15.27:0/1933810213 learned_addr learned my addr 172.21.15.27:0/1933810213 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:58.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.875+0000 7fcceffff640 1 --2- 172.21.15.27:0/1933810213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccf8102ba0 0x7fccf8103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:58.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.875+0000 7fccf77fe640 1 --2- 172.21.15.27:0/1933810213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fccf8103560 0x7fccf81044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:58.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.876+0000 7fccf6ffd640 1 -- 172.21.15.27:0/1933810213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fccf8103560 msgr2=0x7fccf81044c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:58.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.876+0000 7fccf6ffd640 1 --2- 172.21.15.27:0/1933810213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fccf8103560 0x7fccf81044c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:58.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.876+0000 7fccf6ffd640 1 -- 172.21.15.27:0/1933810213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccf8102ba0 msgr2=0x7fccf8103f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:58.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.876+0000 7fccf6ffd640 1 --2- 172.21.15.27:0/1933810213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccf8102ba0 0x7fccf8103f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:58.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.876+0000 7fccf6ffd640 1 -- 172.21.15.27:0/1933810213 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf8104d50 con 0x7fccf81019b0 2024-05-21T03:23:58.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.876+0000 7fccf6ffd640 1 --2- 172.21.15.27:0/1933810213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccf81019b0 0x7fccf8105940 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcce800d950 tx=0x7fcce800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:58.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.876+0000 7fccf4ff9640 1 -- 172.21.15.27:0/1933810213 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8017070 con 0x7fccf81019b0 2024-05-21T03:23:58.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.876+0000 7fccf4ff9640 1 -- 172.21.15.27:0/1933810213 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcce8009c30 con 0x7fccf81019b0 2024-05-21T03:23:58.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.876+0000 7fccfd269640 1 -- 172.21.15.27:0/1933810213 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf81a2d10 con 0x7fccf81019b0 2024-05-21T03:23:58.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.876+0000 7fccf4ff9640 1 -- 172.21.15.27:0/1933810213 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce800f620 con 0x7fccf81019b0 2024-05-21T03:23:58.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.876+0000 7fccfd269640 1 -- 172.21.15.27:0/1933810213 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf81a31b0 con 0x7fccf81019b0 2024-05-21T03:23:58.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.877+0000 7fccfd269640 1 -- 172.21.15.27:0/1933810213 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccf81a3380 con 0x7fccf81019b0 2024-05-21T03:23:58.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.879+0000 7fccf4ff9640 1 -- 172.21.15.27:0/1933810213 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7fcce801a070 con 0x7fccf81019b0 2024-05-21T03:23:58.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.879+0000 7fccf4ff9640 1 --2- 172.21.15.27:0/1933810213 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fcce40b28f0 0x7fcce40b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:58.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.879+0000 7fcceffff640 1 -- 172.21.15.27:0/1933810213 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fcce40b28f0 msgr2=0x7fcce40b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3683408243 2024-05-21T03:23:58.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.879+0000 7fccf4ff9640 1 -- 172.21.15.27:0/1933810213 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fcce80e1c50 con 0x7fccf81019b0 2024-05-21T03:23:58.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.879+0000 7fcceffff640 1 --2- 172.21.15.27:0/1933810213 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fcce40b28f0 0x7fcce40b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T03:23:58.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:58.887+0000 7fccf4ff9640 1 -- 172.21.15.27:0/1933810213 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcce8009de0 con 0x7fccf81019b0 2024-05-21T03:23:58.907 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:58 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/1016363937' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-21T03:23:58.907 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:23:58 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:23:58.546+0000 7fb35ad12200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T03:23:58.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:58 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/1016363937' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-21T03:23:58.927 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:23:58 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:23:58.618+0000 7f04ba2c3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T03:23:59.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.081+0000 7fcceffff640 1 -- 172.21.15.27:0/1933810213 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fcce40b28f0 msgr2=0x7fcce40b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3683408243 2024-05-21T03:23:59.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.081+0000 7fcceffff640 1 --2- 172.21.15.27:0/1933810213 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fcce40b28f0 0x7fcce40b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T03:23:59.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.082+0000 7fccfd269640 1 -- 172.21.15.27:0/1933810213 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7fccf81a35f0 con 0x7fccf81019b0 2024-05-21T03:23:59.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.086+0000 7fccf4ff9640 1 -- 172.21.15.27:0/1933810213 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v28)=0 v28) ==== 129+0+0 (secure 0 0 0) 0x7fcce80a9300 con 0x7fccf81019b0 2024-05-21T03:23:59.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.088+0000 7fccfd269640 1 -- 172.21.15.27:0/1933810213 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fcce40b28f0 msgr2=0x7fcce40b4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T03:23:59.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.088+0000 7fccfd269640 1 --2- 172.21.15.27:0/1933810213 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fcce40b28f0 0x7fcce40b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.088+0000 7fccfd269640 1 -- 172.21.15.27:0/1933810213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccf81019b0 msgr2=0x7fccf8105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:59.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.088+0000 7fccfd269640 1 --2- 172.21.15.27:0/1933810213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccf81019b0 0x7fccf8105940 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcce800d950 tx=0x7fcce800de10 comp rx=0 tx=0).stop 2024-05-21T03:23:59.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.089+0000 7fccfd269640 1 -- 172.21.15.27:0/1933810213 shutdown_connections 2024-05-21T03:23:59.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.089+0000 7fccfd269640 1 --2- 172.21.15.27:0/1933810213 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fcce40b28f0 0x7fcce40b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.089+0000 7fccfd269640 1 --2- 172.21.15.27:0/1933810213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fccf8103560 0x7fccf81044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.089+0000 7fccfd269640 1 --2- 172.21.15.27:0/1933810213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccf8102ba0 0x7fccf8103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.089+0000 7fccfd269640 1 --2- 172.21.15.27:0/1933810213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccf81019b0 0x7fccf8105940 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.089+0000 7fccfd269640 1 -- 172.21.15.27:0/1933810213 >> 172.21.15.27:0/1933810213 conn(0x7fccf80fcf20 msgr2=0x7fccf80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:59.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.089+0000 7fccfd269640 1 -- 172.21.15.27:0/1933810213 shutdown_connections 2024-05-21T03:23:59.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.089+0000 7fccfd269640 1 -- 172.21.15.27:0/1933810213 wait complete. 2024-05-21T03:23:59.099 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-05-21T03:23:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.207+0000 7f2f9568c640 1 -- 172.21.15.27:0/3472589256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f90103570 msgr2=0x7f2f90109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.207+0000 7f2f9568c640 1 --2- 172.21.15.27:0/3472589256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f90103570 0x7f2f90109e00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009ca0 tx=0x7f2f8802a8c0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.207+0000 7f2f9568c640 1 -- 172.21.15.27:0/3472589256 shutdown_connections 2024-05-21T03:23:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.207+0000 7f2f9568c640 1 --2- 172.21.15.27:0/3472589256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f90103570 0x7f2f90109e00 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.207+0000 7f2f9568c640 1 --2- 172.21.15.27:0/3472589256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f90102bb0 0x7f2f90103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.207+0000 7f2f9568c640 1 --2- 172.21.15.27:0/3472589256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f901019c0 0x7f2f90101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.207+0000 7f2f9568c640 1 -- 172.21.15.27:0/3472589256 >> 172.21.15.27:0/3472589256 conn(0x7f2f900fcf30 msgr2=0x7f2f900ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:59.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.207+0000 7f2f9568c640 1 -- 172.21.15.27:0/3472589256 shutdown_connections 2024-05-21T03:23:59.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.207+0000 7f2f9568c640 1 -- 172.21.15.27:0/3472589256 wait complete. 2024-05-21T03:23:59.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.207+0000 7f2f9568c640 1 Processor -- start 2024-05-21T03:23:59.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.207+0000 7f2f9568c640 1 -- start start 2024-05-21T03:23:59.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.208+0000 7f2f9568c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f901019c0 0x7f2f90198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:59.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.208+0000 7f2f9568c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f90102bb0 0x7f2f90199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:59.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.208+0000 7f2f9568c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f90103570 0x7f2f901a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:59.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.208+0000 7f2f9568c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9010e610 con 0x7f2f901019c0 2024-05-21T03:23:59.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.208+0000 7f2f9568c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9010e490 con 0x7f2f90103570 2024-05-21T03:23:59.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.208+0000 7f2f9568c640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9010e790 con 0x7f2f90102bb0 2024-05-21T03:23:59.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.208+0000 7f2f8effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f901019c0 0x7f2f90198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:59.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.208+0000 7f2f8effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f901019c0 0x7f2f90198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54454/0 (socket says 172.21.15.27:54454) 2024-05-21T03:23:59.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.208+0000 7f2f8effd640 1 -- 172.21.15.27:0/733695237 learned_addr learned my addr 172.21.15.27:0/733695237 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:59.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.208+0000 7f2f8e7fc640 1 --2- 172.21.15.27:0/733695237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f90102bb0 0x7f2f90199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:59.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.208+0000 7f2f8f7fe640 1 --2- 172.21.15.27:0/733695237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f90103570 0x7f2f901a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:59.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.208+0000 7f2f8effd640 1 -- 172.21.15.27:0/733695237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f90102bb0 msgr2=0x7f2f90199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:59.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.208+0000 7f2f8effd640 1 --2- 172.21.15.27:0/733695237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f90102bb0 0x7f2f90199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.208+0000 7f2f8effd640 1 -- 172.21.15.27:0/733695237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f90103570 msgr2=0x7f2f901a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:59.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.208+0000 7f2f8effd640 1 --2- 172.21.15.27:0/733695237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f90103570 0x7f2f901a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.208+0000 7f2f8effd640 1 -- 172.21.15.27:0/733695237 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f901a0840 con 0x7f2f901019c0 2024-05-21T03:23:59.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.208+0000 7f2f8f7fe640 1 --2- 172.21.15.27:0/733695237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f90103570 0x7f2f901a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:23:59.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.208+0000 7f2f8effd640 1 --2- 172.21.15.27:0/733695237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f901019c0 0x7f2f90198c30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000e9c0 tx=0x7f2f8000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:59.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.208+0000 7f2f87fff640 1 -- 172.21.15.27:0/733695237 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8000cd80 con 0x7f2f901019c0 2024-05-21T03:23:59.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.208+0000 7f2f87fff640 1 -- 172.21.15.27:0/733695237 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f800035f0 con 0x7f2f901019c0 2024-05-21T03:23:59.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.208+0000 7f2f87fff640 1 -- 172.21.15.27:0/733695237 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f80010690 con 0x7f2f901019c0 2024-05-21T03:23:59.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.208+0000 7f2f9568c640 1 -- 172.21.15.27:0/733695237 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f901a0b30 con 0x7f2f901019c0 2024-05-21T03:23:59.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.209+0000 7f2f9568c640 1 -- 172.21.15.27:0/733695237 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f901a1080 con 0x7f2f901019c0 2024-05-21T03:23:59.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.210+0000 7f2f9568c640 1 -- 172.21.15.27:0/733695237 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f90108ce0 con 0x7f2f901019c0 2024-05-21T03:23:59.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.211+0000 7f2f87fff640 1 -- 172.21.15.27:0/733695237 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f2f800037a0 con 0x7f2f901019c0 2024-05-21T03:23:59.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.212+0000 7f2f87fff640 1 --2- 172.21.15.27:0/733695237 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f2f7c0b28f0 0x7f2f7c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:59.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.212+0000 7f2f8e7fc640 1 -- 172.21.15.27:0/733695237 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f2f7c0b28f0 msgr2=0x7f2f7c0b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3683408243 2024-05-21T03:23:59.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.212+0000 7f2f87fff640 1 -- 172.21.15.27:0/733695237 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f2f800e2890 con 0x7f2f901019c0 2024-05-21T03:23:59.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.212+0000 7f2f8e7fc640 1 --2- 172.21.15.27:0/733695237 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f2f7c0b28f0 0x7f2f7c0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T03:23:59.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.215+0000 7f2f87fff640 1 -- 172.21.15.27:0/733695237 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2f800aa590 con 0x7f2f901019c0 2024-05-21T03:23:59.248 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:23:58 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:23:58.907+0000 7fb35ad12200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T03:23:59.248 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:23:59 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:23:59.001+0000 7fb35ad12200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T03:23:59.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.409+0000 7f2f9568c640 1 -- 172.21.15.27:0/733695237 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f2f9002dba0 con 0x7f2f901019c0 2024-05-21T03:23:59.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.413+0000 7f2f8e7fc640 1 -- 172.21.15.27:0/733695237 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f2f7c0b28f0 msgr2=0x7f2f7c0b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3683408243 2024-05-21T03:23:59.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.413+0000 7f2f8e7fc640 1 --2- 172.21.15.27:0/733695237 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f2f7c0b28f0 0x7f2f7c0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T03:23:59.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.413+0000 7f2f87fff640 1 -- 172.21.15.27:0/733695237 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7f2f800a9e90 con 0x7f2f901019c0 2024-05-21T03:23:59.414 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:23:59 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:23:59.022+0000 7f04ba2c3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T03:23:59.414 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:23:59 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:23:59.414+0000 7f04ba2c3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T03:23:59.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.415+0000 7f2f9568c640 1 -- 172.21.15.27:0/733695237 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f2f7c0b28f0 msgr2=0x7f2f7c0b4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T03:23:59.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.415+0000 7f2f9568c640 1 --2- 172.21.15.27:0/733695237 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f2f7c0b28f0 0x7f2f7c0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.415+0000 7f2f9568c640 1 -- 172.21.15.27:0/733695237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f901019c0 msgr2=0x7f2f90198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:59.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.415+0000 7f2f9568c640 1 --2- 172.21.15.27:0/733695237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f901019c0 0x7f2f90198c30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000e9c0 tx=0x7f2f8000ee80 comp rx=0 tx=0).stop 2024-05-21T03:23:59.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.415+0000 7f2f9568c640 1 -- 172.21.15.27:0/733695237 shutdown_connections 2024-05-21T03:23:59.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.415+0000 7f2f9568c640 1 --2- 172.21.15.27:0/733695237 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f2f7c0b28f0 0x7f2f7c0b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.415+0000 7f2f9568c640 1 --2- 172.21.15.27:0/733695237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f90103570 0x7f2f901a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.415+0000 7f2f9568c640 1 --2- 172.21.15.27:0/733695237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f90102bb0 0x7f2f90199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.415+0000 7f2f9568c640 1 --2- 172.21.15.27:0/733695237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f901019c0 0x7f2f90198c30 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.415+0000 7f2f9568c640 1 -- 172.21.15.27:0/733695237 >> 172.21.15.27:0/733695237 conn(0x7f2f900fcf30 msgr2=0x7f2f90105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:59.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.415+0000 7f2f9568c640 1 -- 172.21.15.27:0/733695237 shutdown_connections 2024-05-21T03:23:59.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.415+0000 7f2f9568c640 1 -- 172.21.15.27:0/733695237 wait complete. 2024-05-21T03:23:59.426 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-05-21T03:23:59.498 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:23:59 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:23:59.142+0000 7ff69aaf4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T03:23:59.499 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:23:59 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:23:59.241+0000 7ff69aaf4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T03:23:59.521 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:23:59 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:23:59.248+0000 7fb35ad12200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T03:23:59.521 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:23:59 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:23:59.380+0000 7fb35ad12200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T03:23:59.521 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:23:59 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:23:59.478+0000 7fb35ad12200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T03:23:59.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.534+0000 7f4642ab4640 1 -- 172.21.15.27:0/3324925413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f463c102ba0 msgr2=0x7f463c103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:59.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.534+0000 7f4642ab4640 1 --2- 172.21.15.27:0/3324925413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f463c102ba0 0x7f463c103020 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4628009900 tx=0x7f462802a410 comp rx=0 tx=0).stop 2024-05-21T03:23:59.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.534+0000 7f4642ab4640 1 -- 172.21.15.27:0/3324925413 shutdown_connections 2024-05-21T03:23:59.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.534+0000 7f4642ab4640 1 --2- 172.21.15.27:0/3324925413 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f463c103560 0x7f463c109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.534+0000 7f4642ab4640 1 --2- 172.21.15.27:0/3324925413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f463c102ba0 0x7f463c103020 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.534+0000 7f4642ab4640 1 --2- 172.21.15.27:0/3324925413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f463c1019b0 0x7f463c101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.534+0000 7f4642ab4640 1 -- 172.21.15.27:0/3324925413 >> 172.21.15.27:0/3324925413 conn(0x7f463c0fcf20 msgr2=0x7f463c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:59.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.534+0000 7f4642ab4640 1 -- 172.21.15.27:0/3324925413 shutdown_connections 2024-05-21T03:23:59.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.534+0000 7f4642ab4640 1 -- 172.21.15.27:0/3324925413 wait complete. 2024-05-21T03:23:59.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.534+0000 7f4642ab4640 1 Processor -- start 2024-05-21T03:23:59.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.535+0000 7f4642ab4640 1 -- start start 2024-05-21T03:23:59.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.535+0000 7f4642ab4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f463c1019b0 0x7f463c06ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:59.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.535+0000 7f4642ab4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f463c102ba0 0x7f463c19da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:59.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.535+0000 7f4642ab4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f463c103560 0x7f463c19ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:59.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.535+0000 7f4642ab4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f463c10d4c0 con 0x7f463c103560 2024-05-21T03:23:59.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.535+0000 7f4642ab4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f463c10d340 con 0x7f463c102ba0 2024-05-21T03:23:59.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.535+0000 7f4642ab4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f463c10d640 con 0x7f463c1019b0 2024-05-21T03:23:59.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.535+0000 7f4641051640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f463c103560 0x7f463c19ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:59.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.535+0000 7f4641051640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f463c103560 0x7f463c19ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54472/0 (socket says 172.21.15.27:54472) 2024-05-21T03:23:59.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.535+0000 7f4641051640 1 -- 172.21.15.27:0/3072376918 learned_addr learned my addr 172.21.15.27:0/3072376918 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:59.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.535+0000 7f463bfff640 1 --2- 172.21.15.27:0/3072376918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f463c102ba0 0x7f463c19da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:59.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.535+0000 7f4640850640 1 --2- 172.21.15.27:0/3072376918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f463c1019b0 0x7f463c06ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:59.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.535+0000 7f4641051640 1 -- 172.21.15.27:0/3072376918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f463c1019b0 msgr2=0x7f463c06ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:59.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.535+0000 7f4641051640 1 --2- 172.21.15.27:0/3072376918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f463c1019b0 0x7f463c06ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.535+0000 7f4641051640 1 -- 172.21.15.27:0/3072376918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f463c102ba0 msgr2=0x7f463c19da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:59.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.535+0000 7f4641051640 1 --2- 172.21.15.27:0/3072376918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f463c102ba0 0x7f463c19da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.535+0000 7f4641051640 1 -- 172.21.15.27:0/3072376918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463c1a08a0 con 0x7f463c103560 2024-05-21T03:23:59.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.535+0000 7f4641051640 1 --2- 172.21.15.27:0/3072376918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f463c103560 0x7f463c19ffe0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f463400eb10 tx=0x7f463400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:59.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.535+0000 7f4639ffb640 1 -- 172.21.15.27:0/3072376918 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4634004b20 con 0x7f463c103560 2024-05-21T03:23:59.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.535+0000 7f4639ffb640 1 -- 172.21.15.27:0/3072376918 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4634004cc0 con 0x7f463c103560 2024-05-21T03:23:59.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.536+0000 7f4642ab4640 1 -- 172.21.15.27:0/3072376918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f463c1a0b90 con 0x7f463c103560 2024-05-21T03:23:59.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.536+0000 7f4639ffb640 1 -- 172.21.15.27:0/3072376918 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4634010670 con 0x7f463c103560 2024-05-21T03:23:59.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.536+0000 7f4642ab4640 1 -- 172.21.15.27:0/3072376918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f463c1a1060 con 0x7f463c103560 2024-05-21T03:23:59.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.537+0000 7f4642ab4640 1 -- 172.21.15.27:0/3072376918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f463c108da0 con 0x7f463c103560 2024-05-21T03:23:59.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.538+0000 7f4639ffb640 1 -- 172.21.15.27:0/3072376918 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f4634005000 con 0x7f463c103560 2024-05-21T03:23:59.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.539+0000 7f4639ffb640 1 --2- 172.21.15.27:0/3072376918 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f462c0b28f0 0x7f462c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:59.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.539+0000 7f4640850640 1 -- 172.21.15.27:0/3072376918 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f462c0b28f0 msgr2=0x7f462c0b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3683408243 2024-05-21T03:23:59.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.539+0000 7f4639ffb640 1 -- 172.21.15.27:0/3072376918 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f46340e2990 con 0x7f463c103560 2024-05-21T03:23:59.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.539+0000 7f4640850640 1 --2- 172.21.15.27:0/3072376918 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f462c0b28f0 0x7f462c0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T03:23:59.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.543+0000 7f4639ffb640 1 -- 172.21.15.27:0/3072376918 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4634054df0 con 0x7f463c103560 2024-05-21T03:23:59.676 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:23:59 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:23:59.521+0000 7f04ba2c3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T03:23:59.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.738+0000 7f4642ab4640 1 -- 172.21.15.27:0/3072376918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f463c02dc40 con 0x7f463c103560 2024-05-21T03:23:59.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.740+0000 7f4640850640 1 -- 172.21.15.27:0/3072376918 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f462c0b28f0 msgr2=0x7f462c0b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3683408243 2024-05-21T03:23:59.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.740+0000 7f4640850640 1 --2- 172.21.15.27:0/3072376918 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f462c0b28f0 0x7f462c0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T03:23:59.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.744+0000 7f4639ffb640 1 -- 172.21.15.27:0/3072376918 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7f46340aa690 con 0x7f463c103560 2024-05-21T03:23:59.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.746+0000 7f4642ab4640 1 -- 172.21.15.27:0/3072376918 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f462c0b28f0 msgr2=0x7f462c0b4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T03:23:59.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.746+0000 7f4642ab4640 1 --2- 172.21.15.27:0/3072376918 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f462c0b28f0 0x7f462c0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.746+0000 7f4642ab4640 1 -- 172.21.15.27:0/3072376918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f463c103560 msgr2=0x7f463c19ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:59.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.746+0000 7f4642ab4640 1 --2- 172.21.15.27:0/3072376918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f463c103560 0x7f463c19ffe0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f463400eb10 tx=0x7f463400c2d0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.746+0000 7f4642ab4640 1 -- 172.21.15.27:0/3072376918 shutdown_connections 2024-05-21T03:23:59.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.746+0000 7f4642ab4640 1 --2- 172.21.15.27:0/3072376918 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7f462c0b28f0 0x7f462c0b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.746+0000 7f4642ab4640 1 --2- 172.21.15.27:0/3072376918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f463c103560 0x7f463c19ffe0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.746+0000 7f4642ab4640 1 --2- 172.21.15.27:0/3072376918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f463c102ba0 0x7f463c19da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.746+0000 7f4642ab4640 1 --2- 172.21.15.27:0/3072376918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f463c1019b0 0x7f463c06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.746+0000 7f4642ab4640 1 -- 172.21.15.27:0/3072376918 >> 172.21.15.27:0/3072376918 conn(0x7f463c0fcf20 msgr2=0x7f463c105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:59.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.746+0000 7f4642ab4640 1 -- 172.21.15.27:0/3072376918 shutdown_connections 2024-05-21T03:23:59.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.746+0000 7f4642ab4640 1 -- 172.21.15.27:0/3072376918 wait complete. 2024-05-21T03:23:59.757 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-05-21T03:23:59.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:59 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/1016363937' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-21T03:23:59.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:59 smithi039 ceph-mon[36429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:23:59.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:59 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/1933810213' entity='client.admin' 2024-05-21T03:23:59.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:23:59 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/733695237' entity='client.admin' 2024-05-21T03:23:59.769 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:23:59 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:23:59.498+0000 7ff69aaf4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T03:23:59.769 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:23:59 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:23:59.640+0000 7ff69aaf4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T03:23:59.769 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:23:59 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:23:59.741+0000 7ff69aaf4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T03:23:59.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.867+0000 7fe8176bd640 1 -- 172.21.15.27:0/249606422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe810105ec0 msgr2=0x7fe8101082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:59.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.867+0000 7fe8176bd640 1 --2- 172.21.15.27:0/249606422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe810105ec0 0x7fe8101082b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe800009900 tx=0x7fe80002a410 comp rx=0 tx=0).stop 2024-05-21T03:23:59.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.867+0000 7fe8176bd640 1 -- 172.21.15.27:0/249606422 shutdown_connections 2024-05-21T03:23:59.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.867+0000 7fe8176bd640 1 --2- 172.21.15.27:0/249606422 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8101087f0 0x7fe81010ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.867+0000 7fe8176bd640 1 --2- 172.21.15.27:0/249606422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe810105ec0 0x7fe8101082b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.867+0000 7fe8176bd640 1 --2- 172.21.15.27:0/249606422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe810103590 0x7fe810105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.867+0000 7fe8176bd640 1 -- 172.21.15.27:0/249606422 >> 172.21.15.27:0/249606422 conn(0x7fe8100fcf60 msgr2=0x7fe8100ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:23:59.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.867+0000 7fe8176bd640 1 -- 172.21.15.27:0/249606422 shutdown_connections 2024-05-21T03:23:59.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.867+0000 7fe8176bd640 1 -- 172.21.15.27:0/249606422 wait complete. 2024-05-21T03:23:59.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.867+0000 7fe8176bd640 1 Processor -- start 2024-05-21T03:23:59.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.867+0000 7fe8176bd640 1 -- start start 2024-05-21T03:23:59.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.868+0000 7fe8176bd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe810103590 0x7fe810102f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:59.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.868+0000 7fe8176bd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe810105ec0 0x7fe810101540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:59.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.868+0000 7fe8176bd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8101087f0 0x7fe810101a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:59.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.868+0000 7fe8176bd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe81010e600 con 0x7fe8101087f0 2024-05-21T03:23:59.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.868+0000 7fe8176bd640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe81010e480 con 0x7fe810105ec0 2024-05-21T03:23:59.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.868+0000 7fe8176bd640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe81010e780 con 0x7fe810103590 2024-05-21T03:23:59.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.868+0000 7fe815c5a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8101087f0 0x7fe810101a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:59.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.868+0000 7fe815c5a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8101087f0 0x7fe810101a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54488/0 (socket says 172.21.15.27:54488) 2024-05-21T03:23:59.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.868+0000 7fe815c5a640 1 -- 172.21.15.27:0/727535088 learned_addr learned my addr 172.21.15.27:0/727535088 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:23:59.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.868+0000 7fe815459640 1 --2- 172.21.15.27:0/727535088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe810103590 0x7fe810102f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:59.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.868+0000 7fe814c58640 1 --2- 172.21.15.27:0/727535088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe810105ec0 0x7fe810101540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:23:59.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.868+0000 7fe815c5a640 1 -- 172.21.15.27:0/727535088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe810103590 msgr2=0x7fe810102f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:59.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.868+0000 7fe815c5a640 1 --2- 172.21.15.27:0/727535088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe810103590 0x7fe810102f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.868+0000 7fe815c5a640 1 -- 172.21.15.27:0/727535088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe810105ec0 msgr2=0x7fe810101540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:23:59.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.868+0000 7fe815c5a640 1 --2- 172.21.15.27:0/727535088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe810105ec0 0x7fe810101540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:23:59.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.868+0000 7fe815c5a640 1 -- 172.21.15.27:0/727535088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe810102310 con 0x7fe8101087f0 2024-05-21T03:23:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.868+0000 7fe815c5a640 1 --2- 172.21.15.27:0/727535088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8101087f0 0x7fe810101a80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe80800eb10 tx=0x7fe80800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:23:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.868+0000 7fe8067fc640 1 -- 172.21.15.27:0/727535088 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe808004b20 con 0x7fe8101087f0 2024-05-21T03:23:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.868+0000 7fe8176bd640 1 -- 172.21.15.27:0/727535088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8101a2d90 con 0x7fe8101087f0 2024-05-21T03:23:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.869+0000 7fe8067fc640 1 -- 172.21.15.27:0/727535088 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe808004cc0 con 0x7fe8101087f0 2024-05-21T03:23:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.869+0000 7fe8067fc640 1 -- 172.21.15.27:0/727535088 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe808010690 con 0x7fe8101087f0 2024-05-21T03:23:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.869+0000 7fe8176bd640 1 -- 172.21.15.27:0/727535088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8101a3250 con 0x7fe8101087f0 2024-05-21T03:23:59.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.870+0000 7fe8176bd640 1 -- 172.21.15.27:0/727535088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8100fea50 con 0x7fe8101087f0 2024-05-21T03:23:59.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.871+0000 7fe8067fc640 1 -- 172.21.15.27:0/727535088 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7fe808005000 con 0x7fe8101087f0 2024-05-21T03:23:59.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.872+0000 7fe8067fc640 1 --2- 172.21.15.27:0/727535088 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe7fc0b28f0 0x7fe7fc0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:23:59.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.872+0000 7fe815459640 1 -- 172.21.15.27:0/727535088 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe7fc0b28f0 msgr2=0x7fe7fc0b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3683408243 2024-05-21T03:23:59.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.873+0000 7fe8067fc640 1 -- 172.21.15.27:0/727535088 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fe8080e2a90 con 0x7fe8101087f0 2024-05-21T03:23:59.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.873+0000 7fe815459640 1 --2- 172.21.15.27:0/727535088 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe7fc0b28f0 0x7fe7fc0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T03:23:59.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:23:59.875+0000 7fe8067fc640 1 -- 172.21.15.27:0/727535088 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe808054df0 con 0x7fe8101087f0 2024-05-21T03:24:00.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:59 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/1016363937' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-21T03:24:00.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:59 smithi165 ceph-mon[36269]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:24:00.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:59 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/1933810213' entity='client.admin' 2024-05-21T03:24:00.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:23:59 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/733695237' entity='client.admin' 2024-05-21T03:24:00.022 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:23:59 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:23:59.810+0000 7fb35ad12200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T03:24:00.022 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:23:59 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:23:59.904+0000 7fb35ad12200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T03:24:00.047 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:59 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/1016363937' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-21T03:24:00.047 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:59 smithi027 ceph-mon[32179]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:24:00.047 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:59 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/1933810213' entity='client.admin' 2024-05-21T03:24:00.047 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:23:59 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/733695237' entity='client.admin' 2024-05-21T03:24:00.048 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:23:59 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:23:59.793+0000 7f04ba2c3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T03:24:00.048 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:23:59 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:23:59.941+0000 7f04ba2c3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T03:24:00.048 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:24:00 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:24:00.047+0000 7f04ba2c3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T03:24:00.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:00.074+0000 7fe815459640 1 -- 172.21.15.27:0/727535088 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe7fc0b28f0 msgr2=0x7fe7fc0b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3683408243 2024-05-21T03:24:00.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:00.074+0000 7fe815459640 1 --2- 172.21.15.27:0/727535088 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe7fc0b28f0 0x7fe7fc0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T03:24:00.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:00.094+0000 7fe8176bd640 1 -- 172.21.15.27:0/727535088 --> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fe810062e90 con 0x7fe7fc0b28f0 2024-05-21T03:24:00.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:00.475+0000 7fe815459640 1 -- 172.21.15.27:0/727535088 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe7fc0b28f0 msgr2=0x7fe7fc0b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3683408243 2024-05-21T03:24:00.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:00.475+0000 7fe815459640 1 --2- 172.21.15.27:0/727535088 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe7fc0b28f0 0x7fe7fc0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-21T03:24:00.519 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:24:00 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:24:00.080+0000 7ff69aaf4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T03:24:00.519 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:24:00 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:24:00.176+0000 7ff69aaf4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T03:24:00.676 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:24:00 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:24:00.391+0000 7f04ba2c3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T03:24:00.677 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:24:00 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:24:00.491+0000 7f04ba2c3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T03:24:00.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:00.760+0000 7fe8067fc640 1 -- 172.21.15.27:0/727535088 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7fe8080a62c0 con 0x7fe8101087f0 2024-05-21T03:24:00.771 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:24:00 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:24:00.510+0000 7fb35ad12200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T03:24:00.771 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:24:00 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z[38113]: 2024-05-21T03:24:00.729+0000 7fb35ad12200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T03:24:00.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:00 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/3072376918' entity='client.admin' 2024-05-21T03:24:00.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:00 smithi165 ceph-mon[36269]: Standby manager daemon z restarted 2024-05-21T03:24:00.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:00 smithi165 ceph-mon[36269]: Standby manager daemon z started 2024-05-21T03:24:00.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:00 smithi165 ceph-mon[36269]: from='mgr.? 172.21.15.165:0/1281143288' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-21T03:24:00.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:00 smithi165 ceph-mon[36269]: from='mgr.? 172.21.15.165:0/1281143288' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T03:24:00.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:00 smithi165 ceph-mon[36269]: from='mgr.? 172.21.15.165:0/1281143288' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-21T03:24:00.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:00 smithi165 ceph-mon[36269]: from='mgr.? 172.21.15.165:0/1281143288' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T03:24:01.019 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:24:00 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:24:00.812+0000 7ff69aaf4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T03:24:01.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:00 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/3072376918' entity='client.admin' 2024-05-21T03:24:01.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:00 smithi039 ceph-mon[36429]: Standby manager daemon z restarted 2024-05-21T03:24:01.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:00 smithi039 ceph-mon[36429]: Standby manager daemon z started 2024-05-21T03:24:01.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:00 smithi039 ceph-mon[36429]: from='mgr.? 172.21.15.165:0/1281143288' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-21T03:24:01.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:00 smithi039 ceph-mon[36429]: from='mgr.? 172.21.15.165:0/1281143288' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T03:24:01.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:00 smithi039 ceph-mon[36429]: from='mgr.? 172.21.15.165:0/1281143288' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-21T03:24:01.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:00 smithi039 ceph-mon[36429]: from='mgr.? 172.21.15.165:0/1281143288' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T03:24:01.142 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:00 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3072376918' entity='client.admin' 2024-05-21T03:24:01.143 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:00 smithi027 ceph-mon[32179]: Standby manager daemon z restarted 2024-05-21T03:24:01.143 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:00 smithi027 ceph-mon[32179]: Standby manager daemon z started 2024-05-21T03:24:01.143 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:00 smithi027 ceph-mon[32179]: from='mgr.? 172.21.15.165:0/1281143288' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-21T03:24:01.143 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:00 smithi027 ceph-mon[32179]: from='mgr.? 172.21.15.165:0/1281143288' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T03:24:01.143 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:00 smithi027 ceph-mon[32179]: from='mgr.? 172.21.15.165:0/1281143288' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-21T03:24:01.143 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:00 smithi027 ceph-mon[32179]: from='mgr.? 172.21.15.165:0/1281143288' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T03:24:01.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:01.276+0000 7fe815459640 1 -- 172.21.15.27:0/727535088 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe7fc0b28f0 msgr2=0x7fe7fc0b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3683408243 2024-05-21T03:24:01.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:01.276+0000 7fe815459640 1 --2- 172.21.15.27:0/727535088 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe7fc0b28f0 0x7fe7fc0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-21T03:24:01.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:01.395+0000 7fe8067fc640 1 -- 172.21.15.27:0/727535088 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mgrmap(e 21) ==== 149337+0+0 (secure 0 0 0) 0x7fe8080e0590 con 0x7fe8101087f0 2024-05-21T03:24:01.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:01.395+0000 7fe8067fc640 1 -- 172.21.15.27:0/727535088 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe7fc0b28f0 msgr2=0x7fe7fc0b4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T03:24:01.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:01.395+0000 7fe8067fc640 1 --2- 172.21.15.27:0/727535088 >> [v2:172.21.15.27:6800/3683408243,v1:172.21.15.27:6801/3683408243] conn(0x7fe7fc0b28f0 0x7fe7fc0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:01.427 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:24:01 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:24:01.143+0000 7f04ba2c3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T03:24:01.427 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:24:01 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:24:01.379+0000 7f04ba2c3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T03:24:01.519 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:24:01 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y[37800]: 2024-05-21T03:24:01.038+0000 7ff69aaf4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T03:24:01.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-05-21T03:24:01.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: Standby manager daemon y restarted 2024-05-21T03:24:01.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: Standby manager daemon y started 2024-05-21T03:24:01.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.? 172.21.15.39:0/2504586966' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-21T03:24:01.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.? 172.21.15.39:0/2504586966' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T03:24:01.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.? 172.21.15.39:0/2504586966' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-21T03:24:01.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.? 172.21.15.39:0/2504586966' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T03:24:01.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: Active manager daemon x restarted 2024-05-21T03:24:01.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: Activating manager daemon x 2024-05-21T03:24:01.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: osdmap e83: 12 total, 12 up, 12 in 2024-05-21T03:24:01.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: mgrmap e21: x(active, starting, since 0.0129888s), standbys: z, y 2024-05-21T03:24:01.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T03:24:01.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:24:01.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T03:24:01.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-21T03:24:01.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-21T03:24:01.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-21T03:24:01.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T03:24:01.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T03:24:01.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T03:24:01.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T03:24:01.929 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T03:24:01.929 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T03:24:01.929 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T03:24:01.929 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T03:24:01.929 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T03:24:01.929 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-21T03:24:01.929 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-21T03:24:01.929 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:24:01 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:24:01.651+0000 7f0418acf700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T03:24:01.929 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:24:01 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:24:01.651+0000 7f0418acf700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T03:24:01.929 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:24:01 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:24:01.651+0000 7f0418acf700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T03:24:01.929 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:24:01 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:24:01.651+0000 7f0418acf700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T03:24:01.930 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:24:01 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:24:01.651+0000 7f0418acf700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T03:24:01.930 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:24:01 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:24:01.652+0000 7f041bad5700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T03:24:01.930 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:24:01 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:24:01.652+0000 7f041bad5700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T03:24:01.930 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:24:01 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:24:01.652+0000 7f041bad5700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T03:24:01.930 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:24:01 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:24:01.652+0000 7f041bad5700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T03:24:01.930 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:24:01 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:24:01.652+0000 7f041bad5700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T03:24:01.930 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-21T03:24:01.930 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T03:24:01.930 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T03:24:01.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T03:24:01.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T03:24:01.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T03:24:01.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T03:24:01.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T03:24:01.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T03:24:01.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T03:24:01.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T03:24:01.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T03:24:01.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T03:24:01.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-21T03:24:01.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-21T03:24:01.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-21T03:24:01.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: Manager daemon x is now available 2024-05-21T03:24:01.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:24:01.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-21T03:24:01.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:01 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-21T03:24:02.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-05-21T03:24:02.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: Standby manager daemon y restarted 2024-05-21T03:24:02.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: Standby manager daemon y started 2024-05-21T03:24:02.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.? 172.21.15.39:0/2504586966' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-21T03:24:02.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.? 172.21.15.39:0/2504586966' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T03:24:02.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.? 172.21.15.39:0/2504586966' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-21T03:24:02.021 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.? 172.21.15.39:0/2504586966' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T03:24:02.021 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: Active manager daemon x restarted 2024-05-21T03:24:02.021 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: Activating manager daemon x 2024-05-21T03:24:02.021 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: osdmap e83: 12 total, 12 up, 12 in 2024-05-21T03:24:02.021 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: mgrmap e21: x(active, starting, since 0.0129888s), standbys: z, y 2024-05-21T03:24:02.021 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T03:24:02.021 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:24:02.021 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T03:24:02.021 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-21T03:24:02.021 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-21T03:24:02.021 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-21T03:24:02.021 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T03:24:02.021 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T03:24:02.021 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T03:24:02.022 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T03:24:02.022 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T03:24:02.022 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T03:24:02.022 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T03:24:02.022 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T03:24:02.022 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T03:24:02.022 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-21T03:24:02.022 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-21T03:24:02.022 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-21T03:24:02.022 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T03:24:02.022 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T03:24:02.022 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T03:24:02.022 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T03:24:02.023 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T03:24:02.023 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T03:24:02.023 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T03:24:02.023 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T03:24:02.023 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T03:24:02.023 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T03:24:02.023 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T03:24:02.023 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T03:24:02.023 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-21T03:24:02.023 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-21T03:24:02.023 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-21T03:24:02.023 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: Manager daemon x is now available 2024-05-21T03:24:02.023 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:24:02.024 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-21T03:24:02.024 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:01 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-21T03:24:02.025 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-05-21T03:24:02.025 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: Standby manager daemon y restarted 2024-05-21T03:24:02.025 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: Standby manager daemon y started 2024-05-21T03:24:02.025 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.? 172.21.15.39:0/2504586966' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-21T03:24:02.025 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.? 172.21.15.39:0/2504586966' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T03:24:02.025 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.? 172.21.15.39:0/2504586966' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-21T03:24:02.025 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.? 172.21.15.39:0/2504586966' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T03:24:02.025 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: Active manager daemon x restarted 2024-05-21T03:24:02.025 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: Activating manager daemon x 2024-05-21T03:24:02.025 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: osdmap e83: 12 total, 12 up, 12 in 2024-05-21T03:24:02.025 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: mgrmap e21: x(active, starting, since 0.0129888s), standbys: z, y 2024-05-21T03:24:02.026 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T03:24:02.026 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T03:24:02.026 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T03:24:02.026 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-21T03:24:02.026 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-21T03:24:02.026 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-21T03:24:02.026 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T03:24:02.026 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T03:24:02.026 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T03:24:02.026 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T03:24:02.026 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T03:24:02.026 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T03:24:02.026 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T03:24:02.027 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T03:24:02.027 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T03:24:02.027 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-21T03:24:02.027 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-21T03:24:02.027 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-21T03:24:02.027 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T03:24:02.027 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T03:24:02.027 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T03:24:02.027 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T03:24:02.027 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T03:24:02.027 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T03:24:02.027 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T03:24:02.027 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T03:24:02.028 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T03:24:02.028 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T03:24:02.028 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T03:24:02.028 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T03:24:02.028 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-21T03:24:02.028 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-21T03:24:02.028 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-21T03:24:02.029 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: Manager daemon x is now available 2024-05-21T03:24:02.029 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:24:02.029 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-21T03:24:02.029 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:01 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-21T03:24:02.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.402+0000 7fe8067fc640 1 -- 172.21.15.27:0/727535088 <== mon.0 v2:172.21.15.27:3300/0 9 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7fe808086c40 con 0x7fe8101087f0 2024-05-21T03:24:02.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.403+0000 7fe8067fc640 1 --2- 172.21.15.27:0/727535088 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fe7fc039d90 0x7fe7fc03a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:02.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.403+0000 7fe8067fc640 1 -- 172.21.15.27:0/727535088 --> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fe80800e1a0 con 0x7fe7fc039d90 2024-05-21T03:24:02.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.410+0000 7fe815459640 1 --2- 172.21.15.27:0/727535088 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fe7fc039d90 0x7fe7fc03a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:02.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.412+0000 7fe815459640 1 --2- 172.21.15.27:0/727535088 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fe7fc039d90 0x7fe7fc03a170 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe80c0097f0 tx=0x7fe80c005f90 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:02.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.485+0000 7fe8067fc640 1 -- 172.21.15.27:0/727535088 <== mgr.24913 v2:172.21.15.27:6800/1477999924 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe80800e1a0 con 0x7fe7fc039d90 2024-05-21T03:24:02.484 INFO:teuthology.orchestra.run.smithi027.stdout:Schedule set for path / 2024-05-21T03:24:02.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.487+0000 7fe8176bd640 1 -- 172.21.15.27:0/727535088 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fe7fc039d90 msgr2=0x7fe7fc03a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:02.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.487+0000 7fe8176bd640 1 --2- 172.21.15.27:0/727535088 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fe7fc039d90 0x7fe7fc03a170 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe80c0097f0 tx=0x7fe80c005f90 comp rx=0 tx=0).stop 2024-05-21T03:24:02.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.487+0000 7fe8176bd640 1 -- 172.21.15.27:0/727535088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8101087f0 msgr2=0x7fe810101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:02.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.487+0000 7fe8176bd640 1 --2- 172.21.15.27:0/727535088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8101087f0 0x7fe810101a80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe80800eb10 tx=0x7fe80800c2d0 comp rx=0 tx=0).stop 2024-05-21T03:24:02.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.487+0000 7fe815c5a640 1 -- 172.21.15.27:0/727535088 reap_dead start 2024-05-21T03:24:02.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.487+0000 7fe8176bd640 1 -- 172.21.15.27:0/727535088 shutdown_connections 2024-05-21T03:24:02.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.487+0000 7fe8176bd640 1 -- 172.21.15.27:0/727535088 >> 172.21.15.27:0/727535088 conn(0x7fe8100fcf60 msgr2=0x7fe810104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:02.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.487+0000 7fe8176bd640 1 -- 172.21.15.27:0/727535088 shutdown_connections 2024-05-21T03:24:02.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.487+0000 7fe8176bd640 1 -- 172.21.15.27:0/727535088 wait complete. 2024-05-21T03:24:02.498 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-05-21T03:24:02.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.609+0000 7fc6cb0d1640 1 -- 172.21.15.27:0/2802013091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6c41019b0 msgr2=0x7fc6c4101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:02.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.609+0000 7fc6cb0d1640 1 --2- 172.21.15.27:0/2802013091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6c41019b0 0x7fc6c4101dd0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009900 tx=0x7fc6bc02a410 comp rx=0 tx=0).stop 2024-05-21T03:24:02.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.609+0000 7fc6cb0d1640 1 -- 172.21.15.27:0/2802013091 shutdown_connections 2024-05-21T03:24:02.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.609+0000 7fc6cb0d1640 1 --2- 172.21.15.27:0/2802013091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6c4103560 0x7fc6c4109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:02.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.609+0000 7fc6cb0d1640 1 --2- 172.21.15.27:0/2802013091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6c4102ba0 0x7fc6c4103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:02.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.609+0000 7fc6cb0d1640 1 --2- 172.21.15.27:0/2802013091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6c41019b0 0x7fc6c4101dd0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:02.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.609+0000 7fc6cb0d1640 1 -- 172.21.15.27:0/2802013091 >> 172.21.15.27:0/2802013091 conn(0x7fc6c40fcf20 msgr2=0x7fc6c40ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:02.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.610+0000 7fc6cb0d1640 1 -- 172.21.15.27:0/2802013091 shutdown_connections 2024-05-21T03:24:02.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.610+0000 7fc6cb0d1640 1 -- 172.21.15.27:0/2802013091 wait complete. 2024-05-21T03:24:02.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.610+0000 7fc6cb0d1640 1 Processor -- start 2024-05-21T03:24:02.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.610+0000 7fc6cb0d1640 1 -- start start 2024-05-21T03:24:02.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.610+0000 7fc6cb0d1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6c41019b0 0x7fc6c4198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:02.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.610+0000 7fc6cb0d1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6c4102ba0 0x7fc6c41991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:02.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.610+0000 7fc6cb0d1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6c4103560 0x7fc6c41a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:02.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.610+0000 7fc6cb0d1640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c410d4b0 con 0x7fc6c4103560 2024-05-21T03:24:02.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.610+0000 7fc6cb0d1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c410d330 con 0x7fc6c41019b0 2024-05-21T03:24:02.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.610+0000 7fc6cb0d1640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c410d630 con 0x7fc6c4102ba0 2024-05-21T03:24:02.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.610+0000 7fc6c8e6d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6c41019b0 0x7fc6c4198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:02.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.610+0000 7fc6c966e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6c4103560 0x7fc6c41a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:02.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.610+0000 7fc6c8e6d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6c41019b0 0x7fc6c4198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.27:47276/0 (socket says 172.21.15.27:47276) 2024-05-21T03:24:02.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.610+0000 7fc6c8e6d640 1 -- 172.21.15.27:0/183988086 learned_addr learned my addr 172.21.15.27:0/183988086 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:02.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.611+0000 7fc6c3fff640 1 --2- 172.21.15.27:0/183988086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6c4102ba0 0x7fc6c41991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:02.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.611+0000 7fc6c8e6d640 1 -- 172.21.15.27:0/183988086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6c4102ba0 msgr2=0x7fc6c41991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:02.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.611+0000 7fc6c8e6d640 1 --2- 172.21.15.27:0/183988086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6c4102ba0 0x7fc6c41991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:02.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.611+0000 7fc6c8e6d640 1 -- 172.21.15.27:0/183988086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6c4103560 msgr2=0x7fc6c41a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:02.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.611+0000 7fc6c8e6d640 1 --2- 172.21.15.27:0/183988086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6c4103560 0x7fc6c41a0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:02.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.611+0000 7fc6c8e6d640 1 -- 172.21.15.27:0/183988086 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c41a08b0 con 0x7fc6c41019b0 2024-05-21T03:24:02.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.611+0000 7fc6c966e640 1 --2- 172.21.15.27:0/183988086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6c4103560 0x7fc6c41a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:24:02.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.611+0000 7fc6c8e6d640 1 --2- 172.21.15.27:0/183988086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6c41019b0 0x7fc6c4198c70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc006010 tx=0x7fc6bc0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:02.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.611+0000 7fc6c1ffb640 1 -- 172.21.15.27:0/183988086 <== mon.2 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc03f030 con 0x7fc6c41019b0 2024-05-21T03:24:02.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.611+0000 7fc6c1ffb640 1 -- 172.21.15.27:0/183988086 <== mon.2 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6bc0318f0 con 0x7fc6c41019b0 2024-05-21T03:24:02.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.611+0000 7fc6cb0d1640 1 -- 172.21.15.27:0/183988086 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c41a0b40 con 0x7fc6c41019b0 2024-05-21T03:24:02.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.611+0000 7fc6c1ffb640 1 -- 172.21.15.27:0/183988086 <== mon.2 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc035070 con 0x7fc6c41019b0 2024-05-21T03:24:02.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.612+0000 7fc6cb0d1640 1 -- 172.21.15.27:0/183988086 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c41a1030 con 0x7fc6c41019b0 2024-05-21T03:24:02.617 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.612+0000 7fc6cb0d1640 1 -- 172.21.15.27:0/183988086 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6c41a1200 con 0x7fc6c41019b0 2024-05-21T03:24:02.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.614+0000 7fc6c1ffb640 1 -- 172.21.15.27:0/183988086 <== mon.2 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7fc6bc042030 con 0x7fc6c41019b0 2024-05-21T03:24:02.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.615+0000 7fc6c1ffb640 1 --2- 172.21.15.27:0/183988086 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fc6b00b2690 0x7fc6b00b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:02.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.615+0000 7fc6c3fff640 1 --2- 172.21.15.27:0/183988086 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fc6b00b2690 0x7fc6b00b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:02.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.615+0000 7fc6c1ffb640 1 -- 172.21.15.27:0/183988086 <== mon.2 v2:172.21.15.39:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fc6bc102860 con 0x7fc6c41019b0 2024-05-21T03:24:02.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.615+0000 7fc6c3fff640 1 --2- 172.21.15.27:0/183988086 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fc6b00b2690 0x7fc6b00b4b50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc6c419a170 tx=0x7fc6b4009680 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:02.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.618+0000 7fc6c1ffb640 1 -- 172.21.15.27:0/183988086 <== mon.2 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc6bc0ca0c0 con 0x7fc6c41019b0 2024-05-21T03:24:02.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.839+0000 7fc6cb0d1640 1 -- 172.21.15.27:0/183988086 --> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fc6c40ff150 con 0x7fc6b00b2690 2024-05-21T03:24:02.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.897+0000 7fc6c1ffb640 1 -- 172.21.15.27:0/183988086 <== mgr.24913 v2:172.21.15.27:6800/1477999924 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fc6c40ff150 con 0x7fc6b00b2690 2024-05-21T03:24:02.896 INFO:teuthology.orchestra.run.smithi027.stdout:Retention added to path / 2024-05-21T03:24:02.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.899+0000 7fc6cb0d1640 1 -- 172.21.15.27:0/183988086 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fc6b00b2690 msgr2=0x7fc6b00b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:02.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.899+0000 7fc6cb0d1640 1 --2- 172.21.15.27:0/183988086 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fc6b00b2690 0x7fc6b00b4b50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc6c419a170 tx=0x7fc6b4009680 comp rx=0 tx=0).stop 2024-05-21T03:24:02.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.899+0000 7fc6cb0d1640 1 -- 172.21.15.27:0/183988086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6c41019b0 msgr2=0x7fc6c4198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:02.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.899+0000 7fc6cb0d1640 1 --2- 172.21.15.27:0/183988086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6c41019b0 0x7fc6c4198c70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc006010 tx=0x7fc6bc0042f0 comp rx=0 tx=0).stop 2024-05-21T03:24:02.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.899+0000 7fc6cb0d1640 1 -- 172.21.15.27:0/183988086 shutdown_connections 2024-05-21T03:24:02.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.899+0000 7fc6cb0d1640 1 --2- 172.21.15.27:0/183988086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6c4103560 0x7fc6c41a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:02.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.899+0000 7fc6cb0d1640 1 --2- 172.21.15.27:0/183988086 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fc6b00b2690 0x7fc6b00b4b50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:02.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.899+0000 7fc6cb0d1640 1 --2- 172.21.15.27:0/183988086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6c4102ba0 0x7fc6c41991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:02.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.899+0000 7fc6cb0d1640 1 --2- 172.21.15.27:0/183988086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6c41019b0 0x7fc6c4198c70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:02.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.899+0000 7fc6cb0d1640 1 -- 172.21.15.27:0/183988086 >> 172.21.15.27:0/183988086 conn(0x7fc6c40fcf20 msgr2=0x7fc6c40fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:02.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.899+0000 7fc6cb0d1640 1 -- 172.21.15.27:0/183988086 shutdown_connections 2024-05-21T03:24:02.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:02.899+0000 7fc6cb0d1640 1 -- 172.21.15.27:0/183988086 wait complete. 2024-05-21T03:24:02.909 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-21T03:24:03.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.017+0000 7f1cc5f7f640 1 -- 172.21.15.27:0/3380482323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cc0103560 msgr2=0x7f1cc0109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:03.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.017+0000 7f1cc5f7f640 1 --2- 172.21.15.27:0/3380482323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cc0103560 0x7f1cc0109df0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8009ca0 tx=0x7f1cb802a8c0 comp rx=0 tx=0).stop 2024-05-21T03:24:03.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.017+0000 7f1cc5f7f640 1 -- 172.21.15.27:0/3380482323 shutdown_connections 2024-05-21T03:24:03.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.017+0000 7f1cc5f7f640 1 --2- 172.21.15.27:0/3380482323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cc0103560 0x7f1cc0109df0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:03.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.017+0000 7f1cc5f7f640 1 --2- 172.21.15.27:0/3380482323 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cc0102ba0 0x7f1cc0103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:03.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.017+0000 7f1cc5f7f640 1 --2- 172.21.15.27:0/3380482323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cc01019b0 0x7f1cc0101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:03.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.017+0000 7f1cc5f7f640 1 -- 172.21.15.27:0/3380482323 >> 172.21.15.27:0/3380482323 conn(0x7f1cc00fcf20 msgr2=0x7f1cc00ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:03.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.017+0000 7f1cc5f7f640 1 -- 172.21.15.27:0/3380482323 shutdown_connections 2024-05-21T03:24:03.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.017+0000 7f1cc5f7f640 1 -- 172.21.15.27:0/3380482323 wait complete. 2024-05-21T03:24:03.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.018+0000 7f1cc5f7f640 1 Processor -- start 2024-05-21T03:24:03.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.018+0000 7f1cc5f7f640 1 -- start start 2024-05-21T03:24:03.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.018+0000 7f1cc5f7f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cc01019b0 0x7f1cc0198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:03.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.018+0000 7f1cc5f7f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cc0102ba0 0x7f1cc01990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:03.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.018+0000 7f1cc5f7f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cc0103560 0x7f1cc01a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:03.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.018+0000 7f1cc5f7f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc010d4c0 con 0x7f1cc01019b0 2024-05-21T03:24:03.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.018+0000 7f1cc5f7f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc010d340 con 0x7f1cc0102ba0 2024-05-21T03:24:03.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.018+0000 7f1cc5f7f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc010d640 con 0x7f1cc0103560 2024-05-21T03:24:03.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.018+0000 7f1cbf7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cc01019b0 0x7f1cc0198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:03.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.018+0000 7f1cbf7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cc01019b0 0x7f1cc0198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55798/0 (socket says 172.21.15.27:55798) 2024-05-21T03:24:03.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.018+0000 7f1cbf7fe640 1 -- 172.21.15.27:0/3244739466 learned_addr learned my addr 172.21.15.27:0/3244739466 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:03.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.018+0000 7f1cbeffd640 1 --2- 172.21.15.27:0/3244739466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cc0102ba0 0x7f1cc01990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:03.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.018+0000 7f1cbffff640 1 --2- 172.21.15.27:0/3244739466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cc0103560 0x7f1cc01a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:03.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.019+0000 7f1cbf7fe640 1 -- 172.21.15.27:0/3244739466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cc0103560 msgr2=0x7f1cc01a0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:03.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.019+0000 7f1cbf7fe640 1 --2- 172.21.15.27:0/3244739466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cc0103560 0x7f1cc01a0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:03.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.019+0000 7f1cbf7fe640 1 -- 172.21.15.27:0/3244739466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cc0102ba0 msgr2=0x7f1cc01990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:03.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.019+0000 7f1cbf7fe640 1 --2- 172.21.15.27:0/3244739466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cc0102ba0 0x7f1cc01990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:03.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.019+0000 7f1cbf7fe640 1 -- 172.21.15.27:0/3244739466 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc01a0780 con 0x7f1cc01019b0 2024-05-21T03:24:03.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.019+0000 7f1cbeffd640 1 --2- 172.21.15.27:0/3244739466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cc0102ba0 0x7f1cc01990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:24:03.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.019+0000 7f1cbf7fe640 1 --2- 172.21.15.27:0/3244739466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cc01019b0 0x7f1cc0198b60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1cb000d770 tx=0x7f1cb000db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:03.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.019+0000 7f1cbcff9640 1 -- 172.21.15.27:0/3244739466 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0017070 con 0x7f1cc01019b0 2024-05-21T03:24:03.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.019+0000 7f1cc5f7f640 1 -- 172.21.15.27:0/3244739466 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc01a0a70 con 0x7f1cc01019b0 2024-05-21T03:24:03.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.019+0000 7f1cbcff9640 1 -- 172.21.15.27:0/3244739466 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cb0009cd0 con 0x7f1cc01019b0 2024-05-21T03:24:03.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.019+0000 7f1cbcff9640 1 -- 172.21.15.27:0/3244739466 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0003c60 con 0x7f1cc01019b0 2024-05-21T03:24:03.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.019+0000 7f1cc5f7f640 1 -- 172.21.15.27:0/3244739466 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc01a0fc0 con 0x7f1cc01019b0 2024-05-21T03:24:03.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.020+0000 7f1cc5f7f640 1 -- 172.21.15.27:0/3244739466 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cc00507e0 con 0x7f1cc01019b0 2024-05-21T03:24:03.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.021+0000 7f1cbcff9640 1 -- 172.21.15.27:0/3244739466 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7f1cb0004340 con 0x7f1cc01019b0 2024-05-21T03:24:03.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.022+0000 7f1cbcff9640 1 --2- 172.21.15.27:0/3244739466 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f1cac0b2690 0x7f1cac0b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:03.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.022+0000 7f1cbcff9640 1 -- 172.21.15.27:0/3244739466 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f1cb00e1c10 con 0x7f1cc01019b0 2024-05-21T03:24:03.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.022+0000 7f1cbeffd640 1 --2- 172.21.15.27:0/3244739466 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f1cac0b2690 0x7f1cac0b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:03.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.023+0000 7f1cbeffd640 1 --2- 172.21.15.27:0/3244739466 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f1cac0b2690 0x7f1cac0b4b50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1cc019a070 tx=0x7f1ca800a2d0 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:03.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.026+0000 7f1cbcff9640 1 -- 172.21.15.27:0/3244739466 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1cb00a9b00 con 0x7f1cc01019b0 2024-05-21T03:24:03.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.252+0000 7f1cc5f7f640 1 -- 172.21.15.27:0/3244739466 --> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f1cc00feff0 con 0x7f1cac0b2690 2024-05-21T03:24:03.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.260+0000 7f1cbcff9640 1 -- 172.21.15.27:0/3244739466 <== mgr.24913 v2:172.21.15.27:6800/1477999924 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f1cc00feff0 con 0x7f1cac0b2690 2024-05-21T03:24:03.260 INFO:teuthology.orchestra.run.smithi027.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-21T00:00:00", "created": "2024-05-21T03:24:02", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-05-21T03:24:03.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.263+0000 7f1cc5f7f640 1 -- 172.21.15.27:0/3244739466 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f1cac0b2690 msgr2=0x7f1cac0b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:03.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.263+0000 7f1cc5f7f640 1 --2- 172.21.15.27:0/3244739466 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f1cac0b2690 0x7f1cac0b4b50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1cc019a070 tx=0x7f1ca800a2d0 comp rx=0 tx=0).stop 2024-05-21T03:24:03.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.263+0000 7f1cc5f7f640 1 -- 172.21.15.27:0/3244739466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cc01019b0 msgr2=0x7f1cc0198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:03.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.263+0000 7f1cc5f7f640 1 --2- 172.21.15.27:0/3244739466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cc01019b0 0x7f1cc0198b60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1cb000d770 tx=0x7f1cb000db70 comp rx=0 tx=0).stop 2024-05-21T03:24:03.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.263+0000 7f1cc5f7f640 1 -- 172.21.15.27:0/3244739466 shutdown_connections 2024-05-21T03:24:03.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.263+0000 7f1cc5f7f640 1 --2- 172.21.15.27:0/3244739466 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f1cac0b2690 0x7f1cac0b4b50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:03.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.263+0000 7f1cc5f7f640 1 --2- 172.21.15.27:0/3244739466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cc0103560 0x7f1cc01a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:03.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.263+0000 7f1cc5f7f640 1 --2- 172.21.15.27:0/3244739466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cc0102ba0 0x7f1cc01990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:03.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.263+0000 7f1cc5f7f640 1 --2- 172.21.15.27:0/3244739466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cc01019b0 0x7f1cc0198b60 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:03.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.263+0000 7f1cc5f7f640 1 -- 172.21.15.27:0/3244739466 >> 172.21.15.27:0/3244739466 conn(0x7f1cc00fcf20 msgr2=0x7f1cc00fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:03.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.263+0000 7f1cc5f7f640 1 -- 172.21.15.27:0/3244739466 shutdown_connections 2024-05-21T03:24:03.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.263+0000 7f1cc5f7f640 1 -- 172.21.15.27:0/3244739466 wait complete. 2024-05-21T03:24:03.277 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-21T03:24:03.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.399+0000 7f82cbfaf640 1 -- 172.21.15.27:0/1448367724 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82c4102ad0 msgr2=0x7f82c4102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:03.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.399+0000 7f82cbfaf640 1 --2- 172.21.15.27:0/1448367724 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82c4102ad0 0x7f82c4102ef0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009900 tx=0x7f82c002a410 comp rx=0 tx=0).stop 2024-05-21T03:24:03.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.399+0000 7f82cbfaf640 1 -- 172.21.15.27:0/1448367724 shutdown_connections 2024-05-21T03:24:03.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.399+0000 7f82cbfaf640 1 --2- 172.21.15.27:0/1448367724 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82c4104680 0x7f82c410af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:03.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.399+0000 7f82cbfaf640 1 --2- 172.21.15.27:0/1448367724 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82c4103cc0 0x7f82c4104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:03.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.399+0000 7f82cbfaf640 1 --2- 172.21.15.27:0/1448367724 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82c4102ad0 0x7f82c4102ef0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:03.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.399+0000 7f82cbfaf640 1 -- 172.21.15.27:0/1448367724 >> 172.21.15.27:0/1448367724 conn(0x7f82c40fe040 msgr2=0x7f82c4100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:03.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.399+0000 7f82cbfaf640 1 -- 172.21.15.27:0/1448367724 shutdown_connections 2024-05-21T03:24:03.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.399+0000 7f82cbfaf640 1 -- 172.21.15.27:0/1448367724 wait complete. 2024-05-21T03:24:03.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.399+0000 7f82cbfaf640 1 Processor -- start 2024-05-21T03:24:03.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.400+0000 7f82cbfaf640 1 -- start start 2024-05-21T03:24:03.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.400+0000 7f82cbfaf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82c4102ad0 0x7f82c4198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:03.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.400+0000 7f82cbfaf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82c4103cc0 0x7f82c4199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:03.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.400+0000 7f82cbfaf640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82c4104680 0x7f82c41a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:03.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.400+0000 7f82cbfaf640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f82c410e5d0 con 0x7f82c4102ad0 2024-05-21T03:24:03.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.400+0000 7f82cbfaf640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f82c410e450 con 0x7f82c4103cc0 2024-05-21T03:24:03.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.400+0000 7f82cbfaf640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f82c410e750 con 0x7f82c4104680 2024-05-21T03:24:03.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.400+0000 7f82c9d4b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82c4102ad0 0x7f82c4198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:03.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.400+0000 7f82c9d4b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82c4102ad0 0x7f82c4198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55818/0 (socket says 172.21.15.27:55818) 2024-05-21T03:24:03.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.400+0000 7f82c9d4b640 1 -- 172.21.15.27:0/2563781381 learned_addr learned my addr 172.21.15.27:0/2563781381 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:03.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.400+0000 7f82c954a640 1 --2- 172.21.15.27:0/2563781381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82c4103cc0 0x7f82c4199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:03.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.400+0000 7f82ca54c640 1 --2- 172.21.15.27:0/2563781381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82c4104680 0x7f82c41a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:03.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.400+0000 7f82c9d4b640 1 -- 172.21.15.27:0/2563781381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82c4104680 msgr2=0x7f82c41a0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:03.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.400+0000 7f82c9d4b640 1 --2- 172.21.15.27:0/2563781381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82c4104680 0x7f82c41a0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:03.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.400+0000 7f82c9d4b640 1 -- 172.21.15.27:0/2563781381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82c4103cc0 msgr2=0x7f82c4199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:03.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.400+0000 7f82c9d4b640 1 --2- 172.21.15.27:0/2563781381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82c4103cc0 0x7f82c4199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:03.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.400+0000 7f82c9d4b640 1 -- 172.21.15.27:0/2563781381 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c41a08f0 con 0x7f82c4102ad0 2024-05-21T03:24:03.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.400+0000 7f82c954a640 1 --2- 172.21.15.27:0/2563781381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82c4103cc0 0x7f82c4199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:24:03.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.400+0000 7f82c9d4b640 1 --2- 172.21.15.27:0/2563781381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82c4102ad0 0x7f82c4198d00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f82c0006010 tx=0x7f82c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:03.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.401+0000 7f82baffd640 1 -- 172.21.15.27:0/2563781381 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c003f030 con 0x7f82c4102ad0 2024-05-21T03:24:03.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.401+0000 7f82baffd640 1 -- 172.21.15.27:0/2563781381 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82c00318f0 con 0x7f82c4102ad0 2024-05-21T03:24:03.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.401+0000 7f82cbfaf640 1 -- 172.21.15.27:0/2563781381 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c41a0b80 con 0x7f82c4102ad0 2024-05-21T03:24:03.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.401+0000 7f82baffd640 1 -- 172.21.15.27:0/2563781381 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c0035070 con 0x7f82c4102ad0 2024-05-21T03:24:03.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.401+0000 7f82cbfaf640 1 -- 172.21.15.27:0/2563781381 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c41a1070 con 0x7f82c4102ad0 2024-05-21T03:24:03.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.402+0000 7f82cbfaf640 1 -- 172.21.15.27:0/2563781381 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82c4109ec0 con 0x7f82c4102ad0 2024-05-21T03:24:03.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.403+0000 7f82baffd640 1 -- 172.21.15.27:0/2563781381 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7f82c0042030 con 0x7f82c4102ad0 2024-05-21T03:24:03.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.404+0000 7f82baffd640 1 --2- 172.21.15.27:0/2563781381 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f82b00b2690 0x7f82b00b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:03.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.404+0000 7f82c954a640 1 --2- 172.21.15.27:0/2563781381 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f82b00b2690 0x7f82b00b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:03.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.404+0000 7f82baffd640 1 -- 172.21.15.27:0/2563781381 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f82c01028b0 con 0x7f82c4102ad0 2024-05-21T03:24:03.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.405+0000 7f82c954a640 1 --2- 172.21.15.27:0/2563781381 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f82b00b2690 0x7f82b00b4b50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f82c419a1b0 tx=0x7f82b400b390 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:03.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.408+0000 7f82baffd640 1 -- 172.21.15.27:0/2563781381 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f82c00ca7a0 con 0x7f82c4102ad0 2024-05-21T03:24:03.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:03 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:03.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:03 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:03.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:03 smithi165 ceph-mon[36269]: mgrmap e22: x(active, since 1.02188s), standbys: z, y 2024-05-21T03:24:03.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:03 smithi165 ceph-mon[36269]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:24:03.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:03 smithi165 ceph-mon[36269]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:24:03.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.672+0000 7f82cbfaf640 1 -- 172.21.15.27:0/2563781381 --> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f82c4062e90 con 0x7f82b00b2690 2024-05-21T03:24:03.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:03 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:03.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:03 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:03.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:03 smithi027 ceph-mon[32179]: mgrmap e22: x(active, since 1.02188s), standbys: z, y 2024-05-21T03:24:03.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:03 smithi027 ceph-mon[32179]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:24:03.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:03 smithi027 ceph-mon[32179]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:24:03.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.680+0000 7f82baffd640 1 -- 172.21.15.27:0/2563781381 <== mgr.24913 v2:172.21.15.27:6800/1477999924 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f82c4062e90 con 0x7f82b00b2690 2024-05-21T03:24:03.679 INFO:teuthology.orchestra.run.smithi027.stdout:/ 1m 6m3h 2024-05-21T03:24:03.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.682+0000 7f82cbfaf640 1 -- 172.21.15.27:0/2563781381 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f82b00b2690 msgr2=0x7f82b00b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.682+0000 7f82cbfaf640 1 --2- 172.21.15.27:0/2563781381 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f82b00b2690 0x7f82b00b4b50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f82c419a1b0 tx=0x7f82b400b390 comp rx=0 tx=0).stop 2024-05-21T03:24:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.682+0000 7f82cbfaf640 1 -- 172.21.15.27:0/2563781381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82c4102ad0 msgr2=0x7f82c4198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.682+0000 7f82cbfaf640 1 --2- 172.21.15.27:0/2563781381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82c4102ad0 0x7f82c4198d00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f82c0006010 tx=0x7f82c00042f0 comp rx=0 tx=0).stop 2024-05-21T03:24:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.682+0000 7f82cbfaf640 1 -- 172.21.15.27:0/2563781381 shutdown_connections 2024-05-21T03:24:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.682+0000 7f82cbfaf640 1 --2- 172.21.15.27:0/2563781381 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f82b00b2690 0x7f82b00b4b50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.682+0000 7f82cbfaf640 1 --2- 172.21.15.27:0/2563781381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82c4104680 0x7f82c41a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.682+0000 7f82cbfaf640 1 --2- 172.21.15.27:0/2563781381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82c4103cc0 0x7f82c4199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.682+0000 7f82cbfaf640 1 --2- 172.21.15.27:0/2563781381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82c4102ad0 0x7f82c4198d00 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.682+0000 7f82cbfaf640 1 -- 172.21.15.27:0/2563781381 >> 172.21.15.27:0/2563781381 conn(0x7f82c40fe040 msgr2=0x7f82c4107090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.682+0000 7f82cbfaf640 1 -- 172.21.15.27:0/2563781381 shutdown_connections 2024-05-21T03:24:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:03.682+0000 7f82cbfaf640 1 -- 172.21.15.27:0/2563781381 wait complete. 2024-05-21T03:24:03.693 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-05-21T03:24:03.722 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-05-21T03:24:03.733 INFO:teuthology.run_tasks:Running task quiescer... 2024-05-21T03:24:03.746 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-21T03:24:03.746 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-21T03:24:03.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:03 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:03.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:03 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:03.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:03 smithi039 ceph-mon[36429]: mgrmap e22: x(active, since 1.02188s), standbys: z, y 2024-05-21T03:24:03.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:03 smithi039 ceph-mon[36429]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:24:03.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:03 smithi039 ceph-mon[36429]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:24:04.026 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:24:04.900 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:04 smithi027 ceph-mon[32179]: [21/May/2024:03:24:03] ENGINE Bus STARTING 2024-05-21T03:24:04.900 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:04 smithi027 ceph-mon[32179]: [21/May/2024:03:24:03] ENGINE Serving on https://172.21.15.27:7150 2024-05-21T03:24:04.900 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:04 smithi027 ceph-mon[32179]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:24:04.901 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:04 smithi027 ceph-mon[32179]: [21/May/2024:03:24:03] ENGINE Serving on http://172.21.15.27:8765 2024-05-21T03:24:04.901 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:04 smithi027 ceph-mon[32179]: [21/May/2024:03:24:03] ENGINE Bus STARTED 2024-05-21T03:24:04.901 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:04 smithi027 ceph-mon[32179]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:24:04.901 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:04 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:04.901 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:04 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:04.901 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:04 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:04.901 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:04 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:04.901 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:04 smithi027 ceph-mon[32179]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:24:04.901 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:04 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:04.901 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:04 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:04.901 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:04 smithi027 ceph-mon[32179]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-05-21T03:24:05.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:04 smithi039 ceph-mon[36429]: [21/May/2024:03:24:03] ENGINE Bus STARTING 2024-05-21T03:24:05.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:04 smithi039 ceph-mon[36429]: [21/May/2024:03:24:03] ENGINE Serving on https://172.21.15.27:7150 2024-05-21T03:24:05.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:04 smithi039 ceph-mon[36429]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:24:05.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:04 smithi039 ceph-mon[36429]: [21/May/2024:03:24:03] ENGINE Serving on http://172.21.15.27:8765 2024-05-21T03:24:05.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:04 smithi039 ceph-mon[36429]: [21/May/2024:03:24:03] ENGINE Bus STARTED 2024-05-21T03:24:05.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:04 smithi039 ceph-mon[36429]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:24:05.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:04 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:05.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:04 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:05.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:04 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:05.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:04 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:05.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:04 smithi039 ceph-mon[36429]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:24:05.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:04 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:05.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:04 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:05.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:04 smithi039 ceph-mon[36429]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-05-21T03:24:05.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:04 smithi165 ceph-mon[36269]: [21/May/2024:03:24:03] ENGINE Bus STARTING 2024-05-21T03:24:05.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:04 smithi165 ceph-mon[36269]: [21/May/2024:03:24:03] ENGINE Serving on https://172.21.15.27:7150 2024-05-21T03:24:05.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:04 smithi165 ceph-mon[36269]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:24:05.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:04 smithi165 ceph-mon[36269]: [21/May/2024:03:24:03] ENGINE Serving on http://172.21.15.27:8765 2024-05-21T03:24:05.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:04 smithi165 ceph-mon[36269]: [21/May/2024:03:24:03] ENGINE Bus STARTED 2024-05-21T03:24:05.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:04 smithi165 ceph-mon[36269]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:24:05.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:04 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:05.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:04 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:05.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:04 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:05.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:04 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:05.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:04 smithi165 ceph-mon[36269]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:24:05.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:04 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:05.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:04 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:05.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:04 smithi165 ceph-mon[36269]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-05-21T03:24:05.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.726+0000 7ff9976f6700 1 -- 172.21.15.27:0/1675394236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff990108920 msgr2=0x7ff9901104a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:05.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.726+0000 7ff9976f6700 1 --2- 172.21.15.27:0/1675394236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff990108920 0x7ff9901104a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff98800b0d0 tx=0x7ff98800b3e0 comp rx=0 tx=0).stop 2024-05-21T03:24:05.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.726+0000 7ff9976f6700 1 -- 172.21.15.27:0/1675394236 shutdown_connections 2024-05-21T03:24:05.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.726+0000 7ff9976f6700 1 --2- 172.21.15.27:0/1675394236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff990108920 0x7ff9901104a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:05.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.726+0000 7ff9976f6700 1 --2- 172.21.15.27:0/1675394236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff990103850 0x7ff990103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:05.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.726+0000 7ff9976f6700 1 --2- 172.21.15.27:0/1675394236 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff990104200 0x7ff9901081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:05.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.726+0000 7ff9976f6700 1 -- 172.21.15.27:0/1675394236 >> 172.21.15.27:0/1675394236 conn(0x7ff99007a620 msgr2=0x7ff99007aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:05.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.726+0000 7ff9976f6700 1 -- 172.21.15.27:0/1675394236 shutdown_connections 2024-05-21T03:24:05.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.726+0000 7ff9976f6700 1 -- 172.21.15.27:0/1675394236 wait complete. 2024-05-21T03:24:05.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.727+0000 7ff9976f6700 1 Processor -- start 2024-05-21T03:24:05.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.727+0000 7ff9976f6700 1 -- start start 2024-05-21T03:24:05.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.727+0000 7ff9976f6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff990103850 0x7ff990075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:05.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.727+0000 7ff9976f6700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff990104200 0x7ff990075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:05.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.727+0000 7ff9976f6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff990108920 0x7ff9900762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:05.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.727+0000 7ff9976f6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff990113750 con 0x7ff990108920 2024-05-21T03:24:05.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.727+0000 7ff9976f6700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff9901135d0 con 0x7ff990103850 2024-05-21T03:24:05.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.727+0000 7ff9976f6700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff9901138d0 con 0x7ff990104200 2024-05-21T03:24:05.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.728+0000 7ff995492700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff990103850 0x7ff990075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.728+0000 7ff995c93700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff990108920 0x7ff9900762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.728+0000 7ff995492700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff990103850 0x7ff990075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.27:47340/0 (socket says 172.21.15.27:47340) 2024-05-21T03:24:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.728+0000 7ff995492700 1 -- 172.21.15.27:0/4167038081 learned_addr learned my addr 172.21.15.27:0/4167038081 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.728+0000 7ff994c91700 1 --2- 172.21.15.27:0/4167038081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff990104200 0x7ff990075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.728+0000 7ff995492700 1 -- 172.21.15.27:0/4167038081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff990104200 msgr2=0x7ff990075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.728+0000 7ff995492700 1 --2- 172.21.15.27:0/4167038081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff990104200 0x7ff990075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.728+0000 7ff995492700 1 -- 172.21.15.27:0/4167038081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff990108920 msgr2=0x7ff9900762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.728+0000 7ff995492700 1 --2- 172.21.15.27:0/4167038081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff990108920 0x7ff9900762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.728+0000 7ff995492700 1 -- 172.21.15.27:0/4167038081 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9901a80c0 con 0x7ff990103850 2024-05-21T03:24:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.728+0000 7ff995c93700 1 --2- 172.21.15.27:0/4167038081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff990108920 0x7ff9900762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:24:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.728+0000 7ff995492700 1 --2- 172.21.15.27:0/4167038081 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff990103850 0x7ff990075840 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00d980 tx=0x7ff98c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.728+0000 7ff9867fc700 1 -- 172.21.15.27:0/4167038081 <== mon.2 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c016070 con 0x7ff990103850 2024-05-21T03:24:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.728+0000 7ff9976f6700 1 -- 172.21.15.27:0/4167038081 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9901a83b0 con 0x7ff990103850 2024-05-21T03:24:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.728+0000 7ff9867fc700 1 -- 172.21.15.27:0/4167038081 <== mon.2 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff98c004830 con 0x7ff990103850 2024-05-21T03:24:05.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.728+0000 7ff9867fc700 1 -- 172.21.15.27:0/4167038081 <== mon.2 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c005540 con 0x7ff990103850 2024-05-21T03:24:05.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.729+0000 7ff9976f6700 1 -- 172.21.15.27:0/4167038081 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9901a8940 con 0x7ff990103850 2024-05-21T03:24:05.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.730+0000 7ff9976f6700 1 -- 172.21.15.27:0/4167038081 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff99006a230 con 0x7ff990103850 2024-05-21T03:24:05.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.732+0000 7ff9867fc700 1 -- 172.21.15.27:0/4167038081 <== mon.2 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) ==== 149772+0+0 (secure 0 0 0) 0x7ff98c004220 con 0x7ff990103850 2024-05-21T03:24:05.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.733+0000 7ff9867fc700 1 --2- 172.21.15.27:0/4167038081 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7ff97c0b2a70 0x7ff97c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:05.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.733+0000 7ff994c91700 1 --2- 172.21.15.27:0/4167038081 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7ff97c0b2a70 0x7ff97c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:05.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.733+0000 7ff9867fc700 1 -- 172.21.15.27:0/4167038081 <== mon.2 v2:172.21.15.39:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7ff98c0e28c0 con 0x7ff990103850 2024-05-21T03:24:05.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.733+0000 7ff994c91700 1 --2- 172.21.15.27:0/4167038081 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7ff97c0b2a70 0x7ff97c0b4f30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff980009680 tx=0x7ff980009260 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:05.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.735+0000 7ff9867fc700 1 -- 172.21.15.27:0/4167038081 <== mon.2 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff98c01f020 con 0x7ff990103850 2024-05-21T03:24:05.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:05 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:05.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:05 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:05.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:05 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:05.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:05 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:05.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:05 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:05.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:05 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:05.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:05 smithi027 ceph-mon[32179]: Adjusting osd_memory_target on smithi039 to 176.2M 2024-05-21T03:24:05.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:05 smithi027 ceph-mon[32179]: Unable to set osd_memory_target on smithi039 to 184789606: error parsing value: Value '184789606' is below minimum 939524096 2024-05-21T03:24:05.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:05 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:05.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:05 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:05.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:05 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:05.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:05 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:05.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:05 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:05.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:05 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:05.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:05 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:05.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:05 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:05.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:05 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:05.929 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:05 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:05.929 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:05 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:05.929 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:05 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:05.929 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:05 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:24:05.929 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:05 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:24:05.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.958+0000 7ff9976f6700 1 -- 172.21.15.27:0/4167038081 --> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff990063980 con 0x7ff97c0b2a70 2024-05-21T03:24:05.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.966+0000 7ff9867fc700 1 -- 172.21.15.27:0/4167038081 <== mgr.24913 v2:172.21.15.27:6800/1477999924 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257795 (secure 0 0 0) 0x7ff990063980 con 0x7ff97c0b2a70 2024-05-21T03:24:05.970 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:24:05.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.970+0000 7ff9976f6700 1 -- 172.21.15.27:0/4167038081 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7ff97c0b2a70 msgr2=0x7ff97c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:05.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.970+0000 7ff9976f6700 1 --2- 172.21.15.27:0/4167038081 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7ff97c0b2a70 0x7ff97c0b4f30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff980009680 tx=0x7ff980009260 comp rx=0 tx=0).stop 2024-05-21T03:24:05.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.970+0000 7ff9976f6700 1 -- 172.21.15.27:0/4167038081 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff990103850 msgr2=0x7ff990075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:05.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.970+0000 7ff9976f6700 1 --2- 172.21.15.27:0/4167038081 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff990103850 0x7ff990075840 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00d980 tx=0x7ff98c00dc90 comp rx=0 tx=0).stop 2024-05-21T03:24:05.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.970+0000 7ff9976f6700 1 -- 172.21.15.27:0/4167038081 shutdown_connections 2024-05-21T03:24:05.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.970+0000 7ff9976f6700 1 --2- 172.21.15.27:0/4167038081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff990108920 0x7ff9900762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:05.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.970+0000 7ff9976f6700 1 --2- 172.21.15.27:0/4167038081 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7ff97c0b2a70 0x7ff97c0b4f30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:05.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.970+0000 7ff9976f6700 1 --2- 172.21.15.27:0/4167038081 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff990103850 0x7ff990075840 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:05.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.970+0000 7ff9976f6700 1 --2- 172.21.15.27:0/4167038081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff990104200 0x7ff990075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:05.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.970+0000 7ff9976f6700 1 -- 172.21.15.27:0/4167038081 >> 172.21.15.27:0/4167038081 conn(0x7ff99007a620 msgr2=0x7ff990102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:05.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.970+0000 7ff9976f6700 1 -- 172.21.15.27:0/4167038081 shutdown_connections 2024-05-21T03:24:05.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:05.970+0000 7ff9976f6700 1 -- 172.21.15.27:0/4167038081 wait complete. 2024-05-21T03:24:05.973 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-05-21T03:24:06.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:05 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:06.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:05 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:06.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:05 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:06.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:05 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:06.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:05 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:06.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:05 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:06.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:05 smithi039 ceph-mon[36429]: Adjusting osd_memory_target on smithi039 to 176.2M 2024-05-21T03:24:06.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:05 smithi039 ceph-mon[36429]: Unable to set osd_memory_target on smithi039 to 184789606: error parsing value: Value '184789606' is below minimum 939524096 2024-05-21T03:24:06.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:05 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:06.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:05 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:06.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:05 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:06.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:05 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:06.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:05 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:06.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:05 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:06.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:05 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:06.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:05 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:06.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:05 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:06.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:05 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:06.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:05 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:06.021 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:05 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:06.021 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:05 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:24:06.021 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:05 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:24:06.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:05 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:06.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:05 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:06.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:05 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:06.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:05 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:06.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:05 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:06.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:05 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:06.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:05 smithi165 ceph-mon[36269]: Adjusting osd_memory_target on smithi039 to 176.2M 2024-05-21T03:24:06.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:05 smithi165 ceph-mon[36269]: Unable to set osd_memory_target on smithi039 to 184789606: error parsing value: Value '184789606' is below minimum 939524096 2024-05-21T03:24:06.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:05 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:06.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:05 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:06.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:05 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:06.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:05 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:06.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:05 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:06.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:05 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:06.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:05 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:06.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:05 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:06.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:05 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:06.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:05 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:06.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:05 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:06.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:05 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:24:06.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:05 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:24:06.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:05 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:24:06.689 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-05-21T03:24:05.402330+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191782,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2122,"num_read_kb":1216,"num_write":503,"num_write_kb":3621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2953518,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":900,"ondisk_log_size":900,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":363844,"kb_used_data":14480,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124577980,"statfs":{"total":1151940427776,"available":1151567851520,"internally_reserved":0,"allocated":14827520,"data_stored":8859821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19139,"internal_metadata":357414205},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.987180"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656903+0000","last_change":"2024-05-21T03:20:32.149961+0000","last_active":"2024-05-21T03:24:01.656903+0000","last_peered":"2024-05-21T03:24:01.656903+0000","last_clean":"2024-05-21T03:24:01.656903+0000","last_became_active":"2024-05-21T03:20:32.149552+0000","last_became_peered":"2024-05-21T03:20:32.149552+0000","last_unstale":"2024-05-21T03:24:01.656903+0000","last_undegraded":"2024-05-21T03:24:01.656903+0000","last_fullsized":"2024-05-21T03:24:01.656903+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:20:38.543551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656992+0000","last_change":"2024-05-21T03:20:32.146413+0000","last_active":"2024-05-21T03:24:01.656992+0000","last_peered":"2024-05-21T03:24:01.656992+0000","last_clean":"2024-05-21T03:24:01.656992+0000","last_became_active":"2024-05-21T03:20:32.146006+0000","last_became_peered":"2024-05-21T03:20:32.146006+0000","last_unstale":"2024-05-21T03:24:01.656992+0000","last_undegraded":"2024-05-21T03:24:01.656992+0000","last_fullsized":"2024-05-21T03:24:01.656992+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:05:12.888550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656984+0000","last_change":"2024-05-21T03:20:32.153654+0000","last_active":"2024-05-21T03:24:01.656984+0000","last_peered":"2024-05-21T03:24:01.656984+0000","last_clean":"2024-05-21T03:24:01.656984+0000","last_became_active":"2024-05-21T03:20:32.153296+0000","last_became_peered":"2024-05-21T03:20:32.153296+0000","last_unstale":"2024-05-21T03:24:01.656984+0000","last_undegraded":"2024-05-21T03:24:01.656984+0000","last_fullsized":"2024-05-21T03:24:01.656984+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:16:09.423236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.389156+0000","last_change":"2024-05-21T03:20:32.138672+0000","last_active":"2024-05-21T03:24:01.389156+0000","last_peered":"2024-05-21T03:24:01.389156+0000","last_clean":"2024-05-21T03:24:01.389156+0000","last_became_active":"2024-05-21T03:20:32.138163+0000","last_became_peered":"2024-05-21T03:20:32.138163+0000","last_unstale":"2024-05-21T03:24:01.389156+0000","last_undegraded":"2024-05-21T03:24:01.389156+0000","last_fullsized":"2024-05-21T03:24:01.389156+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:26:55.012897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491536+0000","last_change":"2024-05-21T03:20:32.153950+0000","last_active":"2024-05-21T03:24:01.491536+0000","last_peered":"2024-05-21T03:24:01.491536+0000","last_clean":"2024-05-21T03:24:01.491536+0000","last_became_active":"2024-05-21T03:20:32.153567+0000","last_became_peered":"2024-05-21T03:20:32.153567+0000","last_unstale":"2024-05-21T03:24:01.491536+0000","last_undegraded":"2024-05-21T03:24:01.491536+0000","last_fullsized":"2024-05-21T03:24:01.491536+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:28:34.621193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.502688+0000","last_change":"2024-05-21T03:20:32.140038+0000","last_active":"2024-05-21T03:24:01.502688+0000","last_peered":"2024-05-21T03:24:01.502688+0000","last_clean":"2024-05-21T03:24:01.502688+0000","last_became_active":"2024-05-21T03:20:32.139690+0000","last_became_peered":"2024-05-21T03:20:32.139690+0000","last_unstale":"2024-05-21T03:24:01.502688+0000","last_undegraded":"2024-05-21T03:24:01.502688+0000","last_fullsized":"2024-05-21T03:24:01.502688+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:31:24.270755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656862+0000","last_change":"2024-05-21T03:20:32.143735+0000","last_active":"2024-05-21T03:24:01.656862+0000","last_peered":"2024-05-21T03:24:01.656862+0000","last_clean":"2024-05-21T03:24:01.656862+0000","last_became_active":"2024-05-21T03:20:32.143397+0000","last_became_peered":"2024-05-21T03:20:32.143397+0000","last_unstale":"2024-05-21T03:24:01.656862+0000","last_undegraded":"2024-05-21T03:24:01.656862+0000","last_fullsized":"2024-05-21T03:24:01.656862+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:40:01.951608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491334+0000","last_change":"2024-05-21T03:20:32.156999+0000","last_active":"2024-05-21T03:24:01.491334+0000","last_peered":"2024-05-21T03:24:01.491334+0000","last_clean":"2024-05-21T03:24:01.491334+0000","last_became_active":"2024-05-21T03:20:32.156677+0000","last_became_peered":"2024-05-21T03:20:32.156677+0000","last_unstale":"2024-05-21T03:24:01.491334+0000","last_undegraded":"2024-05-21T03:24:01.491334+0000","last_fullsized":"2024-05-21T03:24:01.491334+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:33:24.225667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.660261+0000","last_change":"2024-05-21T03:20:32.145039+0000","last_active":"2024-05-21T03:24:01.660261+0000","last_peered":"2024-05-21T03:24:01.660261+0000","last_clean":"2024-05-21T03:24:01.660261+0000","last_became_active":"2024-05-21T03:20:32.140926+0000","last_became_peered":"2024-05-21T03:20:32.140926+0000","last_unstale":"2024-05-21T03:24:01.660261+0000","last_undegraded":"2024-05-21T03:24:01.660261+0000","last_fullsized":"2024-05-21T03:24:01.660261+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:11:27.049426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.503068+0000","last_change":"2024-05-21T03:20:32.145840+0000","last_active":"2024-05-21T03:24:01.503068+0000","last_peered":"2024-05-21T03:24:01.503068+0000","last_clean":"2024-05-21T03:24:01.503068+0000","last_became_active":"2024-05-21T03:20:32.145291+0000","last_became_peered":"2024-05-21T03:20:32.145291+0000","last_unstale":"2024-05-21T03:24:01.503068+0000","last_undegraded":"2024-05-21T03:24:01.503068+0000","last_fullsized":"2024-05-21T03:24:01.503068+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:22:27.696588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657241+0000","last_change":"2024-05-21T03:20:32.142522+0000","last_active":"2024-05-21T03:24:01.657241+0000","last_peered":"2024-05-21T03:24:01.657241+0000","last_clean":"2024-05-21T03:24:01.657241+0000","last_became_active":"2024-05-21T03:20:32.142165+0000","last_became_peered":"2024-05-21T03:20:32.142165+0000","last_unstale":"2024-05-21T03:24:01.657241+0000","last_undegraded":"2024-05-21T03:24:01.657241+0000","last_fullsized":"2024-05-21T03:24:01.657241+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:58:39.085834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.389012+0000","last_change":"2024-05-21T03:20:32.142526+0000","last_active":"2024-05-21T03:24:01.389012+0000","last_peered":"2024-05-21T03:24:01.389012+0000","last_clean":"2024-05-21T03:24:01.389012+0000","last_became_active":"2024-05-21T03:20:32.141143+0000","last_became_peered":"2024-05-21T03:20:32.141143+0000","last_unstale":"2024-05-21T03:24:01.389012+0000","last_undegraded":"2024-05-21T03:24:01.389012+0000","last_fullsized":"2024-05-21T03:24:01.389012+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:35:11.336796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659855+0000","last_change":"2024-05-21T03:20:32.135631+0000","last_active":"2024-05-21T03:24:01.659855+0000","last_peered":"2024-05-21T03:24:01.659855+0000","last_clean":"2024-05-21T03:24:01.659855+0000","last_became_active":"2024-05-21T03:20:32.135302+0000","last_became_peered":"2024-05-21T03:20:32.135302+0000","last_unstale":"2024-05-21T03:24:01.659855+0000","last_undegraded":"2024-05-21T03:24:01.659855+0000","last_fullsized":"2024-05-21T03:24:01.659855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:05:55.000690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656928+0000","last_change":"2024-05-21T03:20:32.146580+0000","last_active":"2024-05-21T03:24:01.656928+0000","last_peered":"2024-05-21T03:24:01.656928+0000","last_clean":"2024-05-21T03:24:01.656928+0000","last_became_active":"2024-05-21T03:20:32.145803+0000","last_became_peered":"2024-05-21T03:20:32.145803+0000","last_unstale":"2024-05-21T03:24:01.656928+0000","last_undegraded":"2024-05-21T03:24:01.656928+0000","last_fullsized":"2024-05-21T03:24:01.656928+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:34:09.108799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656841+0000","last_change":"2024-05-21T03:21:14.559498+0000","last_active":"2024-05-21T03:24:01.656841+0000","last_peered":"2024-05-21T03:24:01.656841+0000","last_clean":"2024-05-21T03:24:01.656841+0000","last_became_active":"2024-05-21T03:21:14.559039+0000","last_became_peered":"2024-05-21T03:21:14.559039+0000","last_unstale":"2024-05-21T03:24:01.656841+0000","last_undegraded":"2024-05-21T03:24:01.656841+0000","last_fullsized":"2024-05-21T03:24:01.656841+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:04:36.676857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659265+0000","last_change":"2024-05-21T03:20:32.148944+0000","last_active":"2024-05-21T03:24:01.659265+0000","last_peered":"2024-05-21T03:24:01.659265+0000","last_clean":"2024-05-21T03:24:01.659265+0000","last_became_active":"2024-05-21T03:20:32.148502+0000","last_became_peered":"2024-05-21T03:20:32.148502+0000","last_unstale":"2024-05-21T03:24:01.659265+0000","last_undegraded":"2024-05-21T03:24:01.659265+0000","last_fullsized":"2024-05-21T03:24:01.659265+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:11:00.178105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657063+0000","last_change":"2024-05-21T03:20:32.150609+0000","last_active":"2024-05-21T03:24:01.657063+0000","last_peered":"2024-05-21T03:24:01.657063+0000","last_clean":"2024-05-21T03:24:01.657063+0000","last_became_active":"2024-05-21T03:20:32.150314+0000","last_became_peered":"2024-05-21T03:20:32.150314+0000","last_unstale":"2024-05-21T03:24:01.657063+0000","last_undegraded":"2024-05-21T03:24:01.657063+0000","last_fullsized":"2024-05-21T03:24:01.657063+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:34:21.869304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.387785+0000","last_change":"2024-05-21T03:20:32.142472+0000","last_active":"2024-05-21T03:24:01.387785+0000","last_peered":"2024-05-21T03:24:01.387785+0000","last_clean":"2024-05-21T03:24:01.387785+0000","last_became_active":"2024-05-21T03:20:32.142161+0000","last_became_peered":"2024-05-21T03:20:32.142161+0000","last_unstale":"2024-05-21T03:24:01.387785+0000","last_undegraded":"2024-05-21T03:24:01.387785+0000","last_fullsized":"2024-05-21T03:24:01.387785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:26:57.281655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.389229+0000","last_change":"2024-05-21T03:20:32.153987+0000","last_active":"2024-05-21T03:24:01.389229+0000","last_peered":"2024-05-21T03:24:01.389229+0000","last_clean":"2024-05-21T03:24:01.389229+0000","last_became_active":"2024-05-21T03:20:32.153617+0000","last_became_peered":"2024-05-21T03:20:32.153617+0000","last_unstale":"2024-05-21T03:24:01.389229+0000","last_undegraded":"2024-05-21T03:24:01.389229+0000","last_fullsized":"2024-05-21T03:24:01.389229+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:57:07.341758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'15","reported_seq":236,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656514+0000","last_change":"2024-05-21T03:20:29.099928+0000","last_active":"2024-05-21T03:24:01.656514+0000","last_peered":"2024-05-21T03:24:01.656514+0000","last_clean":"2024-05-21T03:24:01.656514+0000","last_became_active":"2024-05-21T03:20:29.099592+0000","last_became_peered":"2024-05-21T03:20:29.099592+0000","last_unstale":"2024-05-21T03:24:01.656514+0000","last_undegraded":"2024-05-21T03:24:01.656514+0000","last_fullsized":"2024-05-21T03:24:01.656514+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:35:34.843325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":193,"num_read_kb":193,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491831+0000","last_change":"2024-05-21T03:20:32.154962+0000","last_active":"2024-05-21T03:24:01.491831+0000","last_peered":"2024-05-21T03:24:01.491831+0000","last_clean":"2024-05-21T03:24:01.491831+0000","last_became_active":"2024-05-21T03:20:32.154620+0000","last_became_peered":"2024-05-21T03:20:32.154620+0000","last_unstale":"2024-05-21T03:24:01.491831+0000","last_undegraded":"2024-05-21T03:24:01.491831+0000","last_fullsized":"2024-05-21T03:24:01.491831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:56:21.553272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.492006+0000","last_change":"2024-05-21T03:20:29.097312+0000","last_active":"2024-05-21T03:24:01.492006+0000","last_peered":"2024-05-21T03:24:01.492006+0000","last_clean":"2024-05-21T03:24:01.492006+0000","last_became_active":"2024-05-21T03:20:29.096977+0000","last_became_peered":"2024-05-21T03:20:29.096977+0000","last_unstale":"2024-05-21T03:24:01.492006+0000","last_undegraded":"2024-05-21T03:24:01.492006+0000","last_fullsized":"2024-05-21T03:24:01.492006+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:28:45.467446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657332+0000","last_change":"2024-05-21T03:20:32.150035+0000","last_active":"2024-05-21T03:24:01.657332+0000","last_peered":"2024-05-21T03:24:01.657332+0000","last_clean":"2024-05-21T03:24:01.657332+0000","last_became_active":"2024-05-21T03:20:32.149688+0000","last_became_peered":"2024-05-21T03:20:32.149688+0000","last_unstale":"2024-05-21T03:24:01.657332+0000","last_undegraded":"2024-05-21T03:24:01.657332+0000","last_fullsized":"2024-05-21T03:24:01.657332+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:27:00.410770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656922+0000","last_change":"2024-05-21T03:21:14.572832+0000","last_active":"2024-05-21T03:24:01.656922+0000","last_peered":"2024-05-21T03:24:01.656922+0000","last_clean":"2024-05-21T03:24:01.656922+0000","last_became_active":"2024-05-21T03:21:14.559381+0000","last_became_peered":"2024-05-21T03:21:14.559381+0000","last_unstale":"2024-05-21T03:24:01.656922+0000","last_undegraded":"2024-05-21T03:24:01.656922+0000","last_fullsized":"2024-05-21T03:24:01.656922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:02:53.004031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659270+0000","last_change":"2024-05-21T03:20:32.150301+0000","last_active":"2024-05-21T03:24:01.659270+0000","last_peered":"2024-05-21T03:24:01.659270+0000","last_clean":"2024-05-21T03:24:01.659270+0000","last_became_active":"2024-05-21T03:20:32.148475+0000","last_became_peered":"2024-05-21T03:20:32.148475+0000","last_unstale":"2024-05-21T03:24:01.659270+0000","last_undegraded":"2024-05-21T03:24:01.659270+0000","last_fullsized":"2024-05-21T03:24:01.659270+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:59:30.339660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491705+0000","last_change":"2024-05-21T03:20:29.098861+0000","last_active":"2024-05-21T03:24:01.491705+0000","last_peered":"2024-05-21T03:24:01.491705+0000","last_clean":"2024-05-21T03:24:01.491705+0000","last_became_active":"2024-05-21T03:20:29.097874+0000","last_became_peered":"2024-05-21T03:20:29.097874+0000","last_unstale":"2024-05-21T03:24:01.491705+0000","last_undegraded":"2024-05-21T03:24:01.491705+0000","last_fullsized":"2024-05-21T03:24:01.491705+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:25:36.227339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.499470+0000","last_change":"2024-05-21T03:20:32.142820+0000","last_active":"2024-05-21T03:24:01.499470+0000","last_peered":"2024-05-21T03:24:01.499470+0000","last_clean":"2024-05-21T03:24:01.499470+0000","last_became_active":"2024-05-21T03:20:32.142396+0000","last_became_peered":"2024-05-21T03:20:32.142396+0000","last_unstale":"2024-05-21T03:24:01.499470+0000","last_undegraded":"2024-05-21T03:24:01.499470+0000","last_fullsized":"2024-05-21T03:24:01.499470+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:30:14.452389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":228,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.499417+0000","last_change":"2024-05-21T03:20:29.103153+0000","last_active":"2024-05-21T03:24:01.499417+0000","last_peered":"2024-05-21T03:24:01.499417+0000","last_clean":"2024-05-21T03:24:01.499417+0000","last_became_active":"2024-05-21T03:20:29.102819+0000","last_became_peered":"2024-05-21T03:20:29.102819+0000","last_unstale":"2024-05-21T03:24:01.499417+0000","last_undegraded":"2024-05-21T03:24:01.499417+0000","last_fullsized":"2024-05-21T03:24:01.499417+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:47:49.843926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":194,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491481+0000","last_change":"2024-05-21T03:20:32.142263+0000","last_active":"2024-05-21T03:24:01.491481+0000","last_peered":"2024-05-21T03:24:01.491481+0000","last_clean":"2024-05-21T03:24:01.491481+0000","last_became_active":"2024-05-21T03:20:32.141820+0000","last_became_peered":"2024-05-21T03:20:32.141820+0000","last_unstale":"2024-05-21T03:24:01.491481+0000","last_undegraded":"2024-05-21T03:24:01.491481+0000","last_fullsized":"2024-05-21T03:24:01.491481+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:45:20.021237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.502625+0000","last_change":"2024-05-21T03:20:29.100088+0000","last_active":"2024-05-21T03:24:01.502625+0000","last_peered":"2024-05-21T03:24:01.502625+0000","last_clean":"2024-05-21T03:24:01.502625+0000","last_became_active":"2024-05-21T03:20:29.097343+0000","last_became_peered":"2024-05-21T03:20:29.097343+0000","last_unstale":"2024-05-21T03:24:01.502625+0000","last_undegraded":"2024-05-21T03:24:01.502625+0000","last_fullsized":"2024-05-21T03:24:01.502625+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:00:35.682393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.387512+0000","last_change":"2024-05-21T03:21:14.562281+0000","last_active":"2024-05-21T03:24:01.387512+0000","last_peered":"2024-05-21T03:24:01.387512+0000","last_clean":"2024-05-21T03:24:01.387512+0000","last_became_active":"2024-05-21T03:21:14.561857+0000","last_became_peered":"2024-05-21T03:21:14.561857+0000","last_unstale":"2024-05-21T03:24:01.387512+0000","last_undegraded":"2024-05-21T03:24:01.387512+0000","last_fullsized":"2024-05-21T03:24:01.387512+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:37:35.283529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659672+0000","last_change":"2024-05-21T03:20:29.097945+0000","last_active":"2024-05-21T03:24:01.659672+0000","last_peered":"2024-05-21T03:24:01.659672+0000","last_clean":"2024-05-21T03:24:01.659672+0000","last_became_active":"2024-05-21T03:20:29.097624+0000","last_became_peered":"2024-05-21T03:20:29.097624+0000","last_unstale":"2024-05-21T03:24:01.659672+0000","last_undegraded":"2024-05-21T03:24:01.659672+0000","last_fullsized":"2024-05-21T03:24:01.659672+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:01:01.694893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491268+0000","last_change":"2024-05-21T03:20:32.148955+0000","last_active":"2024-05-21T03:24:01.491268+0000","last_peered":"2024-05-21T03:24:01.491268+0000","last_clean":"2024-05-21T03:24:01.491268+0000","last_became_active":"2024-05-21T03:20:32.148631+0000","last_became_peered":"2024-05-21T03:20:32.148631+0000","last_unstale":"2024-05-21T03:24:01.491268+0000","last_undegraded":"2024-05-21T03:24:01.491268+0000","last_fullsized":"2024-05-21T03:24:01.491268+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:36:25.820580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.499174+0000","last_change":"2024-05-21T03:20:29.100902+0000","last_active":"2024-05-21T03:24:01.499174+0000","last_peered":"2024-05-21T03:24:01.499174+0000","last_clean":"2024-05-21T03:24:01.499174+0000","last_became_active":"2024-05-21T03:20:29.100570+0000","last_became_peered":"2024-05-21T03:20:29.100570+0000","last_unstale":"2024-05-21T03:24:01.499174+0000","last_undegraded":"2024-05-21T03:24:01.499174+0000","last_fullsized":"2024-05-21T03:24:01.499174+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:58:36.651338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.387428+0000","last_change":"2024-05-21T03:20:32.150724+0000","last_active":"2024-05-21T03:24:01.387428+0000","last_peered":"2024-05-21T03:24:01.387428+0000","last_clean":"2024-05-21T03:24:01.387428+0000","last_became_active":"2024-05-21T03:20:32.150371+0000","last_became_peered":"2024-05-21T03:20:32.150371+0000","last_unstale":"2024-05-21T03:24:01.387428+0000","last_undegraded":"2024-05-21T03:24:01.387428+0000","last_fullsized":"2024-05-21T03:24:01.387428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:44:32.321192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'6","reported_seq":419,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.747249+0000","last_change":"2024-05-21T03:20:29.095045+0000","last_active":"2024-05-21T03:24:01.747249+0000","last_peered":"2024-05-21T03:24:01.747249+0000","last_clean":"2024-05-21T03:24:01.747249+0000","last_became_active":"2024-05-21T03:20:29.094552+0000","last_became_peered":"2024-05-21T03:20:29.094552+0000","last_unstale":"2024-05-21T03:24:01.747249+0000","last_undegraded":"2024-05-21T03:24:01.747249+0000","last_fullsized":"2024-05-21T03:24:01.747249+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:36:15.155125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":384,"num_read_kb":384,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659781+0000","last_change":"2024-05-21T03:20:32.149813+0000","last_active":"2024-05-21T03:24:01.659781+0000","last_peered":"2024-05-21T03:24:01.659781+0000","last_clean":"2024-05-21T03:24:01.659781+0000","last_became_active":"2024-05-21T03:20:32.148422+0000","last_became_peered":"2024-05-21T03:20:32.148422+0000","last_unstale":"2024-05-21T03:24:01.659781+0000","last_undegraded":"2024-05-21T03:24:01.659781+0000","last_fullsized":"2024-05-21T03:24:01.659781+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:55:35.115463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657337+0000","last_change":"2024-05-21T03:20:29.101463+0000","last_active":"2024-05-21T03:24:01.657337+0000","last_peered":"2024-05-21T03:24:01.657337+0000","last_clean":"2024-05-21T03:24:01.657337+0000","last_became_active":"2024-05-21T03:20:29.101127+0000","last_became_peered":"2024-05-21T03:20:29.101127+0000","last_unstale":"2024-05-21T03:24:01.657337+0000","last_undegraded":"2024-05-21T03:24:01.657337+0000","last_fullsized":"2024-05-21T03:24:01.657337+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:58:17.408315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657158+0000","last_change":"2024-05-21T03:21:14.563434+0000","last_active":"2024-05-21T03:24:01.657158+0000","last_peered":"2024-05-21T03:24:01.657158+0000","last_clean":"2024-05-21T03:24:01.657158+0000","last_became_active":"2024-05-21T03:21:14.563081+0000","last_became_peered":"2024-05-21T03:21:14.563081+0000","last_unstale":"2024-05-21T03:24:01.657158+0000","last_undegraded":"2024-05-21T03:24:01.657158+0000","last_fullsized":"2024-05-21T03:24:01.657158+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:28:55.497338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.499583+0000","last_change":"2024-05-21T03:20:29.098725+0000","last_active":"2024-05-21T03:24:01.499583+0000","last_peered":"2024-05-21T03:24:01.499583+0000","last_clean":"2024-05-21T03:24:01.499583+0000","last_became_active":"2024-05-21T03:20:29.098471+0000","last_became_peered":"2024-05-21T03:20:29.098471+0000","last_unstale":"2024-05-21T03:24:01.499583+0000","last_undegraded":"2024-05-21T03:24:01.499583+0000","last_fullsized":"2024-05-21T03:24:01.499583+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:17:11.059825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659799+0000","last_change":"2024-05-21T03:21:14.560975+0000","last_active":"2024-05-21T03:24:01.659799+0000","last_peered":"2024-05-21T03:24:01.659799+0000","last_clean":"2024-05-21T03:24:01.659799+0000","last_became_active":"2024-05-21T03:21:14.560528+0000","last_became_peered":"2024-05-21T03:21:14.560528+0000","last_unstale":"2024-05-21T03:24:01.659799+0000","last_undegraded":"2024-05-21T03:24:01.659799+0000","last_fullsized":"2024-05-21T03:24:01.659799+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:51:47.469291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'39","reported_seq":465,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.748126+0000","last_change":"2024-05-21T03:20:29.099968+0000","last_active":"2024-05-21T03:24:01.748126+0000","last_peered":"2024-05-21T03:24:01.748126+0000","last_clean":"2024-05-21T03:24:01.748126+0000","last_became_active":"2024-05-21T03:20:29.099181+0000","last_became_peered":"2024-05-21T03:20:29.099181+0000","last_unstale":"2024-05-21T03:24:01.748126+0000","last_undegraded":"2024-05-21T03:24:01.748126+0000","last_fullsized":"2024-05-21T03:24:01.748126+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:35:33.934412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66474,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":397,"num_read_kb":263,"num_write":41,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491786+0000","last_change":"2024-05-21T03:20:32.144024+0000","last_active":"2024-05-21T03:24:01.491786+0000","last_peered":"2024-05-21T03:24:01.491786+0000","last_clean":"2024-05-21T03:24:01.491786+0000","last_became_active":"2024-05-21T03:20:32.143668+0000","last_became_peered":"2024-05-21T03:20:32.143668+0000","last_unstale":"2024-05-21T03:24:01.491786+0000","last_undegraded":"2024-05-21T03:24:01.491786+0000","last_fullsized":"2024-05-21T03:24:01.491786+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:11:31.100802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.389042+0000","last_change":"2024-05-21T03:20:29.101770+0000","last_active":"2024-05-21T03:24:01.389042+0000","last_peered":"2024-05-21T03:24:01.389042+0000","last_clean":"2024-05-21T03:24:01.389042+0000","last_became_active":"2024-05-21T03:20:29.100407+0000","last_became_peered":"2024-05-21T03:20:29.100407+0000","last_unstale":"2024-05-21T03:24:01.389042+0000","last_undegraded":"2024-05-21T03:24:01.389042+0000","last_fullsized":"2024-05-21T03:24:01.389042+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:01:33.991246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659520+0000","last_change":"2024-05-21T03:20:32.147479+0000","last_active":"2024-05-21T03:24:01.659520+0000","last_peered":"2024-05-21T03:24:01.659520+0000","last_clean":"2024-05-21T03:24:01.659520+0000","last_became_active":"2024-05-21T03:20:32.147160+0000","last_became_peered":"2024-05-21T03:20:32.147160+0000","last_unstale":"2024-05-21T03:24:01.659520+0000","last_undegraded":"2024-05-21T03:24:01.659520+0000","last_fullsized":"2024-05-21T03:24:01.659520+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:51:59.343380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.499274+0000","last_change":"2024-05-21T03:20:29.101875+0000","last_active":"2024-05-21T03:24:01.499274+0000","last_peered":"2024-05-21T03:24:01.499274+0000","last_clean":"2024-05-21T03:24:01.499274+0000","last_became_active":"2024-05-21T03:20:29.101468+0000","last_became_peered":"2024-05-21T03:20:29.101468+0000","last_unstale":"2024-05-21T03:24:01.499274+0000","last_undegraded":"2024-05-21T03:24:01.499274+0000","last_fullsized":"2024-05-21T03:24:01.499274+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:10:19.577836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657039+0000","last_change":"2024-05-21T03:20:32.139002+0000","last_active":"2024-05-21T03:24:01.657039+0000","last_peered":"2024-05-21T03:24:01.657039+0000","last_clean":"2024-05-21T03:24:01.657039+0000","last_became_active":"2024-05-21T03:20:32.138619+0000","last_became_peered":"2024-05-21T03:20:32.138619+0000","last_unstale":"2024-05-21T03:24:01.657039+0000","last_undegraded":"2024-05-21T03:24:01.657039+0000","last_fullsized":"2024-05-21T03:24:01.657039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:21:36.136283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'8","reported_seq":233,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.747618+0000","last_change":"2024-05-21T03:20:29.101532+0000","last_active":"2024-05-21T03:24:01.747618+0000","last_peered":"2024-05-21T03:24:01.747618+0000","last_clean":"2024-05-21T03:24:01.747618+0000","last_became_active":"2024-05-21T03:20:29.101206+0000","last_became_peered":"2024-05-21T03:20:29.101206+0000","last_unstale":"2024-05-21T03:24:01.747618+0000","last_undegraded":"2024-05-21T03:24:01.747618+0000","last_fullsized":"2024-05-21T03:24:01.747618+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:47:38.555995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":198,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656787+0000","last_change":"2024-05-21T03:20:32.146866+0000","last_active":"2024-05-21T03:24:01.656787+0000","last_peered":"2024-05-21T03:24:01.656787+0000","last_clean":"2024-05-21T03:24:01.656787+0000","last_became_active":"2024-05-21T03:20:32.146463+0000","last_became_peered":"2024-05-21T03:20:32.146463+0000","last_unstale":"2024-05-21T03:24:01.656787+0000","last_undegraded":"2024-05-21T03:24:01.656787+0000","last_fullsized":"2024-05-21T03:24:01.656787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:06:54.359922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491442+0000","last_change":"2024-05-21T03:20:29.103910+0000","last_active":"2024-05-21T03:24:01.491442+0000","last_peered":"2024-05-21T03:24:01.491442+0000","last_clean":"2024-05-21T03:24:01.491442+0000","last_became_active":"2024-05-21T03:20:29.103583+0000","last_became_peered":"2024-05-21T03:20:29.103583+0000","last_unstale":"2024-05-21T03:24:01.491442+0000","last_undegraded":"2024-05-21T03:24:01.491442+0000","last_fullsized":"2024-05-21T03:24:01.491442+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:57:15.665158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.499198+0000","last_change":"2024-05-21T03:20:32.141242+0000","last_active":"2024-05-21T03:24:01.499198+0000","last_peered":"2024-05-21T03:24:01.499198+0000","last_clean":"2024-05-21T03:24:01.499198+0000","last_became_active":"2024-05-21T03:20:32.140892+0000","last_became_peered":"2024-05-21T03:20:32.140892+0000","last_unstale":"2024-05-21T03:24:01.499198+0000","last_undegraded":"2024-05-21T03:24:01.499198+0000","last_fullsized":"2024-05-21T03:24:01.499198+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:42:13.997708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491966+0000","last_change":"2024-05-21T03:20:29.098973+0000","last_active":"2024-05-21T03:24:01.491966+0000","last_peered":"2024-05-21T03:24:01.491966+0000","last_clean":"2024-05-21T03:24:01.491966+0000","last_became_active":"2024-05-21T03:20:29.098694+0000","last_became_peered":"2024-05-21T03:20:29.098694+0000","last_unstale":"2024-05-21T03:24:01.491966+0000","last_undegraded":"2024-05-21T03:24:01.491966+0000","last_fullsized":"2024-05-21T03:24:01.491966+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:55:17.915820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656862+0000","last_change":"2024-05-21T03:20:32.147465+0000","last_active":"2024-05-21T03:24:01.656862+0000","last_peered":"2024-05-21T03:24:01.656862+0000","last_clean":"2024-05-21T03:24:01.656862+0000","last_became_active":"2024-05-21T03:20:32.147073+0000","last_became_peered":"2024-05-21T03:20:32.147073+0000","last_unstale":"2024-05-21T03:24:01.656862+0000","last_undegraded":"2024-05-21T03:24:01.656862+0000","last_fullsized":"2024-05-21T03:24:01.656862+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:29:05.451370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.387615+0000","last_change":"2024-05-21T03:21:14.560568+0000","last_active":"2024-05-21T03:24:01.387615+0000","last_peered":"2024-05-21T03:24:01.387615+0000","last_clean":"2024-05-21T03:24:01.387615+0000","last_became_active":"2024-05-21T03:21:14.560093+0000","last_became_peered":"2024-05-21T03:21:14.560093+0000","last_unstale":"2024-05-21T03:24:01.387615+0000","last_undegraded":"2024-05-21T03:24:01.387615+0000","last_fullsized":"2024-05-21T03:24:01.387615+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:55:39.839942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.502820+0000","last_change":"2024-05-21T03:20:32.146456+0000","last_active":"2024-05-21T03:24:01.502820+0000","last_peered":"2024-05-21T03:24:01.502820+0000","last_clean":"2024-05-21T03:24:01.502820+0000","last_became_active":"2024-05-21T03:20:32.146105+0000","last_became_peered":"2024-05-21T03:20:32.146105+0000","last_unstale":"2024-05-21T03:24:01.502820+0000","last_undegraded":"2024-05-21T03:24:01.502820+0000","last_fullsized":"2024-05-21T03:24:01.502820+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:00:30.550178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'8","reported_seq":232,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.758178+0000","last_change":"2024-05-21T03:20:29.100262+0000","last_active":"2024-05-21T03:24:01.758178+0000","last_peered":"2024-05-21T03:24:01.758178+0000","last_clean":"2024-05-21T03:24:01.758178+0000","last_became_active":"2024-05-21T03:20:29.099917+0000","last_became_peered":"2024-05-21T03:20:29.099917+0000","last_unstale":"2024-05-21T03:24:01.758178+0000","last_undegraded":"2024-05-21T03:24:01.758178+0000","last_fullsized":"2024-05-21T03:24:01.758178+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:55:48.608341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":196,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.502772+0000","last_change":"2024-05-21T03:20:32.142776+0000","last_active":"2024-05-21T03:24:01.502772+0000","last_peered":"2024-05-21T03:24:01.502772+0000","last_clean":"2024-05-21T03:24:01.502772+0000","last_became_active":"2024-05-21T03:20:32.142430+0000","last_became_peered":"2024-05-21T03:20:32.142430+0000","last_unstale":"2024-05-21T03:24:01.502772+0000","last_undegraded":"2024-05-21T03:24:01.502772+0000","last_fullsized":"2024-05-21T03:24:01.502772+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:54:28.682343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.499435+0000","last_change":"2024-05-21T03:20:32.139506+0000","last_active":"2024-05-21T03:24:01.499435+0000","last_peered":"2024-05-21T03:24:01.499435+0000","last_clean":"2024-05-21T03:24:01.499435+0000","last_became_active":"2024-05-21T03:20:32.138136+0000","last_became_peered":"2024-05-21T03:20:32.138136+0000","last_unstale":"2024-05-21T03:24:01.499435+0000","last_undegraded":"2024-05-21T03:24:01.499435+0000","last_fullsized":"2024-05-21T03:24:01.499435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:31:24.996276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'7","reported_seq":229,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.661475+0000","last_change":"2024-05-21T03:20:29.096546+0000","last_active":"2024-05-21T03:24:01.661475+0000","last_peered":"2024-05-21T03:24:01.661475+0000","last_clean":"2024-05-21T03:24:01.661475+0000","last_became_active":"2024-05-21T03:20:29.096184+0000","last_became_peered":"2024-05-21T03:20:29.096184+0000","last_unstale":"2024-05-21T03:24:01.661475+0000","last_undegraded":"2024-05-21T03:24:01.661475+0000","last_fullsized":"2024-05-21T03:24:01.661475+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:09:12.832745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":193,"num_read_kb":193,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491686+0000","last_change":"2024-05-21T03:20:32.156231+0000","last_active":"2024-05-21T03:24:01.491686+0000","last_peered":"2024-05-21T03:24:01.491686+0000","last_clean":"2024-05-21T03:24:01.491686+0000","last_became_active":"2024-05-21T03:20:32.154035+0000","last_became_peered":"2024-05-21T03:20:32.154035+0000","last_unstale":"2024-05-21T03:24:01.491686+0000","last_undegraded":"2024-05-21T03:24:01.491686+0000","last_fullsized":"2024-05-21T03:24:01.491686+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:30:53.820292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.388854+0000","last_change":"2024-05-21T03:20:29.100422+0000","last_active":"2024-05-21T03:24:01.388854+0000","last_peered":"2024-05-21T03:24:01.388854+0000","last_clean":"2024-05-21T03:24:01.388854+0000","last_became_active":"2024-05-21T03:20:29.099993+0000","last_became_peered":"2024-05-21T03:20:29.099993+0000","last_unstale":"2024-05-21T03:24:01.388854+0000","last_undegraded":"2024-05-21T03:24:01.388854+0000","last_fullsized":"2024-05-21T03:24:01.388854+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:03:15.456447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":102,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656886+0000","last_change":"2024-05-21T03:19:05.647808+0000","last_active":"2024-05-21T03:24:01.656886+0000","last_peered":"2024-05-21T03:24:01.656886+0000","last_clean":"2024-05-21T03:24:01.656886+0000","last_became_active":"2024-05-21T03:19:05.641865+0000","last_became_peered":"2024-05-21T03:19:05.641865+0000","last_unstale":"2024-05-21T03:24:01.656886+0000","last_undegraded":"2024-05-21T03:24:01.656886+0000","last_fullsized":"2024-05-21T03:24:01.656886+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:15:42.008493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:15:42.008493+0000","last_clean_scrub_stamp":"2024-05-21T03:15:42.008493+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:26:58.040196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659351+0000","last_change":"2024-05-21T03:20:32.138182+0000","last_active":"2024-05-21T03:24:01.659351+0000","last_peered":"2024-05-21T03:24:01.659351+0000","last_clean":"2024-05-21T03:24:01.659351+0000","last_became_active":"2024-05-21T03:20:32.137842+0000","last_became_peered":"2024-05-21T03:20:32.137842+0000","last_unstale":"2024-05-21T03:24:01.659351+0000","last_undegraded":"2024-05-21T03:24:01.659351+0000","last_fullsized":"2024-05-21T03:24:01.659351+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:43:50.070160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657022+0000","last_change":"2024-05-21T03:20:29.104246+0000","last_active":"2024-05-21T03:24:01.657022+0000","last_peered":"2024-05-21T03:24:01.657022+0000","last_clean":"2024-05-21T03:24:01.657022+0000","last_became_active":"2024-05-21T03:20:29.103910+0000","last_became_peered":"2024-05-21T03:20:29.103910+0000","last_unstale":"2024-05-21T03:24:01.657022+0000","last_undegraded":"2024-05-21T03:24:01.657022+0000","last_fullsized":"2024-05-21T03:24:01.657022+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:39:28.029283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491818+0000","last_change":"2024-05-21T03:20:29.102020+0000","last_active":"2024-05-21T03:24:01.491818+0000","last_peered":"2024-05-21T03:24:01.491818+0000","last_clean":"2024-05-21T03:24:01.491818+0000","last_became_active":"2024-05-21T03:20:29.100429+0000","last_became_peered":"2024-05-21T03:20:29.100429+0000","last_unstale":"2024-05-21T03:24:01.491818+0000","last_undegraded":"2024-05-21T03:24:01.491818+0000","last_fullsized":"2024-05-21T03:24:01.491818+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:09:01.621074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659598+0000","last_change":"2024-05-21T03:20:32.137130+0000","last_active":"2024-05-21T03:24:01.659598+0000","last_peered":"2024-05-21T03:24:01.659598+0000","last_clean":"2024-05-21T03:24:01.659598+0000","last_became_active":"2024-05-21T03:20:32.136821+0000","last_became_peered":"2024-05-21T03:20:32.136821+0000","last_unstale":"2024-05-21T03:24:01.659598+0000","last_undegraded":"2024-05-21T03:24:01.659598+0000","last_fullsized":"2024-05-21T03:24:01.659598+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:53:41.290370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'50","reported_seq":101,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.610537+0000","last_change":"2024-05-21T03:20:29.099869+0000","last_active":"2024-05-21T03:24:01.610537+0000","last_peered":"2024-05-21T03:24:01.610537+0000","last_clean":"2024-05-21T03:24:01.610537+0000","last_became_active":"2024-05-21T03:20:29.099534+0000","last_became_peered":"2024-05-21T03:20:29.099534+0000","last_unstale":"2024-05-21T03:24:01.610537+0000","last_undegraded":"2024-05-21T03:24:01.610537+0000","last_fullsized":"2024-05-21T03:24:01.610537+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:01:11.281544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":89,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491991+0000","last_change":"2024-05-21T03:20:32.153921+0000","last_active":"2024-05-21T03:24:01.491991+0000","last_peered":"2024-05-21T03:24:01.491991+0000","last_clean":"2024-05-21T03:24:01.491991+0000","last_became_active":"2024-05-21T03:20:32.153594+0000","last_became_peered":"2024-05-21T03:20:32.153594+0000","last_unstale":"2024-05-21T03:24:01.491991+0000","last_undegraded":"2024-05-21T03:24:01.491991+0000","last_fullsized":"2024-05-21T03:24:01.491991+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:27:59.426942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.389210+0000","last_change":"2024-05-21T03:20:29.097399+0000","last_active":"2024-05-21T03:24:01.389210+0000","last_peered":"2024-05-21T03:24:01.389210+0000","last_clean":"2024-05-21T03:24:01.389210+0000","last_became_active":"2024-05-21T03:20:29.097010+0000","last_became_peered":"2024-05-21T03:20:29.097010+0000","last_unstale":"2024-05-21T03:24:01.389210+0000","last_undegraded":"2024-05-21T03:24:01.389210+0000","last_fullsized":"2024-05-21T03:24:01.389210+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:54:53.273947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657141+0000","last_change":"2024-05-21T03:21:14.559609+0000","last_active":"2024-05-21T03:24:01.657141+0000","last_peered":"2024-05-21T03:24:01.657141+0000","last_clean":"2024-05-21T03:24:01.657141+0000","last_became_active":"2024-05-21T03:21:14.559218+0000","last_became_peered":"2024-05-21T03:21:14.559218+0000","last_unstale":"2024-05-21T03:24:01.657141+0000","last_undegraded":"2024-05-21T03:24:01.657141+0000","last_fullsized":"2024-05-21T03:24:01.657141+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:32:11.156037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,4],"acting":[0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659469+0000","last_change":"2024-05-21T03:20:29.103526+0000","last_active":"2024-05-21T03:24:01.659469+0000","last_peered":"2024-05-21T03:24:01.659469+0000","last_clean":"2024-05-21T03:24:01.659469+0000","last_became_active":"2024-05-21T03:20:29.103067+0000","last_became_peered":"2024-05-21T03:20:29.103067+0000","last_unstale":"2024-05-21T03:24:01.659469+0000","last_undegraded":"2024-05-21T03:24:01.659469+0000","last_fullsized":"2024-05-21T03:24:01.659469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:54:47.566949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491463+0000","last_change":"2024-05-21T03:20:32.141088+0000","last_active":"2024-05-21T03:24:01.491463+0000","last_peered":"2024-05-21T03:24:01.491463+0000","last_clean":"2024-05-21T03:24:01.491463+0000","last_became_active":"2024-05-21T03:20:32.140744+0000","last_became_peered":"2024-05-21T03:20:32.140744+0000","last_unstale":"2024-05-21T03:24:01.491463+0000","last_undegraded":"2024-05-21T03:24:01.491463+0000","last_fullsized":"2024-05-21T03:24:01.491463+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:57:35.591164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491373+0000","last_change":"2024-05-21T03:20:29.093440+0000","last_active":"2024-05-21T03:24:01.491373+0000","last_peered":"2024-05-21T03:24:01.491373+0000","last_clean":"2024-05-21T03:24:01.491373+0000","last_became_active":"2024-05-21T03:20:29.092935+0000","last_became_peered":"2024-05-21T03:20:29.092935+0000","last_unstale":"2024-05-21T03:24:01.491373+0000","last_undegraded":"2024-05-21T03:24:01.491373+0000","last_fullsized":"2024-05-21T03:24:01.491373+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:18:00.690465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656650+0000","last_change":"2024-05-21T03:20:32.143781+0000","last_active":"2024-05-21T03:24:01.656650+0000","last_peered":"2024-05-21T03:24:01.656650+0000","last_clean":"2024-05-21T03:24:01.656650+0000","last_became_active":"2024-05-21T03:20:32.143391+0000","last_became_peered":"2024-05-21T03:20:32.143391+0000","last_unstale":"2024-05-21T03:24:01.656650+0000","last_undegraded":"2024-05-21T03:24:01.656650+0000","last_fullsized":"2024-05-21T03:24:01.656650+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:00:41.506561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.388944+0000","last_change":"2024-05-21T03:20:29.101342+0000","last_active":"2024-05-21T03:24:01.388944+0000","last_peered":"2024-05-21T03:24:01.388944+0000","last_clean":"2024-05-21T03:24:01.388944+0000","last_became_active":"2024-05-21T03:20:29.100997+0000","last_became_peered":"2024-05-21T03:20:29.100997+0000","last_unstale":"2024-05-21T03:24:01.388944+0000","last_undegraded":"2024-05-21T03:24:01.388944+0000","last_fullsized":"2024-05-21T03:24:01.388944+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:05:31.801669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491844+0000","last_change":"2024-05-21T03:20:32.150563+0000","last_active":"2024-05-21T03:24:01.491844+0000","last_peered":"2024-05-21T03:24:01.491844+0000","last_clean":"2024-05-21T03:24:01.491844+0000","last_became_active":"2024-05-21T03:20:32.150231+0000","last_became_peered":"2024-05-21T03:20:32.150231+0000","last_unstale":"2024-05-21T03:24:01.491844+0000","last_undegraded":"2024-05-21T03:24:01.491844+0000","last_fullsized":"2024-05-21T03:24:01.491844+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:10:00.251284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491635+0000","last_change":"2024-05-21T03:20:29.101576+0000","last_active":"2024-05-21T03:24:01.491635+0000","last_peered":"2024-05-21T03:24:01.491635+0000","last_clean":"2024-05-21T03:24:01.491635+0000","last_became_active":"2024-05-21T03:20:29.101270+0000","last_became_peered":"2024-05-21T03:20:29.101270+0000","last_unstale":"2024-05-21T03:24:01.491635+0000","last_undegraded":"2024-05-21T03:24:01.491635+0000","last_fullsized":"2024-05-21T03:24:01.491635+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:50:10.579901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656976+0000","last_change":"2024-05-21T03:20:32.141340+0000","last_active":"2024-05-21T03:24:01.656976+0000","last_peered":"2024-05-21T03:24:01.656976+0000","last_clean":"2024-05-21T03:24:01.656976+0000","last_became_active":"2024-05-21T03:20:32.140792+0000","last_became_peered":"2024-05-21T03:20:32.140792+0000","last_unstale":"2024-05-21T03:24:01.656976+0000","last_undegraded":"2024-05-21T03:24:01.656976+0000","last_fullsized":"2024-05-21T03:24:01.656976+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:35:05.842929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'10","reported_seq":197,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.822342+0000","last_change":"2024-05-21T03:21:14.566886+0000","last_active":"2024-05-21T03:24:01.822342+0000","last_peered":"2024-05-21T03:24:01.822342+0000","last_clean":"2024-05-21T03:24:01.822342+0000","last_became_active":"2024-05-21T03:21:14.559816+0000","last_became_peered":"2024-05-21T03:21:14.559816+0000","last_unstale":"2024-05-21T03:24:01.822342+0000","last_undegraded":"2024-05-21T03:24:01.822342+0000","last_fullsized":"2024-05-21T03:24:01.822342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:34:43.109424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2792,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":176,"num_read_kb":5,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1561,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657398+0000","last_change":"2024-05-21T03:20:32.146674+0000","last_active":"2024-05-21T03:24:01.657398+0000","last_peered":"2024-05-21T03:24:01.657398+0000","last_clean":"2024-05-21T03:24:01.657398+0000","last_became_active":"2024-05-21T03:20:32.146150+0000","last_became_peered":"2024-05-21T03:20:32.146150+0000","last_unstale":"2024-05-21T03:24:01.657398+0000","last_undegraded":"2024-05-21T03:24:01.657398+0000","last_fullsized":"2024-05-21T03:24:01.657398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:39:09.905640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'41","reported_seq":78,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.607615+0000","last_change":"2024-05-21T03:20:29.097613+0000","last_active":"2024-05-21T03:24:01.607615+0000","last_peered":"2024-05-21T03:24:01.607615+0000","last_clean":"2024-05-21T03:24:01.607615+0000","last_became_active":"2024-05-21T03:20:29.097331+0000","last_became_peered":"2024-05-21T03:20:29.097331+0000","last_unstale":"2024-05-21T03:24:01.607615+0000","last_undegraded":"2024-05-21T03:24:01.607615+0000","last_fullsized":"2024-05-21T03:24:01.607615+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:56:14.720420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657109+0000","last_change":"2024-05-21T03:21:14.566406+0000","last_active":"2024-05-21T03:24:01.657109+0000","last_peered":"2024-05-21T03:24:01.657109+0000","last_clean":"2024-05-21T03:24:01.657109+0000","last_became_active":"2024-05-21T03:21:14.563023+0000","last_became_peered":"2024-05-21T03:21:14.563023+0000","last_unstale":"2024-05-21T03:24:01.657109+0000","last_undegraded":"2024-05-21T03:24:01.657109+0000","last_fullsized":"2024-05-21T03:24:01.657109+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:12:53.125136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.660167+0000","last_change":"2024-05-21T03:20:29.103456+0000","last_active":"2024-05-21T03:24:01.660167+0000","last_peered":"2024-05-21T03:24:01.660167+0000","last_clean":"2024-05-21T03:24:01.660167+0000","last_became_active":"2024-05-21T03:20:29.103008+0000","last_became_peered":"2024-05-21T03:20:29.103008+0000","last_unstale":"2024-05-21T03:24:01.660167+0000","last_undegraded":"2024-05-21T03:24:01.660167+0000","last_fullsized":"2024-05-21T03:24:01.660167+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:02:37.951347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657407+0000","last_change":"2024-05-21T03:20:32.146986+0000","last_active":"2024-05-21T03:24:01.657407+0000","last_peered":"2024-05-21T03:24:01.657407+0000","last_clean":"2024-05-21T03:24:01.657407+0000","last_became_active":"2024-05-21T03:20:32.146355+0000","last_became_peered":"2024-05-21T03:20:32.146355+0000","last_unstale":"2024-05-21T03:24:01.657407+0000","last_undegraded":"2024-05-21T03:24:01.657407+0000","last_fullsized":"2024-05-21T03:24:01.657407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:36:29.503099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656717+0000","last_change":"2024-05-21T03:21:14.559658+0000","last_active":"2024-05-21T03:24:01.656717+0000","last_peered":"2024-05-21T03:24:01.656717+0000","last_clean":"2024-05-21T03:24:01.656717+0000","last_became_active":"2024-05-21T03:21:14.559194+0000","last_became_peered":"2024-05-21T03:21:14.559194+0000","last_unstale":"2024-05-21T03:24:01.656717+0000","last_undegraded":"2024-05-21T03:24:01.656717+0000","last_fullsized":"2024-05-21T03:24:01.656717+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:56:31.709545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.389291+0000","last_change":"2024-05-21T03:20:32.137532+0000","last_active":"2024-05-21T03:24:01.389291+0000","last_peered":"2024-05-21T03:24:01.389291+0000","last_clean":"2024-05-21T03:24:01.389291+0000","last_became_active":"2024-05-21T03:20:32.137182+0000","last_became_peered":"2024-05-21T03:20:32.137182+0000","last_unstale":"2024-05-21T03:24:01.389291+0000","last_undegraded":"2024-05-21T03:24:01.389291+0000","last_fullsized":"2024-05-21T03:24:01.389291+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:25:44.207866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.388938+0000","last_change":"2024-05-21T03:20:32.153912+0000","last_active":"2024-05-21T03:24:01.388938+0000","last_peered":"2024-05-21T03:24:01.388938+0000","last_clean":"2024-05-21T03:24:01.388938+0000","last_became_active":"2024-05-21T03:20:32.153541+0000","last_became_peered":"2024-05-21T03:20:32.153541+0000","last_unstale":"2024-05-21T03:24:01.388938+0000","last_undegraded":"2024-05-21T03:24:01.388938+0000","last_fullsized":"2024-05-21T03:24:01.388938+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:02:58.688378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657148+0000","last_change":"2024-05-21T03:20:32.143689+0000","last_active":"2024-05-21T03:24:01.657148+0000","last_peered":"2024-05-21T03:24:01.657148+0000","last_clean":"2024-05-21T03:24:01.657148+0000","last_became_active":"2024-05-21T03:20:32.143399+0000","last_became_peered":"2024-05-21T03:20:32.143399+0000","last_unstale":"2024-05-21T03:24:01.657148+0000","last_undegraded":"2024-05-21T03:24:01.657148+0000","last_fullsized":"2024-05-21T03:24:01.657148+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:27:55.116599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.660308+0000","last_change":"2024-05-21T03:20:32.141355+0000","last_active":"2024-05-21T03:24:01.660308+0000","last_peered":"2024-05-21T03:24:01.660308+0000","last_clean":"2024-05-21T03:24:01.660308+0000","last_became_active":"2024-05-21T03:20:32.140960+0000","last_became_peered":"2024-05-21T03:20:32.140960+0000","last_unstale":"2024-05-21T03:24:01.660308+0000","last_undegraded":"2024-05-21T03:24:01.660308+0000","last_fullsized":"2024-05-21T03:24:01.660308+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:23:50.287864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656934+0000","last_change":"2024-05-21T03:20:32.142145+0000","last_active":"2024-05-21T03:24:01.656934+0000","last_peered":"2024-05-21T03:24:01.656934+0000","last_clean":"2024-05-21T03:24:01.656934+0000","last_became_active":"2024-05-21T03:20:32.141802+0000","last_became_peered":"2024-05-21T03:20:32.141802+0000","last_unstale":"2024-05-21T03:24:01.656934+0000","last_undegraded":"2024-05-21T03:24:01.656934+0000","last_fullsized":"2024-05-21T03:24:01.656934+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:26:28.707616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.389203+0000","last_change":"2024-05-21T03:20:32.141606+0000","last_active":"2024-05-21T03:24:01.389203+0000","last_peered":"2024-05-21T03:24:01.389203+0000","last_clean":"2024-05-21T03:24:01.389203+0000","last_became_active":"2024-05-21T03:20:32.141258+0000","last_became_peered":"2024-05-21T03:20:32.141258+0000","last_unstale":"2024-05-21T03:24:01.389203+0000","last_undegraded":"2024-05-21T03:24:01.389203+0000","last_fullsized":"2024-05-21T03:24:01.389203+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:22.372723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491954+0000","last_change":"2024-05-21T03:20:32.151621+0000","last_active":"2024-05-21T03:24:01.491954+0000","last_peered":"2024-05-21T03:24:01.491954+0000","last_clean":"2024-05-21T03:24:01.491954+0000","last_became_active":"2024-05-21T03:20:32.151260+0000","last_became_peered":"2024-05-21T03:20:32.151260+0000","last_unstale":"2024-05-21T03:24:01.491954+0000","last_undegraded":"2024-05-21T03:24:01.491954+0000","last_fullsized":"2024-05-21T03:24:01.491954+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:48:59.751179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.499163+0000","last_change":"2024-05-21T03:20:32.153843+0000","last_active":"2024-05-21T03:24:01.499163+0000","last_peered":"2024-05-21T03:24:01.499163+0000","last_clean":"2024-05-21T03:24:01.499163+0000","last_became_active":"2024-05-21T03:20:32.153418+0000","last_became_peered":"2024-05-21T03:20:32.153418+0000","last_unstale":"2024-05-21T03:24:01.499163+0000","last_undegraded":"2024-05-21T03:24:01.499163+0000","last_fullsized":"2024-05-21T03:24:01.499163+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:26:29.663966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491312+0000","last_change":"2024-05-21T03:20:32.153942+0000","last_active":"2024-05-21T03:24:01.491312+0000","last_peered":"2024-05-21T03:24:01.491312+0000","last_clean":"2024-05-21T03:24:01.491312+0000","last_became_active":"2024-05-21T03:20:32.153566+0000","last_became_peered":"2024-05-21T03:20:32.153566+0000","last_unstale":"2024-05-21T03:24:01.491312+0000","last_undegraded":"2024-05-21T03:24:01.491312+0000","last_fullsized":"2024-05-21T03:24:01.491312+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:06.899039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659626+0000","last_change":"2024-05-21T03:20:32.153243+0000","last_active":"2024-05-21T03:24:01.659626+0000","last_peered":"2024-05-21T03:24:01.659626+0000","last_clean":"2024-05-21T03:24:01.659626+0000","last_became_active":"2024-05-21T03:20:32.152386+0000","last_became_peered":"2024-05-21T03:20:32.152386+0000","last_unstale":"2024-05-21T03:24:01.659626+0000","last_undegraded":"2024-05-21T03:24:01.659626+0000","last_fullsized":"2024-05-21T03:24:01.659626+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:25:28.935183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659802+0000","last_change":"2024-05-21T03:20:32.142536+0000","last_active":"2024-05-21T03:24:01.659802+0000","last_peered":"2024-05-21T03:24:01.659802+0000","last_clean":"2024-05-21T03:24:01.659802+0000","last_became_active":"2024-05-21T03:20:32.141793+0000","last_became_peered":"2024-05-21T03:20:32.141793+0000","last_unstale":"2024-05-21T03:24:01.659802+0000","last_undegraded":"2024-05-21T03:24:01.659802+0000","last_fullsized":"2024-05-21T03:24:01.659802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:21:43.547103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656778+0000","last_change":"2024-05-21T03:20:32.140368+0000","last_active":"2024-05-21T03:24:01.656778+0000","last_peered":"2024-05-21T03:24:01.656778+0000","last_clean":"2024-05-21T03:24:01.656778+0000","last_became_active":"2024-05-21T03:20:32.139747+0000","last_became_peered":"2024-05-21T03:20:32.139747+0000","last_unstale":"2024-05-21T03:24:01.656778+0000","last_undegraded":"2024-05-21T03:24:01.656778+0000","last_fullsized":"2024-05-21T03:24:01.656778+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:05:44.851191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":601180,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2041,"num_read_kb":1147,"num_write":378,"num_write_kb":1469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1561,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1803540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":306,"ondisk_log_size":306,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645369,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":917504,"data_stored":444066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841598,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28736,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716416,"statfs":{"total":95995035648,"available":95965609984,"internally_reserved":0,"allocated":1761280,"data_stored":1232470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005124,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28328,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716824,"statfs":{"total":95995035648,"available":95966027776,"internally_reserved":0,"allocated":1339392,"data_stored":835707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201351,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28320,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716832,"statfs":{"total":95995035648,"available":95966035968,"internally_reserved":0,"allocated":1331200,"data_stored":835968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397582,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32680,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712472,"statfs":{"total":95995035648,"available":95961571328,"internally_reserved":0,"allocated":1536000,"data_stored":1036725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561106,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":433638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574956,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":510401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738481,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":445052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411431,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":508528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149793,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28448,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716704,"statfs":{"total":95995035648,"available":95965904896,"internally_reserved":0,"allocated":1466368,"data_stored":966615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953565,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":933888,"data_stored":444494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757336,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37160,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707992,"statfs":{"total":95995035648,"available":95956983808,"internally_reserved":0,"allocated":1683456,"data_stored":1166157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":67276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":134420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":200733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T03:24:06.828 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-21T03:24:06.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:06 smithi027 ceph-mon[32179]: Adjusting osd_memory_target on smithi027 to 176.1M 2024-05-21T03:24:06.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:06 smithi027 ceph-mon[32179]: Unable to set osd_memory_target on smithi027 to 184745164: error parsing value: Value '184745164' is below minimum 939524096 2024-05-21T03:24:06.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:06 smithi027 ceph-mon[32179]: Adjusting osd_memory_target on smithi165 to 176.2M 2024-05-21T03:24:06.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:06 smithi027 ceph-mon[32179]: Unable to set osd_memory_target on smithi165 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-05-21T03:24:06.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:06 smithi027 ceph-mon[32179]: Updating smithi027:/etc/ceph/ceph.conf 2024-05-21T03:24:06.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:06 smithi027 ceph-mon[32179]: Updating smithi039:/etc/ceph/ceph.conf 2024-05-21T03:24:06.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:06 smithi027 ceph-mon[32179]: Updating smithi165:/etc/ceph/ceph.conf 2024-05-21T03:24:06.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:06 smithi027 ceph-mon[32179]: Updating smithi027:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T03:24:06.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:06 smithi027 ceph-mon[32179]: Updating smithi039:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T03:24:06.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:06 smithi027 ceph-mon[32179]: Updating smithi165:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T03:24:06.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:06 smithi027 ceph-mon[32179]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:24:06.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:06 smithi027 ceph-mon[32179]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-05-21T03:24:06.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:06 smithi027 ceph-mon[32179]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-05-21T03:24:06.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:06 smithi027 ceph-mon[32179]: Updating smithi039:/etc/ceph/ceph.client.admin.keyring 2024-05-21T03:24:06.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:06 smithi027 ceph-mon[32179]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-05-21T03:24:06.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:06 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:06.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:06 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:06.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:06 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:06.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:06 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:06.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:06 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:06.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:06 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:06.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:06 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:06.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:06 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:24:06.929 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:06 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:24:06.929 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:06 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:24:06.929 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:06 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:07.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:06 smithi039 ceph-mon[36429]: Adjusting osd_memory_target on smithi027 to 176.1M 2024-05-21T03:24:07.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:06 smithi039 ceph-mon[36429]: Unable to set osd_memory_target on smithi027 to 184745164: error parsing value: Value '184745164' is below minimum 939524096 2024-05-21T03:24:07.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:06 smithi039 ceph-mon[36429]: Adjusting osd_memory_target on smithi165 to 176.2M 2024-05-21T03:24:07.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:06 smithi039 ceph-mon[36429]: Unable to set osd_memory_target on smithi165 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-05-21T03:24:07.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:06 smithi039 ceph-mon[36429]: Updating smithi027:/etc/ceph/ceph.conf 2024-05-21T03:24:07.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:06 smithi039 ceph-mon[36429]: Updating smithi039:/etc/ceph/ceph.conf 2024-05-21T03:24:07.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:06 smithi039 ceph-mon[36429]: Updating smithi165:/etc/ceph/ceph.conf 2024-05-21T03:24:07.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:06 smithi039 ceph-mon[36429]: Updating smithi027:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T03:24:07.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:06 smithi039 ceph-mon[36429]: Updating smithi039:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T03:24:07.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:06 smithi039 ceph-mon[36429]: Updating smithi165:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T03:24:07.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:06 smithi039 ceph-mon[36429]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:24:07.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:06 smithi039 ceph-mon[36429]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-05-21T03:24:07.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:06 smithi039 ceph-mon[36429]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-05-21T03:24:07.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:06 smithi039 ceph-mon[36429]: Updating smithi039:/etc/ceph/ceph.client.admin.keyring 2024-05-21T03:24:07.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:06 smithi039 ceph-mon[36429]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-05-21T03:24:07.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:06 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:07.021 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:06 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:07.021 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:06 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:07.021 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:06 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:07.021 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:06 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:07.021 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:06 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:07.021 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:06 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:07.021 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:06 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:24:07.021 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:06 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:24:07.021 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:06 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:24:07.021 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:06 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:07.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:06 smithi165 ceph-mon[36269]: Adjusting osd_memory_target on smithi027 to 176.1M 2024-05-21T03:24:07.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:06 smithi165 ceph-mon[36269]: Unable to set osd_memory_target on smithi027 to 184745164: error parsing value: Value '184745164' is below minimum 939524096 2024-05-21T03:24:07.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:06 smithi165 ceph-mon[36269]: Adjusting osd_memory_target on smithi165 to 176.2M 2024-05-21T03:24:07.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:06 smithi165 ceph-mon[36269]: Unable to set osd_memory_target on smithi165 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-05-21T03:24:07.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:06 smithi165 ceph-mon[36269]: Updating smithi027:/etc/ceph/ceph.conf 2024-05-21T03:24:07.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:06 smithi165 ceph-mon[36269]: Updating smithi039:/etc/ceph/ceph.conf 2024-05-21T03:24:07.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:06 smithi165 ceph-mon[36269]: Updating smithi165:/etc/ceph/ceph.conf 2024-05-21T03:24:07.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:06 smithi165 ceph-mon[36269]: Updating smithi027:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T03:24:07.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:06 smithi165 ceph-mon[36269]: Updating smithi039:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T03:24:07.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:06 smithi165 ceph-mon[36269]: Updating smithi165:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T03:24:07.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:06 smithi165 ceph-mon[36269]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:24:07.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:06 smithi165 ceph-mon[36269]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-05-21T03:24:07.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:06 smithi165 ceph-mon[36269]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-05-21T03:24:07.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:06 smithi165 ceph-mon[36269]: Updating smithi039:/etc/ceph/ceph.client.admin.keyring 2024-05-21T03:24:07.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:06 smithi165 ceph-mon[36269]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-05-21T03:24:07.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:06 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:07.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:06 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:07.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:06 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:07.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:06 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:07.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:06 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:07.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:06 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:07.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:06 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:07.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:06 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:24:07.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:06 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:24:07.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:06 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:24:07.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:06 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:24:07.078 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:24:07.868 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:07 smithi027 ceph-mon[32179]: Updating smithi165:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T03:24:07.869 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:07 smithi027 ceph-mon[32179]: Updating smithi039:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T03:24:07.869 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:07 smithi027 ceph-mon[32179]: Updating smithi027:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T03:24:07.869 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:07 smithi027 ceph-mon[32179]: from='client.25007 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:24:08.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:07 smithi039 ceph-mon[36429]: Updating smithi165:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T03:24:08.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:07 smithi039 ceph-mon[36429]: Updating smithi039:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T03:24:08.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:07 smithi039 ceph-mon[36429]: Updating smithi027:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T03:24:08.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:07 smithi039 ceph-mon[36429]: from='client.25007 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:24:08.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:07 smithi165 ceph-mon[36269]: Updating smithi165:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T03:24:08.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:07 smithi165 ceph-mon[36269]: Updating smithi039:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T03:24:08.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:07 smithi165 ceph-mon[36269]: Updating smithi027:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T03:24:08.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:07 smithi165 ceph-mon[36269]: from='client.25007 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:24:08.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.629+0000 7f76d9992700 1 -- 172.21.15.27:0/524254998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76d41037f0 msgr2=0x7f76d4103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:08.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.629+0000 7f76d9992700 1 --2- 172.21.15.27:0/524254998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76d41037f0 0x7f76d4103bd0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f76c40099c0 tx=0x7f76c4009cd0 comp rx=0 tx=0).stop 2024-05-21T03:24:08.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.629+0000 7f76d9992700 1 -- 172.21.15.27:0/524254998 shutdown_connections 2024-05-21T03:24:08.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.629+0000 7f76d9992700 1 --2- 172.21.15.27:0/524254998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76d41088f0 0x7f76d4110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:08.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.629+0000 7f76d9992700 1 --2- 172.21.15.27:0/524254998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76d41037f0 0x7f76d4103bd0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:08.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.629+0000 7f76d9992700 1 --2- 172.21.15.27:0/524254998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76d41041a0 0x7f76d41081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:08.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.629+0000 7f76d9992700 1 -- 172.21.15.27:0/524254998 >> 172.21.15.27:0/524254998 conn(0x7f76d407a6a0 msgr2=0x7f76d407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:08.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.629+0000 7f76d9992700 1 -- 172.21.15.27:0/524254998 shutdown_connections 2024-05-21T03:24:08.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.629+0000 7f76d9992700 1 -- 172.21.15.27:0/524254998 wait complete. 2024-05-21T03:24:08.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.629+0000 7f76d9992700 1 Processor -- start 2024-05-21T03:24:08.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.630+0000 7f76d9992700 1 -- start start 2024-05-21T03:24:08.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.630+0000 7f76d9992700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76d41037f0 0x7f76d419bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:08.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.630+0000 7f76d9992700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76d41041a0 0x7f76d419c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:08.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.630+0000 7f76d9992700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76d41088f0 0x7f76d41a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:08.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.630+0000 7f76d9992700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f76d4113760 con 0x7f76d41037f0 2024-05-21T03:24:08.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.630+0000 7f76d9992700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f76d41135e0 con 0x7f76d41041a0 2024-05-21T03:24:08.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.630+0000 7f76d9992700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f76d41138e0 con 0x7f76d41088f0 2024-05-21T03:24:08.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.630+0000 7f76d2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76d41037f0 0x7f76d419bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:08.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.630+0000 7f76d2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76d41037f0 0x7f76d419bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55862/0 (socket says 172.21.15.27:55862) 2024-05-21T03:24:08.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.630+0000 7f76d2ffd700 1 -- 172.21.15.27:0/1928697785 learned_addr learned my addr 172.21.15.27:0/1928697785 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:08.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.630+0000 7f76d37fe700 1 --2- 172.21.15.27:0/1928697785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76d41088f0 0x7f76d41a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:08.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.630+0000 7f76d27fc700 1 --2- 172.21.15.27:0/1928697785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76d41041a0 0x7f76d419c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:08.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.631+0000 7f76d2ffd700 1 -- 172.21.15.27:0/1928697785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76d41088f0 msgr2=0x7f76d41a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:08.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.631+0000 7f76d2ffd700 1 --2- 172.21.15.27:0/1928697785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76d41088f0 0x7f76d41a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:08.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.631+0000 7f76d2ffd700 1 -- 172.21.15.27:0/1928697785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76d41041a0 msgr2=0x7f76d419c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:08.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.631+0000 7f76d2ffd700 1 --2- 172.21.15.27:0/1928697785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76d41041a0 0x7f76d419c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:08.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.631+0000 7f76d2ffd700 1 -- 172.21.15.27:0/1928697785 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d41a0e00 con 0x7f76d41037f0 2024-05-21T03:24:08.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.631+0000 7f76d2ffd700 1 --2- 172.21.15.27:0/1928697785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76d41037f0 0x7f76d419bdf0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f76c40098e0 tx=0x7f76c40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:08.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.631+0000 7f76d8990700 1 -- 172.21.15.27:0/1928697785 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c4027070 con 0x7f76d41037f0 2024-05-21T03:24:08.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.631+0000 7f76d9992700 1 -- 172.21.15.27:0/1928697785 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d41a1090 con 0x7f76d41037f0 2024-05-21T03:24:08.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.631+0000 7f76d8990700 1 -- 172.21.15.27:0/1928697785 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76c4003f20 con 0x7f76d41037f0 2024-05-21T03:24:08.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.631+0000 7f76d8990700 1 -- 172.21.15.27:0/1928697785 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c4022d70 con 0x7f76d41037f0 2024-05-21T03:24:08.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.631+0000 7f76d9992700 1 -- 172.21.15.27:0/1928697785 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d41a8970 con 0x7f76d41037f0 2024-05-21T03:24:08.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.632+0000 7f76d9992700 1 -- 172.21.15.27:0/1928697785 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76d4104c80 con 0x7f76d41037f0 2024-05-21T03:24:08.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.634+0000 7f76d8990700 1 -- 172.21.15.27:0/1928697785 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f76c402a070 con 0x7f76d41037f0 2024-05-21T03:24:08.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.634+0000 7f76d8990700 1 --2- 172.21.15.27:0/1928697785 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f76bc0b2a80 0x7f76bc0b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:08.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.634+0000 7f76d27fc700 1 --2- 172.21.15.27:0/1928697785 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f76bc0b2a80 0x7f76bc0b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:08.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.634+0000 7f76d8990700 1 -- 172.21.15.27:0/1928697785 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f76c40e3ea0 con 0x7f76d41037f0 2024-05-21T03:24:08.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.635+0000 7f76d27fc700 1 --2- 172.21.15.27:0/1928697785 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f76bc0b2a80 0x7f76bc0b4f40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f76d4109f40 tx=0x7f76c800a300 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:08.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.638+0000 7f76d8990700 1 -- 172.21.15.27:0/1928697785 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f76c4031020 con 0x7f76d41037f0 2024-05-21T03:24:08.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.853+0000 7f76d9992700 1 -- 172.21.15.27:0/1928697785 --> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f76d410f7d0 con 0x7f76bc0b2a80 2024-05-21T03:24:08.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.860+0000 7f76d8990700 1 -- 172.21.15.27:0/1928697785 <== mgr.24913 v2:172.21.15.27:6800/1477999924 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257796 (secure 0 0 0) 0x7f76d410f7d0 con 0x7f76bc0b2a80 2024-05-21T03:24:08.863 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:24:08.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.863+0000 7f76d9992700 1 -- 172.21.15.27:0/1928697785 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f76bc0b2a80 msgr2=0x7f76bc0b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:08.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.863+0000 7f76d9992700 1 --2- 172.21.15.27:0/1928697785 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f76bc0b2a80 0x7f76bc0b4f40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f76d4109f40 tx=0x7f76c800a300 comp rx=0 tx=0).stop 2024-05-21T03:24:08.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.864+0000 7f76d9992700 1 -- 172.21.15.27:0/1928697785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76d41037f0 msgr2=0x7f76d419bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:08.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.864+0000 7f76d9992700 1 --2- 172.21.15.27:0/1928697785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76d41037f0 0x7f76d419bdf0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f76c40098e0 tx=0x7f76c40057c0 comp rx=0 tx=0).stop 2024-05-21T03:24:08.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.864+0000 7f76d9992700 1 -- 172.21.15.27:0/1928697785 shutdown_connections 2024-05-21T03:24:08.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.864+0000 7f76d9992700 1 --2- 172.21.15.27:0/1928697785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76d41088f0 0x7f76d41a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:08.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.864+0000 7f76d9992700 1 --2- 172.21.15.27:0/1928697785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76d41037f0 0x7f76d419bdf0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:08.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.864+0000 7f76d9992700 1 --2- 172.21.15.27:0/1928697785 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f76bc0b2a80 0x7f76bc0b4f40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:08.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.864+0000 7f76d9992700 1 --2- 172.21.15.27:0/1928697785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76d41041a0 0x7f76d419c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:08.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.864+0000 7f76d9992700 1 -- 172.21.15.27:0/1928697785 >> 172.21.15.27:0/1928697785 conn(0x7f76d407a6a0 msgr2=0x7f76d4102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:08.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.864+0000 7f76d9992700 1 -- 172.21.15.27:0/1928697785 shutdown_connections 2024-05-21T03:24:08.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:08.864+0000 7f76d9992700 1 -- 172.21.15.27:0/1928697785 wait complete. 2024-05-21T03:24:08.866 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-05-21T03:24:09.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:08 smithi039 ceph-mon[36429]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:24:09.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:08 smithi165 ceph-mon[36269]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:24:09.143 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:08 smithi027 ceph-mon[32179]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T03:24:09.497 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-05-21T03:24:07.402675+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191782,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2122,"num_read_kb":1216,"num_write":503,"num_write_kb":3621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2953518,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":900,"ondisk_log_size":900,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364156,"kb_used_data":14672,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124577668,"statfs":{"total":1151940427776,"available":1151567532032,"internally_reserved":0,"allocated":15024128,"data_stored":9056453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19139,"internal_metadata":357414205},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.987525"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656903+0000","last_change":"2024-05-21T03:20:32.149961+0000","last_active":"2024-05-21T03:24:01.656903+0000","last_peered":"2024-05-21T03:24:01.656903+0000","last_clean":"2024-05-21T03:24:01.656903+0000","last_became_active":"2024-05-21T03:20:32.149552+0000","last_became_peered":"2024-05-21T03:20:32.149552+0000","last_unstale":"2024-05-21T03:24:01.656903+0000","last_undegraded":"2024-05-21T03:24:01.656903+0000","last_fullsized":"2024-05-21T03:24:01.656903+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:20:38.543551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656992+0000","last_change":"2024-05-21T03:20:32.146413+0000","last_active":"2024-05-21T03:24:01.656992+0000","last_peered":"2024-05-21T03:24:01.656992+0000","last_clean":"2024-05-21T03:24:01.656992+0000","last_became_active":"2024-05-21T03:20:32.146006+0000","last_became_peered":"2024-05-21T03:20:32.146006+0000","last_unstale":"2024-05-21T03:24:01.656992+0000","last_undegraded":"2024-05-21T03:24:01.656992+0000","last_fullsized":"2024-05-21T03:24:01.656992+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:05:12.888550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656984+0000","last_change":"2024-05-21T03:20:32.153654+0000","last_active":"2024-05-21T03:24:01.656984+0000","last_peered":"2024-05-21T03:24:01.656984+0000","last_clean":"2024-05-21T03:24:01.656984+0000","last_became_active":"2024-05-21T03:20:32.153296+0000","last_became_peered":"2024-05-21T03:20:32.153296+0000","last_unstale":"2024-05-21T03:24:01.656984+0000","last_undegraded":"2024-05-21T03:24:01.656984+0000","last_fullsized":"2024-05-21T03:24:01.656984+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:16:09.423236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.389156+0000","last_change":"2024-05-21T03:20:32.138672+0000","last_active":"2024-05-21T03:24:01.389156+0000","last_peered":"2024-05-21T03:24:01.389156+0000","last_clean":"2024-05-21T03:24:01.389156+0000","last_became_active":"2024-05-21T03:20:32.138163+0000","last_became_peered":"2024-05-21T03:20:32.138163+0000","last_unstale":"2024-05-21T03:24:01.389156+0000","last_undegraded":"2024-05-21T03:24:01.389156+0000","last_fullsized":"2024-05-21T03:24:01.389156+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:26:55.012897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491536+0000","last_change":"2024-05-21T03:20:32.153950+0000","last_active":"2024-05-21T03:24:01.491536+0000","last_peered":"2024-05-21T03:24:01.491536+0000","last_clean":"2024-05-21T03:24:01.491536+0000","last_became_active":"2024-05-21T03:20:32.153567+0000","last_became_peered":"2024-05-21T03:20:32.153567+0000","last_unstale":"2024-05-21T03:24:01.491536+0000","last_undegraded":"2024-05-21T03:24:01.491536+0000","last_fullsized":"2024-05-21T03:24:01.491536+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:28:34.621193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.502688+0000","last_change":"2024-05-21T03:20:32.140038+0000","last_active":"2024-05-21T03:24:01.502688+0000","last_peered":"2024-05-21T03:24:01.502688+0000","last_clean":"2024-05-21T03:24:01.502688+0000","last_became_active":"2024-05-21T03:20:32.139690+0000","last_became_peered":"2024-05-21T03:20:32.139690+0000","last_unstale":"2024-05-21T03:24:01.502688+0000","last_undegraded":"2024-05-21T03:24:01.502688+0000","last_fullsized":"2024-05-21T03:24:01.502688+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:31:24.270755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656862+0000","last_change":"2024-05-21T03:20:32.143735+0000","last_active":"2024-05-21T03:24:01.656862+0000","last_peered":"2024-05-21T03:24:01.656862+0000","last_clean":"2024-05-21T03:24:01.656862+0000","last_became_active":"2024-05-21T03:20:32.143397+0000","last_became_peered":"2024-05-21T03:20:32.143397+0000","last_unstale":"2024-05-21T03:24:01.656862+0000","last_undegraded":"2024-05-21T03:24:01.656862+0000","last_fullsized":"2024-05-21T03:24:01.656862+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:40:01.951608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491334+0000","last_change":"2024-05-21T03:20:32.156999+0000","last_active":"2024-05-21T03:24:01.491334+0000","last_peered":"2024-05-21T03:24:01.491334+0000","last_clean":"2024-05-21T03:24:01.491334+0000","last_became_active":"2024-05-21T03:20:32.156677+0000","last_became_peered":"2024-05-21T03:20:32.156677+0000","last_unstale":"2024-05-21T03:24:01.491334+0000","last_undegraded":"2024-05-21T03:24:01.491334+0000","last_fullsized":"2024-05-21T03:24:01.491334+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:33:24.225667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.660261+0000","last_change":"2024-05-21T03:20:32.145039+0000","last_active":"2024-05-21T03:24:01.660261+0000","last_peered":"2024-05-21T03:24:01.660261+0000","last_clean":"2024-05-21T03:24:01.660261+0000","last_became_active":"2024-05-21T03:20:32.140926+0000","last_became_peered":"2024-05-21T03:20:32.140926+0000","last_unstale":"2024-05-21T03:24:01.660261+0000","last_undegraded":"2024-05-21T03:24:01.660261+0000","last_fullsized":"2024-05-21T03:24:01.660261+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:11:27.049426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.503068+0000","last_change":"2024-05-21T03:20:32.145840+0000","last_active":"2024-05-21T03:24:01.503068+0000","last_peered":"2024-05-21T03:24:01.503068+0000","last_clean":"2024-05-21T03:24:01.503068+0000","last_became_active":"2024-05-21T03:20:32.145291+0000","last_became_peered":"2024-05-21T03:20:32.145291+0000","last_unstale":"2024-05-21T03:24:01.503068+0000","last_undegraded":"2024-05-21T03:24:01.503068+0000","last_fullsized":"2024-05-21T03:24:01.503068+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:22:27.696588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657241+0000","last_change":"2024-05-21T03:20:32.142522+0000","last_active":"2024-05-21T03:24:01.657241+0000","last_peered":"2024-05-21T03:24:01.657241+0000","last_clean":"2024-05-21T03:24:01.657241+0000","last_became_active":"2024-05-21T03:20:32.142165+0000","last_became_peered":"2024-05-21T03:20:32.142165+0000","last_unstale":"2024-05-21T03:24:01.657241+0000","last_undegraded":"2024-05-21T03:24:01.657241+0000","last_fullsized":"2024-05-21T03:24:01.657241+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:58:39.085834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.389012+0000","last_change":"2024-05-21T03:20:32.142526+0000","last_active":"2024-05-21T03:24:01.389012+0000","last_peered":"2024-05-21T03:24:01.389012+0000","last_clean":"2024-05-21T03:24:01.389012+0000","last_became_active":"2024-05-21T03:20:32.141143+0000","last_became_peered":"2024-05-21T03:20:32.141143+0000","last_unstale":"2024-05-21T03:24:01.389012+0000","last_undegraded":"2024-05-21T03:24:01.389012+0000","last_fullsized":"2024-05-21T03:24:01.389012+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:35:11.336796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659855+0000","last_change":"2024-05-21T03:20:32.135631+0000","last_active":"2024-05-21T03:24:01.659855+0000","last_peered":"2024-05-21T03:24:01.659855+0000","last_clean":"2024-05-21T03:24:01.659855+0000","last_became_active":"2024-05-21T03:20:32.135302+0000","last_became_peered":"2024-05-21T03:20:32.135302+0000","last_unstale":"2024-05-21T03:24:01.659855+0000","last_undegraded":"2024-05-21T03:24:01.659855+0000","last_fullsized":"2024-05-21T03:24:01.659855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:05:55.000690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656928+0000","last_change":"2024-05-21T03:20:32.146580+0000","last_active":"2024-05-21T03:24:01.656928+0000","last_peered":"2024-05-21T03:24:01.656928+0000","last_clean":"2024-05-21T03:24:01.656928+0000","last_became_active":"2024-05-21T03:20:32.145803+0000","last_became_peered":"2024-05-21T03:20:32.145803+0000","last_unstale":"2024-05-21T03:24:01.656928+0000","last_undegraded":"2024-05-21T03:24:01.656928+0000","last_fullsized":"2024-05-21T03:24:01.656928+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:34:09.108799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656841+0000","last_change":"2024-05-21T03:21:14.559498+0000","last_active":"2024-05-21T03:24:01.656841+0000","last_peered":"2024-05-21T03:24:01.656841+0000","last_clean":"2024-05-21T03:24:01.656841+0000","last_became_active":"2024-05-21T03:21:14.559039+0000","last_became_peered":"2024-05-21T03:21:14.559039+0000","last_unstale":"2024-05-21T03:24:01.656841+0000","last_undegraded":"2024-05-21T03:24:01.656841+0000","last_fullsized":"2024-05-21T03:24:01.656841+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:04:36.676857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659265+0000","last_change":"2024-05-21T03:20:32.148944+0000","last_active":"2024-05-21T03:24:01.659265+0000","last_peered":"2024-05-21T03:24:01.659265+0000","last_clean":"2024-05-21T03:24:01.659265+0000","last_became_active":"2024-05-21T03:20:32.148502+0000","last_became_peered":"2024-05-21T03:20:32.148502+0000","last_unstale":"2024-05-21T03:24:01.659265+0000","last_undegraded":"2024-05-21T03:24:01.659265+0000","last_fullsized":"2024-05-21T03:24:01.659265+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:11:00.178105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657063+0000","last_change":"2024-05-21T03:20:32.150609+0000","last_active":"2024-05-21T03:24:01.657063+0000","last_peered":"2024-05-21T03:24:01.657063+0000","last_clean":"2024-05-21T03:24:01.657063+0000","last_became_active":"2024-05-21T03:20:32.150314+0000","last_became_peered":"2024-05-21T03:20:32.150314+0000","last_unstale":"2024-05-21T03:24:01.657063+0000","last_undegraded":"2024-05-21T03:24:01.657063+0000","last_fullsized":"2024-05-21T03:24:01.657063+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:34:21.869304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.387785+0000","last_change":"2024-05-21T03:20:32.142472+0000","last_active":"2024-05-21T03:24:01.387785+0000","last_peered":"2024-05-21T03:24:01.387785+0000","last_clean":"2024-05-21T03:24:01.387785+0000","last_became_active":"2024-05-21T03:20:32.142161+0000","last_became_peered":"2024-05-21T03:20:32.142161+0000","last_unstale":"2024-05-21T03:24:01.387785+0000","last_undegraded":"2024-05-21T03:24:01.387785+0000","last_fullsized":"2024-05-21T03:24:01.387785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:26:57.281655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.389229+0000","last_change":"2024-05-21T03:20:32.153987+0000","last_active":"2024-05-21T03:24:01.389229+0000","last_peered":"2024-05-21T03:24:01.389229+0000","last_clean":"2024-05-21T03:24:01.389229+0000","last_became_active":"2024-05-21T03:20:32.153617+0000","last_became_peered":"2024-05-21T03:20:32.153617+0000","last_unstale":"2024-05-21T03:24:01.389229+0000","last_undegraded":"2024-05-21T03:24:01.389229+0000","last_fullsized":"2024-05-21T03:24:01.389229+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:57:07.341758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'15","reported_seq":236,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656514+0000","last_change":"2024-05-21T03:20:29.099928+0000","last_active":"2024-05-21T03:24:01.656514+0000","last_peered":"2024-05-21T03:24:01.656514+0000","last_clean":"2024-05-21T03:24:01.656514+0000","last_became_active":"2024-05-21T03:20:29.099592+0000","last_became_peered":"2024-05-21T03:20:29.099592+0000","last_unstale":"2024-05-21T03:24:01.656514+0000","last_undegraded":"2024-05-21T03:24:01.656514+0000","last_fullsized":"2024-05-21T03:24:01.656514+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:35:34.843325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":193,"num_read_kb":193,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491831+0000","last_change":"2024-05-21T03:20:32.154962+0000","last_active":"2024-05-21T03:24:01.491831+0000","last_peered":"2024-05-21T03:24:01.491831+0000","last_clean":"2024-05-21T03:24:01.491831+0000","last_became_active":"2024-05-21T03:20:32.154620+0000","last_became_peered":"2024-05-21T03:20:32.154620+0000","last_unstale":"2024-05-21T03:24:01.491831+0000","last_undegraded":"2024-05-21T03:24:01.491831+0000","last_fullsized":"2024-05-21T03:24:01.491831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:56:21.553272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.492006+0000","last_change":"2024-05-21T03:20:29.097312+0000","last_active":"2024-05-21T03:24:01.492006+0000","last_peered":"2024-05-21T03:24:01.492006+0000","last_clean":"2024-05-21T03:24:01.492006+0000","last_became_active":"2024-05-21T03:20:29.096977+0000","last_became_peered":"2024-05-21T03:20:29.096977+0000","last_unstale":"2024-05-21T03:24:01.492006+0000","last_undegraded":"2024-05-21T03:24:01.492006+0000","last_fullsized":"2024-05-21T03:24:01.492006+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:28:45.467446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657332+0000","last_change":"2024-05-21T03:20:32.150035+0000","last_active":"2024-05-21T03:24:01.657332+0000","last_peered":"2024-05-21T03:24:01.657332+0000","last_clean":"2024-05-21T03:24:01.657332+0000","last_became_active":"2024-05-21T03:20:32.149688+0000","last_became_peered":"2024-05-21T03:20:32.149688+0000","last_unstale":"2024-05-21T03:24:01.657332+0000","last_undegraded":"2024-05-21T03:24:01.657332+0000","last_fullsized":"2024-05-21T03:24:01.657332+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:27:00.410770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656922+0000","last_change":"2024-05-21T03:21:14.572832+0000","last_active":"2024-05-21T03:24:01.656922+0000","last_peered":"2024-05-21T03:24:01.656922+0000","last_clean":"2024-05-21T03:24:01.656922+0000","last_became_active":"2024-05-21T03:21:14.559381+0000","last_became_peered":"2024-05-21T03:21:14.559381+0000","last_unstale":"2024-05-21T03:24:01.656922+0000","last_undegraded":"2024-05-21T03:24:01.656922+0000","last_fullsized":"2024-05-21T03:24:01.656922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:02:53.004031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659270+0000","last_change":"2024-05-21T03:20:32.150301+0000","last_active":"2024-05-21T03:24:01.659270+0000","last_peered":"2024-05-21T03:24:01.659270+0000","last_clean":"2024-05-21T03:24:01.659270+0000","last_became_active":"2024-05-21T03:20:32.148475+0000","last_became_peered":"2024-05-21T03:20:32.148475+0000","last_unstale":"2024-05-21T03:24:01.659270+0000","last_undegraded":"2024-05-21T03:24:01.659270+0000","last_fullsized":"2024-05-21T03:24:01.659270+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:59:30.339660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491705+0000","last_change":"2024-05-21T03:20:29.098861+0000","last_active":"2024-05-21T03:24:01.491705+0000","last_peered":"2024-05-21T03:24:01.491705+0000","last_clean":"2024-05-21T03:24:01.491705+0000","last_became_active":"2024-05-21T03:20:29.097874+0000","last_became_peered":"2024-05-21T03:20:29.097874+0000","last_unstale":"2024-05-21T03:24:01.491705+0000","last_undegraded":"2024-05-21T03:24:01.491705+0000","last_fullsized":"2024-05-21T03:24:01.491705+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:25:36.227339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.499470+0000","last_change":"2024-05-21T03:20:32.142820+0000","last_active":"2024-05-21T03:24:01.499470+0000","last_peered":"2024-05-21T03:24:01.499470+0000","last_clean":"2024-05-21T03:24:01.499470+0000","last_became_active":"2024-05-21T03:20:32.142396+0000","last_became_peered":"2024-05-21T03:20:32.142396+0000","last_unstale":"2024-05-21T03:24:01.499470+0000","last_undegraded":"2024-05-21T03:24:01.499470+0000","last_fullsized":"2024-05-21T03:24:01.499470+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:30:14.452389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":228,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.499417+0000","last_change":"2024-05-21T03:20:29.103153+0000","last_active":"2024-05-21T03:24:01.499417+0000","last_peered":"2024-05-21T03:24:01.499417+0000","last_clean":"2024-05-21T03:24:01.499417+0000","last_became_active":"2024-05-21T03:20:29.102819+0000","last_became_peered":"2024-05-21T03:20:29.102819+0000","last_unstale":"2024-05-21T03:24:01.499417+0000","last_undegraded":"2024-05-21T03:24:01.499417+0000","last_fullsized":"2024-05-21T03:24:01.499417+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:47:49.843926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":194,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491481+0000","last_change":"2024-05-21T03:20:32.142263+0000","last_active":"2024-05-21T03:24:01.491481+0000","last_peered":"2024-05-21T03:24:01.491481+0000","last_clean":"2024-05-21T03:24:01.491481+0000","last_became_active":"2024-05-21T03:20:32.141820+0000","last_became_peered":"2024-05-21T03:20:32.141820+0000","last_unstale":"2024-05-21T03:24:01.491481+0000","last_undegraded":"2024-05-21T03:24:01.491481+0000","last_fullsized":"2024-05-21T03:24:01.491481+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:45:20.021237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.502625+0000","last_change":"2024-05-21T03:20:29.100088+0000","last_active":"2024-05-21T03:24:01.502625+0000","last_peered":"2024-05-21T03:24:01.502625+0000","last_clean":"2024-05-21T03:24:01.502625+0000","last_became_active":"2024-05-21T03:20:29.097343+0000","last_became_peered":"2024-05-21T03:20:29.097343+0000","last_unstale":"2024-05-21T03:24:01.502625+0000","last_undegraded":"2024-05-21T03:24:01.502625+0000","last_fullsized":"2024-05-21T03:24:01.502625+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:00:35.682393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.387512+0000","last_change":"2024-05-21T03:21:14.562281+0000","last_active":"2024-05-21T03:24:01.387512+0000","last_peered":"2024-05-21T03:24:01.387512+0000","last_clean":"2024-05-21T03:24:01.387512+0000","last_became_active":"2024-05-21T03:21:14.561857+0000","last_became_peered":"2024-05-21T03:21:14.561857+0000","last_unstale":"2024-05-21T03:24:01.387512+0000","last_undegraded":"2024-05-21T03:24:01.387512+0000","last_fullsized":"2024-05-21T03:24:01.387512+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:37:35.283529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659672+0000","last_change":"2024-05-21T03:20:29.097945+0000","last_active":"2024-05-21T03:24:01.659672+0000","last_peered":"2024-05-21T03:24:01.659672+0000","last_clean":"2024-05-21T03:24:01.659672+0000","last_became_active":"2024-05-21T03:20:29.097624+0000","last_became_peered":"2024-05-21T03:20:29.097624+0000","last_unstale":"2024-05-21T03:24:01.659672+0000","last_undegraded":"2024-05-21T03:24:01.659672+0000","last_fullsized":"2024-05-21T03:24:01.659672+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:01:01.694893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491268+0000","last_change":"2024-05-21T03:20:32.148955+0000","last_active":"2024-05-21T03:24:01.491268+0000","last_peered":"2024-05-21T03:24:01.491268+0000","last_clean":"2024-05-21T03:24:01.491268+0000","last_became_active":"2024-05-21T03:20:32.148631+0000","last_became_peered":"2024-05-21T03:20:32.148631+0000","last_unstale":"2024-05-21T03:24:01.491268+0000","last_undegraded":"2024-05-21T03:24:01.491268+0000","last_fullsized":"2024-05-21T03:24:01.491268+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:36:25.820580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.499174+0000","last_change":"2024-05-21T03:20:29.100902+0000","last_active":"2024-05-21T03:24:01.499174+0000","last_peered":"2024-05-21T03:24:01.499174+0000","last_clean":"2024-05-21T03:24:01.499174+0000","last_became_active":"2024-05-21T03:20:29.100570+0000","last_became_peered":"2024-05-21T03:20:29.100570+0000","last_unstale":"2024-05-21T03:24:01.499174+0000","last_undegraded":"2024-05-21T03:24:01.499174+0000","last_fullsized":"2024-05-21T03:24:01.499174+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:58:36.651338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.387428+0000","last_change":"2024-05-21T03:20:32.150724+0000","last_active":"2024-05-21T03:24:01.387428+0000","last_peered":"2024-05-21T03:24:01.387428+0000","last_clean":"2024-05-21T03:24:01.387428+0000","last_became_active":"2024-05-21T03:20:32.150371+0000","last_became_peered":"2024-05-21T03:20:32.150371+0000","last_unstale":"2024-05-21T03:24:01.387428+0000","last_undegraded":"2024-05-21T03:24:01.387428+0000","last_fullsized":"2024-05-21T03:24:01.387428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:44:32.321192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'6","reported_seq":419,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.747249+0000","last_change":"2024-05-21T03:20:29.095045+0000","last_active":"2024-05-21T03:24:01.747249+0000","last_peered":"2024-05-21T03:24:01.747249+0000","last_clean":"2024-05-21T03:24:01.747249+0000","last_became_active":"2024-05-21T03:20:29.094552+0000","last_became_peered":"2024-05-21T03:20:29.094552+0000","last_unstale":"2024-05-21T03:24:01.747249+0000","last_undegraded":"2024-05-21T03:24:01.747249+0000","last_fullsized":"2024-05-21T03:24:01.747249+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:36:15.155125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":384,"num_read_kb":384,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659781+0000","last_change":"2024-05-21T03:20:32.149813+0000","last_active":"2024-05-21T03:24:01.659781+0000","last_peered":"2024-05-21T03:24:01.659781+0000","last_clean":"2024-05-21T03:24:01.659781+0000","last_became_active":"2024-05-21T03:20:32.148422+0000","last_became_peered":"2024-05-21T03:20:32.148422+0000","last_unstale":"2024-05-21T03:24:01.659781+0000","last_undegraded":"2024-05-21T03:24:01.659781+0000","last_fullsized":"2024-05-21T03:24:01.659781+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:55:35.115463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657337+0000","last_change":"2024-05-21T03:20:29.101463+0000","last_active":"2024-05-21T03:24:01.657337+0000","last_peered":"2024-05-21T03:24:01.657337+0000","last_clean":"2024-05-21T03:24:01.657337+0000","last_became_active":"2024-05-21T03:20:29.101127+0000","last_became_peered":"2024-05-21T03:20:29.101127+0000","last_unstale":"2024-05-21T03:24:01.657337+0000","last_undegraded":"2024-05-21T03:24:01.657337+0000","last_fullsized":"2024-05-21T03:24:01.657337+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:58:17.408315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657158+0000","last_change":"2024-05-21T03:21:14.563434+0000","last_active":"2024-05-21T03:24:01.657158+0000","last_peered":"2024-05-21T03:24:01.657158+0000","last_clean":"2024-05-21T03:24:01.657158+0000","last_became_active":"2024-05-21T03:21:14.563081+0000","last_became_peered":"2024-05-21T03:21:14.563081+0000","last_unstale":"2024-05-21T03:24:01.657158+0000","last_undegraded":"2024-05-21T03:24:01.657158+0000","last_fullsized":"2024-05-21T03:24:01.657158+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:28:55.497338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.499583+0000","last_change":"2024-05-21T03:20:29.098725+0000","last_active":"2024-05-21T03:24:01.499583+0000","last_peered":"2024-05-21T03:24:01.499583+0000","last_clean":"2024-05-21T03:24:01.499583+0000","last_became_active":"2024-05-21T03:20:29.098471+0000","last_became_peered":"2024-05-21T03:20:29.098471+0000","last_unstale":"2024-05-21T03:24:01.499583+0000","last_undegraded":"2024-05-21T03:24:01.499583+0000","last_fullsized":"2024-05-21T03:24:01.499583+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:17:11.059825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659799+0000","last_change":"2024-05-21T03:21:14.560975+0000","last_active":"2024-05-21T03:24:01.659799+0000","last_peered":"2024-05-21T03:24:01.659799+0000","last_clean":"2024-05-21T03:24:01.659799+0000","last_became_active":"2024-05-21T03:21:14.560528+0000","last_became_peered":"2024-05-21T03:21:14.560528+0000","last_unstale":"2024-05-21T03:24:01.659799+0000","last_undegraded":"2024-05-21T03:24:01.659799+0000","last_fullsized":"2024-05-21T03:24:01.659799+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:51:47.469291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'39","reported_seq":465,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.748126+0000","last_change":"2024-05-21T03:20:29.099968+0000","last_active":"2024-05-21T03:24:01.748126+0000","last_peered":"2024-05-21T03:24:01.748126+0000","last_clean":"2024-05-21T03:24:01.748126+0000","last_became_active":"2024-05-21T03:20:29.099181+0000","last_became_peered":"2024-05-21T03:20:29.099181+0000","last_unstale":"2024-05-21T03:24:01.748126+0000","last_undegraded":"2024-05-21T03:24:01.748126+0000","last_fullsized":"2024-05-21T03:24:01.748126+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:35:33.934412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66474,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":397,"num_read_kb":263,"num_write":41,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491786+0000","last_change":"2024-05-21T03:20:32.144024+0000","last_active":"2024-05-21T03:24:01.491786+0000","last_peered":"2024-05-21T03:24:01.491786+0000","last_clean":"2024-05-21T03:24:01.491786+0000","last_became_active":"2024-05-21T03:20:32.143668+0000","last_became_peered":"2024-05-21T03:20:32.143668+0000","last_unstale":"2024-05-21T03:24:01.491786+0000","last_undegraded":"2024-05-21T03:24:01.491786+0000","last_fullsized":"2024-05-21T03:24:01.491786+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:11:31.100802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.389042+0000","last_change":"2024-05-21T03:20:29.101770+0000","last_active":"2024-05-21T03:24:01.389042+0000","last_peered":"2024-05-21T03:24:01.389042+0000","last_clean":"2024-05-21T03:24:01.389042+0000","last_became_active":"2024-05-21T03:20:29.100407+0000","last_became_peered":"2024-05-21T03:20:29.100407+0000","last_unstale":"2024-05-21T03:24:01.389042+0000","last_undegraded":"2024-05-21T03:24:01.389042+0000","last_fullsized":"2024-05-21T03:24:01.389042+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:01:33.991246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659520+0000","last_change":"2024-05-21T03:20:32.147479+0000","last_active":"2024-05-21T03:24:01.659520+0000","last_peered":"2024-05-21T03:24:01.659520+0000","last_clean":"2024-05-21T03:24:01.659520+0000","last_became_active":"2024-05-21T03:20:32.147160+0000","last_became_peered":"2024-05-21T03:20:32.147160+0000","last_unstale":"2024-05-21T03:24:01.659520+0000","last_undegraded":"2024-05-21T03:24:01.659520+0000","last_fullsized":"2024-05-21T03:24:01.659520+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:51:59.343380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.499274+0000","last_change":"2024-05-21T03:20:29.101875+0000","last_active":"2024-05-21T03:24:01.499274+0000","last_peered":"2024-05-21T03:24:01.499274+0000","last_clean":"2024-05-21T03:24:01.499274+0000","last_became_active":"2024-05-21T03:20:29.101468+0000","last_became_peered":"2024-05-21T03:20:29.101468+0000","last_unstale":"2024-05-21T03:24:01.499274+0000","last_undegraded":"2024-05-21T03:24:01.499274+0000","last_fullsized":"2024-05-21T03:24:01.499274+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:10:19.577836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657039+0000","last_change":"2024-05-21T03:20:32.139002+0000","last_active":"2024-05-21T03:24:01.657039+0000","last_peered":"2024-05-21T03:24:01.657039+0000","last_clean":"2024-05-21T03:24:01.657039+0000","last_became_active":"2024-05-21T03:20:32.138619+0000","last_became_peered":"2024-05-21T03:20:32.138619+0000","last_unstale":"2024-05-21T03:24:01.657039+0000","last_undegraded":"2024-05-21T03:24:01.657039+0000","last_fullsized":"2024-05-21T03:24:01.657039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:21:36.136283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'8","reported_seq":233,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.747618+0000","last_change":"2024-05-21T03:20:29.101532+0000","last_active":"2024-05-21T03:24:01.747618+0000","last_peered":"2024-05-21T03:24:01.747618+0000","last_clean":"2024-05-21T03:24:01.747618+0000","last_became_active":"2024-05-21T03:20:29.101206+0000","last_became_peered":"2024-05-21T03:20:29.101206+0000","last_unstale":"2024-05-21T03:24:01.747618+0000","last_undegraded":"2024-05-21T03:24:01.747618+0000","last_fullsized":"2024-05-21T03:24:01.747618+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:47:38.555995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":198,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656787+0000","last_change":"2024-05-21T03:20:32.146866+0000","last_active":"2024-05-21T03:24:01.656787+0000","last_peered":"2024-05-21T03:24:01.656787+0000","last_clean":"2024-05-21T03:24:01.656787+0000","last_became_active":"2024-05-21T03:20:32.146463+0000","last_became_peered":"2024-05-21T03:20:32.146463+0000","last_unstale":"2024-05-21T03:24:01.656787+0000","last_undegraded":"2024-05-21T03:24:01.656787+0000","last_fullsized":"2024-05-21T03:24:01.656787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:06:54.359922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491442+0000","last_change":"2024-05-21T03:20:29.103910+0000","last_active":"2024-05-21T03:24:01.491442+0000","last_peered":"2024-05-21T03:24:01.491442+0000","last_clean":"2024-05-21T03:24:01.491442+0000","last_became_active":"2024-05-21T03:20:29.103583+0000","last_became_peered":"2024-05-21T03:20:29.103583+0000","last_unstale":"2024-05-21T03:24:01.491442+0000","last_undegraded":"2024-05-21T03:24:01.491442+0000","last_fullsized":"2024-05-21T03:24:01.491442+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:57:15.665158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.499198+0000","last_change":"2024-05-21T03:20:32.141242+0000","last_active":"2024-05-21T03:24:01.499198+0000","last_peered":"2024-05-21T03:24:01.499198+0000","last_clean":"2024-05-21T03:24:01.499198+0000","last_became_active":"2024-05-21T03:20:32.140892+0000","last_became_peered":"2024-05-21T03:20:32.140892+0000","last_unstale":"2024-05-21T03:24:01.499198+0000","last_undegraded":"2024-05-21T03:24:01.499198+0000","last_fullsized":"2024-05-21T03:24:01.499198+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:42:13.997708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491966+0000","last_change":"2024-05-21T03:20:29.098973+0000","last_active":"2024-05-21T03:24:01.491966+0000","last_peered":"2024-05-21T03:24:01.491966+0000","last_clean":"2024-05-21T03:24:01.491966+0000","last_became_active":"2024-05-21T03:20:29.098694+0000","last_became_peered":"2024-05-21T03:20:29.098694+0000","last_unstale":"2024-05-21T03:24:01.491966+0000","last_undegraded":"2024-05-21T03:24:01.491966+0000","last_fullsized":"2024-05-21T03:24:01.491966+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:55:17.915820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656862+0000","last_change":"2024-05-21T03:20:32.147465+0000","last_active":"2024-05-21T03:24:01.656862+0000","last_peered":"2024-05-21T03:24:01.656862+0000","last_clean":"2024-05-21T03:24:01.656862+0000","last_became_active":"2024-05-21T03:20:32.147073+0000","last_became_peered":"2024-05-21T03:20:32.147073+0000","last_unstale":"2024-05-21T03:24:01.656862+0000","last_undegraded":"2024-05-21T03:24:01.656862+0000","last_fullsized":"2024-05-21T03:24:01.656862+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:29:05.451370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.387615+0000","last_change":"2024-05-21T03:21:14.560568+0000","last_active":"2024-05-21T03:24:01.387615+0000","last_peered":"2024-05-21T03:24:01.387615+0000","last_clean":"2024-05-21T03:24:01.387615+0000","last_became_active":"2024-05-21T03:21:14.560093+0000","last_became_peered":"2024-05-21T03:21:14.560093+0000","last_unstale":"2024-05-21T03:24:01.387615+0000","last_undegraded":"2024-05-21T03:24:01.387615+0000","last_fullsized":"2024-05-21T03:24:01.387615+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:55:39.839942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.502820+0000","last_change":"2024-05-21T03:20:32.146456+0000","last_active":"2024-05-21T03:24:01.502820+0000","last_peered":"2024-05-21T03:24:01.502820+0000","last_clean":"2024-05-21T03:24:01.502820+0000","last_became_active":"2024-05-21T03:20:32.146105+0000","last_became_peered":"2024-05-21T03:20:32.146105+0000","last_unstale":"2024-05-21T03:24:01.502820+0000","last_undegraded":"2024-05-21T03:24:01.502820+0000","last_fullsized":"2024-05-21T03:24:01.502820+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:00:30.550178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'8","reported_seq":232,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.758178+0000","last_change":"2024-05-21T03:20:29.100262+0000","last_active":"2024-05-21T03:24:01.758178+0000","last_peered":"2024-05-21T03:24:01.758178+0000","last_clean":"2024-05-21T03:24:01.758178+0000","last_became_active":"2024-05-21T03:20:29.099917+0000","last_became_peered":"2024-05-21T03:20:29.099917+0000","last_unstale":"2024-05-21T03:24:01.758178+0000","last_undegraded":"2024-05-21T03:24:01.758178+0000","last_fullsized":"2024-05-21T03:24:01.758178+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:55:48.608341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":196,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.502772+0000","last_change":"2024-05-21T03:20:32.142776+0000","last_active":"2024-05-21T03:24:01.502772+0000","last_peered":"2024-05-21T03:24:01.502772+0000","last_clean":"2024-05-21T03:24:01.502772+0000","last_became_active":"2024-05-21T03:20:32.142430+0000","last_became_peered":"2024-05-21T03:20:32.142430+0000","last_unstale":"2024-05-21T03:24:01.502772+0000","last_undegraded":"2024-05-21T03:24:01.502772+0000","last_fullsized":"2024-05-21T03:24:01.502772+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:54:28.682343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.499435+0000","last_change":"2024-05-21T03:20:32.139506+0000","last_active":"2024-05-21T03:24:01.499435+0000","last_peered":"2024-05-21T03:24:01.499435+0000","last_clean":"2024-05-21T03:24:01.499435+0000","last_became_active":"2024-05-21T03:20:32.138136+0000","last_became_peered":"2024-05-21T03:20:32.138136+0000","last_unstale":"2024-05-21T03:24:01.499435+0000","last_undegraded":"2024-05-21T03:24:01.499435+0000","last_fullsized":"2024-05-21T03:24:01.499435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:31:24.996276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'7","reported_seq":229,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.661475+0000","last_change":"2024-05-21T03:20:29.096546+0000","last_active":"2024-05-21T03:24:01.661475+0000","last_peered":"2024-05-21T03:24:01.661475+0000","last_clean":"2024-05-21T03:24:01.661475+0000","last_became_active":"2024-05-21T03:20:29.096184+0000","last_became_peered":"2024-05-21T03:20:29.096184+0000","last_unstale":"2024-05-21T03:24:01.661475+0000","last_undegraded":"2024-05-21T03:24:01.661475+0000","last_fullsized":"2024-05-21T03:24:01.661475+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:09:12.832745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":193,"num_read_kb":193,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491686+0000","last_change":"2024-05-21T03:20:32.156231+0000","last_active":"2024-05-21T03:24:01.491686+0000","last_peered":"2024-05-21T03:24:01.491686+0000","last_clean":"2024-05-21T03:24:01.491686+0000","last_became_active":"2024-05-21T03:20:32.154035+0000","last_became_peered":"2024-05-21T03:20:32.154035+0000","last_unstale":"2024-05-21T03:24:01.491686+0000","last_undegraded":"2024-05-21T03:24:01.491686+0000","last_fullsized":"2024-05-21T03:24:01.491686+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:30:53.820292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.388854+0000","last_change":"2024-05-21T03:20:29.100422+0000","last_active":"2024-05-21T03:24:01.388854+0000","last_peered":"2024-05-21T03:24:01.388854+0000","last_clean":"2024-05-21T03:24:01.388854+0000","last_became_active":"2024-05-21T03:20:29.099993+0000","last_became_peered":"2024-05-21T03:20:29.099993+0000","last_unstale":"2024-05-21T03:24:01.388854+0000","last_undegraded":"2024-05-21T03:24:01.388854+0000","last_fullsized":"2024-05-21T03:24:01.388854+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:03:15.456447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":102,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656886+0000","last_change":"2024-05-21T03:19:05.647808+0000","last_active":"2024-05-21T03:24:01.656886+0000","last_peered":"2024-05-21T03:24:01.656886+0000","last_clean":"2024-05-21T03:24:01.656886+0000","last_became_active":"2024-05-21T03:19:05.641865+0000","last_became_peered":"2024-05-21T03:19:05.641865+0000","last_unstale":"2024-05-21T03:24:01.656886+0000","last_undegraded":"2024-05-21T03:24:01.656886+0000","last_fullsized":"2024-05-21T03:24:01.656886+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:15:42.008493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:15:42.008493+0000","last_clean_scrub_stamp":"2024-05-21T03:15:42.008493+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:26:58.040196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659351+0000","last_change":"2024-05-21T03:20:32.138182+0000","last_active":"2024-05-21T03:24:01.659351+0000","last_peered":"2024-05-21T03:24:01.659351+0000","last_clean":"2024-05-21T03:24:01.659351+0000","last_became_active":"2024-05-21T03:20:32.137842+0000","last_became_peered":"2024-05-21T03:20:32.137842+0000","last_unstale":"2024-05-21T03:24:01.659351+0000","last_undegraded":"2024-05-21T03:24:01.659351+0000","last_fullsized":"2024-05-21T03:24:01.659351+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:43:50.070160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657022+0000","last_change":"2024-05-21T03:20:29.104246+0000","last_active":"2024-05-21T03:24:01.657022+0000","last_peered":"2024-05-21T03:24:01.657022+0000","last_clean":"2024-05-21T03:24:01.657022+0000","last_became_active":"2024-05-21T03:20:29.103910+0000","last_became_peered":"2024-05-21T03:20:29.103910+0000","last_unstale":"2024-05-21T03:24:01.657022+0000","last_undegraded":"2024-05-21T03:24:01.657022+0000","last_fullsized":"2024-05-21T03:24:01.657022+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:39:28.029283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491818+0000","last_change":"2024-05-21T03:20:29.102020+0000","last_active":"2024-05-21T03:24:01.491818+0000","last_peered":"2024-05-21T03:24:01.491818+0000","last_clean":"2024-05-21T03:24:01.491818+0000","last_became_active":"2024-05-21T03:20:29.100429+0000","last_became_peered":"2024-05-21T03:20:29.100429+0000","last_unstale":"2024-05-21T03:24:01.491818+0000","last_undegraded":"2024-05-21T03:24:01.491818+0000","last_fullsized":"2024-05-21T03:24:01.491818+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:09:01.621074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659598+0000","last_change":"2024-05-21T03:20:32.137130+0000","last_active":"2024-05-21T03:24:01.659598+0000","last_peered":"2024-05-21T03:24:01.659598+0000","last_clean":"2024-05-21T03:24:01.659598+0000","last_became_active":"2024-05-21T03:20:32.136821+0000","last_became_peered":"2024-05-21T03:20:32.136821+0000","last_unstale":"2024-05-21T03:24:01.659598+0000","last_undegraded":"2024-05-21T03:24:01.659598+0000","last_fullsized":"2024-05-21T03:24:01.659598+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:53:41.290370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'50","reported_seq":101,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.610537+0000","last_change":"2024-05-21T03:20:29.099869+0000","last_active":"2024-05-21T03:24:01.610537+0000","last_peered":"2024-05-21T03:24:01.610537+0000","last_clean":"2024-05-21T03:24:01.610537+0000","last_became_active":"2024-05-21T03:20:29.099534+0000","last_became_peered":"2024-05-21T03:20:29.099534+0000","last_unstale":"2024-05-21T03:24:01.610537+0000","last_undegraded":"2024-05-21T03:24:01.610537+0000","last_fullsized":"2024-05-21T03:24:01.610537+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:01:11.281544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":89,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491991+0000","last_change":"2024-05-21T03:20:32.153921+0000","last_active":"2024-05-21T03:24:01.491991+0000","last_peered":"2024-05-21T03:24:01.491991+0000","last_clean":"2024-05-21T03:24:01.491991+0000","last_became_active":"2024-05-21T03:20:32.153594+0000","last_became_peered":"2024-05-21T03:20:32.153594+0000","last_unstale":"2024-05-21T03:24:01.491991+0000","last_undegraded":"2024-05-21T03:24:01.491991+0000","last_fullsized":"2024-05-21T03:24:01.491991+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:27:59.426942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.389210+0000","last_change":"2024-05-21T03:20:29.097399+0000","last_active":"2024-05-21T03:24:01.389210+0000","last_peered":"2024-05-21T03:24:01.389210+0000","last_clean":"2024-05-21T03:24:01.389210+0000","last_became_active":"2024-05-21T03:20:29.097010+0000","last_became_peered":"2024-05-21T03:20:29.097010+0000","last_unstale":"2024-05-21T03:24:01.389210+0000","last_undegraded":"2024-05-21T03:24:01.389210+0000","last_fullsized":"2024-05-21T03:24:01.389210+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:54:53.273947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657141+0000","last_change":"2024-05-21T03:21:14.559609+0000","last_active":"2024-05-21T03:24:01.657141+0000","last_peered":"2024-05-21T03:24:01.657141+0000","last_clean":"2024-05-21T03:24:01.657141+0000","last_became_active":"2024-05-21T03:21:14.559218+0000","last_became_peered":"2024-05-21T03:21:14.559218+0000","last_unstale":"2024-05-21T03:24:01.657141+0000","last_undegraded":"2024-05-21T03:24:01.657141+0000","last_fullsized":"2024-05-21T03:24:01.657141+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:32:11.156037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,4],"acting":[0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659469+0000","last_change":"2024-05-21T03:20:29.103526+0000","last_active":"2024-05-21T03:24:01.659469+0000","last_peered":"2024-05-21T03:24:01.659469+0000","last_clean":"2024-05-21T03:24:01.659469+0000","last_became_active":"2024-05-21T03:20:29.103067+0000","last_became_peered":"2024-05-21T03:20:29.103067+0000","last_unstale":"2024-05-21T03:24:01.659469+0000","last_undegraded":"2024-05-21T03:24:01.659469+0000","last_fullsized":"2024-05-21T03:24:01.659469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:54:47.566949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491463+0000","last_change":"2024-05-21T03:20:32.141088+0000","last_active":"2024-05-21T03:24:01.491463+0000","last_peered":"2024-05-21T03:24:01.491463+0000","last_clean":"2024-05-21T03:24:01.491463+0000","last_became_active":"2024-05-21T03:20:32.140744+0000","last_became_peered":"2024-05-21T03:20:32.140744+0000","last_unstale":"2024-05-21T03:24:01.491463+0000","last_undegraded":"2024-05-21T03:24:01.491463+0000","last_fullsized":"2024-05-21T03:24:01.491463+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:57:35.591164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491373+0000","last_change":"2024-05-21T03:20:29.093440+0000","last_active":"2024-05-21T03:24:01.491373+0000","last_peered":"2024-05-21T03:24:01.491373+0000","last_clean":"2024-05-21T03:24:01.491373+0000","last_became_active":"2024-05-21T03:20:29.092935+0000","last_became_peered":"2024-05-21T03:20:29.092935+0000","last_unstale":"2024-05-21T03:24:01.491373+0000","last_undegraded":"2024-05-21T03:24:01.491373+0000","last_fullsized":"2024-05-21T03:24:01.491373+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:18:00.690465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656650+0000","last_change":"2024-05-21T03:20:32.143781+0000","last_active":"2024-05-21T03:24:01.656650+0000","last_peered":"2024-05-21T03:24:01.656650+0000","last_clean":"2024-05-21T03:24:01.656650+0000","last_became_active":"2024-05-21T03:20:32.143391+0000","last_became_peered":"2024-05-21T03:20:32.143391+0000","last_unstale":"2024-05-21T03:24:01.656650+0000","last_undegraded":"2024-05-21T03:24:01.656650+0000","last_fullsized":"2024-05-21T03:24:01.656650+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:00:41.506561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.388944+0000","last_change":"2024-05-21T03:20:29.101342+0000","last_active":"2024-05-21T03:24:01.388944+0000","last_peered":"2024-05-21T03:24:01.388944+0000","last_clean":"2024-05-21T03:24:01.388944+0000","last_became_active":"2024-05-21T03:20:29.100997+0000","last_became_peered":"2024-05-21T03:20:29.100997+0000","last_unstale":"2024-05-21T03:24:01.388944+0000","last_undegraded":"2024-05-21T03:24:01.388944+0000","last_fullsized":"2024-05-21T03:24:01.388944+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:05:31.801669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491844+0000","last_change":"2024-05-21T03:20:32.150563+0000","last_active":"2024-05-21T03:24:01.491844+0000","last_peered":"2024-05-21T03:24:01.491844+0000","last_clean":"2024-05-21T03:24:01.491844+0000","last_became_active":"2024-05-21T03:20:32.150231+0000","last_became_peered":"2024-05-21T03:20:32.150231+0000","last_unstale":"2024-05-21T03:24:01.491844+0000","last_undegraded":"2024-05-21T03:24:01.491844+0000","last_fullsized":"2024-05-21T03:24:01.491844+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:10:00.251284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491635+0000","last_change":"2024-05-21T03:20:29.101576+0000","last_active":"2024-05-21T03:24:01.491635+0000","last_peered":"2024-05-21T03:24:01.491635+0000","last_clean":"2024-05-21T03:24:01.491635+0000","last_became_active":"2024-05-21T03:20:29.101270+0000","last_became_peered":"2024-05-21T03:20:29.101270+0000","last_unstale":"2024-05-21T03:24:01.491635+0000","last_undegraded":"2024-05-21T03:24:01.491635+0000","last_fullsized":"2024-05-21T03:24:01.491635+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:50:10.579901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656976+0000","last_change":"2024-05-21T03:20:32.141340+0000","last_active":"2024-05-21T03:24:01.656976+0000","last_peered":"2024-05-21T03:24:01.656976+0000","last_clean":"2024-05-21T03:24:01.656976+0000","last_became_active":"2024-05-21T03:20:32.140792+0000","last_became_peered":"2024-05-21T03:20:32.140792+0000","last_unstale":"2024-05-21T03:24:01.656976+0000","last_undegraded":"2024-05-21T03:24:01.656976+0000","last_fullsized":"2024-05-21T03:24:01.656976+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:35:05.842929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'10","reported_seq":197,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.822342+0000","last_change":"2024-05-21T03:21:14.566886+0000","last_active":"2024-05-21T03:24:01.822342+0000","last_peered":"2024-05-21T03:24:01.822342+0000","last_clean":"2024-05-21T03:24:01.822342+0000","last_became_active":"2024-05-21T03:21:14.559816+0000","last_became_peered":"2024-05-21T03:21:14.559816+0000","last_unstale":"2024-05-21T03:24:01.822342+0000","last_undegraded":"2024-05-21T03:24:01.822342+0000","last_fullsized":"2024-05-21T03:24:01.822342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:34:43.109424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2792,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":176,"num_read_kb":5,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1561,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657398+0000","last_change":"2024-05-21T03:20:32.146674+0000","last_active":"2024-05-21T03:24:01.657398+0000","last_peered":"2024-05-21T03:24:01.657398+0000","last_clean":"2024-05-21T03:24:01.657398+0000","last_became_active":"2024-05-21T03:20:32.146150+0000","last_became_peered":"2024-05-21T03:20:32.146150+0000","last_unstale":"2024-05-21T03:24:01.657398+0000","last_undegraded":"2024-05-21T03:24:01.657398+0000","last_fullsized":"2024-05-21T03:24:01.657398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:39:09.905640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'41","reported_seq":78,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.607615+0000","last_change":"2024-05-21T03:20:29.097613+0000","last_active":"2024-05-21T03:24:01.607615+0000","last_peered":"2024-05-21T03:24:01.607615+0000","last_clean":"2024-05-21T03:24:01.607615+0000","last_became_active":"2024-05-21T03:20:29.097331+0000","last_became_peered":"2024-05-21T03:20:29.097331+0000","last_unstale":"2024-05-21T03:24:01.607615+0000","last_undegraded":"2024-05-21T03:24:01.607615+0000","last_fullsized":"2024-05-21T03:24:01.607615+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:56:14.720420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657109+0000","last_change":"2024-05-21T03:21:14.566406+0000","last_active":"2024-05-21T03:24:01.657109+0000","last_peered":"2024-05-21T03:24:01.657109+0000","last_clean":"2024-05-21T03:24:01.657109+0000","last_became_active":"2024-05-21T03:21:14.563023+0000","last_became_peered":"2024-05-21T03:21:14.563023+0000","last_unstale":"2024-05-21T03:24:01.657109+0000","last_undegraded":"2024-05-21T03:24:01.657109+0000","last_fullsized":"2024-05-21T03:24:01.657109+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:12:53.125136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.660167+0000","last_change":"2024-05-21T03:20:29.103456+0000","last_active":"2024-05-21T03:24:01.660167+0000","last_peered":"2024-05-21T03:24:01.660167+0000","last_clean":"2024-05-21T03:24:01.660167+0000","last_became_active":"2024-05-21T03:20:29.103008+0000","last_became_peered":"2024-05-21T03:20:29.103008+0000","last_unstale":"2024-05-21T03:24:01.660167+0000","last_undegraded":"2024-05-21T03:24:01.660167+0000","last_fullsized":"2024-05-21T03:24:01.660167+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:02:37.951347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657407+0000","last_change":"2024-05-21T03:20:32.146986+0000","last_active":"2024-05-21T03:24:01.657407+0000","last_peered":"2024-05-21T03:24:01.657407+0000","last_clean":"2024-05-21T03:24:01.657407+0000","last_became_active":"2024-05-21T03:20:32.146355+0000","last_became_peered":"2024-05-21T03:20:32.146355+0000","last_unstale":"2024-05-21T03:24:01.657407+0000","last_undegraded":"2024-05-21T03:24:01.657407+0000","last_fullsized":"2024-05-21T03:24:01.657407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:36:29.503099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656717+0000","last_change":"2024-05-21T03:21:14.559658+0000","last_active":"2024-05-21T03:24:01.656717+0000","last_peered":"2024-05-21T03:24:01.656717+0000","last_clean":"2024-05-21T03:24:01.656717+0000","last_became_active":"2024-05-21T03:21:14.559194+0000","last_became_peered":"2024-05-21T03:21:14.559194+0000","last_unstale":"2024-05-21T03:24:01.656717+0000","last_undegraded":"2024-05-21T03:24:01.656717+0000","last_fullsized":"2024-05-21T03:24:01.656717+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:56:31.709545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.389291+0000","last_change":"2024-05-21T03:20:32.137532+0000","last_active":"2024-05-21T03:24:01.389291+0000","last_peered":"2024-05-21T03:24:01.389291+0000","last_clean":"2024-05-21T03:24:01.389291+0000","last_became_active":"2024-05-21T03:20:32.137182+0000","last_became_peered":"2024-05-21T03:20:32.137182+0000","last_unstale":"2024-05-21T03:24:01.389291+0000","last_undegraded":"2024-05-21T03:24:01.389291+0000","last_fullsized":"2024-05-21T03:24:01.389291+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:25:44.207866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.388938+0000","last_change":"2024-05-21T03:20:32.153912+0000","last_active":"2024-05-21T03:24:01.388938+0000","last_peered":"2024-05-21T03:24:01.388938+0000","last_clean":"2024-05-21T03:24:01.388938+0000","last_became_active":"2024-05-21T03:20:32.153541+0000","last_became_peered":"2024-05-21T03:20:32.153541+0000","last_unstale":"2024-05-21T03:24:01.388938+0000","last_undegraded":"2024-05-21T03:24:01.388938+0000","last_fullsized":"2024-05-21T03:24:01.388938+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:02:58.688378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657148+0000","last_change":"2024-05-21T03:20:32.143689+0000","last_active":"2024-05-21T03:24:01.657148+0000","last_peered":"2024-05-21T03:24:01.657148+0000","last_clean":"2024-05-21T03:24:01.657148+0000","last_became_active":"2024-05-21T03:20:32.143399+0000","last_became_peered":"2024-05-21T03:20:32.143399+0000","last_unstale":"2024-05-21T03:24:01.657148+0000","last_undegraded":"2024-05-21T03:24:01.657148+0000","last_fullsized":"2024-05-21T03:24:01.657148+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:27:55.116599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.660308+0000","last_change":"2024-05-21T03:20:32.141355+0000","last_active":"2024-05-21T03:24:01.660308+0000","last_peered":"2024-05-21T03:24:01.660308+0000","last_clean":"2024-05-21T03:24:01.660308+0000","last_became_active":"2024-05-21T03:20:32.140960+0000","last_became_peered":"2024-05-21T03:20:32.140960+0000","last_unstale":"2024-05-21T03:24:01.660308+0000","last_undegraded":"2024-05-21T03:24:01.660308+0000","last_fullsized":"2024-05-21T03:24:01.660308+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:23:50.287864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656934+0000","last_change":"2024-05-21T03:20:32.142145+0000","last_active":"2024-05-21T03:24:01.656934+0000","last_peered":"2024-05-21T03:24:01.656934+0000","last_clean":"2024-05-21T03:24:01.656934+0000","last_became_active":"2024-05-21T03:20:32.141802+0000","last_became_peered":"2024-05-21T03:20:32.141802+0000","last_unstale":"2024-05-21T03:24:01.656934+0000","last_undegraded":"2024-05-21T03:24:01.656934+0000","last_fullsized":"2024-05-21T03:24:01.656934+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:26:28.707616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.389203+0000","last_change":"2024-05-21T03:20:32.141606+0000","last_active":"2024-05-21T03:24:01.389203+0000","last_peered":"2024-05-21T03:24:01.389203+0000","last_clean":"2024-05-21T03:24:01.389203+0000","last_became_active":"2024-05-21T03:20:32.141258+0000","last_became_peered":"2024-05-21T03:20:32.141258+0000","last_unstale":"2024-05-21T03:24:01.389203+0000","last_undegraded":"2024-05-21T03:24:01.389203+0000","last_fullsized":"2024-05-21T03:24:01.389203+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:22.372723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491954+0000","last_change":"2024-05-21T03:20:32.151621+0000","last_active":"2024-05-21T03:24:01.491954+0000","last_peered":"2024-05-21T03:24:01.491954+0000","last_clean":"2024-05-21T03:24:01.491954+0000","last_became_active":"2024-05-21T03:20:32.151260+0000","last_became_peered":"2024-05-21T03:20:32.151260+0000","last_unstale":"2024-05-21T03:24:01.491954+0000","last_undegraded":"2024-05-21T03:24:01.491954+0000","last_fullsized":"2024-05-21T03:24:01.491954+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:48:59.751179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.499163+0000","last_change":"2024-05-21T03:20:32.153843+0000","last_active":"2024-05-21T03:24:01.499163+0000","last_peered":"2024-05-21T03:24:01.499163+0000","last_clean":"2024-05-21T03:24:01.499163+0000","last_became_active":"2024-05-21T03:20:32.153418+0000","last_became_peered":"2024-05-21T03:20:32.153418+0000","last_unstale":"2024-05-21T03:24:01.499163+0000","last_undegraded":"2024-05-21T03:24:01.499163+0000","last_fullsized":"2024-05-21T03:24:01.499163+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:26:29.663966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491312+0000","last_change":"2024-05-21T03:20:32.153942+0000","last_active":"2024-05-21T03:24:01.491312+0000","last_peered":"2024-05-21T03:24:01.491312+0000","last_clean":"2024-05-21T03:24:01.491312+0000","last_became_active":"2024-05-21T03:20:32.153566+0000","last_became_peered":"2024-05-21T03:20:32.153566+0000","last_unstale":"2024-05-21T03:24:01.491312+0000","last_undegraded":"2024-05-21T03:24:01.491312+0000","last_fullsized":"2024-05-21T03:24:01.491312+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:06.899039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659626+0000","last_change":"2024-05-21T03:20:32.153243+0000","last_active":"2024-05-21T03:24:01.659626+0000","last_peered":"2024-05-21T03:24:01.659626+0000","last_clean":"2024-05-21T03:24:01.659626+0000","last_became_active":"2024-05-21T03:20:32.152386+0000","last_became_peered":"2024-05-21T03:20:32.152386+0000","last_unstale":"2024-05-21T03:24:01.659626+0000","last_undegraded":"2024-05-21T03:24:01.659626+0000","last_fullsized":"2024-05-21T03:24:01.659626+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:25:28.935183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659802+0000","last_change":"2024-05-21T03:20:32.142536+0000","last_active":"2024-05-21T03:24:01.659802+0000","last_peered":"2024-05-21T03:24:01.659802+0000","last_clean":"2024-05-21T03:24:01.659802+0000","last_became_active":"2024-05-21T03:20:32.141793+0000","last_became_peered":"2024-05-21T03:20:32.141793+0000","last_unstale":"2024-05-21T03:24:01.659802+0000","last_undegraded":"2024-05-21T03:24:01.659802+0000","last_fullsized":"2024-05-21T03:24:01.659802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:21:43.547103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656778+0000","last_change":"2024-05-21T03:20:32.140368+0000","last_active":"2024-05-21T03:24:01.656778+0000","last_peered":"2024-05-21T03:24:01.656778+0000","last_clean":"2024-05-21T03:24:01.656778+0000","last_became_active":"2024-05-21T03:20:32.139747+0000","last_became_peered":"2024-05-21T03:20:32.139747+0000","last_unstale":"2024-05-21T03:24:01.656778+0000","last_undegraded":"2024-05-21T03:24:01.656778+0000","last_fullsized":"2024-05-21T03:24:01.656778+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:05:44.851191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":601180,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2041,"num_read_kb":1147,"num_write":378,"num_write_kb":1469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1561,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2371584,"data_stored":2000172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":306,"ondisk_log_size":306,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645369,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":917504,"data_stored":444066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841598,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28736,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716416,"statfs":{"total":95995035648,"available":95965609984,"internally_reserved":0,"allocated":1761280,"data_stored":1232470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005124,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28328,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716824,"statfs":{"total":95995035648,"available":95966027776,"internally_reserved":0,"allocated":1339392,"data_stored":835707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201351,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28320,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716832,"statfs":{"total":95995035648,"available":95966035968,"internally_reserved":0,"allocated":1331200,"data_stored":835968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397582,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32680,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712472,"statfs":{"total":95995035648,"available":95961571328,"internally_reserved":0,"allocated":1536000,"data_stored":1036725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561106,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":433638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574956,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":510401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738481,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":445052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411431,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":508528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149794,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28760,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716392,"statfs":{"total":95995035648,"available":95965585408,"internally_reserved":0,"allocated":1662976,"data_stored":1163247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953565,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":933888,"data_stored":444494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757336,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37160,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707992,"statfs":{"total":95995035648,"available":95956983808,"internally_reserved":0,"allocated":1683456,"data_stored":1166157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":67276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":134420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":200733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T03:24:09.502 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-21T03:24:09.502 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-21T03:24:09.754 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:24:11.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:10 smithi039 ceph-mon[36429]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:24:11.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:10 smithi039 ceph-mon[36429]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 131 KiB/s wr, 72 op/s 2024-05-21T03:24:11.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:10 smithi165 ceph-mon[36269]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:24:11.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:10 smithi165 ceph-mon[36269]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 131 KiB/s wr, 72 op/s 2024-05-21T03:24:11.027 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:10 smithi027 ceph-mon[32179]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:24:11.027 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:10 smithi027 ceph-mon[32179]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 131 KiB/s wr, 72 op/s 2024-05-21T03:24:11.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.402+0000 7f2bb2811700 1 -- 172.21.15.27:0/3608452916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bac1084f0 msgr2=0x7f2bac1088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:11.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.402+0000 7f2bb2811700 1 --2- 172.21.15.27:0/3608452916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bac1084f0 0x7f2bac1088d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c009990 tx=0x7f2b9c009ca0 comp rx=0 tx=0).stop 2024-05-21T03:24:11.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.402+0000 7f2bb2811700 1 -- 172.21.15.27:0/3608452916 shutdown_connections 2024-05-21T03:24:11.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.402+0000 7f2bb2811700 1 --2- 172.21.15.27:0/3608452916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2bac102e20 0x7f2bac10f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:11.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.402+0000 7f2bb2811700 1 --2- 172.21.15.27:0/3608452916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bac1084f0 0x7f2bac1088d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:11.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.402+0000 7f2bb2811700 1 --2- 172.21.15.27:0/3608452916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bac102460 0x7f2bac1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:11.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.402+0000 7f2bb2811700 1 -- 172.21.15.27:0/3608452916 >> 172.21.15.27:0/3608452916 conn(0x7f2bac0794e0 msgr2=0x7f2bac0798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:11.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.402+0000 7f2bb2811700 1 -- 172.21.15.27:0/3608452916 shutdown_connections 2024-05-21T03:24:11.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.402+0000 7f2bb2811700 1 -- 172.21.15.27:0/3608452916 wait complete. 2024-05-21T03:24:11.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.403+0000 7f2bb2811700 1 Processor -- start 2024-05-21T03:24:11.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.403+0000 7f2bb2811700 1 -- start start 2024-05-21T03:24:11.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.403+0000 7f2bb2811700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2bac102460 0x7f2bac199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:11.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.403+0000 7f2bb2811700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bac102e20 0x7f2bac19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:11.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.403+0000 7f2bb2811700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bac1084f0 0x7f2bac19e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:11.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.403+0000 7f2bb2811700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac112700 con 0x7f2bac1084f0 2024-05-21T03:24:11.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.403+0000 7f2bb2811700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac112580 con 0x7f2bac102e20 2024-05-21T03:24:11.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.403+0000 7f2bb2811700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac112880 con 0x7f2bac102460 2024-05-21T03:24:11.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.403+0000 7f2bb0dae700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bac1084f0 0x7f2bac19e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:11.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.403+0000 7f2bb0dae700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bac1084f0 0x7f2bac19e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55878/0 (socket says 172.21.15.27:55878) 2024-05-21T03:24:11.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.403+0000 7f2bb0dae700 1 -- 172.21.15.27:0/498111053 learned_addr learned my addr 172.21.15.27:0/498111053 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:11.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.403+0000 7f2ba3fff700 1 --2- 172.21.15.27:0/498111053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bac102e20 0x7f2bac19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:11.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.403+0000 7f2babfff700 1 --2- 172.21.15.27:0/498111053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2bac102460 0x7f2bac199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:11.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.404+0000 7f2bb0dae700 1 -- 172.21.15.27:0/498111053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2bac102460 msgr2=0x7f2bac199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:11.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.404+0000 7f2bb0dae700 1 --2- 172.21.15.27:0/498111053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2bac102460 0x7f2bac199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:11.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.404+0000 7f2bb0dae700 1 -- 172.21.15.27:0/498111053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bac102e20 msgr2=0x7f2bac19a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:11.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.404+0000 7f2bb0dae700 1 --2- 172.21.15.27:0/498111053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bac102e20 0x7f2bac19a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:11.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.404+0000 7f2bb0dae700 1 -- 172.21.15.27:0/498111053 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bac19ed80 con 0x7f2bac1084f0 2024-05-21T03:24:11.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.404+0000 7f2bb0dae700 1 --2- 172.21.15.27:0/498111053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bac1084f0 0x7f2bac19e640 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2ba400b970 tx=0x7f2ba400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:11.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.404+0000 7f2ba9ffb700 1 -- 172.21.15.27:0/498111053 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba400d860 con 0x7f2bac1084f0 2024-05-21T03:24:11.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.404+0000 7f2ba9ffb700 1 -- 172.21.15.27:0/498111053 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ba400e430 con 0x7f2bac1084f0 2024-05-21T03:24:11.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.404+0000 7f2bb2811700 1 -- 172.21.15.27:0/498111053 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bac19f070 con 0x7f2bac1084f0 2024-05-21T03:24:11.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.404+0000 7f2ba9ffb700 1 -- 172.21.15.27:0/498111053 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4017660 con 0x7f2bac1084f0 2024-05-21T03:24:11.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.404+0000 7f2bb2811700 1 -- 172.21.15.27:0/498111053 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bac1a68f0 con 0x7f2bac1084f0 2024-05-21T03:24:11.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.405+0000 7f2bb2811700 1 -- 172.21.15.27:0/498111053 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bac06a230 con 0x7f2bac1084f0 2024-05-21T03:24:11.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.407+0000 7f2ba9ffb700 1 -- 172.21.15.27:0/498111053 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f2ba400da00 con 0x7f2bac1084f0 2024-05-21T03:24:11.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.407+0000 7f2ba9ffb700 1 --2- 172.21.15.27:0/498111053 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f2b980b2a80 0x7f2b980b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:11.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.407+0000 7f2babfff700 1 --2- 172.21.15.27:0/498111053 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f2b980b2a80 0x7f2b980b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:11.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.407+0000 7f2ba9ffb700 1 -- 172.21.15.27:0/498111053 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f2ba40e27d0 con 0x7f2bac1084f0 2024-05-21T03:24:11.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.408+0000 7f2babfff700 1 --2- 172.21.15.27:0/498111053 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f2b980b2a80 0x7f2b980b4f40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c00b5c0 tx=0x7f2b9c005960 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:11.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.411+0000 7f2ba9ffb700 1 -- 172.21.15.27:0/498111053 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2ba4013030 con 0x7f2bac1084f0 2024-05-21T03:24:11.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.620+0000 7f2bb2811700 1 -- 172.21.15.27:0/498111053 --> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2bac063980 con 0x7f2b980b2a80 2024-05-21T03:24:11.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.627+0000 7f2ba9ffb700 1 -- 172.21.15.27:0/498111053 <== mgr.24913 v2:172.21.15.27:6800/1477999924 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257821 (secure 0 0 0) 0x7f2bac063980 con 0x7f2b980b2a80 2024-05-21T03:24:11.631 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:24:11.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.630+0000 7f2bb2811700 1 -- 172.21.15.27:0/498111053 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f2b980b2a80 msgr2=0x7f2b980b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:11.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.630+0000 7f2bb2811700 1 --2- 172.21.15.27:0/498111053 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f2b980b2a80 0x7f2b980b4f40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c00b5c0 tx=0x7f2b9c005960 comp rx=0 tx=0).stop 2024-05-21T03:24:11.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.630+0000 7f2bb2811700 1 -- 172.21.15.27:0/498111053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bac1084f0 msgr2=0x7f2bac19e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:11.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.630+0000 7f2bb2811700 1 --2- 172.21.15.27:0/498111053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bac1084f0 0x7f2bac19e640 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2ba400b970 tx=0x7f2ba400bc80 comp rx=0 tx=0).stop 2024-05-21T03:24:11.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.631+0000 7f2bb2811700 1 -- 172.21.15.27:0/498111053 shutdown_connections 2024-05-21T03:24:11.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.631+0000 7f2bb2811700 1 --2- 172.21.15.27:0/498111053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bac1084f0 0x7f2bac19e640 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:11.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.631+0000 7f2bb2811700 1 --2- 172.21.15.27:0/498111053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2bac102460 0x7f2bac199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:11.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.631+0000 7f2bb2811700 1 --2- 172.21.15.27:0/498111053 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f2b980b2a80 0x7f2b980b4f40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:11.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.631+0000 7f2bb2811700 1 --2- 172.21.15.27:0/498111053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bac102e20 0x7f2bac19a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:11.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.631+0000 7f2bb2811700 1 -- 172.21.15.27:0/498111053 >> 172.21.15.27:0/498111053 conn(0x7f2bac0794e0 msgr2=0x7f2bac06bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:11.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.631+0000 7f2bb2811700 1 -- 172.21.15.27:0/498111053 shutdown_connections 2024-05-21T03:24:11.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:11.631+0000 7f2bb2811700 1 -- 172.21.15.27:0/498111053 wait complete. 2024-05-21T03:24:11.634 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-05-21T03:24:12.244 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-05-21T03:24:11.403684+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1388414,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2488,"num_read_kb":1650,"num_write":645,"num_write_kb":4537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2953518,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":988,"ondisk_log_size":988,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364608,"kb_used_data":15072,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124577216,"statfs":{"total":1151940427776,"available":1151567069184,"internally_reserved":0,"allocated":15433728,"data_stored":9460266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19139,"internal_metadata":357414205},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":366,"num_read_kb":434,"num_write":142,"num_write_kb":916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.988534"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656903+0000","last_change":"2024-05-21T03:20:32.149961+0000","last_active":"2024-05-21T03:24:01.656903+0000","last_peered":"2024-05-21T03:24:01.656903+0000","last_clean":"2024-05-21T03:24:01.656903+0000","last_became_active":"2024-05-21T03:20:32.149552+0000","last_became_peered":"2024-05-21T03:20:32.149552+0000","last_unstale":"2024-05-21T03:24:01.656903+0000","last_undegraded":"2024-05-21T03:24:01.656903+0000","last_fullsized":"2024-05-21T03:24:01.656903+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:20:38.543551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656992+0000","last_change":"2024-05-21T03:20:32.146413+0000","last_active":"2024-05-21T03:24:01.656992+0000","last_peered":"2024-05-21T03:24:01.656992+0000","last_clean":"2024-05-21T03:24:01.656992+0000","last_became_active":"2024-05-21T03:20:32.146006+0000","last_became_peered":"2024-05-21T03:20:32.146006+0000","last_unstale":"2024-05-21T03:24:01.656992+0000","last_undegraded":"2024-05-21T03:24:01.656992+0000","last_fullsized":"2024-05-21T03:24:01.656992+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:05:12.888550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656984+0000","last_change":"2024-05-21T03:20:32.153654+0000","last_active":"2024-05-21T03:24:01.656984+0000","last_peered":"2024-05-21T03:24:01.656984+0000","last_clean":"2024-05-21T03:24:01.656984+0000","last_became_active":"2024-05-21T03:20:32.153296+0000","last_became_peered":"2024-05-21T03:20:32.153296+0000","last_unstale":"2024-05-21T03:24:01.656984+0000","last_undegraded":"2024-05-21T03:24:01.656984+0000","last_fullsized":"2024-05-21T03:24:01.656984+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:16:09.423236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.389156+0000","last_change":"2024-05-21T03:20:32.138672+0000","last_active":"2024-05-21T03:24:01.389156+0000","last_peered":"2024-05-21T03:24:01.389156+0000","last_clean":"2024-05-21T03:24:01.389156+0000","last_became_active":"2024-05-21T03:20:32.138163+0000","last_became_peered":"2024-05-21T03:20:32.138163+0000","last_unstale":"2024-05-21T03:24:01.389156+0000","last_undegraded":"2024-05-21T03:24:01.389156+0000","last_fullsized":"2024-05-21T03:24:01.389156+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:26:55.012897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491536+0000","last_change":"2024-05-21T03:20:32.153950+0000","last_active":"2024-05-21T03:24:01.491536+0000","last_peered":"2024-05-21T03:24:01.491536+0000","last_clean":"2024-05-21T03:24:01.491536+0000","last_became_active":"2024-05-21T03:20:32.153567+0000","last_became_peered":"2024-05-21T03:20:32.153567+0000","last_unstale":"2024-05-21T03:24:01.491536+0000","last_undegraded":"2024-05-21T03:24:01.491536+0000","last_fullsized":"2024-05-21T03:24:01.491536+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:28:34.621193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.502688+0000","last_change":"2024-05-21T03:20:32.140038+0000","last_active":"2024-05-21T03:24:01.502688+0000","last_peered":"2024-05-21T03:24:01.502688+0000","last_clean":"2024-05-21T03:24:01.502688+0000","last_became_active":"2024-05-21T03:20:32.139690+0000","last_became_peered":"2024-05-21T03:20:32.139690+0000","last_unstale":"2024-05-21T03:24:01.502688+0000","last_undegraded":"2024-05-21T03:24:01.502688+0000","last_fullsized":"2024-05-21T03:24:01.502688+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:31:24.270755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656862+0000","last_change":"2024-05-21T03:20:32.143735+0000","last_active":"2024-05-21T03:24:01.656862+0000","last_peered":"2024-05-21T03:24:01.656862+0000","last_clean":"2024-05-21T03:24:01.656862+0000","last_became_active":"2024-05-21T03:20:32.143397+0000","last_became_peered":"2024-05-21T03:20:32.143397+0000","last_unstale":"2024-05-21T03:24:01.656862+0000","last_undegraded":"2024-05-21T03:24:01.656862+0000","last_fullsized":"2024-05-21T03:24:01.656862+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:40:01.951608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491334+0000","last_change":"2024-05-21T03:20:32.156999+0000","last_active":"2024-05-21T03:24:01.491334+0000","last_peered":"2024-05-21T03:24:01.491334+0000","last_clean":"2024-05-21T03:24:01.491334+0000","last_became_active":"2024-05-21T03:20:32.156677+0000","last_became_peered":"2024-05-21T03:20:32.156677+0000","last_unstale":"2024-05-21T03:24:01.491334+0000","last_undegraded":"2024-05-21T03:24:01.491334+0000","last_fullsized":"2024-05-21T03:24:01.491334+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:33:24.225667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.660261+0000","last_change":"2024-05-21T03:20:32.145039+0000","last_active":"2024-05-21T03:24:01.660261+0000","last_peered":"2024-05-21T03:24:01.660261+0000","last_clean":"2024-05-21T03:24:01.660261+0000","last_became_active":"2024-05-21T03:20:32.140926+0000","last_became_peered":"2024-05-21T03:20:32.140926+0000","last_unstale":"2024-05-21T03:24:01.660261+0000","last_undegraded":"2024-05-21T03:24:01.660261+0000","last_fullsized":"2024-05-21T03:24:01.660261+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:11:27.049426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.503068+0000","last_change":"2024-05-21T03:20:32.145840+0000","last_active":"2024-05-21T03:24:01.503068+0000","last_peered":"2024-05-21T03:24:01.503068+0000","last_clean":"2024-05-21T03:24:01.503068+0000","last_became_active":"2024-05-21T03:20:32.145291+0000","last_became_peered":"2024-05-21T03:20:32.145291+0000","last_unstale":"2024-05-21T03:24:01.503068+0000","last_undegraded":"2024-05-21T03:24:01.503068+0000","last_fullsized":"2024-05-21T03:24:01.503068+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:22:27.696588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657241+0000","last_change":"2024-05-21T03:20:32.142522+0000","last_active":"2024-05-21T03:24:01.657241+0000","last_peered":"2024-05-21T03:24:01.657241+0000","last_clean":"2024-05-21T03:24:01.657241+0000","last_became_active":"2024-05-21T03:20:32.142165+0000","last_became_peered":"2024-05-21T03:20:32.142165+0000","last_unstale":"2024-05-21T03:24:01.657241+0000","last_undegraded":"2024-05-21T03:24:01.657241+0000","last_fullsized":"2024-05-21T03:24:01.657241+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:58:39.085834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.389012+0000","last_change":"2024-05-21T03:20:32.142526+0000","last_active":"2024-05-21T03:24:01.389012+0000","last_peered":"2024-05-21T03:24:01.389012+0000","last_clean":"2024-05-21T03:24:01.389012+0000","last_became_active":"2024-05-21T03:20:32.141143+0000","last_became_peered":"2024-05-21T03:20:32.141143+0000","last_unstale":"2024-05-21T03:24:01.389012+0000","last_undegraded":"2024-05-21T03:24:01.389012+0000","last_fullsized":"2024-05-21T03:24:01.389012+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:35:11.336796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659855+0000","last_change":"2024-05-21T03:20:32.135631+0000","last_active":"2024-05-21T03:24:01.659855+0000","last_peered":"2024-05-21T03:24:01.659855+0000","last_clean":"2024-05-21T03:24:01.659855+0000","last_became_active":"2024-05-21T03:20:32.135302+0000","last_became_peered":"2024-05-21T03:20:32.135302+0000","last_unstale":"2024-05-21T03:24:01.659855+0000","last_undegraded":"2024-05-21T03:24:01.659855+0000","last_fullsized":"2024-05-21T03:24:01.659855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:05:55.000690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656928+0000","last_change":"2024-05-21T03:20:32.146580+0000","last_active":"2024-05-21T03:24:01.656928+0000","last_peered":"2024-05-21T03:24:01.656928+0000","last_clean":"2024-05-21T03:24:01.656928+0000","last_became_active":"2024-05-21T03:20:32.145803+0000","last_became_peered":"2024-05-21T03:20:32.145803+0000","last_unstale":"2024-05-21T03:24:01.656928+0000","last_undegraded":"2024-05-21T03:24:01.656928+0000","last_fullsized":"2024-05-21T03:24:01.656928+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:34:09.108799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656841+0000","last_change":"2024-05-21T03:21:14.559498+0000","last_active":"2024-05-21T03:24:01.656841+0000","last_peered":"2024-05-21T03:24:01.656841+0000","last_clean":"2024-05-21T03:24:01.656841+0000","last_became_active":"2024-05-21T03:21:14.559039+0000","last_became_peered":"2024-05-21T03:21:14.559039+0000","last_unstale":"2024-05-21T03:24:01.656841+0000","last_undegraded":"2024-05-21T03:24:01.656841+0000","last_fullsized":"2024-05-21T03:24:01.656841+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:04:36.676857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659265+0000","last_change":"2024-05-21T03:20:32.148944+0000","last_active":"2024-05-21T03:24:01.659265+0000","last_peered":"2024-05-21T03:24:01.659265+0000","last_clean":"2024-05-21T03:24:01.659265+0000","last_became_active":"2024-05-21T03:20:32.148502+0000","last_became_peered":"2024-05-21T03:20:32.148502+0000","last_unstale":"2024-05-21T03:24:01.659265+0000","last_undegraded":"2024-05-21T03:24:01.659265+0000","last_fullsized":"2024-05-21T03:24:01.659265+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:11:00.178105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657063+0000","last_change":"2024-05-21T03:20:32.150609+0000","last_active":"2024-05-21T03:24:01.657063+0000","last_peered":"2024-05-21T03:24:01.657063+0000","last_clean":"2024-05-21T03:24:01.657063+0000","last_became_active":"2024-05-21T03:20:32.150314+0000","last_became_peered":"2024-05-21T03:20:32.150314+0000","last_unstale":"2024-05-21T03:24:01.657063+0000","last_undegraded":"2024-05-21T03:24:01.657063+0000","last_fullsized":"2024-05-21T03:24:01.657063+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:34:21.869304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.387785+0000","last_change":"2024-05-21T03:20:32.142472+0000","last_active":"2024-05-21T03:24:01.387785+0000","last_peered":"2024-05-21T03:24:01.387785+0000","last_clean":"2024-05-21T03:24:01.387785+0000","last_became_active":"2024-05-21T03:20:32.142161+0000","last_became_peered":"2024-05-21T03:20:32.142161+0000","last_unstale":"2024-05-21T03:24:01.387785+0000","last_undegraded":"2024-05-21T03:24:01.387785+0000","last_fullsized":"2024-05-21T03:24:01.387785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:26:57.281655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.389229+0000","last_change":"2024-05-21T03:20:32.153987+0000","last_active":"2024-05-21T03:24:01.389229+0000","last_peered":"2024-05-21T03:24:01.389229+0000","last_clean":"2024-05-21T03:24:01.389229+0000","last_became_active":"2024-05-21T03:20:32.153617+0000","last_became_peered":"2024-05-21T03:20:32.153617+0000","last_unstale":"2024-05-21T03:24:01.389229+0000","last_undegraded":"2024-05-21T03:24:01.389229+0000","last_fullsized":"2024-05-21T03:24:01.389229+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:57:07.341758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'15","reported_seq":241,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:05.762719+0000","last_change":"2024-05-21T03:20:29.099928+0000","last_active":"2024-05-21T03:24:05.762719+0000","last_peered":"2024-05-21T03:24:05.762719+0000","last_clean":"2024-05-21T03:24:05.762719+0000","last_became_active":"2024-05-21T03:20:29.099592+0000","last_became_peered":"2024-05-21T03:20:29.099592+0000","last_unstale":"2024-05-21T03:24:05.762719+0000","last_undegraded":"2024-05-21T03:24:05.762719+0000","last_fullsized":"2024-05-21T03:24:05.762719+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:35:34.843325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":198,"num_read_kb":198,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491831+0000","last_change":"2024-05-21T03:20:32.154962+0000","last_active":"2024-05-21T03:24:01.491831+0000","last_peered":"2024-05-21T03:24:01.491831+0000","last_clean":"2024-05-21T03:24:01.491831+0000","last_became_active":"2024-05-21T03:20:32.154620+0000","last_became_peered":"2024-05-21T03:20:32.154620+0000","last_unstale":"2024-05-21T03:24:01.491831+0000","last_undegraded":"2024-05-21T03:24:01.491831+0000","last_fullsized":"2024-05-21T03:24:01.491831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:56:21.553272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.492006+0000","last_change":"2024-05-21T03:20:29.097312+0000","last_active":"2024-05-21T03:24:01.492006+0000","last_peered":"2024-05-21T03:24:01.492006+0000","last_clean":"2024-05-21T03:24:01.492006+0000","last_became_active":"2024-05-21T03:20:29.096977+0000","last_became_peered":"2024-05-21T03:20:29.096977+0000","last_unstale":"2024-05-21T03:24:01.492006+0000","last_undegraded":"2024-05-21T03:24:01.492006+0000","last_fullsized":"2024-05-21T03:24:01.492006+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:28:45.467446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657332+0000","last_change":"2024-05-21T03:20:32.150035+0000","last_active":"2024-05-21T03:24:01.657332+0000","last_peered":"2024-05-21T03:24:01.657332+0000","last_clean":"2024-05-21T03:24:01.657332+0000","last_became_active":"2024-05-21T03:20:32.149688+0000","last_became_peered":"2024-05-21T03:20:32.149688+0000","last_unstale":"2024-05-21T03:24:01.657332+0000","last_undegraded":"2024-05-21T03:24:01.657332+0000","last_fullsized":"2024-05-21T03:24:01.657332+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:27:00.410770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656922+0000","last_change":"2024-05-21T03:21:14.572832+0000","last_active":"2024-05-21T03:24:01.656922+0000","last_peered":"2024-05-21T03:24:01.656922+0000","last_clean":"2024-05-21T03:24:01.656922+0000","last_became_active":"2024-05-21T03:21:14.559381+0000","last_became_peered":"2024-05-21T03:21:14.559381+0000","last_unstale":"2024-05-21T03:24:01.656922+0000","last_undegraded":"2024-05-21T03:24:01.656922+0000","last_fullsized":"2024-05-21T03:24:01.656922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:02:53.004031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659270+0000","last_change":"2024-05-21T03:20:32.150301+0000","last_active":"2024-05-21T03:24:01.659270+0000","last_peered":"2024-05-21T03:24:01.659270+0000","last_clean":"2024-05-21T03:24:01.659270+0000","last_became_active":"2024-05-21T03:20:32.148475+0000","last_became_peered":"2024-05-21T03:20:32.148475+0000","last_unstale":"2024-05-21T03:24:01.659270+0000","last_undegraded":"2024-05-21T03:24:01.659270+0000","last_fullsized":"2024-05-21T03:24:01.659270+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:59:30.339660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491705+0000","last_change":"2024-05-21T03:20:29.098861+0000","last_active":"2024-05-21T03:24:01.491705+0000","last_peered":"2024-05-21T03:24:01.491705+0000","last_clean":"2024-05-21T03:24:01.491705+0000","last_became_active":"2024-05-21T03:20:29.097874+0000","last_became_peered":"2024-05-21T03:20:29.097874+0000","last_unstale":"2024-05-21T03:24:01.491705+0000","last_undegraded":"2024-05-21T03:24:01.491705+0000","last_fullsized":"2024-05-21T03:24:01.491705+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:25:36.227339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.499470+0000","last_change":"2024-05-21T03:20:32.142820+0000","last_active":"2024-05-21T03:24:01.499470+0000","last_peered":"2024-05-21T03:24:01.499470+0000","last_clean":"2024-05-21T03:24:01.499470+0000","last_became_active":"2024-05-21T03:20:32.142396+0000","last_became_peered":"2024-05-21T03:20:32.142396+0000","last_unstale":"2024-05-21T03:24:01.499470+0000","last_undegraded":"2024-05-21T03:24:01.499470+0000","last_fullsized":"2024-05-21T03:24:01.499470+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:30:14.452389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":233,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:05.752989+0000","last_change":"2024-05-21T03:20:29.103153+0000","last_active":"2024-05-21T03:24:05.752989+0000","last_peered":"2024-05-21T03:24:05.752989+0000","last_clean":"2024-05-21T03:24:05.752989+0000","last_became_active":"2024-05-21T03:20:29.102819+0000","last_became_peered":"2024-05-21T03:20:29.102819+0000","last_unstale":"2024-05-21T03:24:05.752989+0000","last_undegraded":"2024-05-21T03:24:05.752989+0000","last_fullsized":"2024-05-21T03:24:05.752989+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:47:49.843926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":199,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491481+0000","last_change":"2024-05-21T03:20:32.142263+0000","last_active":"2024-05-21T03:24:01.491481+0000","last_peered":"2024-05-21T03:24:01.491481+0000","last_clean":"2024-05-21T03:24:01.491481+0000","last_became_active":"2024-05-21T03:20:32.141820+0000","last_became_peered":"2024-05-21T03:20:32.141820+0000","last_unstale":"2024-05-21T03:24:01.491481+0000","last_undegraded":"2024-05-21T03:24:01.491481+0000","last_fullsized":"2024-05-21T03:24:01.491481+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:45:20.021237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.502625+0000","last_change":"2024-05-21T03:20:29.100088+0000","last_active":"2024-05-21T03:24:01.502625+0000","last_peered":"2024-05-21T03:24:01.502625+0000","last_clean":"2024-05-21T03:24:01.502625+0000","last_became_active":"2024-05-21T03:20:29.097343+0000","last_became_peered":"2024-05-21T03:20:29.097343+0000","last_unstale":"2024-05-21T03:24:01.502625+0000","last_undegraded":"2024-05-21T03:24:01.502625+0000","last_fullsized":"2024-05-21T03:24:01.502625+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:00:35.682393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.387512+0000","last_change":"2024-05-21T03:21:14.562281+0000","last_active":"2024-05-21T03:24:01.387512+0000","last_peered":"2024-05-21T03:24:01.387512+0000","last_clean":"2024-05-21T03:24:01.387512+0000","last_became_active":"2024-05-21T03:21:14.561857+0000","last_became_peered":"2024-05-21T03:21:14.561857+0000","last_unstale":"2024-05-21T03:24:01.387512+0000","last_undegraded":"2024-05-21T03:24:01.387512+0000","last_fullsized":"2024-05-21T03:24:01.387512+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:37:35.283529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659672+0000","last_change":"2024-05-21T03:20:29.097945+0000","last_active":"2024-05-21T03:24:01.659672+0000","last_peered":"2024-05-21T03:24:01.659672+0000","last_clean":"2024-05-21T03:24:01.659672+0000","last_became_active":"2024-05-21T03:20:29.097624+0000","last_became_peered":"2024-05-21T03:20:29.097624+0000","last_unstale":"2024-05-21T03:24:01.659672+0000","last_undegraded":"2024-05-21T03:24:01.659672+0000","last_fullsized":"2024-05-21T03:24:01.659672+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:01:01.694893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491268+0000","last_change":"2024-05-21T03:20:32.148955+0000","last_active":"2024-05-21T03:24:01.491268+0000","last_peered":"2024-05-21T03:24:01.491268+0000","last_clean":"2024-05-21T03:24:01.491268+0000","last_became_active":"2024-05-21T03:20:32.148631+0000","last_became_peered":"2024-05-21T03:20:32.148631+0000","last_unstale":"2024-05-21T03:24:01.491268+0000","last_undegraded":"2024-05-21T03:24:01.491268+0000","last_fullsized":"2024-05-21T03:24:01.491268+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:36:25.820580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.499174+0000","last_change":"2024-05-21T03:20:29.100902+0000","last_active":"2024-05-21T03:24:01.499174+0000","last_peered":"2024-05-21T03:24:01.499174+0000","last_clean":"2024-05-21T03:24:01.499174+0000","last_became_active":"2024-05-21T03:20:29.100570+0000","last_became_peered":"2024-05-21T03:20:29.100570+0000","last_unstale":"2024-05-21T03:24:01.499174+0000","last_undegraded":"2024-05-21T03:24:01.499174+0000","last_fullsized":"2024-05-21T03:24:01.499174+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:58:36.651338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.387428+0000","last_change":"2024-05-21T03:20:32.150724+0000","last_active":"2024-05-21T03:24:01.387428+0000","last_peered":"2024-05-21T03:24:01.387428+0000","last_clean":"2024-05-21T03:24:01.387428+0000","last_became_active":"2024-05-21T03:20:32.150371+0000","last_became_peered":"2024-05-21T03:20:32.150371+0000","last_unstale":"2024-05-21T03:24:01.387428+0000","last_undegraded":"2024-05-21T03:24:01.387428+0000","last_fullsized":"2024-05-21T03:24:01.387428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:44:32.321192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'6","reported_seq":429,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:06.753618+0000","last_change":"2024-05-21T03:20:29.095045+0000","last_active":"2024-05-21T03:24:06.753618+0000","last_peered":"2024-05-21T03:24:06.753618+0000","last_clean":"2024-05-21T03:24:06.753618+0000","last_became_active":"2024-05-21T03:20:29.094552+0000","last_became_peered":"2024-05-21T03:20:29.094552+0000","last_unstale":"2024-05-21T03:24:06.753618+0000","last_undegraded":"2024-05-21T03:24:06.753618+0000","last_fullsized":"2024-05-21T03:24:06.753618+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:36:15.155125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":394,"num_read_kb":394,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659781+0000","last_change":"2024-05-21T03:20:32.149813+0000","last_active":"2024-05-21T03:24:01.659781+0000","last_peered":"2024-05-21T03:24:01.659781+0000","last_clean":"2024-05-21T03:24:01.659781+0000","last_became_active":"2024-05-21T03:20:32.148422+0000","last_became_peered":"2024-05-21T03:20:32.148422+0000","last_unstale":"2024-05-21T03:24:01.659781+0000","last_undegraded":"2024-05-21T03:24:01.659781+0000","last_fullsized":"2024-05-21T03:24:01.659781+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:55:35.115463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657337+0000","last_change":"2024-05-21T03:20:29.101463+0000","last_active":"2024-05-21T03:24:01.657337+0000","last_peered":"2024-05-21T03:24:01.657337+0000","last_clean":"2024-05-21T03:24:01.657337+0000","last_became_active":"2024-05-21T03:20:29.101127+0000","last_became_peered":"2024-05-21T03:20:29.101127+0000","last_unstale":"2024-05-21T03:24:01.657337+0000","last_undegraded":"2024-05-21T03:24:01.657337+0000","last_fullsized":"2024-05-21T03:24:01.657337+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:58:17.408315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657158+0000","last_change":"2024-05-21T03:21:14.563434+0000","last_active":"2024-05-21T03:24:01.657158+0000","last_peered":"2024-05-21T03:24:01.657158+0000","last_clean":"2024-05-21T03:24:01.657158+0000","last_became_active":"2024-05-21T03:21:14.563081+0000","last_became_peered":"2024-05-21T03:21:14.563081+0000","last_unstale":"2024-05-21T03:24:01.657158+0000","last_undegraded":"2024-05-21T03:24:01.657158+0000","last_fullsized":"2024-05-21T03:24:01.657158+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:28:55.497338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.499583+0000","last_change":"2024-05-21T03:20:29.098725+0000","last_active":"2024-05-21T03:24:01.499583+0000","last_peered":"2024-05-21T03:24:01.499583+0000","last_clean":"2024-05-21T03:24:01.499583+0000","last_became_active":"2024-05-21T03:20:29.098471+0000","last_became_peered":"2024-05-21T03:20:29.098471+0000","last_unstale":"2024-05-21T03:24:01.499583+0000","last_undegraded":"2024-05-21T03:24:01.499583+0000","last_fullsized":"2024-05-21T03:24:01.499583+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:17:11.059825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659799+0000","last_change":"2024-05-21T03:21:14.560975+0000","last_active":"2024-05-21T03:24:01.659799+0000","last_peered":"2024-05-21T03:24:01.659799+0000","last_clean":"2024-05-21T03:24:01.659799+0000","last_became_active":"2024-05-21T03:21:14.560528+0000","last_became_peered":"2024-05-21T03:21:14.560528+0000","last_unstale":"2024-05-21T03:24:01.659799+0000","last_undegraded":"2024-05-21T03:24:01.659799+0000","last_fullsized":"2024-05-21T03:24:01.659799+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:51:47.469291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'39","reported_seq":475,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:06.753362+0000","last_change":"2024-05-21T03:20:29.099968+0000","last_active":"2024-05-21T03:24:06.753362+0000","last_peered":"2024-05-21T03:24:06.753362+0000","last_clean":"2024-05-21T03:24:06.753362+0000","last_became_active":"2024-05-21T03:20:29.099181+0000","last_became_peered":"2024-05-21T03:20:29.099181+0000","last_unstale":"2024-05-21T03:24:06.753362+0000","last_undegraded":"2024-05-21T03:24:06.753362+0000","last_fullsized":"2024-05-21T03:24:06.753362+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:35:33.934412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66474,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":407,"num_read_kb":268,"num_write":41,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491786+0000","last_change":"2024-05-21T03:20:32.144024+0000","last_active":"2024-05-21T03:24:01.491786+0000","last_peered":"2024-05-21T03:24:01.491786+0000","last_clean":"2024-05-21T03:24:01.491786+0000","last_became_active":"2024-05-21T03:20:32.143668+0000","last_became_peered":"2024-05-21T03:20:32.143668+0000","last_unstale":"2024-05-21T03:24:01.491786+0000","last_undegraded":"2024-05-21T03:24:01.491786+0000","last_fullsized":"2024-05-21T03:24:01.491786+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:11:31.100802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.389042+0000","last_change":"2024-05-21T03:20:29.101770+0000","last_active":"2024-05-21T03:24:01.389042+0000","last_peered":"2024-05-21T03:24:01.389042+0000","last_clean":"2024-05-21T03:24:01.389042+0000","last_became_active":"2024-05-21T03:20:29.100407+0000","last_became_peered":"2024-05-21T03:20:29.100407+0000","last_unstale":"2024-05-21T03:24:01.389042+0000","last_undegraded":"2024-05-21T03:24:01.389042+0000","last_fullsized":"2024-05-21T03:24:01.389042+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:01:33.991246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659520+0000","last_change":"2024-05-21T03:20:32.147479+0000","last_active":"2024-05-21T03:24:01.659520+0000","last_peered":"2024-05-21T03:24:01.659520+0000","last_clean":"2024-05-21T03:24:01.659520+0000","last_became_active":"2024-05-21T03:20:32.147160+0000","last_became_peered":"2024-05-21T03:20:32.147160+0000","last_unstale":"2024-05-21T03:24:01.659520+0000","last_undegraded":"2024-05-21T03:24:01.659520+0000","last_fullsized":"2024-05-21T03:24:01.659520+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:51:59.343380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.499274+0000","last_change":"2024-05-21T03:20:29.101875+0000","last_active":"2024-05-21T03:24:01.499274+0000","last_peered":"2024-05-21T03:24:01.499274+0000","last_clean":"2024-05-21T03:24:01.499274+0000","last_became_active":"2024-05-21T03:20:29.101468+0000","last_became_peered":"2024-05-21T03:20:29.101468+0000","last_unstale":"2024-05-21T03:24:01.499274+0000","last_undegraded":"2024-05-21T03:24:01.499274+0000","last_fullsized":"2024-05-21T03:24:01.499274+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:10:19.577836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657039+0000","last_change":"2024-05-21T03:20:32.139002+0000","last_active":"2024-05-21T03:24:01.657039+0000","last_peered":"2024-05-21T03:24:01.657039+0000","last_clean":"2024-05-21T03:24:01.657039+0000","last_became_active":"2024-05-21T03:20:32.138619+0000","last_became_peered":"2024-05-21T03:20:32.138619+0000","last_unstale":"2024-05-21T03:24:01.657039+0000","last_undegraded":"2024-05-21T03:24:01.657039+0000","last_fullsized":"2024-05-21T03:24:01.657039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:21:36.136283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'8","reported_seq":238,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:05.752720+0000","last_change":"2024-05-21T03:20:29.101532+0000","last_active":"2024-05-21T03:24:05.752720+0000","last_peered":"2024-05-21T03:24:05.752720+0000","last_clean":"2024-05-21T03:24:05.752720+0000","last_became_active":"2024-05-21T03:20:29.101206+0000","last_became_peered":"2024-05-21T03:20:29.101206+0000","last_unstale":"2024-05-21T03:24:05.752720+0000","last_undegraded":"2024-05-21T03:24:05.752720+0000","last_fullsized":"2024-05-21T03:24:05.752720+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T12:47:38.555995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":203,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656787+0000","last_change":"2024-05-21T03:20:32.146866+0000","last_active":"2024-05-21T03:24:01.656787+0000","last_peered":"2024-05-21T03:24:01.656787+0000","last_clean":"2024-05-21T03:24:01.656787+0000","last_became_active":"2024-05-21T03:20:32.146463+0000","last_became_peered":"2024-05-21T03:20:32.146463+0000","last_unstale":"2024-05-21T03:24:01.656787+0000","last_undegraded":"2024-05-21T03:24:01.656787+0000","last_fullsized":"2024-05-21T03:24:01.656787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:06:54.359922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491442+0000","last_change":"2024-05-21T03:20:29.103910+0000","last_active":"2024-05-21T03:24:01.491442+0000","last_peered":"2024-05-21T03:24:01.491442+0000","last_clean":"2024-05-21T03:24:01.491442+0000","last_became_active":"2024-05-21T03:20:29.103583+0000","last_became_peered":"2024-05-21T03:20:29.103583+0000","last_unstale":"2024-05-21T03:24:01.491442+0000","last_undegraded":"2024-05-21T03:24:01.491442+0000","last_fullsized":"2024-05-21T03:24:01.491442+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:57:15.665158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.499198+0000","last_change":"2024-05-21T03:20:32.141242+0000","last_active":"2024-05-21T03:24:01.499198+0000","last_peered":"2024-05-21T03:24:01.499198+0000","last_clean":"2024-05-21T03:24:01.499198+0000","last_became_active":"2024-05-21T03:20:32.140892+0000","last_became_peered":"2024-05-21T03:20:32.140892+0000","last_unstale":"2024-05-21T03:24:01.499198+0000","last_undegraded":"2024-05-21T03:24:01.499198+0000","last_fullsized":"2024-05-21T03:24:01.499198+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:42:13.997708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'3","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491997+0000","last_change":"2024-05-21T03:20:29.098973+0000","last_active":"2024-05-21T03:24:01.491997+0000","last_peered":"2024-05-21T03:24:01.491997+0000","last_clean":"2024-05-21T03:24:01.491997+0000","last_became_active":"2024-05-21T03:20:29.098694+0000","last_became_peered":"2024-05-21T03:20:29.098694+0000","last_unstale":"2024-05-21T03:24:01.491997+0000","last_undegraded":"2024-05-21T03:24:01.491997+0000","last_fullsized":"2024-05-21T03:24:01.491997+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:55:17.915820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656862+0000","last_change":"2024-05-21T03:20:32.147465+0000","last_active":"2024-05-21T03:24:01.656862+0000","last_peered":"2024-05-21T03:24:01.656862+0000","last_clean":"2024-05-21T03:24:01.656862+0000","last_became_active":"2024-05-21T03:20:32.147073+0000","last_became_peered":"2024-05-21T03:20:32.147073+0000","last_unstale":"2024-05-21T03:24:01.656862+0000","last_undegraded":"2024-05-21T03:24:01.656862+0000","last_fullsized":"2024-05-21T03:24:01.656862+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:29:05.451370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.387615+0000","last_change":"2024-05-21T03:21:14.560568+0000","last_active":"2024-05-21T03:24:01.387615+0000","last_peered":"2024-05-21T03:24:01.387615+0000","last_clean":"2024-05-21T03:24:01.387615+0000","last_became_active":"2024-05-21T03:21:14.560093+0000","last_became_peered":"2024-05-21T03:21:14.560093+0000","last_unstale":"2024-05-21T03:24:01.387615+0000","last_undegraded":"2024-05-21T03:24:01.387615+0000","last_fullsized":"2024-05-21T03:24:01.387615+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:55:39.839942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.502820+0000","last_change":"2024-05-21T03:20:32.146456+0000","last_active":"2024-05-21T03:24:01.502820+0000","last_peered":"2024-05-21T03:24:01.502820+0000","last_clean":"2024-05-21T03:24:01.502820+0000","last_became_active":"2024-05-21T03:20:32.146105+0000","last_became_peered":"2024-05-21T03:20:32.146105+0000","last_unstale":"2024-05-21T03:24:01.502820+0000","last_undegraded":"2024-05-21T03:24:01.502820+0000","last_fullsized":"2024-05-21T03:24:01.502820+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:00:30.550178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'8","reported_seq":237,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:05.763201+0000","last_change":"2024-05-21T03:20:29.100262+0000","last_active":"2024-05-21T03:24:05.763201+0000","last_peered":"2024-05-21T03:24:05.763201+0000","last_clean":"2024-05-21T03:24:05.763201+0000","last_became_active":"2024-05-21T03:20:29.099917+0000","last_became_peered":"2024-05-21T03:20:29.099917+0000","last_unstale":"2024-05-21T03:24:05.763201+0000","last_undegraded":"2024-05-21T03:24:05.763201+0000","last_fullsized":"2024-05-21T03:24:05.763201+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:55:48.608341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":201,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.502772+0000","last_change":"2024-05-21T03:20:32.142776+0000","last_active":"2024-05-21T03:24:01.502772+0000","last_peered":"2024-05-21T03:24:01.502772+0000","last_clean":"2024-05-21T03:24:01.502772+0000","last_became_active":"2024-05-21T03:20:32.142430+0000","last_became_peered":"2024-05-21T03:20:32.142430+0000","last_unstale":"2024-05-21T03:24:01.502772+0000","last_undegraded":"2024-05-21T03:24:01.502772+0000","last_fullsized":"2024-05-21T03:24:01.502772+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:54:28.682343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.499435+0000","last_change":"2024-05-21T03:20:32.139506+0000","last_active":"2024-05-21T03:24:01.499435+0000","last_peered":"2024-05-21T03:24:01.499435+0000","last_clean":"2024-05-21T03:24:01.499435+0000","last_became_active":"2024-05-21T03:20:32.138136+0000","last_became_peered":"2024-05-21T03:20:32.138136+0000","last_unstale":"2024-05-21T03:24:01.499435+0000","last_undegraded":"2024-05-21T03:24:01.499435+0000","last_fullsized":"2024-05-21T03:24:01.499435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:31:24.996276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'7","reported_seq":234,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:05.752358+0000","last_change":"2024-05-21T03:20:29.096546+0000","last_active":"2024-05-21T03:24:05.752358+0000","last_peered":"2024-05-21T03:24:05.752358+0000","last_clean":"2024-05-21T03:24:05.752358+0000","last_became_active":"2024-05-21T03:20:29.096184+0000","last_became_peered":"2024-05-21T03:20:29.096184+0000","last_unstale":"2024-05-21T03:24:05.752358+0000","last_undegraded":"2024-05-21T03:24:05.752358+0000","last_fullsized":"2024-05-21T03:24:05.752358+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:09:12.832745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":198,"num_read_kb":198,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491686+0000","last_change":"2024-05-21T03:20:32.156231+0000","last_active":"2024-05-21T03:24:01.491686+0000","last_peered":"2024-05-21T03:24:01.491686+0000","last_clean":"2024-05-21T03:24:01.491686+0000","last_became_active":"2024-05-21T03:20:32.154035+0000","last_became_peered":"2024-05-21T03:20:32.154035+0000","last_unstale":"2024-05-21T03:24:01.491686+0000","last_undegraded":"2024-05-21T03:24:01.491686+0000","last_fullsized":"2024-05-21T03:24:01.491686+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:30:53.820292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.388854+0000","last_change":"2024-05-21T03:20:29.100422+0000","last_active":"2024-05-21T03:24:01.388854+0000","last_peered":"2024-05-21T03:24:01.388854+0000","last_clean":"2024-05-21T03:24:01.388854+0000","last_became_active":"2024-05-21T03:20:29.099993+0000","last_became_peered":"2024-05-21T03:20:29.099993+0000","last_unstale":"2024-05-21T03:24:01.388854+0000","last_undegraded":"2024-05-21T03:24:01.388854+0000","last_fullsized":"2024-05-21T03:24:01.388854+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:03:15.456447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"83'99","reported_seq":130,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:03.495299+0000","last_change":"2024-05-21T03:19:05.647808+0000","last_active":"2024-05-21T03:24:03.495299+0000","last_peered":"2024-05-21T03:24:03.495299+0000","last_clean":"2024-05-21T03:24:03.495299+0000","last_became_active":"2024-05-21T03:19:05.641865+0000","last_became_peered":"2024-05-21T03:19:05.641865+0000","last_unstale":"2024-05-21T03:24:03.495299+0000","last_undegraded":"2024-05-21T03:24:03.495299+0000","last_fullsized":"2024-05-21T03:24:03.495299+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:15:42.008493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:15:42.008493+0000","last_clean_scrub_stamp":"2024-05-21T03:15:42.008493+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:26:58.040196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659351+0000","last_change":"2024-05-21T03:20:32.138182+0000","last_active":"2024-05-21T03:24:01.659351+0000","last_peered":"2024-05-21T03:24:01.659351+0000","last_clean":"2024-05-21T03:24:01.659351+0000","last_became_active":"2024-05-21T03:20:32.137842+0000","last_became_peered":"2024-05-21T03:20:32.137842+0000","last_unstale":"2024-05-21T03:24:01.659351+0000","last_undegraded":"2024-05-21T03:24:01.659351+0000","last_fullsized":"2024-05-21T03:24:01.659351+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:43:50.070160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657022+0000","last_change":"2024-05-21T03:20:29.104246+0000","last_active":"2024-05-21T03:24:01.657022+0000","last_peered":"2024-05-21T03:24:01.657022+0000","last_clean":"2024-05-21T03:24:01.657022+0000","last_became_active":"2024-05-21T03:20:29.103910+0000","last_became_peered":"2024-05-21T03:20:29.103910+0000","last_unstale":"2024-05-21T03:24:01.657022+0000","last_undegraded":"2024-05-21T03:24:01.657022+0000","last_fullsized":"2024-05-21T03:24:01.657022+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:39:28.029283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491818+0000","last_change":"2024-05-21T03:20:29.102020+0000","last_active":"2024-05-21T03:24:01.491818+0000","last_peered":"2024-05-21T03:24:01.491818+0000","last_clean":"2024-05-21T03:24:01.491818+0000","last_became_active":"2024-05-21T03:20:29.100429+0000","last_became_peered":"2024-05-21T03:20:29.100429+0000","last_unstale":"2024-05-21T03:24:01.491818+0000","last_undegraded":"2024-05-21T03:24:01.491818+0000","last_fullsized":"2024-05-21T03:24:01.491818+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:09:01.621074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659598+0000","last_change":"2024-05-21T03:20:32.137130+0000","last_active":"2024-05-21T03:24:01.659598+0000","last_peered":"2024-05-21T03:24:01.659598+0000","last_clean":"2024-05-21T03:24:01.659598+0000","last_became_active":"2024-05-21T03:20:32.136821+0000","last_became_peered":"2024-05-21T03:20:32.136821+0000","last_unstale":"2024-05-21T03:24:01.659598+0000","last_undegraded":"2024-05-21T03:24:01.659598+0000","last_fullsized":"2024-05-21T03:24:01.659598+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:53:41.290370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'103","reported_seq":223,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:03.678190+0000","last_change":"2024-05-21T03:20:29.099869+0000","last_active":"2024-05-21T03:24:03.678190+0000","last_peered":"2024-05-21T03:24:03.678190+0000","last_clean":"2024-05-21T03:24:03.678190+0000","last_became_active":"2024-05-21T03:20:29.099534+0000","last_became_peered":"2024-05-21T03:20:29.099534+0000","last_unstale":"2024-05-21T03:24:03.678190+0000","last_undegraded":"2024-05-21T03:24:03.678190+0000","last_fullsized":"2024-05-21T03:24:03.678190+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:01:11.281544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":188,"num_write_kb":1285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491991+0000","last_change":"2024-05-21T03:20:32.153921+0000","last_active":"2024-05-21T03:24:01.491991+0000","last_peered":"2024-05-21T03:24:01.491991+0000","last_clean":"2024-05-21T03:24:01.491991+0000","last_became_active":"2024-05-21T03:20:32.153594+0000","last_became_peered":"2024-05-21T03:20:32.153594+0000","last_unstale":"2024-05-21T03:24:01.491991+0000","last_undegraded":"2024-05-21T03:24:01.491991+0000","last_fullsized":"2024-05-21T03:24:01.491991+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:27:59.426942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.389210+0000","last_change":"2024-05-21T03:20:29.097399+0000","last_active":"2024-05-21T03:24:01.389210+0000","last_peered":"2024-05-21T03:24:01.389210+0000","last_clean":"2024-05-21T03:24:01.389210+0000","last_became_active":"2024-05-21T03:20:29.097010+0000","last_became_peered":"2024-05-21T03:20:29.097010+0000","last_unstale":"2024-05-21T03:24:01.389210+0000","last_undegraded":"2024-05-21T03:24:01.389210+0000","last_fullsized":"2024-05-21T03:24:01.389210+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:54:53.273947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657141+0000","last_change":"2024-05-21T03:21:14.559609+0000","last_active":"2024-05-21T03:24:01.657141+0000","last_peered":"2024-05-21T03:24:01.657141+0000","last_clean":"2024-05-21T03:24:01.657141+0000","last_became_active":"2024-05-21T03:21:14.559218+0000","last_became_peered":"2024-05-21T03:21:14.559218+0000","last_unstale":"2024-05-21T03:24:01.657141+0000","last_undegraded":"2024-05-21T03:24:01.657141+0000","last_fullsized":"2024-05-21T03:24:01.657141+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T13:32:11.156037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,4],"acting":[0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659469+0000","last_change":"2024-05-21T03:20:29.103526+0000","last_active":"2024-05-21T03:24:01.659469+0000","last_peered":"2024-05-21T03:24:01.659469+0000","last_clean":"2024-05-21T03:24:01.659469+0000","last_became_active":"2024-05-21T03:20:29.103067+0000","last_became_peered":"2024-05-21T03:20:29.103067+0000","last_unstale":"2024-05-21T03:24:01.659469+0000","last_undegraded":"2024-05-21T03:24:01.659469+0000","last_fullsized":"2024-05-21T03:24:01.659469+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:54:47.566949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491463+0000","last_change":"2024-05-21T03:20:32.141088+0000","last_active":"2024-05-21T03:24:01.491463+0000","last_peered":"2024-05-21T03:24:01.491463+0000","last_clean":"2024-05-21T03:24:01.491463+0000","last_became_active":"2024-05-21T03:20:32.140744+0000","last_became_peered":"2024-05-21T03:20:32.140744+0000","last_unstale":"2024-05-21T03:24:01.491463+0000","last_undegraded":"2024-05-21T03:24:01.491463+0000","last_fullsized":"2024-05-21T03:24:01.491463+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:57:35.591164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491373+0000","last_change":"2024-05-21T03:20:29.093440+0000","last_active":"2024-05-21T03:24:01.491373+0000","last_peered":"2024-05-21T03:24:01.491373+0000","last_clean":"2024-05-21T03:24:01.491373+0000","last_became_active":"2024-05-21T03:20:29.092935+0000","last_became_peered":"2024-05-21T03:20:29.092935+0000","last_unstale":"2024-05-21T03:24:01.491373+0000","last_undegraded":"2024-05-21T03:24:01.491373+0000","last_fullsized":"2024-05-21T03:24:01.491373+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T10:18:00.690465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656650+0000","last_change":"2024-05-21T03:20:32.143781+0000","last_active":"2024-05-21T03:24:01.656650+0000","last_peered":"2024-05-21T03:24:01.656650+0000","last_clean":"2024-05-21T03:24:01.656650+0000","last_became_active":"2024-05-21T03:20:32.143391+0000","last_became_peered":"2024-05-21T03:20:32.143391+0000","last_unstale":"2024-05-21T03:24:01.656650+0000","last_undegraded":"2024-05-21T03:24:01.656650+0000","last_fullsized":"2024-05-21T03:24:01.656650+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:00:41.506561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'4","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.389141+0000","last_change":"2024-05-21T03:20:29.101342+0000","last_active":"2024-05-21T03:24:01.389141+0000","last_peered":"2024-05-21T03:24:01.389141+0000","last_clean":"2024-05-21T03:24:01.389141+0000","last_became_active":"2024-05-21T03:20:29.100997+0000","last_became_peered":"2024-05-21T03:20:29.100997+0000","last_unstale":"2024-05-21T03:24:01.389141+0000","last_undegraded":"2024-05-21T03:24:01.389141+0000","last_fullsized":"2024-05-21T03:24:01.389141+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:05:31.801669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491844+0000","last_change":"2024-05-21T03:20:32.150563+0000","last_active":"2024-05-21T03:24:01.491844+0000","last_peered":"2024-05-21T03:24:01.491844+0000","last_clean":"2024-05-21T03:24:01.491844+0000","last_became_active":"2024-05-21T03:20:32.150231+0000","last_became_peered":"2024-05-21T03:20:32.150231+0000","last_unstale":"2024-05-21T03:24:01.491844+0000","last_undegraded":"2024-05-21T03:24:01.491844+0000","last_fullsized":"2024-05-21T03:24:01.491844+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:10:00.251284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491635+0000","last_change":"2024-05-21T03:20:29.101576+0000","last_active":"2024-05-21T03:24:01.491635+0000","last_peered":"2024-05-21T03:24:01.491635+0000","last_clean":"2024-05-21T03:24:01.491635+0000","last_became_active":"2024-05-21T03:20:29.101270+0000","last_became_peered":"2024-05-21T03:20:29.101270+0000","last_unstale":"2024-05-21T03:24:01.491635+0000","last_undegraded":"2024-05-21T03:24:01.491635+0000","last_fullsized":"2024-05-21T03:24:01.491635+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:50:10.579901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656976+0000","last_change":"2024-05-21T03:20:32.141340+0000","last_active":"2024-05-21T03:24:01.656976+0000","last_peered":"2024-05-21T03:24:01.656976+0000","last_clean":"2024-05-21T03:24:01.656976+0000","last_became_active":"2024-05-21T03:20:32.140792+0000","last_became_peered":"2024-05-21T03:20:32.140792+0000","last_unstale":"2024-05-21T03:24:01.656976+0000","last_undegraded":"2024-05-21T03:24:01.656976+0000","last_fullsized":"2024-05-21T03:24:01.656976+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:35:05.842929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'10","reported_seq":202,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:05.828704+0000","last_change":"2024-05-21T03:21:14.566886+0000","last_active":"2024-05-21T03:24:05.828704+0000","last_peered":"2024-05-21T03:24:05.828704+0000","last_clean":"2024-05-21T03:24:05.828704+0000","last_became_active":"2024-05-21T03:21:14.559816+0000","last_became_peered":"2024-05-21T03:21:14.559816+0000","last_unstale":"2024-05-21T03:24:05.828704+0000","last_undegraded":"2024-05-21T03:24:05.828704+0000","last_fullsized":"2024-05-21T03:24:05.828704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:34:43.109424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2792,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":181,"num_read_kb":5,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1561,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657398+0000","last_change":"2024-05-21T03:20:32.146674+0000","last_active":"2024-05-21T03:24:01.657398+0000","last_peered":"2024-05-21T03:24:01.657398+0000","last_clean":"2024-05-21T03:24:01.657398+0000","last_became_active":"2024-05-21T03:20:32.146150+0000","last_became_peered":"2024-05-21T03:20:32.146150+0000","last_unstale":"2024-05-21T03:24:01.657398+0000","last_undegraded":"2024-05-21T03:24:01.657398+0000","last_fullsized":"2024-05-21T03:24:01.657398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:39:09.905640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'67","reported_seq":127,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:03.257139+0000","last_change":"2024-05-21T03:20:29.097613+0000","last_active":"2024-05-21T03:24:03.257139+0000","last_peered":"2024-05-21T03:24:03.257139+0000","last_clean":"2024-05-21T03:24:03.257139+0000","last_became_active":"2024-05-21T03:20:29.097331+0000","last_became_peered":"2024-05-21T03:20:29.097331+0000","last_unstale":"2024-05-21T03:24:03.257139+0000","last_undegraded":"2024-05-21T03:24:03.257139+0000","last_fullsized":"2024-05-21T03:24:03.257139+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T09:56:14.720420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657109+0000","last_change":"2024-05-21T03:21:14.566406+0000","last_active":"2024-05-21T03:24:01.657109+0000","last_peered":"2024-05-21T03:24:01.657109+0000","last_clean":"2024-05-21T03:24:01.657109+0000","last_became_active":"2024-05-21T03:21:14.563023+0000","last_became_peered":"2024-05-21T03:21:14.563023+0000","last_unstale":"2024-05-21T03:24:01.657109+0000","last_undegraded":"2024-05-21T03:24:01.657109+0000","last_fullsized":"2024-05-21T03:24:01.657109+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:12:53.125136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.660167+0000","last_change":"2024-05-21T03:20:29.103456+0000","last_active":"2024-05-21T03:24:01.660167+0000","last_peered":"2024-05-21T03:24:01.660167+0000","last_clean":"2024-05-21T03:24:01.660167+0000","last_became_active":"2024-05-21T03:20:29.103008+0000","last_became_peered":"2024-05-21T03:20:29.103008+0000","last_unstale":"2024-05-21T03:24:01.660167+0000","last_undegraded":"2024-05-21T03:24:01.660167+0000","last_fullsized":"2024-05-21T03:24:01.660167+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:28.075646+0000","last_clean_scrub_stamp":"2024-05-21T03:20:28.075646+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:02:37.951347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657407+0000","last_change":"2024-05-21T03:20:32.146986+0000","last_active":"2024-05-21T03:24:01.657407+0000","last_peered":"2024-05-21T03:24:01.657407+0000","last_clean":"2024-05-21T03:24:01.657407+0000","last_became_active":"2024-05-21T03:20:32.146355+0000","last_became_peered":"2024-05-21T03:20:32.146355+0000","last_unstale":"2024-05-21T03:24:01.657407+0000","last_undegraded":"2024-05-21T03:24:01.657407+0000","last_fullsized":"2024-05-21T03:24:01.657407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:36:29.503099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656717+0000","last_change":"2024-05-21T03:21:14.559658+0000","last_active":"2024-05-21T03:24:01.656717+0000","last_peered":"2024-05-21T03:24:01.656717+0000","last_clean":"2024-05-21T03:24:01.656717+0000","last_became_active":"2024-05-21T03:21:14.559194+0000","last_became_peered":"2024-05-21T03:21:14.559194+0000","last_unstale":"2024-05-21T03:24:01.656717+0000","last_undegraded":"2024-05-21T03:24:01.656717+0000","last_fullsized":"2024-05-21T03:24:01.656717+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:56:31.709545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.389291+0000","last_change":"2024-05-21T03:20:32.137532+0000","last_active":"2024-05-21T03:24:01.389291+0000","last_peered":"2024-05-21T03:24:01.389291+0000","last_clean":"2024-05-21T03:24:01.389291+0000","last_became_active":"2024-05-21T03:20:32.137182+0000","last_became_peered":"2024-05-21T03:20:32.137182+0000","last_unstale":"2024-05-21T03:24:01.389291+0000","last_undegraded":"2024-05-21T03:24:01.389291+0000","last_fullsized":"2024-05-21T03:24:01.389291+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:25:44.207866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.388938+0000","last_change":"2024-05-21T03:20:32.153912+0000","last_active":"2024-05-21T03:24:01.388938+0000","last_peered":"2024-05-21T03:24:01.388938+0000","last_clean":"2024-05-21T03:24:01.388938+0000","last_became_active":"2024-05-21T03:20:32.153541+0000","last_became_peered":"2024-05-21T03:20:32.153541+0000","last_unstale":"2024-05-21T03:24:01.388938+0000","last_undegraded":"2024-05-21T03:24:01.388938+0000","last_fullsized":"2024-05-21T03:24:01.388938+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T11:02:58.688378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.657148+0000","last_change":"2024-05-21T03:20:32.143689+0000","last_active":"2024-05-21T03:24:01.657148+0000","last_peered":"2024-05-21T03:24:01.657148+0000","last_clean":"2024-05-21T03:24:01.657148+0000","last_became_active":"2024-05-21T03:20:32.143399+0000","last_became_peered":"2024-05-21T03:20:32.143399+0000","last_unstale":"2024-05-21T03:24:01.657148+0000","last_undegraded":"2024-05-21T03:24:01.657148+0000","last_fullsized":"2024-05-21T03:24:01.657148+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T07:27:55.116599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.660308+0000","last_change":"2024-05-21T03:20:32.141355+0000","last_active":"2024-05-21T03:24:01.660308+0000","last_peered":"2024-05-21T03:24:01.660308+0000","last_clean":"2024-05-21T03:24:01.660308+0000","last_became_active":"2024-05-21T03:20:32.140960+0000","last_became_peered":"2024-05-21T03:20:32.140960+0000","last_unstale":"2024-05-21T03:24:01.660308+0000","last_undegraded":"2024-05-21T03:24:01.660308+0000","last_fullsized":"2024-05-21T03:24:01.660308+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:23:50.287864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656934+0000","last_change":"2024-05-21T03:20:32.142145+0000","last_active":"2024-05-21T03:24:01.656934+0000","last_peered":"2024-05-21T03:24:01.656934+0000","last_clean":"2024-05-21T03:24:01.656934+0000","last_became_active":"2024-05-21T03:20:32.141802+0000","last_became_peered":"2024-05-21T03:20:32.141802+0000","last_unstale":"2024-05-21T03:24:01.656934+0000","last_undegraded":"2024-05-21T03:24:01.656934+0000","last_fullsized":"2024-05-21T03:24:01.656934+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:26:28.707616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.389203+0000","last_change":"2024-05-21T03:20:32.141606+0000","last_active":"2024-05-21T03:24:01.389203+0000","last_peered":"2024-05-21T03:24:01.389203+0000","last_clean":"2024-05-21T03:24:01.389203+0000","last_became_active":"2024-05-21T03:20:32.141258+0000","last_became_peered":"2024-05-21T03:20:32.141258+0000","last_unstale":"2024-05-21T03:24:01.389203+0000","last_undegraded":"2024-05-21T03:24:01.389203+0000","last_fullsized":"2024-05-21T03:24:01.389203+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:22.372723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491954+0000","last_change":"2024-05-21T03:20:32.151621+0000","last_active":"2024-05-21T03:24:01.491954+0000","last_peered":"2024-05-21T03:24:01.491954+0000","last_clean":"2024-05-21T03:24:01.491954+0000","last_became_active":"2024-05-21T03:20:32.151260+0000","last_became_peered":"2024-05-21T03:20:32.151260+0000","last_unstale":"2024-05-21T03:24:01.491954+0000","last_undegraded":"2024-05-21T03:24:01.491954+0000","last_fullsized":"2024-05-21T03:24:01.491954+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:48:59.751179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.499163+0000","last_change":"2024-05-21T03:20:32.153843+0000","last_active":"2024-05-21T03:24:01.499163+0000","last_peered":"2024-05-21T03:24:01.499163+0000","last_clean":"2024-05-21T03:24:01.499163+0000","last_became_active":"2024-05-21T03:20:32.153418+0000","last_became_peered":"2024-05-21T03:20:32.153418+0000","last_unstale":"2024-05-21T03:24:01.499163+0000","last_undegraded":"2024-05-21T03:24:01.499163+0000","last_fullsized":"2024-05-21T03:24:01.499163+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:26:29.663966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.491312+0000","last_change":"2024-05-21T03:20:32.153942+0000","last_active":"2024-05-21T03:24:01.491312+0000","last_peered":"2024-05-21T03:24:01.491312+0000","last_clean":"2024-05-21T03:24:01.491312+0000","last_became_active":"2024-05-21T03:20:32.153566+0000","last_became_peered":"2024-05-21T03:20:32.153566+0000","last_unstale":"2024-05-21T03:24:01.491312+0000","last_undegraded":"2024-05-21T03:24:01.491312+0000","last_fullsized":"2024-05-21T03:24:01.491312+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:34:06.899039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659626+0000","last_change":"2024-05-21T03:20:32.153243+0000","last_active":"2024-05-21T03:24:01.659626+0000","last_peered":"2024-05-21T03:24:01.659626+0000","last_clean":"2024-05-21T03:24:01.659626+0000","last_became_active":"2024-05-21T03:20:32.152386+0000","last_became_peered":"2024-05-21T03:20:32.152386+0000","last_unstale":"2024-05-21T03:24:01.659626+0000","last_undegraded":"2024-05-21T03:24:01.659626+0000","last_fullsized":"2024-05-21T03:24:01.659626+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T08:25:28.935183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.659802+0000","last_change":"2024-05-21T03:20:32.142536+0000","last_active":"2024-05-21T03:24:01.659802+0000","last_peered":"2024-05-21T03:24:01.659802+0000","last_clean":"2024-05-21T03:24:01.659802+0000","last_became_active":"2024-05-21T03:20:32.141793+0000","last_became_peered":"2024-05-21T03:20:32.141793+0000","last_unstale":"2024-05-21T03:24:01.659802+0000","last_undegraded":"2024-05-21T03:24:01.659802+0000","last_fullsized":"2024-05-21T03:24:01.659802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:21:43.547103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T03:24:01.656778+0000","last_change":"2024-05-21T03:20:32.140368+0000","last_active":"2024-05-21T03:24:01.656778+0000","last_peered":"2024-05-21T03:24:01.656778+0000","last_clean":"2024-05-21T03:24:01.656778+0000","last_became_active":"2024-05-21T03:20:32.139747+0000","last_became_peered":"2024-05-21T03:20:32.139747+0000","last_unstale":"2024-05-21T03:24:01.656778+0000","last_undegraded":"2024-05-21T03:24:01.656778+0000","last_fullsized":"2024-05-21T03:24:01.656778+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T03:20:31.097469+0000","last_clean_scrub_stamp":"2024-05-21T03:20:31.097469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T14:05:44.851191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":797812,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2344,"num_read_kb":1402,"num_write":508,"num_write_kb":2385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1561,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2764800,"data_stored":2393436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":387,"ondisk_log_size":387,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645370,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":917504,"data_stored":444066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841599,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29048,"kb_used_data":1912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716104,"statfs":{"total":95995035648,"available":95965290496,"internally_reserved":0,"allocated":1957888,"data_stored":1429102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005125,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28324,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716828,"statfs":{"total":95995035648,"available":95966031872,"internally_reserved":0,"allocated":1339392,"data_stored":835707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201352,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28316,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716836,"statfs":{"total":95995035648,"available":95966040064,"internally_reserved":0,"allocated":1331200,"data_stored":835968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397583,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32680,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712472,"statfs":{"total":95995035648,"available":95961571328,"internally_reserved":0,"allocated":1536000,"data_stored":1036725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561107,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":443014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":510401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738482,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":445052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411432,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":508528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149794,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28760,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716392,"statfs":{"total":95995035648,"available":95965585408,"internally_reserved":0,"allocated":1662976,"data_stored":1163247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953566,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":445667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757337,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37292,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707860,"statfs":{"total":95995035648,"available":95956848640,"internally_reserved":0,"allocated":1880064,"data_stored":1362789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":67276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":331052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":397365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T03:24:12.248 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:24:12.502 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:24:12.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:12 smithi027 ceph-mon[32179]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 102 KiB/s wr, 56 op/s 2024-05-21T03:24:12.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:12 smithi027 ceph-mon[32179]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:24:13.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:12 smithi039 ceph-mon[36429]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 102 KiB/s wr, 56 op/s 2024-05-21T03:24:13.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:12 smithi039 ceph-mon[36429]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:24:13.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:12 smithi165 ceph-mon[36269]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 102 KiB/s wr, 56 op/s 2024-05-21T03:24:13.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:12 smithi165 ceph-mon[36269]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T03:24:14.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.116+0000 7fbeb11a3700 1 -- 172.21.15.27:0/248434584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeac108600 msgr2=0x7fbeac1089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:14.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.116+0000 7fbeb11a3700 1 --2- 172.21.15.27:0/248434584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeac108600 0x7fbeac1089e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c009990 tx=0x7fbe9c009ca0 comp rx=0 tx=0).stop 2024-05-21T03:24:14.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.116+0000 7fbeb11a3700 1 -- 172.21.15.27:0/248434584 shutdown_connections 2024-05-21T03:24:14.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.116+0000 7fbeb11a3700 1 --2- 172.21.15.27:0/248434584 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbeac1028a0 0x7fbeac110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:14.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.116+0000 7fbeb11a3700 1 --2- 172.21.15.27:0/248434584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeac108600 0x7fbeac1089e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:14.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.116+0000 7fbeb11a3700 1 --2- 172.21.15.27:0/248434584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeac101ee0 0x7fbeac102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:14.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.116+0000 7fbeb11a3700 1 -- 172.21.15.27:0/248434584 >> 172.21.15.27:0/248434584 conn(0x7fbeac077760 msgr2=0x7fbeac077b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:14.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.116+0000 7fbeb11a3700 1 -- 172.21.15.27:0/248434584 shutdown_connections 2024-05-21T03:24:14.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.116+0000 7fbeb11a3700 1 -- 172.21.15.27:0/248434584 wait complete. 2024-05-21T03:24:14.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.116+0000 7fbeb11a3700 1 Processor -- start 2024-05-21T03:24:14.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.117+0000 7fbeb11a3700 1 -- start start 2024-05-21T03:24:14.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.117+0000 7fbeb11a3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeac101ee0 0x7fbeac1030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:14.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.117+0000 7fbeb11a3700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbeac1028a0 0x7fbeac103600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:14.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.117+0000 7fbeb11a3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeac108600 0x7fbeac103b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:14.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.117+0000 7fbeb11a3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac113730 con 0x7fbeac101ee0 2024-05-21T03:24:14.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.117+0000 7fbeb11a3700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac1135b0 con 0x7fbeac108600 2024-05-21T03:24:14.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.117+0000 7fbeb11a3700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac1138b0 con 0x7fbeac1028a0 2024-05-21T03:24:14.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.117+0000 7fbeaad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeac101ee0 0x7fbeac1030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:14.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.117+0000 7fbeaad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeac101ee0 0x7fbeac1030c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57278/0 (socket says 172.21.15.27:57278) 2024-05-21T03:24:14.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.117+0000 7fbeaad9d700 1 -- 172.21.15.27:0/4144186084 learned_addr learned my addr 172.21.15.27:0/4144186084 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:14.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.117+0000 7fbeab59e700 1 --2- 172.21.15.27:0/4144186084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeac108600 0x7fbeac103b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:14.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.117+0000 7fbeaa59c700 1 --2- 172.21.15.27:0/4144186084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbeac1028a0 0x7fbeac103600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:14.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.117+0000 7fbeaad9d700 1 -- 172.21.15.27:0/4144186084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbeac1028a0 msgr2=0x7fbeac103600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:14.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.117+0000 7fbeaad9d700 1 --2- 172.21.15.27:0/4144186084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbeac1028a0 0x7fbeac103600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:14.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.117+0000 7fbeaad9d700 1 -- 172.21.15.27:0/4144186084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeac108600 msgr2=0x7fbeac103b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:14.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.117+0000 7fbeaad9d700 1 --2- 172.21.15.27:0/4144186084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeac108600 0x7fbeac103b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:14.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.117+0000 7fbeaad9d700 1 -- 172.21.15.27:0/4144186084 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeac1a8100 con 0x7fbeac101ee0 2024-05-21T03:24:14.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.118+0000 7fbeaad9d700 1 --2- 172.21.15.27:0/4144186084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeac101ee0 0x7fbeac1030c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c000c00 tx=0x7fbe9c005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:14.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.118+0000 7fbe93fff700 1 -- 172.21.15.27:0/4144186084 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c027070 con 0x7fbeac101ee0 2024-05-21T03:24:14.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.118+0000 7fbeb11a3700 1 -- 172.21.15.27:0/4144186084 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeac1a8390 con 0x7fbeac101ee0 2024-05-21T03:24:14.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.118+0000 7fbe93fff700 1 -- 172.21.15.27:0/4144186084 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe9c003fe0 con 0x7fbeac101ee0 2024-05-21T03:24:14.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.118+0000 7fbe93fff700 1 -- 172.21.15.27:0/4144186084 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c022c30 con 0x7fbeac101ee0 2024-05-21T03:24:14.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.118+0000 7fbeb11a3700 1 -- 172.21.15.27:0/4144186084 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeac1a88c0 con 0x7fbeac101ee0 2024-05-21T03:24:14.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.119+0000 7fbeb11a3700 1 -- 172.21.15.27:0/4144186084 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeac06a230 con 0x7fbeac101ee0 2024-05-21T03:24:14.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.120+0000 7fbe93fff700 1 -- 172.21.15.27:0/4144186084 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fbe9c02a070 con 0x7fbeac101ee0 2024-05-21T03:24:14.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.121+0000 7fbe93fff700 1 --2- 172.21.15.27:0/4144186084 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fbe940b2a80 0x7fbe940b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:14.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.121+0000 7fbeaa59c700 1 --2- 172.21.15.27:0/4144186084 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fbe940b2a80 0x7fbe940b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:14.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.121+0000 7fbe93fff700 1 -- 172.21.15.27:0/4144186084 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fbe9c0e3c70 con 0x7fbeac101ee0 2024-05-21T03:24:14.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.122+0000 7fbeaa59c700 1 --2- 172.21.15.27:0/4144186084 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fbe940b2a80 0x7fbe940b4f40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbea0005950 tx=0x7fbea00058e0 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:14.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.124+0000 7fbe93fff700 1 -- 172.21.15.27:0/4144186084 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbe9c031020 con 0x7fbeac101ee0 2024-05-21T03:24:14.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.384+0000 7fbeb11a3700 1 -- 172.21.15.27:0/4144186084 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbeac02e560 con 0x7fbeac101ee0 2024-05-21T03:24:14.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.385+0000 7fbe93fff700 1 -- 172.21.15.27:0/4144186084 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11924 (secure 0 0 0) 0x7fbe9c013070 con 0x7fbeac101ee0 2024-05-21T03:24:14.385 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:24:14.386 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/148854869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":148854869},{"type":"v1","addr":"172.21.15.27:6835","nonce":148854869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/4145544730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":4145544730},{"type":"v1","addr":"172.21.15.27:6837","nonce":4145544730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T03:20:34.055985+0000","modified":"2024-05-21T03:23:58.657936+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24472,"mds_2":24457,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/745486764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":745486764},{"type":"v1","addr":"172.21.15.27:6839","nonce":745486764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6841/3982752351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3982752351},{"type":"v1","addr":"172.21.15.27:6841","nonce":3982752351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/3497521608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3497521608},{"type":"v1","addr":"172.21.15.39:6835","nonce":3497521608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.165:6835/3576910218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3576910218},{"type":"v1","addr":"172.21.15.165:6835","nonce":3576910218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/3673846852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3673846852},{"type":"v1","addr":"172.21.15.39:6837","nonce":3673846852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6837/691356139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":691356139},{"type":"v1","addr":"172.21.15.165:6837","nonce":691356139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6839/4158170814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4158170814},{"type":"v1","addr":"172.21.15.165:6839","nonce":4158170814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6839/287607819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":287607819},{"type":"v1","addr":"172.21.15.39:6839","nonce":287607819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24457,24452,24451,24472]},"id":1}]} 2024-05-21T03:24:14.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.387+0000 7fbeb11a3700 1 -- 172.21.15.27:0/4144186084 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fbe940b2a80 msgr2=0x7fbe940b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:14.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.387+0000 7fbeb11a3700 1 --2- 172.21.15.27:0/4144186084 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fbe940b2a80 0x7fbe940b4f40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbea0005950 tx=0x7fbea00058e0 comp rx=0 tx=0).stop 2024-05-21T03:24:14.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.387+0000 7fbeb11a3700 1 -- 172.21.15.27:0/4144186084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeac101ee0 msgr2=0x7fbeac1030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:14.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.387+0000 7fbeb11a3700 1 --2- 172.21.15.27:0/4144186084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeac101ee0 0x7fbeac1030c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c000c00 tx=0x7fbe9c005ef0 comp rx=0 tx=0).stop 2024-05-21T03:24:14.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.387+0000 7fbeb11a3700 1 -- 172.21.15.27:0/4144186084 shutdown_connections 2024-05-21T03:24:14.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.387+0000 7fbeb11a3700 1 --2- 172.21.15.27:0/4144186084 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fbe940b2a80 0x7fbe940b4f40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:14.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.387+0000 7fbeb11a3700 1 --2- 172.21.15.27:0/4144186084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeac108600 0x7fbeac103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:14.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.387+0000 7fbeb11a3700 1 --2- 172.21.15.27:0/4144186084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeac101ee0 0x7fbeac1030c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:14.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.387+0000 7fbeb11a3700 1 --2- 172.21.15.27:0/4144186084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbeac1028a0 0x7fbeac103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:14.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.387+0000 7fbeb11a3700 1 -- 172.21.15.27:0/4144186084 >> 172.21.15.27:0/4144186084 conn(0x7fbeac077760 msgr2=0x7fbeac100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:14.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.388+0000 7fbeb11a3700 1 -- 172.21.15.27:0/4144186084 shutdown_connections 2024-05-21T03:24:14.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:14.388+0000 7fbeb11a3700 1 -- 172.21.15.27:0/4144186084 wait complete. 2024-05-21T03:24:14.389 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-05-21T03:24:14.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:14 smithi027 ceph-mon[32179]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 83 KiB/s wr, 50 op/s 2024-05-21T03:24:14.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:14 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/4144186084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:24:14.976 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:24:15.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:14 smithi039 ceph-mon[36429]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 83 KiB/s wr, 50 op/s 2024-05-21T03:24:15.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:14 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/4144186084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:24:15.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:14 smithi165 ceph-mon[36269]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 83 KiB/s wr, 50 op/s 2024-05-21T03:24:15.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:14 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/4144186084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:24:15.226 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:24:16.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.840+0000 7fab7adf8700 1 -- 172.21.15.27:0/1208410464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab741088f0 msgr2=0x7fab74110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:16.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.840+0000 7fab7adf8700 1 --2- 172.21.15.27:0/1208410464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab741088f0 0x7fab74110470 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00b0d0 tx=0x7fab6c00b3e0 comp rx=0 tx=0).stop 2024-05-21T03:24:16.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.840+0000 7fab7adf8700 1 -- 172.21.15.27:0/1208410464 shutdown_connections 2024-05-21T03:24:16.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.840+0000 7fab7adf8700 1 --2- 172.21.15.27:0/1208410464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab741088f0 0x7fab74110470 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:16.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.840+0000 7fab7adf8700 1 --2- 172.21.15.27:0/1208410464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab741037f0 0x7fab74103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:16.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.840+0000 7fab7adf8700 1 --2- 172.21.15.27:0/1208410464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab741041a0 0x7fab741081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:16.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.840+0000 7fab7adf8700 1 -- 172.21.15.27:0/1208410464 >> 172.21.15.27:0/1208410464 conn(0x7fab7407a6a0 msgr2=0x7fab7407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:16.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.840+0000 7fab7adf8700 1 -- 172.21.15.27:0/1208410464 shutdown_connections 2024-05-21T03:24:16.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.840+0000 7fab7adf8700 1 -- 172.21.15.27:0/1208410464 wait complete. 2024-05-21T03:24:16.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.841+0000 7fab7adf8700 1 Processor -- start 2024-05-21T03:24:16.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.841+0000 7fab7adf8700 1 -- start start 2024-05-21T03:24:16.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.841+0000 7fab7adf8700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab741037f0 0x7fab74075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:16.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.841+0000 7fab7adf8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab741041a0 0x7fab74075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:16.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.841+0000 7fab7adf8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab741088f0 0x7fab740762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:16.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.841+0000 7fab7adf8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fab741136c0 con 0x7fab741088f0 2024-05-21T03:24:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.841+0000 7fab7adf8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fab74113540 con 0x7fab741041a0 2024-05-21T03:24:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.841+0000 7fab7adf8700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fab74113840 con 0x7fab741037f0 2024-05-21T03:24:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.841+0000 7fab79395700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab741088f0 0x7fab740762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.841+0000 7fab78b94700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab741037f0 0x7fab74075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.841+0000 7fab79395700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab741088f0 0x7fab740762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57310/0 (socket says 172.21.15.27:57310) 2024-05-21T03:24:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.841+0000 7fab79395700 1 -- 172.21.15.27:0/1466540333 learned_addr learned my addr 172.21.15.27:0/1466540333 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.841+0000 7fab73fff700 1 --2- 172.21.15.27:0/1466540333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab741041a0 0x7fab74075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.841+0000 7fab79395700 1 -- 172.21.15.27:0/1466540333 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab741037f0 msgr2=0x7fab74075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.841+0000 7fab79395700 1 --2- 172.21.15.27:0/1466540333 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab741037f0 0x7fab74075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.841+0000 7fab79395700 1 -- 172.21.15.27:0/1466540333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab741041a0 msgr2=0x7fab74075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.841+0000 7fab79395700 1 --2- 172.21.15.27:0/1466540333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab741041a0 0x7fab74075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.842+0000 7fab79395700 1 -- 172.21.15.27:0/1466540333 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab741a8050 con 0x7fab741088f0 2024-05-21T03:24:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.842+0000 7fab78b94700 1 --2- 172.21.15.27:0/1466540333 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab741037f0 0x7fab74075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:24:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.842+0000 7fab79395700 1 --2- 172.21.15.27:0/1466540333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab741088f0 0x7fab740762c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fab6c0095c0 tx=0x7fab6c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.842+0000 7fab71ffb700 1 -- 172.21.15.27:0/1466540333 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c01f070 con 0x7fab741088f0 2024-05-21T03:24:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.842+0000 7fab7adf8700 1 -- 172.21.15.27:0/1466540333 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab741a82e0 con 0x7fab741088f0 2024-05-21T03:24:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.842+0000 7fab71ffb700 1 -- 172.21.15.27:0/1466540333 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab6c007e20 con 0x7fab741088f0 2024-05-21T03:24:16.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.842+0000 7fab71ffb700 1 -- 172.21.15.27:0/1466540333 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c01b4a0 con 0x7fab741088f0 2024-05-21T03:24:16.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.842+0000 7fab7adf8700 1 -- 172.21.15.27:0/1466540333 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab741a8790 con 0x7fab741088f0 2024-05-21T03:24:16.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.843+0000 7fab7adf8700 1 -- 172.21.15.27:0/1466540333 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab7406a230 con 0x7fab741088f0 2024-05-21T03:24:16.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.845+0000 7fab71ffb700 1 -- 172.21.15.27:0/1466540333 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fab6c022070 con 0x7fab741088f0 2024-05-21T03:24:16.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.845+0000 7fab71ffb700 1 --2- 172.21.15.27:0/1466540333 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fab600b2a80 0x7fab600b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:16.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.846+0000 7fab78b94700 1 --2- 172.21.15.27:0/1466540333 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fab600b2a80 0x7fab600b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:16.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.846+0000 7fab71ffb700 1 -- 172.21.15.27:0/1466540333 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fab6c00f020 con 0x7fab741088f0 2024-05-21T03:24:16.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.846+0000 7fab78b94700 1 --2- 172.21.15.27:0/1466540333 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fab600b2a80 0x7fab600b4f40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fab5c007fd0 tx=0x7fab5c0073d0 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:16.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:16.849+0000 7fab71ffb700 1 -- 172.21.15.27:0/1466540333 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fab6c029020 con 0x7fab741088f0 2024-05-21T03:24:16.930 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:16 smithi027 ceph-mon[32179]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-05-21T03:24:17.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:16 smithi039 ceph-mon[36429]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-05-21T03:24:17.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:16 smithi165 ceph-mon[36269]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-05-21T03:24:17.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:17.119+0000 7fab7adf8700 1 -- 172.21.15.27:0/1466540333 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab7402e560 con 0x7fab741088f0 2024-05-21T03:24:17.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:17.120+0000 7fab71ffb700 1 -- 172.21.15.27:0/1466540333 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11924 (secure 0 0 0) 0x7fab6c0ab600 con 0x7fab741088f0 2024-05-21T03:24:17.121 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:24:17.121 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/148854869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":148854869},{"type":"v1","addr":"172.21.15.27:6835","nonce":148854869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/4145544730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":4145544730},{"type":"v1","addr":"172.21.15.27:6837","nonce":4145544730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T03:20:34.055985+0000","modified":"2024-05-21T03:23:58.657936+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24472,"mds_2":24457,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/745486764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":745486764},{"type":"v1","addr":"172.21.15.27:6839","nonce":745486764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6841/3982752351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3982752351},{"type":"v1","addr":"172.21.15.27:6841","nonce":3982752351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/3497521608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3497521608},{"type":"v1","addr":"172.21.15.39:6835","nonce":3497521608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.165:6835/3576910218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3576910218},{"type":"v1","addr":"172.21.15.165:6835","nonce":3576910218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/3673846852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3673846852},{"type":"v1","addr":"172.21.15.39:6837","nonce":3673846852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6837/691356139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":691356139},{"type":"v1","addr":"172.21.15.165:6837","nonce":691356139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6839/4158170814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4158170814},{"type":"v1","addr":"172.21.15.165:6839","nonce":4158170814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6839/287607819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":287607819},{"type":"v1","addr":"172.21.15.39:6839","nonce":287607819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24457,24452,24451,24472]},"id":1}]} 2024-05-21T03:24:17.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:17.122+0000 7fab7adf8700 1 -- 172.21.15.27:0/1466540333 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fab600b2a80 msgr2=0x7fab600b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:17.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:17.122+0000 7fab7adf8700 1 --2- 172.21.15.27:0/1466540333 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fab600b2a80 0x7fab600b4f40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fab5c007fd0 tx=0x7fab5c0073d0 comp rx=0 tx=0).stop 2024-05-21T03:24:17.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:17.122+0000 7fab7adf8700 1 -- 172.21.15.27:0/1466540333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab741088f0 msgr2=0x7fab740762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:17.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:17.122+0000 7fab7adf8700 1 --2- 172.21.15.27:0/1466540333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab741088f0 0x7fab740762c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fab6c0095c0 tx=0x7fab6c007a20 comp rx=0 tx=0).stop 2024-05-21T03:24:17.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:17.123+0000 7fab7adf8700 1 -- 172.21.15.27:0/1466540333 shutdown_connections 2024-05-21T03:24:17.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:17.123+0000 7fab7adf8700 1 --2- 172.21.15.27:0/1466540333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab741088f0 0x7fab740762c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:17.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:17.123+0000 7fab7adf8700 1 --2- 172.21.15.27:0/1466540333 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab741037f0 0x7fab74075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:17.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:17.123+0000 7fab7adf8700 1 --2- 172.21.15.27:0/1466540333 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fab600b2a80 0x7fab600b4f40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:17.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:17.123+0000 7fab7adf8700 1 --2- 172.21.15.27:0/1466540333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab741041a0 0x7fab74075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:17.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:17.123+0000 7fab7adf8700 1 -- 172.21.15.27:0/1466540333 >> 172.21.15.27:0/1466540333 conn(0x7fab7407a6a0 msgr2=0x7fab74101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:17.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:17.123+0000 7fab7adf8700 1 -- 172.21.15.27:0/1466540333 shutdown_connections 2024-05-21T03:24:17.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:17.123+0000 7fab7adf8700 1 -- 172.21.15.27:0/1466540333 wait complete. 2024-05-21T03:24:17.125 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-05-21T03:24:17.792 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-21T03:24:18.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:17 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/1466540333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:24:18.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:17 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/1466540333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:24:18.042 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:24:18.165 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:17 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/1466540333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:24:18.870 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:18 smithi027 ceph-mon[32179]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-05-21T03:24:19.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:18 smithi039 ceph-mon[36429]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-05-21T03:24:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:18 smithi165 ceph-mon[36269]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-05-21T03:24:19.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.807+0000 7f1b30150700 1 -- 172.21.15.27:0/1911785156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b281037f0 msgr2=0x7f1b28103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:19.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.807+0000 7f1b30150700 1 --2- 172.21.15.27:0/1911785156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b281037f0 0x7f1b28103bd0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1b240099c0 tx=0x7f1b24009cd0 comp rx=0 tx=0).stop 2024-05-21T03:24:19.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.807+0000 7f1b30150700 1 -- 172.21.15.27:0/1911785156 shutdown_connections 2024-05-21T03:24:19.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.807+0000 7f1b30150700 1 --2- 172.21.15.27:0/1911785156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b281088f0 0x7f1b28110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:19.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.807+0000 7f1b30150700 1 --2- 172.21.15.27:0/1911785156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b281037f0 0x7f1b28103bd0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:19.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.807+0000 7f1b30150700 1 --2- 172.21.15.27:0/1911785156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b281041a0 0x7f1b281081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:19.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.807+0000 7f1b30150700 1 -- 172.21.15.27:0/1911785156 >> 172.21.15.27:0/1911785156 conn(0x7f1b2807a6a0 msgr2=0x7f1b2807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:19.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.807+0000 7f1b30150700 1 -- 172.21.15.27:0/1911785156 shutdown_connections 2024-05-21T03:24:19.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.807+0000 7f1b30150700 1 -- 172.21.15.27:0/1911785156 wait complete. 2024-05-21T03:24:19.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.808+0000 7f1b30150700 1 Processor -- start 2024-05-21T03:24:19.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.808+0000 7f1b30150700 1 -- start start 2024-05-21T03:24:19.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.808+0000 7f1b30150700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b281037f0 0x7f1b2819f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:19.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.808+0000 7f1b30150700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b281041a0 0x7f1b2819fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:19.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.808+0000 7f1b30150700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b281088f0 0x7f1b28199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:19.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.808+0000 7f1b30150700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1b28113800 con 0x7f1b281037f0 2024-05-21T03:24:19.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.808+0000 7f1b30150700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1b28113680 con 0x7f1b281088f0 2024-05-21T03:24:19.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.808+0000 7f1b30150700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1b28113980 con 0x7f1b281041a0 2024-05-21T03:24:19.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.808+0000 7f1b2deec700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b281037f0 0x7f1b2819f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:19.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.808+0000 7f1b2deec700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b281037f0 0x7f1b2819f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57328/0 (socket says 172.21.15.27:57328) 2024-05-21T03:24:19.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.808+0000 7f1b2deec700 1 -- 172.21.15.27:0/453137611 learned_addr learned my addr 172.21.15.27:0/453137611 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:19.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.808+0000 7f1b2d6eb700 1 --2- 172.21.15.27:0/453137611 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b281041a0 0x7f1b2819fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:19.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.808+0000 7f1b2e6ed700 1 --2- 172.21.15.27:0/453137611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b281088f0 0x7f1b28199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:19.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.808+0000 7f1b2deec700 1 -- 172.21.15.27:0/453137611 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b281041a0 msgr2=0x7f1b2819fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:19.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.808+0000 7f1b2deec700 1 --2- 172.21.15.27:0/453137611 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b281041a0 0x7f1b2819fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:19.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.808+0000 7f1b2deec700 1 -- 172.21.15.27:0/453137611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b281088f0 msgr2=0x7f1b28199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:19.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.808+0000 7f1b2deec700 1 --2- 172.21.15.27:0/453137611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b281088f0 0x7f1b28199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:19.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.808+0000 7f1b2deec700 1 -- 172.21.15.27:0/453137611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b2819a240 con 0x7f1b281037f0 2024-05-21T03:24:19.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.809+0000 7f1b2deec700 1 --2- 172.21.15.27:0/453137611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b281037f0 0x7f1b2819f920 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1b24009990 tx=0x7f1b240047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:19.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.809+0000 7f1b1effd700 1 -- 172.21.15.27:0/453137611 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24027070 con 0x7f1b281037f0 2024-05-21T03:24:19.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.809+0000 7f1b1effd700 1 -- 172.21.15.27:0/453137611 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b24003fa0 con 0x7f1b281037f0 2024-05-21T03:24:19.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.809+0000 7f1b30150700 1 -- 172.21.15.27:0/453137611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b2819a4d0 con 0x7f1b281037f0 2024-05-21T03:24:19.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.809+0000 7f1b1effd700 1 -- 172.21.15.27:0/453137611 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24022d20 con 0x7f1b281037f0 2024-05-21T03:24:19.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.809+0000 7f1b30150700 1 -- 172.21.15.27:0/453137611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b281a67f0 con 0x7f1b281037f0 2024-05-21T03:24:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.810+0000 7f1b30150700 1 -- 172.21.15.27:0/453137611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b2806a230 con 0x7f1b281037f0 2024-05-21T03:24:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.811+0000 7f1b1effd700 1 -- 172.21.15.27:0/453137611 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f1b2402a070 con 0x7f1b281037f0 2024-05-21T03:24:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.812+0000 7f1b1effd700 1 --2- 172.21.15.27:0/453137611 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f1b140b2b00 0x7f1b140b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.812+0000 7f1b2d6eb700 1 --2- 172.21.15.27:0/453137611 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f1b140b2b00 0x7f1b140b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.812+0000 7f1b1effd700 1 -- 172.21.15.27:0/453137611 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f1b240e36a0 con 0x7f1b281037f0 2024-05-21T03:24:19.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.813+0000 7f1b2d6eb700 1 --2- 172.21.15.27:0/453137611 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f1b140b2b00 0x7f1b140b4fc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1b28103180 tx=0x7f1b18009190 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:19.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:19.815+0000 7f1b1effd700 1 -- 172.21.15.27:0/453137611 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1b24013070 con 0x7f1b281037f0 2024-05-21T03:24:20.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:20.037+0000 7f1b30150700 1 -- 172.21.15.27:0/453137611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b2802e560 con 0x7f1b281037f0 2024-05-21T03:24:20.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:20.039+0000 7f1b1effd700 1 -- 172.21.15.27:0/453137611 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+23060 (secure 0 0 0) 0x7f1b240aafa0 con 0x7f1b281037f0 2024-05-21T03:24:20.039 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:24:20.040 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":83,"fsid":"c9ca94be-171f-11ef-bc9a-c7b262605968","created":"2024-05-21T03:12:18.217236+0000","modified":"2024-05-21T03:24:01.382251+0000","last_up_change":"2024-05-21T03:19:30.382150+0000","last_in_change":"2024-05-21T03:19:09.058249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T03:15:41.340194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T03:20:27.507346+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T03:20:30.830169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c745c09f-9a76-4d05-af75-c7554f6d5e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6803","nonce":1399996449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6805","nonce":1399996449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6809","nonce":1399996449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6807","nonce":1399996449}]},"public_addr":"172.21.15.27:6803/1399996449","cluster_addr":"172.21.15.27:6805/1399996449","heartbeat_back_addr":"172.21.15.27:6809/1399996449","heartbeat_front_addr":"172.21.15.27:6807/1399996449","state":["exists","up"]},{"osd":1,"uuid":"8b73aa00-4364-438f-9722-53caa4e9bec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6801","nonce":1776318847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6803","nonce":1776318847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6807","nonce":1776318847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6805","nonce":1776318847}]},"public_addr":"172.21.15.39:6801/1776318847","cluster_addr":"172.21.15.39:6803/1776318847","heartbeat_back_addr":"172.21.15.39:6807/1776318847","heartbeat_front_addr":"172.21.15.39:6805/1776318847","state":["exists","up"]},{"osd":2,"uuid":"0822043e-7d3c-4e98-9dc0-94b98b4d562a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6801","nonce":703322625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6803","nonce":703322625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6807","nonce":703322625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6805","nonce":703322625}]},"public_addr":"172.21.15.165:6801/703322625","cluster_addr":"172.21.15.165:6803/703322625","heartbeat_back_addr":"172.21.15.165:6807/703322625","heartbeat_front_addr":"172.21.15.165:6805/703322625","state":["exists","up"]},{"osd":3,"uuid":"c563b1b2-efcb-4f8c-8121-c1938f83131e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6811","nonce":1933403135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6813","nonce":1933403135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6817","nonce":1933403135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6815","nonce":1933403135}]},"public_addr":"172.21.15.27:6811/1933403135","cluster_addr":"172.21.15.27:6813/1933403135","heartbeat_back_addr":"172.21.15.27:6817/1933403135","heartbeat_front_addr":"172.21.15.27:6815/1933403135","state":["exists","up"]},{"osd":4,"uuid":"0c793b87-b045-4b4a-89e7-87ec36deeb0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6809","nonce":155579331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6811","nonce":155579331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6815","nonce":155579331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6813","nonce":155579331}]},"public_addr":"172.21.15.39:6809/155579331","cluster_addr":"172.21.15.39:6811/155579331","heartbeat_back_addr":"172.21.15.39:6815/155579331","heartbeat_front_addr":"172.21.15.39:6813/155579331","state":["exists","up"]},{"osd":5,"uuid":"367eeb5a-9d75-4644-bf46-317c03f33d8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6809","nonce":386850244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6811","nonce":386850244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6815","nonce":386850244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6813","nonce":386850244}]},"public_addr":"172.21.15.165:6809/386850244","cluster_addr":"172.21.15.165:6811/386850244","heartbeat_back_addr":"172.21.15.165:6815/386850244","heartbeat_front_addr":"172.21.15.165:6813/386850244","state":["exists","up"]},{"osd":6,"uuid":"e000167f-24fd-4115-83f0-8afbfa8d8991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6819","nonce":3531376679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6821","nonce":3531376679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6825","nonce":3531376679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6823","nonce":3531376679}]},"public_addr":"172.21.15.27:6819/3531376679","cluster_addr":"172.21.15.27:6821/3531376679","heartbeat_back_addr":"172.21.15.27:6825/3531376679","heartbeat_front_addr":"172.21.15.27:6823/3531376679","state":["exists","up"]},{"osd":7,"uuid":"4b8d486b-868e-4f33-98a1-b99686adeaaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6817","nonce":1550035852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6819","nonce":1550035852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6823","nonce":1550035852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6821","nonce":1550035852}]},"public_addr":"172.21.15.39:6817/1550035852","cluster_addr":"172.21.15.39:6819/1550035852","heartbeat_back_addr":"172.21.15.39:6823/1550035852","heartbeat_front_addr":"172.21.15.39:6821/1550035852","state":["exists","up"]},{"osd":8,"uuid":"2259f35c-2edd-4331-9a63-8c0c1acbdc71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6817","nonce":3184704971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6819","nonce":3184704971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6823","nonce":3184704971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6821","nonce":3184704971}]},"public_addr":"172.21.15.165:6817/3184704971","cluster_addr":"172.21.15.165:6819/3184704971","heartbeat_back_addr":"172.21.15.165:6823/3184704971","heartbeat_front_addr":"172.21.15.165:6821/3184704971","state":["exists","up"]},{"osd":9,"uuid":"10307ecc-8270-4be2-b62b-9708e09bb207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6827","nonce":2800048793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6829","nonce":2800048793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6833","nonce":2800048793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6831","nonce":2800048793}]},"public_addr":"172.21.15.27:6827/2800048793","cluster_addr":"172.21.15.27:6829/2800048793","heartbeat_back_addr":"172.21.15.27:6833/2800048793","heartbeat_front_addr":"172.21.15.27:6831/2800048793","state":["exists","up"]},{"osd":10,"uuid":"dfe1b8ca-ff8a-4b7f-ade5-ad4f00cf60cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6825","nonce":363500811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6827","nonce":363500811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6831","nonce":363500811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6829","nonce":363500811}]},"public_addr":"172.21.15.39:6825/363500811","cluster_addr":"172.21.15.39:6827/363500811","heartbeat_back_addr":"172.21.15.39:6831/363500811","heartbeat_front_addr":"172.21.15.39:6829/363500811","state":["exists","up"]},{"osd":11,"uuid":"f9531145-5d09-4dbd-a050-3159ee09db24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6825","nonce":2274332243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6827","nonce":2274332243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6831","nonce":2274332243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6829","nonce":2274332243}]},"public_addr":"172.21.15.165:6825/2274332243","cluster_addr":"172.21.15.165:6827/2274332243","heartbeat_back_addr":"172.21.15.165:6831/2274332243","heartbeat_front_addr":"172.21.15.165:6829/2274332243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:14:47.077860+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:15:13.272879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:15:38.046651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:03.376483+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:29.179280+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:53.391261+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:17:18.472829+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:17:44.349925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:18:08.652193+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:18:35.282669+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:19:01.637151+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:19:27.461566+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/744303480":"2024-05-22T03:24:01.382214+0000","172.21.15.27:0/3254504266":"2024-05-22T03:24:01.382214+0000","172.21.15.27:0/4135246381":"2024-05-22T03:24:01.382214+0000","172.21.15.27:6800/1185518061":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/4189166571":"2024-05-22T03:24:01.382214+0000","172.21.15.27:6801/1185518061":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/4194431212":"2024-05-22T03:13:13.249220+0000","172.21.15.27:6801/3683408243":"2024-05-22T03:24:01.382214+0000","172.21.15.27:0/1890032101":"2024-05-22T03:13:13.249220+0000","172.21.15.27:6800/175229478":"2024-05-22T03:12:45.739195+0000","172.21.15.27:0/999325706":"2024-05-22T03:12:45.739195+0000","172.21.15.27:6801/175229478":"2024-05-22T03:12:45.739195+0000","172.21.15.27:6800/3683408243":"2024-05-22T03:24:01.382214+0000","172.21.15.27:0/4184128736":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/3000197506":"2024-05-22T03:12:45.739195+0000","172.21.15.27:0/972677319":"2024-05-22T03:12:45.739195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T03:24:20.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:20.041+0000 7f1b30150700 1 -- 172.21.15.27:0/453137611 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f1b140b2b00 msgr2=0x7f1b140b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:20.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:20.041+0000 7f1b30150700 1 --2- 172.21.15.27:0/453137611 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f1b140b2b00 0x7f1b140b4fc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1b28103180 tx=0x7f1b18009190 comp rx=0 tx=0).stop 2024-05-21T03:24:20.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:20.041+0000 7f1b30150700 1 -- 172.21.15.27:0/453137611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b281037f0 msgr2=0x7f1b2819f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:20.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:20.041+0000 7f1b30150700 1 --2- 172.21.15.27:0/453137611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b281037f0 0x7f1b2819f920 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1b24009990 tx=0x7f1b240047b0 comp rx=0 tx=0).stop 2024-05-21T03:24:20.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:20.041+0000 7f1b30150700 1 -- 172.21.15.27:0/453137611 shutdown_connections 2024-05-21T03:24:20.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:20.041+0000 7f1b30150700 1 --2- 172.21.15.27:0/453137611 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f1b140b2b00 0x7f1b140b4fc0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:20.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:20.041+0000 7f1b30150700 1 --2- 172.21.15.27:0/453137611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b281088f0 0x7f1b28199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:20.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:20.041+0000 7f1b30150700 1 --2- 172.21.15.27:0/453137611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b281037f0 0x7f1b2819f920 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:20.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:20.041+0000 7f1b30150700 1 --2- 172.21.15.27:0/453137611 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b281041a0 0x7f1b2819fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:20.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:20.041+0000 7f1b30150700 1 -- 172.21.15.27:0/453137611 >> 172.21.15.27:0/453137611 conn(0x7f1b2807a6a0 msgr2=0x7f1b281018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:20.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:20.041+0000 7f1b30150700 1 -- 172.21.15.27:0/453137611 shutdown_connections 2024-05-21T03:24:20.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:20.041+0000 7f1b30150700 1 -- 172.21.15.27:0/453137611 wait complete. 2024-05-21T03:24:20.724 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-05-21T03:24:20.982 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:24:21.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:20 smithi039 ceph-mon[36429]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-05-21T03:24:21.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:20 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/453137611' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:24:21.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:20 smithi165 ceph-mon[36269]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-05-21T03:24:21.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:20 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/453137611' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:24:21.104 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:20 smithi027 ceph-mon[32179]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-05-21T03:24:21.105 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:20 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/453137611' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:24:22.875 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:22 smithi027 ceph-mon[32179]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:24:23.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:22 smithi039 ceph-mon[36429]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:24:23.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:22 smithi165 ceph-mon[36269]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:24:23.239 INFO:teuthology.orchestra.run.smithi027.stdout:{"run_dir":"/var/run/ceph"} 2024-05-21T03:24:23.239 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-05-21T03:24:23.240 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 8818 2024-05-21T03:24:23.240 DEBUG:tasks.quiescer:Yielding 2024-05-21T03:24:23.240 INFO:teuthology.run_tasks:Running task workunit... 2024-05-21T03:24:23.254 INFO:tasks.workunit:Pulling workunits from ref 3b748ea08891006c1152513d25ab4a03f7c9a995 2024-05-21T03:24:23.255 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-21T03:24:23.255 INFO:tasks.workunit:timeout=3h 2024-05-21T03:24:23.255 INFO:tasks.workunit:cleanup=True 2024-05-21T03:24:23.255 DEBUG:teuthology.orchestra.run.smithi027:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-21T03:24:23.256 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:24:23.274 INFO:teuthology.orchestra.run.smithi027.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-21T03:24:23.274 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-21T03:24:23.274 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-05-21T03:24:23.274 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T03:24:23.274 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-21T03:24:23.274 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-05-21 03:21:04.429596770 +0000 2024-05-21T03:24:23.274 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-05-21 03:21:04.429596770 +0000 2024-05-21T03:24:23.275 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-05-21 03:23:57.863344738 +0000 2024-05-21T03:24:23.275 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-05-21 03:21:04.429596770 +0000 2024-05-21T03:24:23.275 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-21T03:24:23.275 DEBUG:teuthology.orchestra.run.smithi027:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-21T03:24:23.350 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3b748ea08891006c1152513d25ab4a03f7c9a995 2024-05-21T03:24:23.407 INFO:tasks.workunit.client.0.smithi027.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-21T03:24:23.514 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:24:24.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:24 smithi027 ceph-mon[32179]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T03:24:25.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:24 smithi039 ceph-mon[36429]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T03:24:25.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:24 smithi165 ceph-mon[36269]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T03:24:25.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.168+0000 7fdd09797700 1 -- 172.21.15.27:0/1351483919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd0406c670 msgr2=0x7fdd04110420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:25.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.168+0000 7fdd09797700 1 --2- 172.21.15.27:0/1351483919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd0406c670 0x7fdd04110420 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc00b0a0 tx=0x7fdcfc00b3b0 comp rx=0 tx=0).stop 2024-05-21T03:24:25.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.168+0000 7fdd09797700 1 -- 172.21.15.27:0/1351483919 shutdown_connections 2024-05-21T03:24:25.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.168+0000 7fdd09797700 1 --2- 172.21.15.27:0/1351483919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd0406c670 0x7fdd04110420 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:25.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.168+0000 7fdd09797700 1 --2- 172.21.15.27:0/1351483919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd041095e0 0x7fdd041099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:25.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.168+0000 7fdd09797700 1 --2- 172.21.15.27:0/1351483919 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd0406bcb0 0x7fdd0406c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:25.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.168+0000 7fdd09797700 1 -- 172.21.15.27:0/1351483919 >> 172.21.15.27:0/1351483919 conn(0x7fdd040784b0 msgr2=0x7fdd040788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:25.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.168+0000 7fdd09797700 1 -- 172.21.15.27:0/1351483919 shutdown_connections 2024-05-21T03:24:25.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.168+0000 7fdd09797700 1 -- 172.21.15.27:0/1351483919 wait complete. 2024-05-21T03:24:25.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.168+0000 7fdd09797700 1 Processor -- start 2024-05-21T03:24:25.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.168+0000 7fdd09797700 1 -- start start 2024-05-21T03:24:25.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.169+0000 7fdd09797700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd0406bcb0 0x7fdd04199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:25.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.169+0000 7fdd09797700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd0406c670 0x7fdd0419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:25.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.169+0000 7fdd09797700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd041095e0 0x7fdd0419e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:25.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.169+0000 7fdd09797700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdd04113760 con 0x7fdd041095e0 2024-05-21T03:24:25.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.169+0000 7fdd09797700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdd041135e0 con 0x7fdd0406bcb0 2024-05-21T03:24:25.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.169+0000 7fdd09797700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdd041138e0 con 0x7fdd0406c670 2024-05-21T03:24:25.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.169+0000 7fdd037fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd041095e0 0x7fdd0419e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:25.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.169+0000 7fdd037fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd041095e0 0x7fdd0419e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34810/0 (socket says 172.21.15.27:34810) 2024-05-21T03:24:25.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.169+0000 7fdd037fe700 1 -- 172.21.15.27:0/1172935263 learned_addr learned my addr 172.21.15.27:0/1172935263 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:25.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.169+0000 7fdd02ffd700 1 --2- 172.21.15.27:0/1172935263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd0406bcb0 0x7fdd04199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:25.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.169+0000 7fdd027fc700 1 --2- 172.21.15.27:0/1172935263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd0406c670 0x7fdd0419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:25.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.169+0000 7fdd037fe700 1 -- 172.21.15.27:0/1172935263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd0406c670 msgr2=0x7fdd0419a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:25.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.169+0000 7fdd037fe700 1 --2- 172.21.15.27:0/1172935263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd0406c670 0x7fdd0419a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:25.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.169+0000 7fdd037fe700 1 -- 172.21.15.27:0/1172935263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd0406bcb0 msgr2=0x7fdd04199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:25.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.169+0000 7fdd037fe700 1 --2- 172.21.15.27:0/1172935263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd0406bcb0 0x7fdd04199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:25.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.169+0000 7fdd037fe700 1 -- 172.21.15.27:0/1172935263 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd0419ed50 con 0x7fdd041095e0 2024-05-21T03:24:25.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.169+0000 7fdd037fe700 1 --2- 172.21.15.27:0/1172935263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd041095e0 0x7fdd0419e610 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0062a0 tx=0x7fdcfc004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:25.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.170+0000 7fdcebfff700 1 -- 172.21.15.27:0/1172935263 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc021070 con 0x7fdd041095e0 2024-05-21T03:24:25.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.170+0000 7fdcebfff700 1 -- 172.21.15.27:0/1172935263 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcfc003c00 con 0x7fdd041095e0 2024-05-21T03:24:25.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.170+0000 7fdd09797700 1 -- 172.21.15.27:0/1172935263 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd0419efe0 con 0x7fdd041095e0 2024-05-21T03:24:25.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.170+0000 7fdcebfff700 1 -- 172.21.15.27:0/1172935263 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc004880 con 0x7fdd041095e0 2024-05-21T03:24:25.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.170+0000 7fdd09797700 1 -- 172.21.15.27:0/1172935263 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd041a68c0 con 0x7fdd041095e0 2024-05-21T03:24:25.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.171+0000 7fdd09797700 1 -- 172.21.15.27:0/1172935263 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd0410bae0 con 0x7fdd041095e0 2024-05-21T03:24:25.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.172+0000 7fdcebfff700 1 -- 172.21.15.27:0/1172935263 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fdcfc004020 con 0x7fdd041095e0 2024-05-21T03:24:25.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.173+0000 7fdcebfff700 1 --2- 172.21.15.27:0/1172935263 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fdcec0b2a80 0x7fdcec0b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:25.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.173+0000 7fdd02ffd700 1 --2- 172.21.15.27:0/1172935263 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fdcec0b2a80 0x7fdcec0b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:25.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.173+0000 7fdcebfff700 1 -- 172.21.15.27:0/1172935263 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fdcfc00f020 con 0x7fdd041095e0 2024-05-21T03:24:25.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.173+0000 7fdd02ffd700 1 --2- 172.21.15.27:0/1172935263 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fdcec0b2a80 0x7fdcec0b4f40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4009a10 tx=0x7fdcf4009190 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:25.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.176+0000 7fdcebfff700 1 -- 172.21.15.27:0/1172935263 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdcfc02a020 con 0x7fdd041095e0 2024-05-21T03:24:25.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.445+0000 7fdd09797700 1 -- 172.21.15.27:0/1172935263 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd0419b0a0 con 0x7fdd041095e0 2024-05-21T03:24:25.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.445+0000 7fdcebfff700 1 -- 172.21.15.27:0/1172935263 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11924 (secure 0 0 0) 0x7fdcfc0aba60 con 0x7fdd041095e0 2024-05-21T03:24:25.446 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:24:25.446 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/148854869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":148854869},{"type":"v1","addr":"172.21.15.27:6835","nonce":148854869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/4145544730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":4145544730},{"type":"v1","addr":"172.21.15.27:6837","nonce":4145544730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T03:20:34.055985+0000","modified":"2024-05-21T03:23:58.657936+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24472,"mds_2":24457,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/745486764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":745486764},{"type":"v1","addr":"172.21.15.27:6839","nonce":745486764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6841/3982752351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3982752351},{"type":"v1","addr":"172.21.15.27:6841","nonce":3982752351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/3497521608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3497521608},{"type":"v1","addr":"172.21.15.39:6835","nonce":3497521608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.165:6835/3576910218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3576910218},{"type":"v1","addr":"172.21.15.165:6835","nonce":3576910218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/3673846852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3673846852},{"type":"v1","addr":"172.21.15.39:6837","nonce":3673846852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6837/691356139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":691356139},{"type":"v1","addr":"172.21.15.165:6837","nonce":691356139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6839/4158170814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4158170814},{"type":"v1","addr":"172.21.15.165:6839","nonce":4158170814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6839/287607819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":287607819},{"type":"v1","addr":"172.21.15.39:6839","nonce":287607819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24457,24452,24451,24472]},"id":1}]} 2024-05-21T03:24:25.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.447+0000 7fdd09797700 1 -- 172.21.15.27:0/1172935263 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fdcec0b2a80 msgr2=0x7fdcec0b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:25.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.447+0000 7fdd09797700 1 --2- 172.21.15.27:0/1172935263 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fdcec0b2a80 0x7fdcec0b4f40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4009a10 tx=0x7fdcf4009190 comp rx=0 tx=0).stop 2024-05-21T03:24:25.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.447+0000 7fdd09797700 1 -- 172.21.15.27:0/1172935263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd041095e0 msgr2=0x7fdd0419e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:25.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.447+0000 7fdd09797700 1 --2- 172.21.15.27:0/1172935263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd041095e0 0x7fdd0419e610 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0062a0 tx=0x7fdcfc004e20 comp rx=0 tx=0).stop 2024-05-21T03:24:25.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.447+0000 7fdd09797700 1 -- 172.21.15.27:0/1172935263 shutdown_connections 2024-05-21T03:24:25.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.447+0000 7fdd09797700 1 --2- 172.21.15.27:0/1172935263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd041095e0 0x7fdd0419e610 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:25.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.447+0000 7fdd09797700 1 --2- 172.21.15.27:0/1172935263 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fdcec0b2a80 0x7fdcec0b4f40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:25.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.447+0000 7fdd09797700 1 --2- 172.21.15.27:0/1172935263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd0406bcb0 0x7fdd04199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:25.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.447+0000 7fdd09797700 1 --2- 172.21.15.27:0/1172935263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd0406c670 0x7fdd0419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:25.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.448+0000 7fdd09797700 1 -- 172.21.15.27:0/1172935263 >> 172.21.15.27:0/1172935263 conn(0x7fdd040784b0 msgr2=0x7fdd041012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:25.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.448+0000 7fdd09797700 1 -- 172.21.15.27:0/1172935263 shutdown_connections 2024-05-21T03:24:25.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:25.448+0000 7fdd09797700 1 -- 172.21.15.27:0/1172935263 wait complete. 2024-05-21T03:24:25.451 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-05-21T03:24:26.014 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:25 smithi027 ceph-mon[32179]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:24:26.014 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:25 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/1172935263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:24:26.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:25 smithi039 ceph-mon[36429]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:24:26.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:25 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/1172935263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:24:26.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:25 smithi165 ceph-mon[36269]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:24:26.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:25 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/1172935263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:24:26.044 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-21T03:20:34.055985+0000', 'modified': '2024-05-21T03:23:58.657936+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24476, 'mds_1': 24472, 'mds_2': 24457, 'mds_3': 24452, 'mds_4': 24451}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14613': {'gid': 14613, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.27:6839/745486764', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6838', 'nonce': 745486764}, {'type': 'v1', 'addr': '172.21.15.27:6839', 'nonce': 745486764}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14625': {'gid': 14625, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.27:6841/3982752351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6840', 'nonce': 3982752351}, {'type': 'v1', 'addr': '172.21.15.27:6841', 'nonce': 3982752351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24401': {'gid': 24401, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.39:6833/1102873763', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 1102873763}, {'type': 'v1', 'addr': '172.21.15.39:6833', 'nonce': 1102873763}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24418': {'gid': 24418, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6833/4176762898', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6832', 'nonce': 4176762898}, {'type': 'v1', 'addr': '172.21.15.165:6833', 'nonce': 4176762898}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.39:6835/3497521608', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6834', 'nonce': 3497521608}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 3497521608}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.165:6835/3576910218', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6834', 'nonce': 3576910218}, {'type': 'v1', 'addr': '172.21.15.165:6835', 'nonce': 3576910218}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'h', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.39:6837/3673846852', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6836', 'nonce': 3673846852}, {'type': 'v1', 'addr': '172.21.15.39:6837', 'nonce': 3673846852}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.165:6837/691356139', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6836', 'nonce': 691356139}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 691356139}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.165:6839/4158170814', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6838', 'nonce': 4158170814}, {'type': 'v1', 'addr': '172.21.15.165:6839', 'nonce': 4158170814}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6839/287607819', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6838', 'nonce': 287607819}, {'type': 'v1', 'addr': '172.21.15.39:6839', 'nonce': 287607819}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24476, 'qdb_cluster': [24476, 24457, 24452, 24451, 24472]} 2024-05-21T03:24:26.045 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-21T03:24:26.045 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell mds.f status 2024-05-21T03:24:26.338 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:24:28.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.127+0000 7fd64359e700 1 -- 172.21.15.27:0/2528844387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6441046a0 msgr2=0x7fd644104a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:28.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.127+0000 7fd64359e700 1 --2- 172.21.15.27:0/2528844387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6441046a0 0x7fd644104a80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd634009990 tx=0x7fd634009ca0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.127+0000 7fd64359e700 1 -- 172.21.15.27:0/2528844387 shutdown_connections 2024-05-21T03:24:28.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.127+0000 7fd64359e700 1 --2- 172.21.15.27:0/2528844387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd64410dad0 0x7fd64410ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.127+0000 7fd64359e700 1 --2- 172.21.15.27:0/2528844387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6441046a0 0x7fd644104a80 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.127+0000 7fd64359e700 1 --2- 172.21.15.27:0/2528844387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd644104fc0 0x7fd64410d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.127+0000 7fd64359e700 1 -- 172.21.15.27:0/2528844387 >> 172.21.15.27:0/2528844387 conn(0x7fd6440fdef0 msgr2=0x7fd644100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:28.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.127+0000 7fd64359e700 1 -- 172.21.15.27:0/2528844387 shutdown_connections 2024-05-21T03:24:28.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.127+0000 7fd64359e700 1 -- 172.21.15.27:0/2528844387 wait complete. 2024-05-21T03:24:28.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.128+0000 7fd64359e700 1 Processor -- start 2024-05-21T03:24:28.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.128+0000 7fd64359e700 1 -- start start 2024-05-21T03:24:28.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.128+0000 7fd64359e700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6441046a0 0x7fd644102300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.128+0000 7fd64359e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd644104fc0 0x7fd644102840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.128+0000 7fd64359e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd64410dad0 0x7fd644102d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.128+0000 7fd64359e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd6441136e0 con 0x7fd64410dad0 2024-05-21T03:24:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.129+0000 7fd64359e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd644113560 con 0x7fd644104fc0 2024-05-21T03:24:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.129+0000 7fd64359e700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd644113860 con 0x7fd6441046a0 2024-05-21T03:24:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.129+0000 7fd642d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd64410dad0 0x7fd644102d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.129+0000 7fd642d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd64410dad0 0x7fd644102d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34832/0 (socket says 172.21.15.27:34832) 2024-05-21T03:24:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.129+0000 7fd642d9d700 1 -- 172.21.15.27:0/3251405555 learned_addr learned my addr 172.21.15.27:0/3251405555 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.129+0000 7fd64259c700 1 --2- 172.21.15.27:0/3251405555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6441046a0 0x7fd644102300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.129+0000 7fd641d9b700 1 --2- 172.21.15.27:0/3251405555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd644104fc0 0x7fd644102840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.129+0000 7fd642d9d700 1 -- 172.21.15.27:0/3251405555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6441046a0 msgr2=0x7fd644102300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.129+0000 7fd642d9d700 1 --2- 172.21.15.27:0/3251405555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6441046a0 0x7fd644102300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.129+0000 7fd642d9d700 1 -- 172.21.15.27:0/3251405555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd644104fc0 msgr2=0x7fd644102840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:28.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.129+0000 7fd642d9d700 1 --2- 172.21.15.27:0/3251405555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd644104fc0 0x7fd644102840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.129+0000 7fd642d9d700 1 -- 172.21.15.27:0/3251405555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6441a8110 con 0x7fd64410dad0 2024-05-21T03:24:28.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.129+0000 7fd64259c700 1 --2- 172.21.15.27:0/3251405555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6441046a0 0x7fd644102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:24:28.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.129+0000 7fd642d9d700 1 --2- 172.21.15.27:0/3251405555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd64410dad0 0x7fd644102d80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00ada0 tx=0x7fd63c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:28.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.129+0000 7fd6337fe700 1 -- 172.21.15.27:0/3251405555 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c017020 con 0x7fd64410dad0 2024-05-21T03:24:28.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.129+0000 7fd6337fe700 1 -- 172.21.15.27:0/3251405555 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd63c00ce30 con 0x7fd64410dad0 2024-05-21T03:24:28.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.129+0000 7fd64359e700 1 -- 172.21.15.27:0/3251405555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6441a8400 con 0x7fd64410dad0 2024-05-21T03:24:28.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.129+0000 7fd6337fe700 1 -- 172.21.15.27:0/3251405555 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c0054a0 con 0x7fd64410dad0 2024-05-21T03:24:28.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.130+0000 7fd64359e700 1 -- 172.21.15.27:0/3251405555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6441a88f0 con 0x7fd64410dad0 2024-05-21T03:24:28.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.132+0000 7fd6337fe700 1 -- 172.21.15.27:0/3251405555 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fd63c00c980 con 0x7fd64410dad0 2024-05-21T03:24:28.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.133+0000 7fd6337fe700 1 --2- 172.21.15.27:0/3251405555 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fd62c0b2a80 0x7fd62c0b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:28.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.133+0000 7fd64259c700 1 --2- 172.21.15.27:0/3251405555 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fd62c0b2a80 0x7fd62c0b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:28.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.133+0000 7fd6337fe700 1 -- 172.21.15.27:0/3251405555 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fd63c0e1fa0 con 0x7fd64410dad0 2024-05-21T03:24:28.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.134+0000 7fd64259c700 1 --2- 172.21.15.27:0/3251405555 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fd62c0b2a80 0x7fd62c0b4f40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd634006010 tx=0x7fd63400b540 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:28.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.134+0000 7fd653697b80 1 Processor -- start 2024-05-21T03:24:28.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.134+0000 7fd653697b80 1 -- start start 2024-05-21T03:24:28.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.134+0000 7fd653697b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560eee6cfd90 0x560eee6d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:28.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.134+0000 7fd653697b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560eee6cb630 0x560eee6cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:28.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.134+0000 7fd653697b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560eee6cbff0 0x560eee6d7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:28.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.134+0000 7fd653697b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560eee62ac40 con 0x560eee6cfd90 2024-05-21T03:24:28.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.134+0000 7fd653697b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560eee61a7e0 con 0x560eee6cb630 2024-05-21T03:24:28.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.134+0000 7fd653697b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560eee61e820 con 0x560eee6cbff0 2024-05-21T03:24:28.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.134+0000 7fd64259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560eee6cfd90 0x560eee6d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:28.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.134+0000 7fd64259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560eee6cfd90 0x560eee6d0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34834/0 (socket says 172.21.15.27:34834) 2024-05-21T03:24:28.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.134+0000 7fd64259c700 1 -- 172.21.15.27:0/3133508615 learned_addr learned my addr 172.21.15.27:0/3133508615 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:28.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.135+0000 7fd641d9b700 1 --2- 172.21.15.27:0/3133508615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560eee6cb630 0x560eee6cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:28.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.135+0000 7fd642d9d700 1 --2- 172.21.15.27:0/3133508615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560eee6cbff0 0x560eee6d7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:28.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.135+0000 7fd64259c700 1 -- 172.21.15.27:0/3133508615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560eee6cbff0 msgr2=0x560eee6d7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:28.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.135+0000 7fd64259c700 1 --2- 172.21.15.27:0/3133508615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560eee6cbff0 0x560eee6d7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.135+0000 7fd64259c700 1 -- 172.21.15.27:0/3133508615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560eee6cb630 msgr2=0x560eee6cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:28.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.135+0000 7fd64259c700 1 --2- 172.21.15.27:0/3133508615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560eee6cb630 0x560eee6cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.135+0000 7fd64259c700 1 -- 172.21.15.27:0/3133508615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eee56a880 con 0x560eee6cfd90 2024-05-21T03:24:28.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.135+0000 7fd642d9d700 1 --2- 172.21.15.27:0/3133508615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560eee6cbff0 0x560eee6d7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:24:28.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.135+0000 7fd641d9b700 1 --2- 172.21.15.27:0/3133508615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560eee6cb630 0x560eee6cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:24:28.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.135+0000 7fd64259c700 1 --2- 172.21.15.27:0/3133508615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560eee6cfd90 0x560eee6d0170 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd63400bb00 tx=0x7fd63401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef6e3a989df979f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:28.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.135+0000 7fd630ff9700 1 -- 172.21.15.27:0/3133508615 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd634003860 con 0x560eee6cfd90 2024-05-21T03:24:28.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.135+0000 7fd630ff9700 1 -- 172.21.15.27:0/3133508615 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd634005030 con 0x560eee6cfd90 2024-05-21T03:24:28.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.135+0000 7fd630ff9700 1 -- 172.21.15.27:0/3133508615 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63400f740 con 0x560eee6cfd90 2024-05-21T03:24:28.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.136+0000 7fd653697b80 1 -- 172.21.15.27:0/3133508615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560eee6cfd90 msgr2=0x560eee6d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:28.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.136+0000 7fd653697b80 1 --2- 172.21.15.27:0/3133508615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560eee6cfd90 0x560eee6d0170 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd63400bb00 tx=0x7fd63401f000 comp rx=0 tx=0).stop 2024-05-21T03:24:28.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.136+0000 7fd653697b80 1 -- 172.21.15.27:0/3133508615 shutdown_connections 2024-05-21T03:24:28.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.136+0000 7fd653697b80 1 --2- 172.21.15.27:0/3133508615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560eee6cbff0 0x560eee6d7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.136+0000 7fd653697b80 1 --2- 172.21.15.27:0/3133508615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560eee6cfd90 0x560eee6d0170 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.136+0000 7fd653697b80 1 --2- 172.21.15.27:0/3133508615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560eee6cb630 0x560eee6cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.136+0000 7fd653697b80 1 -- 172.21.15.27:0/3133508615 >> 172.21.15.27:0/3133508615 conn(0x560eee5f9030 msgr2=0x560eee6c5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.137+0000 7fd653697b80 1 -- 172.21.15.27:0/3133508615 shutdown_connections 2024-05-21T03:24:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.137+0000 7fd653697b80 1 -- 172.21.15.27:0/3133508615 wait complete. 2024-05-21T03:24:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.137+0000 7fd653697b80 1 -- start start 2024-05-21T03:24:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.137+0000 7fd653697b80 1 Processor -- start 2024-05-21T03:24:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.137+0000 7fd653697b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560eee6cb630 0x560eee6d7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.137+0000 7fd653697b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560eee6cbff0 0x560eee6d4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.137+0000 7fd653697b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560eee6cfd90 0x560eee6d5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.137+0000 7fd653697b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560eee520f50 con 0x560eee6cfd90 2024-05-21T03:24:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.137+0000 7fd653697b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560eee5bc770 con 0x560eee6cbff0 2024-05-21T03:24:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.137+0000 7fd653697b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560eee6d8300 con 0x560eee6cb630 2024-05-21T03:24:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.137+0000 7fd653697b80 10 client.0 _ll_register_callbacks cb 0x560eee5b58d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:24:28.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.137+0000 7fd642d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560eee6cfd90 0x560eee6d5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:28.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.138+0000 7fd642d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560eee6cfd90 0x560eee6d5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34838/0 (socket says 172.21.15.27:34838) 2024-05-21T03:24:28.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.138+0000 7fd642d9d700 1 -- 172.21.15.27:0/2944568077 learned_addr learned my addr 172.21.15.27:0/2944568077 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:28.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.138+0000 7fd64259c700 1 --2- 172.21.15.27:0/2944568077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560eee6cb630 0x560eee6d7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:28.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.138+0000 7fd641d9b700 1 --2- 172.21.15.27:0/2944568077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560eee6cbff0 0x560eee6d4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:28.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.138+0000 7fd642d9d700 1 -- 172.21.15.27:0/2944568077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560eee6cb630 msgr2=0x560eee6d7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:28.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.138+0000 7fd642d9d700 1 --2- 172.21.15.27:0/2944568077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560eee6cb630 0x560eee6d7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.138+0000 7fd642d9d700 1 -- 172.21.15.27:0/2944568077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560eee6cbff0 msgr2=0x560eee6d4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:28.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.138+0000 7fd642d9d700 1 --2- 172.21.15.27:0/2944568077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560eee6cbff0 0x560eee6d4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.138+0000 7fd642d9d700 1 -- 172.21.15.27:0/2944568077 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eee5b9640 con 0x560eee6cfd90 2024-05-21T03:24:28.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.138+0000 7fd641d9b700 1 --2- 172.21.15.27:0/2944568077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560eee6cbff0 0x560eee6d4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:24:28.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.138+0000 7fd642d9d700 1 --2- 172.21.15.27:0/2944568077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560eee6cfd90 0x560eee6d5240 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd63c0a90d0 tx=0x7fd63c007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86932374b1af32c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:28.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.138+0000 7fd629ffb700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:24:28.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.138+0000 7fd629ffb700 1 -- 172.21.15.27:0/2944568077 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c0a9840 con 0x560eee6cfd90 2024-05-21T03:24:28.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.138+0000 7fd629ffb700 1 -- 172.21.15.27:0/2944568077 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fd63c0a93a0 con 0x560eee6cfd90 2024-05-21T03:24:28.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.138+0000 7fd653697b80 1 -- 172.21.15.27:0/2944568077 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd634005030 con 0x560eee6cfd90 2024-05-21T03:24:28.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.138+0000 7fd629ffb700 1 -- 172.21.15.27:0/2944568077 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd63c0a9140 con 0x560eee6cfd90 2024-05-21T03:24:28.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.138+0000 7fd653697b80 10 client.15405 fetch_fsmap learned FSMap version 29 2024-05-21T03:24:28.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.138+0000 7fd653697b80 1 -- 172.21.15.27:0/2944568077 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560eee6e2ab0 con 0x560eee6cfd90 2024-05-21T03:24:28.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.139+0000 7fd629ffb700 1 -- 172.21.15.27:0/2944568077 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd634005030 con 0x560eee6cfd90 2024-05-21T03:24:28.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.139+0000 7fd629ffb700 1 -- 172.21.15.27:0/2944568077 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd63c012070 con 0x560eee6cfd90 2024-05-21T03:24:28.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.139+0000 7fd653697b80 10 client.15405 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T03:24:28.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.139+0000 7fd653697b80 10 client.15405 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-21T03:24:28.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.139+0000 7fd653697b80 20 client.15405 populate_metadata read hostname 'smithi027' 2024-05-21T03:24:28.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.139+0000 7fd653697b80 1 --2- 172.21.15.27:0/2944568077 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x560eee6e5090 0x560eee6e6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:28.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.139+0000 7fd653697b80 4 client.15405 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:24:28.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.139+0000 7fd653697b80 1 -- 172.21.15.27:0/2944568077 --> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560eee697990 con 0x560eee6e5090 2024-05-21T03:24:28.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.139+0000 7fd64259c700 1 --2- 172.21.15.27:0/2944568077 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x560eee6e5090 0x560eee6e6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:28.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.140+0000 7fd629ffb700 10 client.15405 ms_handle_connect on v2:172.21.15.165:6834/3576910218 2024-05-21T03:24:28.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.140+0000 7fd64259c700 1 --2- 172.21.15.27:0/2944568077 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x560eee6e5090 0x560eee6e6500 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=be9a9221b1787488 server_cookie=72fc3f515152c040 in_seq=0 out_seq=0 2024-05-21T03:24:28.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.141+0000 7fd629ffb700 1 -- 172.21.15.27:0/2944568077 <== mds.4 v2:172.21.15.165:6834/3576910218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560eee697990 con 0x560eee6e5090 2024-05-21T03:24:28.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.141+0000 7fd629ffb700 10 client.15405 handle_command_reply: tid=0 2024-05-21T03:24:28.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.141+0000 7fd653697b80 1 client.15405 shutdown 2024-05-21T03:24:28.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.141+0000 7fd653697b80 1 -- 172.21.15.27:0/2944568077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560eee6cfd90 msgr2=0x560eee6d5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:28.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.141+0000 7fd653697b80 1 --2- 172.21.15.27:0/2944568077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560eee6cfd90 0x560eee6d5240 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd63c0a90d0 tx=0x7fd63c007f50 comp rx=0 tx=0).stop 2024-05-21T03:24:28.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.141+0000 7fd653697b80 1 -- 172.21.15.27:0/2944568077 shutdown_connections 2024-05-21T03:24:28.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.141+0000 7fd653697b80 1 --2- 172.21.15.27:0/2944568077 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x560eee6e5090 0x560eee6e6500 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.141+0000 7fd653697b80 1 --2- 172.21.15.27:0/2944568077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560eee6cfd90 0x560eee6d5240 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.141+0000 7fd653697b80 1 --2- 172.21.15.27:0/2944568077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560eee6cb630 0x560eee6d7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.141+0000 7fd653697b80 1 --2- 172.21.15.27:0/2944568077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560eee6cbff0 0x560eee6d4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.141+0000 7fd653697b80 1 -- 172.21.15.27:0/2944568077 >> 172.21.15.27:0/2944568077 conn(0x560eee5f9030 msgr2=0x560eee6d14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:28.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.141+0000 7fd629ffb700 0 client.15405 ms_handle_reset on v2:172.21.15.165:6834/3576910218 2024-05-21T03:24:28.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.141+0000 7fd653697b80 1 -- 172.21.15.27:0/2944568077 shutdown_connections 2024-05-21T03:24:28.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.141+0000 7fd653697b80 1 -- 172.21.15.27:0/2944568077 wait complete. 2024-05-21T03:24:28.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.141+0000 7fd653697b80 20 client.15405 trim_cache size 0 max 16384 2024-05-21T03:24:28.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.162+0000 7fd653697b80 1 Processor -- start 2024-05-21T03:24:28.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.163+0000 7fd653697b80 1 -- start start 2024-05-21T03:24:28.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.163+0000 7fd653697b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560eee6cb630 0x560eee6cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:28.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.163+0000 7fd653697b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560eee6cbff0 0x560eee6d5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:28.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.163+0000 7fd653697b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560eee6cfd90 0x560eee6c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:28.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.163+0000 7fd653697b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560eee6e4840 con 0x560eee6cb630 2024-05-21T03:24:28.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.163+0000 7fd653697b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560eee6e4f10 con 0x560eee6cbff0 2024-05-21T03:24:28.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.163+0000 7fd653697b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560eee6e55e0 con 0x560eee6cfd90 2024-05-21T03:24:28.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.163+0000 7fd64259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560eee6cb630 0x560eee6cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:28.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.163+0000 7fd64259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560eee6cb630 0x560eee6cf580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34848/0 (socket says 172.21.15.27:34848) 2024-05-21T03:24:28.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.163+0000 7fd64259c700 1 -- 172.21.15.27:0/2223899210 learned_addr learned my addr 172.21.15.27:0/2223899210 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:28.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.163+0000 7fd641d9b700 1 --2- 172.21.15.27:0/2223899210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560eee6cbff0 0x560eee6d5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:28.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.163+0000 7fd642d9d700 1 --2- 172.21.15.27:0/2223899210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560eee6cfd90 0x560eee6c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:28.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.163+0000 7fd64259c700 1 -- 172.21.15.27:0/2223899210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560eee6cfd90 msgr2=0x560eee6c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:28.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.163+0000 7fd64259c700 1 --2- 172.21.15.27:0/2223899210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560eee6cfd90 0x560eee6c4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.163+0000 7fd64259c700 1 -- 172.21.15.27:0/2223899210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560eee6cbff0 msgr2=0x560eee6d5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:28.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.163+0000 7fd64259c700 1 --2- 172.21.15.27:0/2223899210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560eee6cbff0 0x560eee6d5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.163+0000 7fd64259c700 1 -- 172.21.15.27:0/2223899210 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eee6d5c60 con 0x560eee6cb630 2024-05-21T03:24:28.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.163+0000 7fd64259c700 1 --2- 172.21.15.27:0/2223899210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560eee6cb630 0x560eee6cf580 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd6340098e0 tx=0x7fd634019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de3414e4e1d62af server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:28.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.164+0000 7fd62bfff700 1 -- 172.21.15.27:0/2223899210 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6340055a0 con 0x560eee6cb630 2024-05-21T03:24:28.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.164+0000 7fd62bfff700 1 -- 172.21.15.27:0/2223899210 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6340050f0 con 0x560eee6cb630 2024-05-21T03:24:28.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.164+0000 7fd62bfff700 1 -- 172.21.15.27:0/2223899210 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63403aab0 con 0x560eee6cb630 2024-05-21T03:24:28.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.164+0000 7fd653697b80 1 -- 172.21.15.27:0/2223899210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560eee6cb630 msgr2=0x560eee6cf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:28.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.164+0000 7fd653697b80 1 --2- 172.21.15.27:0/2223899210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560eee6cb630 0x560eee6cf580 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd6340098e0 tx=0x7fd634019040 comp rx=0 tx=0).stop 2024-05-21T03:24:28.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.164+0000 7fd653697b80 1 -- 172.21.15.27:0/2223899210 shutdown_connections 2024-05-21T03:24:28.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.164+0000 7fd653697b80 1 --2- 172.21.15.27:0/2223899210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560eee6cfd90 0x560eee6c4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.164+0000 7fd653697b80 1 --2- 172.21.15.27:0/2223899210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560eee6cb630 0x560eee6cf580 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.164+0000 7fd653697b80 1 --2- 172.21.15.27:0/2223899210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560eee6cbff0 0x560eee6d5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.164+0000 7fd653697b80 1 -- 172.21.15.27:0/2223899210 >> 172.21.15.27:0/2223899210 conn(0x560eee5f9030 msgr2=0x560eee6cee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:28.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.164+0000 7fd653697b80 1 -- 172.21.15.27:0/2223899210 shutdown_connections 2024-05-21T03:24:28.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.164+0000 7fd653697b80 1 -- 172.21.15.27:0/2223899210 wait complete. 2024-05-21T03:24:28.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.165+0000 7fd653697b80 1 -- start start 2024-05-21T03:24:28.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.165+0000 7fd653697b80 1 Processor -- start 2024-05-21T03:24:28.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.165+0000 7fd653697b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560eee6cb630 0x560eee6ce400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:28.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.165+0000 7fd653697b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560eee6cbff0 0x560eee6ef970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:28.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.165+0000 7fd653697b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560eee6cfd90 0x560eee6eff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:28.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.165+0000 7fd653697b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560eee6e6380 con 0x560eee6cfd90 2024-05-21T03:24:28.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.165+0000 7fd653697b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560eee6e5cb0 con 0x560eee6cbff0 2024-05-21T03:24:28.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.165+0000 7fd653697b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560eee6e6a50 con 0x560eee6cb630 2024-05-21T03:24:28.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.165+0000 7fd653697b80 10 client.0 _ll_register_callbacks cb 0x560eee5b58d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:24:28.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.165+0000 7fd642d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560eee6cfd90 0x560eee6eff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:28.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.165+0000 7fd64259c700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560eee6cb630 0x560eee6ce400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:28.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.165+0000 7fd642d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560eee6cfd90 0x560eee6eff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34858/0 (socket says 172.21.15.27:34858) 2024-05-21T03:24:28.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.165+0000 7fd642d9d700 1 -- 172.21.15.27:0/1695806252 learned_addr learned my addr 172.21.15.27:0/1695806252 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:28.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.165+0000 7fd641d9b700 1 --2- 172.21.15.27:0/1695806252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560eee6cbff0 0x560eee6ef970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:28.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.165+0000 7fd642d9d700 1 -- 172.21.15.27:0/1695806252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560eee6cb630 msgr2=0x560eee6ce400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:28.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.165+0000 7fd642d9d700 1 --2- 172.21.15.27:0/1695806252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560eee6cb630 0x560eee6ce400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.166+0000 7fd642d9d700 1 -- 172.21.15.27:0/1695806252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560eee6cbff0 msgr2=0x560eee6ef970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:28.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.166+0000 7fd642d9d700 1 --2- 172.21.15.27:0/1695806252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560eee6cbff0 0x560eee6ef970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.166+0000 7fd642d9d700 1 -- 172.21.15.27:0/1695806252 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eee6c52d0 con 0x560eee6cfd90 2024-05-21T03:24:28.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.166+0000 7fd642d9d700 1 --2- 172.21.15.27:0/1695806252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560eee6cfd90 0x560eee6eff30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd63c004ff0 tx=0x7fd63c007ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4578c71b62cdaca9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:28.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.166+0000 7fd629ffb700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:24:28.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.166+0000 7fd629ffb700 1 -- 172.21.15.27:0/1695806252 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c0a8f10 con 0x560eee6cfd90 2024-05-21T03:24:28.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.166+0000 7fd629ffb700 1 -- 172.21.15.27:0/1695806252 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fd63c0a4ab0 con 0x560eee6cfd90 2024-05-21T03:24:28.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.166+0000 7fd653697b80 1 -- 172.21.15.27:0/1695806252 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6340050f0 con 0x560eee6cfd90 2024-05-21T03:24:28.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.166+0000 7fd629ffb700 1 -- 172.21.15.27:0/1695806252 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd63c0b1d00 con 0x560eee6cfd90 2024-05-21T03:24:28.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.166+0000 7fd653697b80 10 client.15411 fetch_fsmap learned FSMap version 29 2024-05-21T03:24:28.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.166+0000 7fd653697b80 1 -- 172.21.15.27:0/1695806252 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560eee750b50 con 0x560eee6cfd90 2024-05-21T03:24:28.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.166+0000 7fd629ffb700 1 -- 172.21.15.27:0/1695806252 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6340050f0 con 0x560eee6cfd90 2024-05-21T03:24:28.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.166+0000 7fd629ffb700 1 -- 172.21.15.27:0/1695806252 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd63c012070 con 0x560eee6cfd90 2024-05-21T03:24:28.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.166+0000 7fd653697b80 10 client.15411 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T03:24:28.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.166+0000 7fd653697b80 10 client.15411 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-21T03:24:28.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.166+0000 7fd653697b80 20 client.15411 populate_metadata read hostname 'smithi027' 2024-05-21T03:24:28.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.166+0000 7fd653697b80 1 --2- 172.21.15.27:0/1695806252 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x560eee755730 0x560eee754440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:28.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.166+0000 7fd653697b80 4 client.15411 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T03:24:28.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.166+0000 7fd653697b80 1 -- 172.21.15.27:0/1695806252 --> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] -- command(tid 0: {"prefix": "status"}) -- 0x560eee62cb90 con 0x560eee755730 2024-05-21T03:24:28.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.167+0000 7fd64259c700 1 --2- 172.21.15.27:0/1695806252 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x560eee755730 0x560eee754440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:28.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.167+0000 7fd64259c700 1 --2- 172.21.15.27:0/1695806252 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x560eee755730 0x560eee754440 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=863efbb50904b691 server_cookie=8e4c4d97adc551cc in_seq=0 out_seq=0 2024-05-21T03:24:28.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.167+0000 7fd629ffb700 10 client.15411 ms_handle_connect on v2:172.21.15.165:6834/3576910218 2024-05-21T03:24:28.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.167+0000 7fd629ffb700 1 -- 172.21.15.27:0/1695806252 <== mds.4 v2:172.21.15.165:6834/3576910218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x560eee62cb90 con 0x560eee755730 2024-05-21T03:24:28.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.167+0000 7fd629ffb700 10 client.15411 handle_command_reply: tid=0 2024-05-21T03:24:28.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.167+0000 7fd653697b80 1 client.15411 shutdown 2024-05-21T03:24:28.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.168+0000 7fd653697b80 1 -- 172.21.15.27:0/1695806252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560eee6cfd90 msgr2=0x560eee6eff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:28.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.168+0000 7fd653697b80 1 --2- 172.21.15.27:0/1695806252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560eee6cfd90 0x560eee6eff30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd63c004ff0 tx=0x7fd63c007ed0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.168+0000 7fd653697b80 1 -- 172.21.15.27:0/1695806252 shutdown_connections 2024-05-21T03:24:28.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.168+0000 7fd653697b80 1 --2- 172.21.15.27:0/1695806252 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x560eee755730 0x560eee754440 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.168+0000 7fd653697b80 1 --2- 172.21.15.27:0/1695806252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560eee6cfd90 0x560eee6eff30 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.168+0000 7fd653697b80 1 --2- 172.21.15.27:0/1695806252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560eee6cb630 0x560eee6ce400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.168+0000 7fd653697b80 1 --2- 172.21.15.27:0/1695806252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560eee6cbff0 0x560eee6ef970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.168+0000 7fd653697b80 1 -- 172.21.15.27:0/1695806252 >> 172.21.15.27:0/1695806252 conn(0x560eee5f9030 msgr2=0x560eee6caeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:28.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.168+0000 7fd629ffb700 0 client.15411 ms_handle_reset on v2:172.21.15.165:6834/3576910218 2024-05-21T03:24:28.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.168+0000 7fd653697b80 1 -- 172.21.15.27:0/1695806252 shutdown_connections 2024-05-21T03:24:28.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.168+0000 7fd653697b80 1 -- 172.21.15.27:0/1695806252 wait complete. 2024-05-21T03:24:28.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.168+0000 7fd653697b80 20 client.15411 trim_cache size 0 max 16384 2024-05-21T03:24:28.174 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:24:28.174 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "c9ca94be-171f-11ef-bc9a-c7b262605968", 2024-05-21T03:24:28.174 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 4, 2024-05-21T03:24:28.174 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24451, 2024-05-21T03:24:28.174 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-05-21T03:24:28.174 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-05-21T03:24:28.174 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-05-21T03:24:28.174 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 218.70727013000001, 2024-05-21T03:24:28.174 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 29, 2024-05-21T03:24:28.174 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 83, 2024-05-21T03:24:28.174 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 82, 2024-05-21T03:24:28.174 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 266.57477637400001 2024-05-21T03:24:28.175 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:24:28.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.168+0000 7fd64359e700 1 -- 172.21.15.27:0/3251405555 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fd62c0b2a80 msgr2=0x7fd62c0b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:28.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.168+0000 7fd64359e700 1 --2- 172.21.15.27:0/3251405555 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fd62c0b2a80 0x7fd62c0b4f40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd634006010 tx=0x7fd63400b540 comp rx=0 tx=0).stop 2024-05-21T03:24:28.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.168+0000 7fd64359e700 1 -- 172.21.15.27:0/3251405555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd64410dad0 msgr2=0x7fd644102d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:28.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.168+0000 7fd64359e700 1 --2- 172.21.15.27:0/3251405555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd64410dad0 0x7fd644102d80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00ada0 tx=0x7fd63c00c330 comp rx=0 tx=0).stop 2024-05-21T03:24:28.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.169+0000 7fd64359e700 1 -- 172.21.15.27:0/3251405555 shutdown_connections 2024-05-21T03:24:28.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.169+0000 7fd64359e700 1 --2- 172.21.15.27:0/3251405555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd64410dad0 0x7fd644102d80 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.169+0000 7fd64359e700 1 --2- 172.21.15.27:0/3251405555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6441046a0 0x7fd644102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.169+0000 7fd64359e700 1 --2- 172.21.15.27:0/3251405555 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fd62c0b2a80 0x7fd62c0b4f40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.169+0000 7fd64359e700 1 --2- 172.21.15.27:0/3251405555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd644104fc0 0x7fd644102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:28.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.169+0000 7fd64359e700 1 -- 172.21.15.27:0/3251405555 >> 172.21.15.27:0/3251405555 conn(0x7fd6440fdef0 msgr2=0x7fd6440fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:28.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.169+0000 7fd64359e700 1 -- 172.21.15.27:0/3251405555 shutdown_connections 2024-05-21T03:24:28.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:28.169+0000 7fd64359e700 1 -- 172.21.15.27:0/3251405555 wait complete. 2024-05-21T03:24:28.722 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:28 smithi039 ceph-mon[36429]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:24:28.724 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:28 smithi165 ceph-mon[36269]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:24:28.838 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:28 smithi027 ceph-mon[32179]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:24:28.871 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell mds.h status 2024-05-21T03:24:29.127 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:24:30.586 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:30 smithi027 ceph-mon[32179]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 938 B/s wr, 12 op/s 2024-05-21T03:24:30.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:30 smithi039 ceph-mon[36429]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 938 B/s wr, 12 op/s 2024-05-21T03:24:30.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:30 smithi165 ceph-mon[36269]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 938 B/s wr, 12 op/s 2024-05-21T03:24:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.849+0000 7f31b4949700 1 -- 172.21.15.27:0/169040028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31b01037f0 msgr2=0x7f31b0103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.849+0000 7f31b4949700 1 --2- 172.21.15.27:0/169040028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31b01037f0 0x7f31b0103bd0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f31a00099c0 tx=0x7f31a0009cd0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.849+0000 7f31b4949700 1 -- 172.21.15.27:0/169040028 shutdown_connections 2024-05-21T03:24:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.849+0000 7f31b4949700 1 --2- 172.21.15.27:0/169040028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31b01088f0 0x7f31b0110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.849+0000 7f31b4949700 1 --2- 172.21.15.27:0/169040028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31b01037f0 0x7f31b0103bd0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.849+0000 7f31b4949700 1 --2- 172.21.15.27:0/169040028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31b01041a0 0x7f31b01081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.849+0000 7f31b4949700 1 -- 172.21.15.27:0/169040028 >> 172.21.15.27:0/169040028 conn(0x7f31b007a6a0 msgr2=0x7f31b007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.849+0000 7f31b4949700 1 -- 172.21.15.27:0/169040028 shutdown_connections 2024-05-21T03:24:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.849+0000 7f31b4949700 1 -- 172.21.15.27:0/169040028 wait complete. 2024-05-21T03:24:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.850+0000 7f31b4949700 1 Processor -- start 2024-05-21T03:24:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.851+0000 7f31b4949700 1 -- start start 2024-05-21T03:24:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.851+0000 7f31b4949700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31b01037f0 0x7f31b0075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:30.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.851+0000 7f31b4949700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31b01041a0 0x7f31b0075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:30.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.851+0000 7f31b4949700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31b01088f0 0x7f31b00762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:30.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.851+0000 7f31b4949700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f31b01136c0 con 0x7f31b01088f0 2024-05-21T03:24:30.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.851+0000 7f31b4949700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f31b0113540 con 0x7f31b01037f0 2024-05-21T03:24:30.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.851+0000 7f31b4949700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f31b0113840 con 0x7f31b01041a0 2024-05-21T03:24:30.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.851+0000 7f31aed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31b01088f0 0x7f31b00762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:30.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.851+0000 7f31a5d9b700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31b01041a0 0x7f31b0075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:30.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.851+0000 7f31a5d9b700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31b01041a0 0x7f31b0075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.27:35362/0 (socket says 172.21.15.27:35362) 2024-05-21T03:24:30.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.851+0000 7f31aed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31b01088f0 0x7f31b00762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34874/0 (socket says 172.21.15.27:34874) 2024-05-21T03:24:30.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.851+0000 7f31a5d9b700 1 -- 172.21.15.27:0/2191183951 learned_addr learned my addr 172.21.15.27:0/2191183951 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:30.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.851+0000 7f31ae59c700 1 --2- 172.21.15.27:0/2191183951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31b01037f0 0x7f31b0075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:30.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.852+0000 7f31a5d9b700 1 -- 172.21.15.27:0/2191183951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31b01037f0 msgr2=0x7f31b0075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:30.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.852+0000 7f31a5d9b700 1 --2- 172.21.15.27:0/2191183951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31b01037f0 0x7f31b0075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.852+0000 7f31a5d9b700 1 -- 172.21.15.27:0/2191183951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31b01088f0 msgr2=0x7f31b00762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:30.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.852+0000 7f31a5d9b700 1 --2- 172.21.15.27:0/2191183951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31b01088f0 0x7f31b00762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.852+0000 7f31a5d9b700 1 -- 172.21.15.27:0/2191183951 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b01a8120 con 0x7f31b01041a0 2024-05-21T03:24:30.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.852+0000 7f31aed9d700 1 --2- 172.21.15.27:0/2191183951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31b01088f0 0x7f31b00762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:24:30.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.852+0000 7f31a5d9b700 1 --2- 172.21.15.27:0/2191183951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31b01041a0 0x7f31b0075d80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f319800e9a0 tx=0x7f319800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:30.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.852+0000 7f31a7fff700 1 -- 172.21.15.27:0/2191183951 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319800ca10 con 0x7f31b01041a0 2024-05-21T03:24:30.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.852+0000 7f31b4949700 1 -- 172.21.15.27:0/2191183951 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31b01a8410 con 0x7f31b01041a0 2024-05-21T03:24:30.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.852+0000 7f31a7fff700 1 -- 172.21.15.27:0/2191183951 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f319800cbb0 con 0x7f31b01041a0 2024-05-21T03:24:30.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.852+0000 7f31a7fff700 1 -- 172.21.15.27:0/2191183951 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3198018680 con 0x7f31b01041a0 2024-05-21T03:24:30.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.852+0000 7f31b4949700 1 -- 172.21.15.27:0/2191183951 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31b01a8950 con 0x7f31b01041a0 2024-05-21T03:24:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.855+0000 7f31a7fff700 1 -- 172.21.15.27:0/2191183951 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f3198010430 con 0x7f31b01041a0 2024-05-21T03:24:30.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.856+0000 7f31a7fff700 1 --2- 172.21.15.27:0/2191183951 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f319c0b2a80 0x7f319c0b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:30.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.856+0000 7f31bf1dfb80 1 Processor -- start 2024-05-21T03:24:30.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.856+0000 7f31ae59c700 1 --2- 172.21.15.27:0/2191183951 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f319c0b2a80 0x7f319c0b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.856+0000 7f31a7fff700 1 -- 172.21.15.27:0/2191183951 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f31980e2910 con 0x7f31b01041a0 2024-05-21T03:24:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.856+0000 7f31bf1dfb80 1 -- start start 2024-05-21T03:24:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.856+0000 7f31bf1dfb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffaa803810 0x55ffaa803bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.856+0000 7f31bf1dfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffaa8041c0 0x55ffaa8078f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.856+0000 7f31bf1dfb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffaa807f10 0x55ffaa808350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.856+0000 7f31bf1dfb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ffaa762c40 con 0x55ffaa803810 2024-05-21T03:24:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.856+0000 7f31bf1dfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ffaa7527e0 con 0x55ffaa8041c0 2024-05-21T03:24:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.856+0000 7f31bf1dfb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ffaa756820 con 0x55ffaa807f10 2024-05-21T03:24:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.856+0000 7f31ae59c700 1 --2- 172.21.15.27:0/2191183951 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f319c0b2a80 0x7f319c0b4f40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f31a0003760 tx=0x7f31a0005fb0 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.856+0000 7f31aed9d700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffaa807f10 0x55ffaa808350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.856+0000 7f31ae59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffaa803810 0x55ffaa803bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.856+0000 7f31aed9d700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffaa807f10 0x55ffaa808350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.27:35364/0 (socket says 172.21.15.27:35364) 2024-05-21T03:24:30.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.857+0000 7f31aed9d700 1 -- 172.21.15.27:0/1441286810 learned_addr learned my addr 172.21.15.27:0/1441286810 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:30.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.857+0000 7f31ae59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffaa803810 0x55ffaa803bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34888/0 (socket says 172.21.15.27:34888) 2024-05-21T03:24:30.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.857+0000 7f31a5d9b700 1 --2- 172.21.15.27:0/1441286810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffaa8041c0 0x55ffaa8078f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:30.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.857+0000 7f31aed9d700 1 -- 172.21.15.27:0/1441286810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffaa8041c0 msgr2=0x55ffaa8078f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:30.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.857+0000 7f31aed9d700 1 --2- 172.21.15.27:0/1441286810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffaa8041c0 0x55ffaa8078f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.857+0000 7f31aed9d700 1 -- 172.21.15.27:0/1441286810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffaa803810 msgr2=0x55ffaa803bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:30.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.857+0000 7f31aed9d700 1 --2- 172.21.15.27:0/1441286810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffaa803810 0x55ffaa803bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.857+0000 7f31aed9d700 1 -- 172.21.15.27:0/1441286810 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffaa6a2880 con 0x55ffaa807f10 2024-05-21T03:24:30.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.857+0000 7f31a5d9b700 1 --2- 172.21.15.27:0/1441286810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffaa8041c0 0x55ffaa8078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:24:30.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.857+0000 7f31aed9d700 1 --2- 172.21.15.27:0/1441286810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffaa807f10 0x55ffaa808350 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f31a800a730 tx=0x7f31a80078d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=506e61debbfe7eea server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:30.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.857+0000 7f31a4d99700 1 -- 172.21.15.27:0/1441286810 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a8014600 con 0x55ffaa807f10 2024-05-21T03:24:30.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.857+0000 7f31a4d99700 1 -- 172.21.15.27:0/1441286810 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31a8019070 con 0x55ffaa807f10 2024-05-21T03:24:30.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.857+0000 7f31a4d99700 1 -- 172.21.15.27:0/1441286810 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a8008a50 con 0x55ffaa807f10 2024-05-21T03:24:30.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.858+0000 7f31bf1dfb80 1 -- 172.21.15.27:0/1441286810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffaa807f10 msgr2=0x55ffaa808350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:30.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.858+0000 7f31bf1dfb80 1 --2- 172.21.15.27:0/1441286810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffaa807f10 0x55ffaa808350 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f31a800a730 tx=0x7f31a80078d0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.858+0000 7f31bf1dfb80 1 -- 172.21.15.27:0/1441286810 shutdown_connections 2024-05-21T03:24:30.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.858+0000 7f31bf1dfb80 1 --2- 172.21.15.27:0/1441286810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffaa807f10 0x55ffaa808350 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.858+0000 7f31bf1dfb80 1 --2- 172.21.15.27:0/1441286810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffaa803810 0x55ffaa803bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.858+0000 7f31bf1dfb80 1 --2- 172.21.15.27:0/1441286810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffaa8041c0 0x55ffaa8078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.858+0000 7f31bf1dfb80 1 -- 172.21.15.27:0/1441286810 >> 172.21.15.27:0/1441286810 conn(0x55ffaa731030 msgr2=0x55ffaa7fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:30.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.858+0000 7f31bf1dfb80 1 -- 172.21.15.27:0/1441286810 shutdown_connections 2024-05-21T03:24:30.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.858+0000 7f31bf1dfb80 1 -- 172.21.15.27:0/1441286810 wait complete. 2024-05-21T03:24:30.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.858+0000 7f31bf1dfb80 1 -- start start 2024-05-21T03:24:30.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.859+0000 7f31bf1dfb80 1 Processor -- start 2024-05-21T03:24:30.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.859+0000 7f31bf1dfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffaa803810 0x55ffaa80f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:30.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.859+0000 7f31bf1dfb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffaa8041c0 0x55ffaa80c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:30.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.859+0000 7f31bf1dfb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffaa807f10 0x55ffaa80cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:30.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.859+0000 7f31bf1dfb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ffaa658f50 con 0x55ffaa807f10 2024-05-21T03:24:30.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.859+0000 7f31bf1dfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ffaa6f4770 con 0x55ffaa803810 2024-05-21T03:24:30.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.859+0000 7f31bf1dfb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ffaa810070 con 0x55ffaa8041c0 2024-05-21T03:24:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.859+0000 7f31bf1dfb80 10 client.0 _ll_register_callbacks cb 0x55ffaa6ed8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:24:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.859+0000 7f31aed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffaa807f10 0x55ffaa80cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.859+0000 7f31aed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffaa807f10 0x55ffaa80cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34902/0 (socket says 172.21.15.27:34902) 2024-05-21T03:24:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.859+0000 7f31aed9d700 1 -- 172.21.15.27:0/1861926587 learned_addr learned my addr 172.21.15.27:0/1861926587 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.859+0000 7f31a5d9b700 1 --2- 172.21.15.27:0/1861926587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffaa8041c0 0x55ffaa80c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.859+0000 7f31ae59c700 1 --2- 172.21.15.27:0/1861926587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffaa803810 0x55ffaa80f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.859+0000 7f31aed9d700 1 -- 172.21.15.27:0/1861926587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffaa8041c0 msgr2=0x55ffaa80c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.859+0000 7f31aed9d700 1 --2- 172.21.15.27:0/1861926587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffaa8041c0 0x55ffaa80c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.859+0000 7f31aed9d700 1 -- 172.21.15.27:0/1861926587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffaa803810 msgr2=0x55ffaa80f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.859+0000 7f31aed9d700 1 --2- 172.21.15.27:0/1861926587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffaa803810 0x55ffaa80f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.859+0000 7f31aed9d700 1 -- 172.21.15.27:0/1861926587 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffaa6f1640 con 0x55ffaa807f10 2024-05-21T03:24:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.860+0000 7f31aed9d700 1 --2- 172.21.15.27:0/1861926587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffaa807f10 0x55ffaa80cc30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f31a8009e60 tx=0x7f31a8009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7518730d5c2c5d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:30.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.860+0000 7f3195ffb700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:24:30.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.860+0000 7f3195ffb700 1 -- 172.21.15.27:0/1861926587 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a8014810 con 0x55ffaa807f10 2024-05-21T03:24:30.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.860+0000 7f3195ffb700 1 -- 172.21.15.27:0/1861926587 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f31a80149b0 con 0x55ffaa807f10 2024-05-21T03:24:30.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.860+0000 7f31bf1dfb80 1 -- 172.21.15.27:0/1861926587 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31a8019070 con 0x55ffaa807f10 2024-05-21T03:24:30.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.860+0000 7f3195ffb700 1 -- 172.21.15.27:0/1861926587 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f31a8014bb0 con 0x55ffaa807f10 2024-05-21T03:24:30.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.860+0000 7f31bf1dfb80 10 client.15423 fetch_fsmap learned FSMap version 29 2024-05-21T03:24:30.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.860+0000 7f31bf1dfb80 1 -- 172.21.15.27:0/1861926587 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ffaa81a7c0 con 0x55ffaa807f10 2024-05-21T03:24:30.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.860+0000 7f3195ffb700 1 -- 172.21.15.27:0/1861926587 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31a8019070 con 0x55ffaa807f10 2024-05-21T03:24:30.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.860+0000 7f3195ffb700 1 -- 172.21.15.27:0/1861926587 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f31a8004020 con 0x55ffaa807f10 2024-05-21T03:24:30.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.860+0000 7f31bf1dfb80 10 client.15423 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T03:24:30.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.860+0000 7f31bf1dfb80 10 client.15423 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-21T03:24:30.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.861+0000 7f31bf1dfb80 20 client.15423 populate_metadata read hostname 'smithi027' 2024-05-21T03:24:30.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.861+0000 7f31bf1dfb80 1 --2- 172.21.15.27:0/1861926587 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x55ffaa81c9c0 0x55ffaa81de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:30.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.861+0000 7f31bf1dfb80 4 client.15423 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:24:30.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.861+0000 7f31bf1dfb80 1 -- 172.21.15.27:0/1861926587 --> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffaa7cf990 con 0x55ffaa81c9c0 2024-05-21T03:24:30.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.861+0000 7f31ae59c700 1 --2- 172.21.15.27:0/1861926587 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x55ffaa81c9c0 0x55ffaa81de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:30.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.861+0000 7f31ae59c700 1 --2- 172.21.15.27:0/1861926587 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x55ffaa81c9c0 0x55ffaa81de30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cbca3162dfbb5125 server_cookie=b0c8211b82cd66f9 in_seq=0 out_seq=0 2024-05-21T03:24:30.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.861+0000 7f3195ffb700 10 client.15423 ms_handle_connect on v2:172.21.15.39:6836/3673846852 2024-05-21T03:24:30.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.862+0000 7f3195ffb700 1 -- 172.21.15.27:0/1861926587 <== mds.3 v2:172.21.15.39:6836/3673846852 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ffaa7cf990 con 0x55ffaa81c9c0 2024-05-21T03:24:30.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.862+0000 7f3195ffb700 10 client.15423 handle_command_reply: tid=0 2024-05-21T03:24:30.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.862+0000 7f31bf1dfb80 1 client.15423 shutdown 2024-05-21T03:24:30.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.862+0000 7f31bf1dfb80 1 -- 172.21.15.27:0/1861926587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffaa807f10 msgr2=0x55ffaa80cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:30.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.862+0000 7f31bf1dfb80 1 --2- 172.21.15.27:0/1861926587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffaa807f10 0x55ffaa80cc30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f31a8009e60 tx=0x7f31a8009e90 comp rx=0 tx=0).stop 2024-05-21T03:24:30.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.862+0000 7f31bf1dfb80 1 -- 172.21.15.27:0/1861926587 shutdown_connections 2024-05-21T03:24:30.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.862+0000 7f31bf1dfb80 1 --2- 172.21.15.27:0/1861926587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffaa807f10 0x55ffaa80cc30 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.862+0000 7f31bf1dfb80 1 --2- 172.21.15.27:0/1861926587 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x55ffaa81c9c0 0x55ffaa81de30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.862+0000 7f31bf1dfb80 1 --2- 172.21.15.27:0/1861926587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffaa803810 0x55ffaa80f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.862+0000 7f31bf1dfb80 1 --2- 172.21.15.27:0/1861926587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffaa8041c0 0x55ffaa80c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.862+0000 7f3195ffb700 0 client.15423 ms_handle_reset on v2:172.21.15.39:6836/3673846852 2024-05-21T03:24:30.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.862+0000 7f31bf1dfb80 1 -- 172.21.15.27:0/1861926587 >> 172.21.15.27:0/1861926587 conn(0x55ffaa731030 msgr2=0x55ffaa7fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:30.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.863+0000 7f31bf1dfb80 1 -- 172.21.15.27:0/1861926587 shutdown_connections 2024-05-21T03:24:30.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.863+0000 7f31bf1dfb80 1 -- 172.21.15.27:0/1861926587 wait complete. 2024-05-21T03:24:30.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.863+0000 7f31bf1dfb80 20 client.15423 trim_cache size 0 max 16384 2024-05-21T03:24:30.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.884+0000 7f31bf1dfb80 1 Processor -- start 2024-05-21T03:24:30.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.884+0000 7f31bf1dfb80 1 -- start start 2024-05-21T03:24:30.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.885+0000 7f31bf1dfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffaa731030 0x55ffaa806f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:30.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.885+0000 7f31bf1dfb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffaa803810 0x55ffaa7fdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:30.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.885+0000 7f31bf1dfb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffaa8041c0 0x55ffaa7fe180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:30.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.885+0000 7f31bf1dfb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ffaa81c170 con 0x55ffaa803810 2024-05-21T03:24:30.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.885+0000 7f31bf1dfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ffaa81c840 con 0x55ffaa731030 2024-05-21T03:24:30.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.885+0000 7f31bf1dfb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ffaa81cf10 con 0x55ffaa8041c0 2024-05-21T03:24:30.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.885+0000 7f31a5d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffaa803810 0x55ffaa7fdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:30.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.885+0000 7f31a5d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffaa803810 0x55ffaa7fdc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34912/0 (socket says 172.21.15.27:34912) 2024-05-21T03:24:30.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.885+0000 7f31a5d9b700 1 -- 172.21.15.27:0/252501869 learned_addr learned my addr 172.21.15.27:0/252501869 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:30.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.885+0000 7f31aed9d700 1 --2- 172.21.15.27:0/252501869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffaa8041c0 0x55ffaa7fe180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:30.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.885+0000 7f31ae59c700 1 --2- 172.21.15.27:0/252501869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffaa731030 0x55ffaa806f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:30.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.885+0000 7f31a5d9b700 1 -- 172.21.15.27:0/252501869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffaa8041c0 msgr2=0x55ffaa7fe180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:30.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.885+0000 7f31a5d9b700 1 --2- 172.21.15.27:0/252501869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffaa8041c0 0x55ffaa7fe180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.885+0000 7f31a5d9b700 1 -- 172.21.15.27:0/252501869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffaa731030 msgr2=0x55ffaa806f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:30.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.885+0000 7f31a5d9b700 1 --2- 172.21.15.27:0/252501869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffaa731030 0x55ffaa806f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.885+0000 7f31a5d9b700 1 -- 172.21.15.27:0/252501869 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffaa7fecf0 con 0x55ffaa803810 2024-05-21T03:24:30.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.885+0000 7f31ae59c700 1 --2- 172.21.15.27:0/252501869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffaa731030 0x55ffaa806f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:24:30.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.885+0000 7f31a5d9b700 1 --2- 172.21.15.27:0/252501869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffaa803810 0x55ffaa7fdc40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f31980a9780 tx=0x7f3198011fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712a35254f6749aa server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:30.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.885+0000 7f3197fff700 1 -- 172.21.15.27:0/252501869 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31980aa180 con 0x55ffaa803810 2024-05-21T03:24:30.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.885+0000 7f3197fff700 1 -- 172.21.15.27:0/252501869 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31980a9a80 con 0x55ffaa803810 2024-05-21T03:24:30.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.885+0000 7f3197fff700 1 -- 172.21.15.27:0/252501869 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31980b26b0 con 0x55ffaa803810 2024-05-21T03:24:30.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.886+0000 7f31bf1dfb80 1 -- 172.21.15.27:0/252501869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffaa803810 msgr2=0x55ffaa7fdc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:30.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.886+0000 7f31bf1dfb80 1 --2- 172.21.15.27:0/252501869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffaa803810 0x55ffaa7fdc40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f31980a9780 tx=0x7f3198011fe0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.886+0000 7f31bf1dfb80 1 -- 172.21.15.27:0/252501869 shutdown_connections 2024-05-21T03:24:30.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.886+0000 7f31bf1dfb80 1 --2- 172.21.15.27:0/252501869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffaa8041c0 0x55ffaa7fe180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.886+0000 7f31bf1dfb80 1 --2- 172.21.15.27:0/252501869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffaa731030 0x55ffaa806f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.886+0000 7f31bf1dfb80 1 --2- 172.21.15.27:0/252501869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffaa803810 0x55ffaa7fdc40 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.886+0000 7f31bf1dfb80 1 -- 172.21.15.27:0/252501869 >> 172.21.15.27:0/252501869 conn(0x55ffaa7faab0 msgr2=0x55ffaa806860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:30.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.886+0000 7f31bf1dfb80 1 -- 172.21.15.27:0/252501869 shutdown_connections 2024-05-21T03:24:30.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.886+0000 7f31bf1dfb80 1 -- 172.21.15.27:0/252501869 wait complete. 2024-05-21T03:24:30.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.886+0000 7f31bf1dfb80 1 -- start start 2024-05-21T03:24:30.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.886+0000 7f31bf1dfb80 1 Processor -- start 2024-05-21T03:24:30.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.887+0000 7f31bf1dfb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffaa731030 0x55ffaa880440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:30.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.887+0000 7f31bf1dfb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffaa803810 0x55ffaa880980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:30.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.887+0000 7f31bf1dfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffaa8041c0 0x55ffaa8238a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:30.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.887+0000 7f31bf1dfb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ffaa81dcb0 con 0x55ffaa731030 2024-05-21T03:24:30.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.887+0000 7f31bf1dfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ffaa81d5e0 con 0x55ffaa8041c0 2024-05-21T03:24:30.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.887+0000 7f31bf1dfb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ffaa81e380 con 0x55ffaa803810 2024-05-21T03:24:30.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.887+0000 7f31bf1dfb80 10 client.0 _ll_register_callbacks cb 0x55ffaa6ed8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:24:30.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.887+0000 7f31ae59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffaa731030 0x55ffaa880440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:30.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.887+0000 7f31ae59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffaa731030 0x55ffaa880440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34922/0 (socket says 172.21.15.27:34922) 2024-05-21T03:24:30.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.887+0000 7f31ae59c700 1 -- 172.21.15.27:0/2797240189 learned_addr learned my addr 172.21.15.27:0/2797240189 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:30.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.887+0000 7f31a5d9b700 1 --2- 172.21.15.27:0/2797240189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffaa803810 0x55ffaa880980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:30.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.887+0000 7f31aed9d700 1 --2- 172.21.15.27:0/2797240189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffaa8041c0 0x55ffaa8238a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:30.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.887+0000 7f31a5d9b700 1 -- 172.21.15.27:0/2797240189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffaa8041c0 msgr2=0x55ffaa8238a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:30.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.887+0000 7f31a5d9b700 1 --2- 172.21.15.27:0/2797240189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffaa8041c0 0x55ffaa8238a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.887+0000 7f31a5d9b700 1 -- 172.21.15.27:0/2797240189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffaa731030 msgr2=0x55ffaa880440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:30.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.887+0000 7f31a5d9b700 1 --2- 172.21.15.27:0/2797240189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffaa731030 0x55ffaa880440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.887+0000 7f31a5d9b700 1 -- 172.21.15.27:0/2797240189 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffaa8240e0 con 0x55ffaa803810 2024-05-21T03:24:30.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.887+0000 7f31ae59c700 1 --2- 172.21.15.27:0/2797240189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffaa731030 0x55ffaa880440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:24:30.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.887+0000 7f31a5d9b700 1 --2- 172.21.15.27:0/2797240189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffaa803810 0x55ffaa880980 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x55ffaa81ea20 tx=0x7f31980b2680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1f4e24180945349 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:30.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.887+0000 7f3195ffb700 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-05-21T03:24:30.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.888+0000 7f3195ffb700 1 -- 172.21.15.27:0/2797240189 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31980b28e0 con 0x55ffaa803810 2024-05-21T03:24:30.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.888+0000 7f31bf1dfb80 1 -- 172.21.15.27:0/2797240189 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31980a9a80 con 0x55ffaa803810 2024-05-21T03:24:30.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.888+0000 7f3195ffb700 1 -- 172.21.15.27:0/2797240189 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f31980cc2c0 con 0x55ffaa803810 2024-05-21T03:24:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.888+0000 7f3195ffb700 1 -- 172.21.15.27:0/2797240189 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f31980aa180 con 0x55ffaa803810 2024-05-21T03:24:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.888+0000 7f31bf1dfb80 10 client.25066 fetch_fsmap learned FSMap version 29 2024-05-21T03:24:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.888+0000 7f31bf1dfb80 1 -- 172.21.15.27:0/2797240189 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ffaa888d00 con 0x55ffaa803810 2024-05-21T03:24:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.888+0000 7f3195ffb700 1 -- 172.21.15.27:0/2797240189 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31980a9a80 con 0x55ffaa803810 2024-05-21T03:24:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.888+0000 7f3195ffb700 1 -- 172.21.15.27:0/2797240189 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f31980108b0 con 0x55ffaa803810 2024-05-21T03:24:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.888+0000 7f31bf1dfb80 10 client.25066 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T03:24:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.888+0000 7f31bf1dfb80 10 client.25066 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-21T03:24:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.888+0000 7f31bf1dfb80 20 client.25066 populate_metadata read hostname 'smithi027' 2024-05-21T03:24:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.888+0000 7f31bf1dfb80 1 --2- 172.21.15.27:0/2797240189 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x55ffaa807f10 0x55ffaa88c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.888+0000 7f31bf1dfb80 4 client.25066 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T03:24:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.888+0000 7f31bf1dfb80 1 -- 172.21.15.27:0/2797240189 --> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] -- command(tid 0: {"prefix": "status"}) -- 0x55ffaa764b90 con 0x55ffaa807f10 2024-05-21T03:24:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.889+0000 7f31ae59c700 1 --2- 172.21.15.27:0/2797240189 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x55ffaa807f10 0x55ffaa88c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.889+0000 7f31ae59c700 1 --2- 172.21.15.27:0/2797240189 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x55ffaa807f10 0x55ffaa88c180 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=19184ddb30de2f58 server_cookie=12fd836337843a65 in_seq=0 out_seq=0 2024-05-21T03:24:30.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.889+0000 7f3195ffb700 10 client.25066 ms_handle_connect on v2:172.21.15.39:6836/3673846852 2024-05-21T03:24:30.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.889+0000 7f3195ffb700 1 -- 172.21.15.27:0/2797240189 <== mds.3 v2:172.21.15.39:6836/3673846852 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ffaa764b90 con 0x55ffaa807f10 2024-05-21T03:24:30.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.889+0000 7f3195ffb700 10 client.25066 handle_command_reply: tid=0 2024-05-21T03:24:30.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.889+0000 7f31bf1dfb80 1 client.25066 shutdown 2024-05-21T03:24:30.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.889+0000 7f31bf1dfb80 1 -- 172.21.15.27:0/2797240189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffaa803810 msgr2=0x55ffaa880980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:30.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.889+0000 7f31bf1dfb80 1 --2- 172.21.15.27:0/2797240189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffaa803810 0x55ffaa880980 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x55ffaa81ea20 tx=0x7f31980b2680 comp rx=0 tx=0).stop 2024-05-21T03:24:30.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.889+0000 7f31bf1dfb80 1 -- 172.21.15.27:0/2797240189 shutdown_connections 2024-05-21T03:24:30.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.889+0000 7f31bf1dfb80 1 --2- 172.21.15.27:0/2797240189 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x55ffaa807f10 0x55ffaa88c180 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.889+0000 7f31bf1dfb80 1 --2- 172.21.15.27:0/2797240189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffaa8041c0 0x55ffaa8238a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.889+0000 7f31bf1dfb80 1 --2- 172.21.15.27:0/2797240189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ffaa731030 0x55ffaa880440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.889+0000 7f31bf1dfb80 1 --2- 172.21.15.27:0/2797240189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffaa803810 0x55ffaa880980 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.889+0000 7f31bf1dfb80 1 -- 172.21.15.27:0/2797240189 >> 172.21.15.27:0/2797240189 conn(0x55ffaa7faab0 msgr2=0x55ffaa8070f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:30.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.889+0000 7f3195ffb700 0 client.25066 ms_handle_reset on v2:172.21.15.39:6836/3673846852 2024-05-21T03:24:30.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.889+0000 7f31bf1dfb80 1 -- 172.21.15.27:0/2797240189 shutdown_connections 2024-05-21T03:24:30.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.889+0000 7f31bf1dfb80 1 -- 172.21.15.27:0/2797240189 wait complete. 2024-05-21T03:24:30.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.890+0000 7f31bf1dfb80 20 client.25066 trim_cache size 0 max 16384 2024-05-21T03:24:30.896 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:24:30.896 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "c9ca94be-171f-11ef-bc9a-c7b262605968", 2024-05-21T03:24:30.896 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 3, 2024-05-21T03:24:30.896 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24452, 2024-05-21T03:24:30.896 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-05-21T03:24:30.896 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-05-21T03:24:30.896 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-05-21T03:24:30.896 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 222.44551544699999, 2024-05-21T03:24:30.897 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 29, 2024-05-21T03:24:30.897 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 83, 2024-05-21T03:24:30.897 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 82, 2024-05-21T03:24:30.897 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 263.64302700299999 2024-05-21T03:24:30.897 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:24:30.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.890+0000 7f31b4949700 1 -- 172.21.15.27:0/2191183951 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f319c0b2a80 msgr2=0x7f319c0b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:30.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.890+0000 7f31b4949700 1 --2- 172.21.15.27:0/2191183951 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f319c0b2a80 0x7f319c0b4f40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f31a0003760 tx=0x7f31a0005fb0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.890+0000 7f31b4949700 1 -- 172.21.15.27:0/2191183951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31b01041a0 msgr2=0x7f31b0075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:30.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.890+0000 7f31b4949700 1 --2- 172.21.15.27:0/2191183951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31b01041a0 0x7f31b0075d80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f319800e9a0 tx=0x7f319800ecb0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.890+0000 7f31b4949700 1 -- 172.21.15.27:0/2191183951 shutdown_connections 2024-05-21T03:24:30.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.890+0000 7f31b4949700 1 --2- 172.21.15.27:0/2191183951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31b01088f0 0x7f31b00762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.890+0000 7f31b4949700 1 --2- 172.21.15.27:0/2191183951 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f319c0b2a80 0x7f319c0b4f40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.890+0000 7f31b4949700 1 --2- 172.21.15.27:0/2191183951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31b01037f0 0x7f31b0075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.890+0000 7f31b4949700 1 --2- 172.21.15.27:0/2191183951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31b01041a0 0x7f31b0075d80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:30.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.890+0000 7f31b4949700 1 -- 172.21.15.27:0/2191183951 >> 172.21.15.27:0/2191183951 conn(0x7f31b007a6a0 msgr2=0x7f31b0101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:30.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.890+0000 7f31b4949700 1 -- 172.21.15.27:0/2191183951 shutdown_connections 2024-05-21T03:24:30.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:30.890+0000 7f31b4949700 1 -- 172.21.15.27:0/2191183951 wait complete. 2024-05-21T03:24:31.690 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell mds.c status 2024-05-21T03:24:31.948 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:24:32.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:32 smithi027 ceph-mon[32179]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-21T03:24:32.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:32 smithi039 ceph-mon[36429]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-21T03:24:32.772 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:32 smithi165 ceph-mon[36269]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-21T03:24:33.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.436+0000 7fe5f42af700 1 -- 172.21.15.27:0/2912511547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5ec1039d0 msgr2=0x7fe5ec10f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:33.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.436+0000 7fe5f42af700 1 --2- 172.21.15.27:0/2912511547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5ec1039d0 0x7fe5ec10f3e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400b0d0 tx=0x7fe5e400b3e0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.437+0000 7fe5f42af700 1 -- 172.21.15.27:0/2912511547 shutdown_connections 2024-05-21T03:24:33.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.437+0000 7fe5f42af700 1 --2- 172.21.15.27:0/2912511547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5ec1039d0 0x7fe5ec10f3e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.437+0000 7fe5f42af700 1 --2- 172.21.15.27:0/2912511547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5ec108730 0x7fe5ec108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.437+0000 7fe5f42af700 1 --2- 172.21.15.27:0/2912511547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5ec103010 0x7fe5ec103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.437+0000 7fe5f42af700 1 -- 172.21.15.27:0/2912511547 >> 172.21.15.27:0/2912511547 conn(0x7fe5ec077780 msgr2=0x7fe5ec077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:33.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.437+0000 7fe5f42af700 1 -- 172.21.15.27:0/2912511547 shutdown_connections 2024-05-21T03:24:33.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.437+0000 7fe5f42af700 1 -- 172.21.15.27:0/2912511547 wait complete. 2024-05-21T03:24:33.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.437+0000 7fe5f42af700 1 Processor -- start 2024-05-21T03:24:33.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.437+0000 7fe5f42af700 1 -- start start 2024-05-21T03:24:33.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.437+0000 7fe5f42af700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5ec103010 0x7fe5ec19f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:33.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.438+0000 7fe5f42af700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5ec1039d0 0x7fe5ec19fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:33.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.438+0000 7fe5f42af700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5ec108730 0x7fe5ec199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:33.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.438+0000 7fe5f42af700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ec1127b0 con 0x7fe5ec1039d0 2024-05-21T03:24:33.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.438+0000 7fe5f42af700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ec112630 con 0x7fe5ec103010 2024-05-21T03:24:33.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.438+0000 7fe5f42af700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ec112930 con 0x7fe5ec108730 2024-05-21T03:24:33.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.438+0000 7fe5f184a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5ec1039d0 0x7fe5ec19fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:33.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.438+0000 7fe5f184a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5ec1039d0 0x7fe5ec19fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47486/0 (socket says 172.21.15.27:47486) 2024-05-21T03:24:33.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.438+0000 7fe5f184a700 1 -- 172.21.15.27:0/3341551382 learned_addr learned my addr 172.21.15.27:0/3341551382 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:33.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.438+0000 7fe5f204b700 1 --2- 172.21.15.27:0/3341551382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5ec103010 0x7fe5ec19f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:33.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.438+0000 7fe5f284c700 1 --2- 172.21.15.27:0/3341551382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5ec108730 0x7fe5ec199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:33.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.438+0000 7fe5f184a700 1 -- 172.21.15.27:0/3341551382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5ec108730 msgr2=0x7fe5ec199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:33.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.438+0000 7fe5f184a700 1 --2- 172.21.15.27:0/3341551382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5ec108730 0x7fe5ec199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.438+0000 7fe5f184a700 1 -- 172.21.15.27:0/3341551382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5ec103010 msgr2=0x7fe5ec19f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:33.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.438+0000 7fe5f184a700 1 --2- 172.21.15.27:0/3341551382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5ec103010 0x7fe5ec19f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.438+0000 7fe5f184a700 1 -- 172.21.15.27:0/3341551382 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5ec19a200 con 0x7fe5ec1039d0 2024-05-21T03:24:33.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.438+0000 7fe5f204b700 1 --2- 172.21.15.27:0/3341551382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5ec103010 0x7fe5ec19f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:24:33.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.438+0000 7fe5f184a700 1 --2- 172.21.15.27:0/3341551382 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5ec1039d0 0x7fe5ec19fe20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00d980 tx=0x7fe5dc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:33.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.438+0000 7fe5e37fe700 1 -- 172.21.15.27:0/3341551382 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc016070 con 0x7fe5ec1039d0 2024-05-21T03:24:33.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.439+0000 7fe5e37fe700 1 -- 172.21.15.27:0/3341551382 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5dc004830 con 0x7fe5ec1039d0 2024-05-21T03:24:33.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.439+0000 7fe5e37fe700 1 -- 172.21.15.27:0/3341551382 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc0054c0 con 0x7fe5ec1039d0 2024-05-21T03:24:33.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.439+0000 7fe5f42af700 1 -- 172.21.15.27:0/3341551382 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5ec19a4f0 con 0x7fe5ec1039d0 2024-05-21T03:24:33.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.439+0000 7fe5f42af700 1 -- 172.21.15.27:0/3341551382 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5ec1a67b0 con 0x7fe5ec1039d0 2024-05-21T03:24:33.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.442+0000 7fe5e37fe700 1 -- 172.21.15.27:0/3341551382 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fe5dc004220 con 0x7fe5ec1039d0 2024-05-21T03:24:33.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.443+0000 7fe5e37fe700 1 --2- 172.21.15.27:0/3341551382 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fe5d80b2a80 0x7fe5d80b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:33.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.443+0000 7fe5f204b700 1 --2- 172.21.15.27:0/3341551382 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fe5d80b2a80 0x7fe5d80b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:33.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.443+0000 7fe5e37fe700 1 -- 172.21.15.27:0/3341551382 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fe5dc0e2690 con 0x7fe5ec1039d0 2024-05-21T03:24:33.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.443+0000 7fe5feb45b80 1 Processor -- start 2024-05-21T03:24:33.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.443+0000 7fe5feb45b80 1 -- start start 2024-05-21T03:24:33.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.443+0000 7fe5f204b700 1 --2- 172.21.15.27:0/3341551382 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fe5d80b2a80 0x7fe5d80b4f40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec19abe0 tx=0x7fe5e800b3c0 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:33.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.443+0000 7fe5feb45b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5569ba684c20 0x5569ba685000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:33.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.443+0000 7fe5feb45b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5569ba6855d0 0x5569ba688d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:33.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.443+0000 7fe5feb45b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5569ba689320 0x5569ba689760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:33.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.443+0000 7fe5feb45b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5569ba5dfc40 con 0x5569ba6855d0 2024-05-21T03:24:33.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.443+0000 7fe5feb45b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5569ba5cf7e0 con 0x5569ba684c20 2024-05-21T03:24:33.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.443+0000 7fe5feb45b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5569ba5d3820 con 0x5569ba689320 2024-05-21T03:24:33.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.444+0000 7fe5f184a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5569ba6855d0 0x5569ba688d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:33.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.444+0000 7fe5f184a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5569ba6855d0 0x5569ba688d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47500/0 (socket says 172.21.15.27:47500) 2024-05-21T03:24:33.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.444+0000 7fe5f184a700 1 -- 172.21.15.27:0/3748887750 learned_addr learned my addr 172.21.15.27:0/3748887750 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:33.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.444+0000 7fe5f204b700 1 --2- 172.21.15.27:0/3748887750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5569ba684c20 0x5569ba685000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:33.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.444+0000 7fe5f284c700 1 --2- 172.21.15.27:0/3748887750 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5569ba689320 0x5569ba689760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:33.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.444+0000 7fe5f184a700 1 -- 172.21.15.27:0/3748887750 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5569ba689320 msgr2=0x5569ba689760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:33.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.444+0000 7fe5f184a700 1 --2- 172.21.15.27:0/3748887750 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5569ba689320 0x5569ba689760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.444+0000 7fe5f184a700 1 -- 172.21.15.27:0/3748887750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5569ba684c20 msgr2=0x5569ba685000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:33.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.444+0000 7fe5f184a700 1 --2- 172.21.15.27:0/3748887750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5569ba684c20 0x5569ba685000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.444+0000 7fe5f184a700 1 -- 172.21.15.27:0/3748887750 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569ba51f880 con 0x5569ba6855d0 2024-05-21T03:24:33.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.444+0000 7fe5f184a700 1 --2- 172.21.15.27:0/3748887750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5569ba6855d0 0x5569ba688d00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00b540 tx=0x7fe5dc0a9310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c02b15e99c2321 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:33.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.444+0000 7fe5e0d70700 1 -- 172.21.15.27:0/3748887750 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc0adbb0 con 0x5569ba6855d0 2024-05-21T03:24:33.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.444+0000 7fe5e0d70700 1 -- 172.21.15.27:0/3748887750 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5dc0b2660 con 0x5569ba6855d0 2024-05-21T03:24:33.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.444+0000 7fe5e0d70700 1 -- 172.21.15.27:0/3748887750 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc0dffd0 con 0x5569ba6855d0 2024-05-21T03:24:33.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.445+0000 7fe5feb45b80 1 -- 172.21.15.27:0/3748887750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5569ba6855d0 msgr2=0x5569ba688d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:33.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.445+0000 7fe5feb45b80 1 --2- 172.21.15.27:0/3748887750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5569ba6855d0 0x5569ba688d00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00b540 tx=0x7fe5dc0a9310 comp rx=0 tx=0).stop 2024-05-21T03:24:33.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.445+0000 7fe5feb45b80 1 -- 172.21.15.27:0/3748887750 shutdown_connections 2024-05-21T03:24:33.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.445+0000 7fe5feb45b80 1 --2- 172.21.15.27:0/3748887750 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5569ba689320 0x5569ba689760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.445+0000 7fe5feb45b80 1 --2- 172.21.15.27:0/3748887750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5569ba684c20 0x5569ba685000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.445+0000 7fe5feb45b80 1 --2- 172.21.15.27:0/3748887750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5569ba6855d0 0x5569ba688d00 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.445+0000 7fe5feb45b80 1 -- 172.21.15.27:0/3748887750 >> 172.21.15.27:0/3748887750 conn(0x5569ba5ae030 msgr2=0x5569ba67ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:33.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.445+0000 7fe5feb45b80 1 -- 172.21.15.27:0/3748887750 shutdown_connections 2024-05-21T03:24:33.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.445+0000 7fe5feb45b80 1 -- 172.21.15.27:0/3748887750 wait complete. 2024-05-21T03:24:33.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.446+0000 7fe5feb45b80 1 -- start start 2024-05-21T03:24:33.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.446+0000 7fe5feb45b80 1 Processor -- start 2024-05-21T03:24:33.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.446+0000 7fe5feb45b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5569ba684c20 0x5569ba6908b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:33.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.446+0000 7fe5feb45b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5569ba6855d0 0x5569ba68dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:33.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.446+0000 7fe5feb45b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5569ba689320 0x5569ba68e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:33.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.446+0000 7fe5feb45b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5569ba4d5f50 con 0x5569ba684c20 2024-05-21T03:24:33.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.446+0000 7fe5feb45b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5569ba571770 con 0x5569ba689320 2024-05-21T03:24:33.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.446+0000 7fe5feb45b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5569ba6914e0 con 0x5569ba6855d0 2024-05-21T03:24:33.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.446+0000 7fe5feb45b80 10 client.0 _ll_register_callbacks cb 0x5569ba56a8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:24:33.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.446+0000 7fe5f204b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5569ba684c20 0x5569ba6908b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:33.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.446+0000 7fe5f204b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5569ba684c20 0x5569ba6908b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47504/0 (socket says 172.21.15.27:47504) 2024-05-21T03:24:33.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.446+0000 7fe5f204b700 1 -- 172.21.15.27:0/4080443633 learned_addr learned my addr 172.21.15.27:0/4080443633 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:33.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.446+0000 7fe5f184a700 1 --2- 172.21.15.27:0/4080443633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5569ba6855d0 0x5569ba68dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:33.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.446+0000 7fe5f284c700 1 --2- 172.21.15.27:0/4080443633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5569ba689320 0x5569ba68e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:33.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.446+0000 7fe5f204b700 1 -- 172.21.15.27:0/4080443633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5569ba6855d0 msgr2=0x5569ba68dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:33.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.446+0000 7fe5f204b700 1 --2- 172.21.15.27:0/4080443633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5569ba6855d0 0x5569ba68dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.446+0000 7fe5f204b700 1 -- 172.21.15.27:0/4080443633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5569ba689320 msgr2=0x5569ba68e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:33.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.446+0000 7fe5f204b700 1 --2- 172.21.15.27:0/4080443633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5569ba689320 0x5569ba68e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.446+0000 7fe5f204b700 1 -- 172.21.15.27:0/4080443633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569ba56e640 con 0x5569ba684c20 2024-05-21T03:24:33.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.446+0000 7fe5f284c700 1 --2- 172.21.15.27:0/4080443633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5569ba689320 0x5569ba68e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:24:33.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.447+0000 7fe5f204b700 1 --2- 172.21.15.27:0/4080443633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5569ba684c20 0x5569ba6908b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe5e80119c0 tx=0x7fe5e80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287215f112355773 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:33.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.447+0000 7fe5d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:24:33.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.447+0000 7fe5d5ffb700 1 -- 172.21.15.27:0/4080443633 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e801f840 con 0x5569ba684c20 2024-05-21T03:24:33.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.447+0000 7fe5d5ffb700 1 -- 172.21.15.27:0/4080443633 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fe5e801f9e0 con 0x5569ba684c20 2024-05-21T03:24:33.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.447+0000 7fe5feb45b80 1 -- 172.21.15.27:0/4080443633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5dc0b2660 con 0x5569ba684c20 2024-05-21T03:24:33.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.447+0000 7fe5d5ffb700 1 -- 172.21.15.27:0/4080443633 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe5e801fbe0 con 0x5569ba684c20 2024-05-21T03:24:33.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.447+0000 7fe5feb45b80 10 client.15441 fetch_fsmap learned FSMap version 29 2024-05-21T03:24:33.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.447+0000 7fe5feb45b80 1 -- 172.21.15.27:0/4080443633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5569ba69bde0 con 0x5569ba684c20 2024-05-21T03:24:33.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.447+0000 7fe5d5ffb700 1 -- 172.21.15.27:0/4080443633 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5dc0b2660 con 0x5569ba684c20 2024-05-21T03:24:33.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.447+0000 7fe5d5ffb700 1 -- 172.21.15.27:0/4080443633 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe5e8028470 con 0x5569ba684c20 2024-05-21T03:24:33.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.448+0000 7fe5feb45b80 10 client.15441 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T03:24:33.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.448+0000 7fe5feb45b80 10 client.15441 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-21T03:24:33.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.448+0000 7fe5feb45b80 20 client.15441 populate_metadata read hostname 'smithi027' 2024-05-21T03:24:33.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.448+0000 7fe5feb45b80 1 --2- 172.21.15.27:0/4080443633 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x5569ba69e3a0 0x5569ba69f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:33.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.448+0000 7fe5feb45b80 4 client.15441 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:24:33.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.448+0000 7fe5feb45b80 1 -- 172.21.15.27:0/4080443633 --> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569ba64c990 con 0x5569ba69e3a0 2024-05-21T03:24:33.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.448+0000 7fe5f184a700 1 --2- 172.21.15.27:0/4080443633 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x5569ba69e3a0 0x5569ba69f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:33.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.448+0000 7fe5d5ffb700 10 client.15441 ms_handle_connect on v2:172.21.15.165:6836/691356139 2024-05-21T03:24:33.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.448+0000 7fe5f184a700 1 --2- 172.21.15.27:0/4080443633 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x5569ba69e3a0 0x5569ba69f810 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=107e349659648993 server_cookie=51e2d103bc86d1c0 in_seq=0 out_seq=0 2024-05-21T03:24:33.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.449+0000 7fe5d5ffb700 1 -- 172.21.15.27:0/4080443633 <== mds.2 v2:172.21.15.165:6836/691356139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5569ba64c990 con 0x5569ba69e3a0 2024-05-21T03:24:33.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.449+0000 7fe5d5ffb700 10 client.15441 handle_command_reply: tid=0 2024-05-21T03:24:33.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.449+0000 7fe5feb45b80 1 client.15441 shutdown 2024-05-21T03:24:33.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.449+0000 7fe5feb45b80 1 -- 172.21.15.27:0/4080443633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5569ba684c20 msgr2=0x5569ba6908b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:33.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.449+0000 7fe5feb45b80 1 --2- 172.21.15.27:0/4080443633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5569ba684c20 0x5569ba6908b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe5e80119c0 tx=0x7fe5e80047c0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.449+0000 7fe5feb45b80 1 -- 172.21.15.27:0/4080443633 shutdown_connections 2024-05-21T03:24:33.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.449+0000 7fe5feb45b80 1 --2- 172.21.15.27:0/4080443633 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x5569ba69e3a0 0x5569ba69f810 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.449+0000 7fe5feb45b80 1 --2- 172.21.15.27:0/4080443633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5569ba689320 0x5569ba68e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.449+0000 7fe5feb45b80 1 --2- 172.21.15.27:0/4080443633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5569ba684c20 0x5569ba6908b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.449+0000 7fe5feb45b80 1 --2- 172.21.15.27:0/4080443633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5569ba6855d0 0x5569ba68dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.449+0000 7fe5feb45b80 1 -- 172.21.15.27:0/4080443633 >> 172.21.15.27:0/4080443633 conn(0x5569ba5ae030 msgr2=0x5569ba68a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:33.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.449+0000 7fe5d5ffb700 0 client.15441 ms_handle_reset on v2:172.21.15.165:6836/691356139 2024-05-21T03:24:33.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.450+0000 7fe5feb45b80 1 -- 172.21.15.27:0/4080443633 shutdown_connections 2024-05-21T03:24:33.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.450+0000 7fe5feb45b80 1 -- 172.21.15.27:0/4080443633 wait complete. 2024-05-21T03:24:33.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.450+0000 7fe5feb45b80 20 client.15441 trim_cache size 0 max 16384 2024-05-21T03:24:33.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.471+0000 7fe5feb45b80 1 Processor -- start 2024-05-21T03:24:33.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.471+0000 7fe5feb45b80 1 -- start start 2024-05-21T03:24:33.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.471+0000 7fe5feb45b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5569ba5ae030 0x5569ba68faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:33.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.471+0000 7fe5feb45b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5569ba684c20 0x5569ba67ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:33.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.471+0000 7fe5feb45b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5569ba6855d0 0x5569ba686c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:33.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.471+0000 7fe5feb45b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5569ba69db50 con 0x5569ba684c20 2024-05-21T03:24:33.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.471+0000 7fe5feb45b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5569ba69e220 con 0x5569ba6855d0 2024-05-21T03:24:33.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.471+0000 7fe5feb45b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5569ba69e8f0 con 0x5569ba5ae030 2024-05-21T03:24:33.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.472+0000 7fe5f184a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5569ba684c20 0x5569ba67ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:33.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.472+0000 7fe5f184a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5569ba684c20 0x5569ba67ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47506/0 (socket says 172.21.15.27:47506) 2024-05-21T03:24:33.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.472+0000 7fe5f184a700 1 -- 172.21.15.27:0/2076175034 learned_addr learned my addr 172.21.15.27:0/2076175034 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:33.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.472+0000 7fe5f204b700 1 --2- 172.21.15.27:0/2076175034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5569ba5ae030 0x5569ba68faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:33.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.472+0000 7fe5f284c700 1 --2- 172.21.15.27:0/2076175034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5569ba6855d0 0x5569ba686c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:33.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.472+0000 7fe5f184a700 1 -- 172.21.15.27:0/2076175034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5569ba5ae030 msgr2=0x5569ba68faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:33.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.472+0000 7fe5f184a700 1 --2- 172.21.15.27:0/2076175034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5569ba5ae030 0x5569ba68faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.472+0000 7fe5f184a700 1 -- 172.21.15.27:0/2076175034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5569ba6855d0 msgr2=0x5569ba686c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:33.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.472+0000 7fe5f184a700 1 --2- 172.21.15.27:0/2076175034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5569ba6855d0 0x5569ba686c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.472+0000 7fe5f184a700 1 -- 172.21.15.27:0/2076175034 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569ba68f110 con 0x5569ba684c20 2024-05-21T03:24:33.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.472+0000 7fe5f204b700 1 --2- 172.21.15.27:0/2076175034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5569ba5ae030 0x5569ba68faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:24:33.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.472+0000 7fe5f284c700 1 --2- 172.21.15.27:0/2076175034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5569ba6855d0 0x5569ba686c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:24:33.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.472+0000 7fe5f184a700 1 --2- 172.21.15.27:0/2076175034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5569ba684c20 0x5569ba67ac40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc0cc2d0 tx=0x7fe5dc091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e88299180a2e01f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:33.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.472+0000 7fe5d7fff700 1 -- 172.21.15.27:0/2076175034 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc005660 con 0x5569ba684c20 2024-05-21T03:24:33.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.472+0000 7fe5d7fff700 1 -- 172.21.15.27:0/2076175034 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5dc004ce0 con 0x5569ba684c20 2024-05-21T03:24:33.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.472+0000 7fe5d7fff700 1 -- 172.21.15.27:0/2076175034 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc0a9a60 con 0x5569ba684c20 2024-05-21T03:24:33.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.473+0000 7fe5feb45b80 1 -- 172.21.15.27:0/2076175034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5569ba684c20 msgr2=0x5569ba67ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:33.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.473+0000 7fe5feb45b80 1 --2- 172.21.15.27:0/2076175034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5569ba684c20 0x5569ba67ac40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc0cc2d0 tx=0x7fe5dc091040 comp rx=0 tx=0).stop 2024-05-21T03:24:33.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.473+0000 7fe5feb45b80 1 -- 172.21.15.27:0/2076175034 shutdown_connections 2024-05-21T03:24:33.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.473+0000 7fe5feb45b80 1 --2- 172.21.15.27:0/2076175034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5569ba6855d0 0x5569ba686c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.473+0000 7fe5feb45b80 1 --2- 172.21.15.27:0/2076175034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5569ba5ae030 0x5569ba68faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.473+0000 7fe5feb45b80 1 --2- 172.21.15.27:0/2076175034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5569ba684c20 0x5569ba67ac40 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.473+0000 7fe5feb45b80 1 -- 172.21.15.27:0/2076175034 >> 172.21.15.27:0/2076175034 conn(0x5569ba677ab0 msgr2=0x5569ba68f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.473+0000 7fe5feb45b80 1 -- 172.21.15.27:0/2076175034 shutdown_connections 2024-05-21T03:24:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.473+0000 7fe5feb45b80 1 -- 172.21.15.27:0/2076175034 wait complete. 2024-05-21T03:24:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.473+0000 7fe5feb45b80 1 -- start start 2024-05-21T03:24:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.474+0000 7fe5feb45b80 1 Processor -- start 2024-05-21T03:24:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.474+0000 7fe5feb45b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5569ba5ae030 0x5569ba6a31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.474+0000 7fe5feb45b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5569ba684c20 0x5569ba6a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.474+0000 7fe5feb45b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5569ba6855d0 0x5569ba6a77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.474+0000 7fe5feb45b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5569ba69f690 con 0x5569ba6855d0 2024-05-21T03:24:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.474+0000 7fe5feb45b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5569ba69efc0 con 0x5569ba5ae030 2024-05-21T03:24:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.474+0000 7fe5feb45b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5569ba69fd60 con 0x5569ba684c20 2024-05-21T03:24:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.474+0000 7fe5feb45b80 10 client.0 _ll_register_callbacks cb 0x5569ba56a8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:24:33.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.474+0000 7fe5f284c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5569ba6855d0 0x5569ba6a77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:33.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.474+0000 7fe5f204b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5569ba5ae030 0x5569ba6a31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:33.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.474+0000 7fe5f284c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5569ba6855d0 0x5569ba6a77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47520/0 (socket says 172.21.15.27:47520) 2024-05-21T03:24:33.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.474+0000 7fe5f284c700 1 -- 172.21.15.27:0/4268431754 learned_addr learned my addr 172.21.15.27:0/4268431754 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:33.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.474+0000 7fe5f184a700 1 --2- 172.21.15.27:0/4268431754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5569ba684c20 0x5569ba6a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:33.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.474+0000 7fe5f204b700 1 -- 172.21.15.27:0/4268431754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5569ba684c20 msgr2=0x5569ba6a36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:33.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.474+0000 7fe5f204b700 1 --2- 172.21.15.27:0/4268431754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5569ba684c20 0x5569ba6a36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.474+0000 7fe5f204b700 1 -- 172.21.15.27:0/4268431754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5569ba6855d0 msgr2=0x5569ba6a77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:33.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.474+0000 7fe5f204b700 1 --2- 172.21.15.27:0/4268431754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5569ba6855d0 0x5569ba6a77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.474+0000 7fe5f204b700 1 -- 172.21.15.27:0/4268431754 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569ba6a7de0 con 0x5569ba5ae030 2024-05-21T03:24:33.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.474+0000 7fe5f284c700 1 --2- 172.21.15.27:0/4268431754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5569ba6855d0 0x5569ba6a77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:24:33.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.475+0000 7fe5f204b700 1 --2- 172.21.15.27:0/4268431754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5569ba5ae030 0x5569ba6a31a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8028840 tx=0x7fe5e8028ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4585850f6be544e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:33.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.475+0000 7fe5d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-21T03:24:33.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.475+0000 7fe5d5ffb700 1 -- 172.21.15.27:0/4268431754 <== mon.2 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e803e060 con 0x5569ba5ae030 2024-05-21T03:24:33.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.475+0000 7fe5feb45b80 1 -- 172.21.15.27:0/4268431754 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5dc004ce0 con 0x5569ba5ae030 2024-05-21T03:24:33.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.475+0000 7fe5d5ffb700 1 -- 172.21.15.27:0/4268431754 <== mon.2 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fe5e8031480 con 0x5569ba5ae030 2024-05-21T03:24:33.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.475+0000 7fe5d5ffb700 1 -- 172.21.15.27:0/4268431754 <== mon.2 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe5e8031680 con 0x5569ba5ae030 2024-05-21T03:24:33.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.475+0000 7fe5feb45b80 10 client.25079 fetch_fsmap learned FSMap version 29 2024-05-21T03:24:33.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.475+0000 7fe5feb45b80 1 -- 172.21.15.27:0/4268431754 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5569ba70a4e0 con 0x5569ba5ae030 2024-05-21T03:24:33.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.475+0000 7fe5d5ffb700 1 -- 172.21.15.27:0/4268431754 <== mon.2 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5dc004ce0 con 0x5569ba5ae030 2024-05-21T03:24:33.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.475+0000 7fe5d5ffb700 1 -- 172.21.15.27:0/4268431754 <== mon.2 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe5e8042760 con 0x5569ba5ae030 2024-05-21T03:24:33.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.475+0000 7fe5feb45b80 10 client.25079 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T03:24:33.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.475+0000 7fe5feb45b80 10 client.25079 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-21T03:24:33.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.475+0000 7fe5feb45b80 20 client.25079 populate_metadata read hostname 'smithi027' 2024-05-21T03:24:33.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.475+0000 7fe5feb45b80 1 --2- 172.21.15.27:0/4268431754 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x5569ba689320 0x5569ba70ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:33.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.475+0000 7fe5feb45b80 4 client.25079 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T03:24:33.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.475+0000 7fe5feb45b80 1 -- 172.21.15.27:0/4268431754 --> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] -- command(tid 0: {"prefix": "status"}) -- 0x5569ba5e1b90 con 0x5569ba689320 2024-05-21T03:24:33.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.476+0000 7fe5f184a700 1 --2- 172.21.15.27:0/4268431754 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x5569ba689320 0x5569ba70ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:33.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.476+0000 7fe5f184a700 1 --2- 172.21.15.27:0/4268431754 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x5569ba689320 0x5569ba70ddc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fdefef422ece0756 server_cookie=b6dcd3afdc2b09a3 in_seq=0 out_seq=0 2024-05-21T03:24:33.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.476+0000 7fe5d5ffb700 10 client.25079 ms_handle_connect on v2:172.21.15.165:6836/691356139 2024-05-21T03:24:33.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.476+0000 7fe5d5ffb700 1 -- 172.21.15.27:0/4268431754 <== mds.2 v2:172.21.15.165:6836/691356139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5569ba5e1b90 con 0x5569ba689320 2024-05-21T03:24:33.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.476+0000 7fe5d5ffb700 10 client.25079 handle_command_reply: tid=0 2024-05-21T03:24:33.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.476+0000 7fe5feb45b80 1 client.25079 shutdown 2024-05-21T03:24:33.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.476+0000 7fe5feb45b80 1 -- 172.21.15.27:0/4268431754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5569ba5ae030 msgr2=0x5569ba6a31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:33.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.476+0000 7fe5feb45b80 1 --2- 172.21.15.27:0/4268431754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5569ba5ae030 0x5569ba6a31a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8028840 tx=0x7fe5e8028ce0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.476+0000 7fe5feb45b80 1 -- 172.21.15.27:0/4268431754 shutdown_connections 2024-05-21T03:24:33.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.477+0000 7fe5feb45b80 1 --2- 172.21.15.27:0/4268431754 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x5569ba689320 0x5569ba70ddc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.477+0000 7fe5feb45b80 1 --2- 172.21.15.27:0/4268431754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5569ba6855d0 0x5569ba6a77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.477+0000 7fe5feb45b80 1 --2- 172.21.15.27:0/4268431754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5569ba5ae030 0x5569ba6a31a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.477+0000 7fe5d5ffb700 0 client.25079 ms_handle_reset on v2:172.21.15.165:6836/691356139 2024-05-21T03:24:33.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.477+0000 7fe5feb45b80 1 --2- 172.21.15.27:0/4268431754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5569ba684c20 0x5569ba6a36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.477+0000 7fe5feb45b80 1 -- 172.21.15.27:0/4268431754 >> 172.21.15.27:0/4268431754 conn(0x5569ba677ab0 msgr2=0x5569ba67feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:33.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.477+0000 7fe5feb45b80 1 -- 172.21.15.27:0/4268431754 shutdown_connections 2024-05-21T03:24:33.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.477+0000 7fe5feb45b80 1 -- 172.21.15.27:0/4268431754 wait complete. 2024-05-21T03:24:33.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.477+0000 7fe5feb45b80 20 client.25079 trim_cache size 0 max 16384 2024-05-21T03:24:33.483 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:24:33.484 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "c9ca94be-171f-11ef-bc9a-c7b262605968", 2024-05-21T03:24:33.484 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 2, 2024-05-21T03:24:33.484 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24457, 2024-05-21T03:24:33.484 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-05-21T03:24:33.484 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-05-21T03:24:33.484 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-05-21T03:24:33.484 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 226.04049188299999, 2024-05-21T03:24:33.484 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 29, 2024-05-21T03:24:33.484 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 83, 2024-05-21T03:24:33.484 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 82, 2024-05-21T03:24:33.484 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 269.13984218500002 2024-05-21T03:24:33.484 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:24:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.477+0000 7fe5f42af700 1 -- 172.21.15.27:0/3341551382 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fe5d80b2a80 msgr2=0x7fe5d80b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.477+0000 7fe5f42af700 1 --2- 172.21.15.27:0/3341551382 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fe5d80b2a80 0x7fe5d80b4f40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec19abe0 tx=0x7fe5e800b3c0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.477+0000 7fe5f42af700 1 -- 172.21.15.27:0/3341551382 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5ec1039d0 msgr2=0x7fe5ec19fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.477+0000 7fe5f42af700 1 --2- 172.21.15.27:0/3341551382 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5ec1039d0 0x7fe5ec19fe20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00d980 tx=0x7fe5dc00dc90 comp rx=0 tx=0).stop 2024-05-21T03:24:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.477+0000 7fe5f42af700 1 -- 172.21.15.27:0/3341551382 shutdown_connections 2024-05-21T03:24:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.477+0000 7fe5f42af700 1 --2- 172.21.15.27:0/3341551382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5ec108730 0x7fe5ec199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.477+0000 7fe5f42af700 1 --2- 172.21.15.27:0/3341551382 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fe5d80b2a80 0x7fe5d80b4f40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.477+0000 7fe5f42af700 1 --2- 172.21.15.27:0/3341551382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5ec103010 0x7fe5ec19f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.477+0000 7fe5f42af700 1 --2- 172.21.15.27:0/3341551382 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5ec1039d0 0x7fe5ec19fe20 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.477+0000 7fe5f42af700 1 -- 172.21.15.27:0/3341551382 >> 172.21.15.27:0/3341551382 conn(0x7fe5ec077780 msgr2=0x7fe5ec101fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.477+0000 7fe5f42af700 1 -- 172.21.15.27:0/3341551382 shutdown_connections 2024-05-21T03:24:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:33.477+0000 7fe5f42af700 1 -- 172.21.15.27:0/3341551382 wait complete. 2024-05-21T03:24:34.231 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell mds.i status 2024-05-21T03:24:34.485 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:24:34.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:34 smithi039 ceph-mon[36429]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 938 B/s wr, 13 op/s 2024-05-21T03:24:34.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:34 smithi165 ceph-mon[36269]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 938 B/s wr, 13 op/s 2024-05-21T03:24:34.830 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:34 smithi027 ceph-mon[32179]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 938 B/s wr, 13 op/s 2024-05-21T03:24:36.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.341+0000 7f5f0979d700 1 -- 172.21.15.27:0/788003004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f0406bab0 msgr2=0x7f5f0406bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:36.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.341+0000 7f5f0979d700 1 --2- 172.21.15.27:0/788003004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f0406bab0 0x7f5f0406bf30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8009990 tx=0x7f5ef8009ca0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.341+0000 7f5f0979d700 1 -- 172.21.15.27:0/788003004 shutdown_connections 2024-05-21T03:24:36.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.341+0000 7f5f0979d700 1 --2- 172.21.15.27:0/788003004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f0406c470 0x7f5f041103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.341+0000 7f5f0979d700 1 --2- 172.21.15.27:0/788003004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f041095a0 0x7f5f04109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.341+0000 7f5f0979d700 1 --2- 172.21.15.27:0/788003004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f0406bab0 0x7f5f0406bf30 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.341+0000 7f5f0979d700 1 -- 172.21.15.27:0/788003004 >> 172.21.15.27:0/788003004 conn(0x7f5f040793b0 msgr2=0x7f5f040797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:36.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.341+0000 7f5f0979d700 1 -- 172.21.15.27:0/788003004 shutdown_connections 2024-05-21T03:24:36.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.341+0000 7f5f0979d700 1 -- 172.21.15.27:0/788003004 wait complete. 2024-05-21T03:24:36.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.342+0000 7f5f0979d700 1 Processor -- start 2024-05-21T03:24:36.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.342+0000 7f5f0979d700 1 -- start start 2024-05-21T03:24:36.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.342+0000 7f5f0979d700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f0406bab0 0x7f5f04102500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:36.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.342+0000 7f5f0979d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f0406c470 0x7f5f04102a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:36.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.342+0000 7f5f0979d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f041095a0 0x7f5f04102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:36.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.342+0000 7f5f0979d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5f041136b0 con 0x7f5f0406c470 2024-05-21T03:24:36.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.342+0000 7f5f0979d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5f04113530 con 0x7f5f041095a0 2024-05-21T03:24:36.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.342+0000 7f5f0979d700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5f04113830 con 0x7f5f0406bab0 2024-05-21T03:24:36.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.343+0000 7f5f027fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f0406c470 0x7f5f04102a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:36.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.343+0000 7f5f027fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f0406c470 0x7f5f04102a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47540/0 (socket says 172.21.15.27:47540) 2024-05-21T03:24:36.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.343+0000 7f5f027fc700 1 -- 172.21.15.27:0/2516895584 learned_addr learned my addr 172.21.15.27:0/2516895584 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:36.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.343+0000 7f5f02ffd700 1 --2- 172.21.15.27:0/2516895584 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f0406bab0 0x7f5f04102500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:36.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.343+0000 7f5f037fe700 1 --2- 172.21.15.27:0/2516895584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f041095a0 0x7f5f04102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:36.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.343+0000 7f5f027fc700 1 -- 172.21.15.27:0/2516895584 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f0406bab0 msgr2=0x7f5f04102500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:36.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.343+0000 7f5f027fc700 1 --2- 172.21.15.27:0/2516895584 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f0406bab0 0x7f5f04102500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.343+0000 7f5f027fc700 1 -- 172.21.15.27:0/2516895584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f041095a0 msgr2=0x7f5f04102f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:36.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.343+0000 7f5f027fc700 1 --2- 172.21.15.27:0/2516895584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f041095a0 0x7f5f04102f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.343+0000 7f5f027fc700 1 -- 172.21.15.27:0/2516895584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f041a8170 con 0x7f5f0406c470 2024-05-21T03:24:36.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.343+0000 7f5f027fc700 1 --2- 172.21.15.27:0/2516895584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f0406c470 0x7f5f04102a40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800b5c0 tx=0x7f5ef80058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:36.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.343+0000 7f5eebfff700 1 -- 172.21.15.27:0/2516895584 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8027070 con 0x7f5f0406c470 2024-05-21T03:24:36.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.343+0000 7f5eebfff700 1 -- 172.21.15.27:0/2516895584 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ef8003fb0 con 0x7f5f0406c470 2024-05-21T03:24:36.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.343+0000 7f5f0979d700 1 -- 172.21.15.27:0/2516895584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f041a8400 con 0x7f5f0406c470 2024-05-21T03:24:36.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.343+0000 7f5eebfff700 1 -- 172.21.15.27:0/2516895584 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8022c80 con 0x7f5f0406c470 2024-05-21T03:24:36.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.343+0000 7f5f0979d700 1 -- 172.21.15.27:0/2516895584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f041a8850 con 0x7f5f0406c470 2024-05-21T03:24:36.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.346+0000 7f5eebfff700 1 -- 172.21.15.27:0/2516895584 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f5ef802a070 con 0x7f5f0406c470 2024-05-21T03:24:36.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.347+0000 7f5eebfff700 1 --2- 172.21.15.27:0/2516895584 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f5eec0b2a80 0x7f5eec0b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:36.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.347+0000 7f5f02ffd700 1 --2- 172.21.15.27:0/2516895584 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f5eec0b2a80 0x7f5eec0b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:36.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.347+0000 7f5eebfff700 1 -- 172.21.15.27:0/2516895584 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f5ef80e3620 con 0x7f5f0406c470 2024-05-21T03:24:36.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.347+0000 7f5f14033b80 1 Processor -- start 2024-05-21T03:24:36.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.347+0000 7f5f14033b80 1 -- start start 2024-05-21T03:24:36.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.348+0000 7f5f14033b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f2d410810 0x558f2d410bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:36.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.348+0000 7f5f14033b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f2d4111c0 0x558f2d4148f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:36.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.348+0000 7f5f14033b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f2d414f10 0x558f2d415350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:36.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.348+0000 7f5f14033b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558f2d36fc40 con 0x558f2d414f10 2024-05-21T03:24:36.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.348+0000 7f5f14033b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558f2d35f7e0 con 0x558f2d410810 2024-05-21T03:24:36.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.348+0000 7f5f14033b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558f2d363820 con 0x558f2d4111c0 2024-05-21T03:24:36.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.348+0000 7f5f02ffd700 1 --2- 172.21.15.27:0/2516895584 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f5eec0b2a80 0x7f5eec0b4f40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4007b60 tx=0x7f5ef40073d0 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:36.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.348+0000 7f5f037fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f2d414f10 0x558f2d415350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:36.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.348+0000 7f5f037fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f2d414f10 0x558f2d415350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47552/0 (socket says 172.21.15.27:47552) 2024-05-21T03:24:36.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.348+0000 7f5f037fe700 1 -- 172.21.15.27:0/2709706193 learned_addr learned my addr 172.21.15.27:0/2709706193 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:36.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.348+0000 7f5f02ffd700 1 --2- 172.21.15.27:0/2709706193 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f2d410810 0x558f2d410bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:36.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.348+0000 7f5f027fc700 1 --2- 172.21.15.27:0/2709706193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f2d4111c0 0x558f2d4148f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:36.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.348+0000 7f5f037fe700 1 -- 172.21.15.27:0/2709706193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f2d4111c0 msgr2=0x558f2d4148f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:36.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.348+0000 7f5f037fe700 1 --2- 172.21.15.27:0/2709706193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f2d4111c0 0x558f2d4148f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.348+0000 7f5f037fe700 1 -- 172.21.15.27:0/2709706193 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f2d410810 msgr2=0x558f2d410bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:36.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.348+0000 7f5f037fe700 1 --2- 172.21.15.27:0/2709706193 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f2d410810 0x558f2d410bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.348+0000 7f5f037fe700 1 -- 172.21.15.27:0/2709706193 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f2d2af880 con 0x558f2d414f10 2024-05-21T03:24:36.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.348+0000 7f5f037fe700 1 --2- 172.21.15.27:0/2709706193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f2d414f10 0x558f2d415350 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5efc00a730 tx=0x7f5efc0078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e19dbdbef34b1c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:36.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.349+0000 7f5ee97fa700 1 -- 172.21.15.27:0/2709706193 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5efc003a00 con 0x558f2d414f10 2024-05-21T03:24:36.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.349+0000 7f5ee97fa700 1 -- 172.21.15.27:0/2709706193 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5efc003ba0 con 0x558f2d414f10 2024-05-21T03:24:36.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.349+0000 7f5ee97fa700 1 -- 172.21.15.27:0/2709706193 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5efc014660 con 0x558f2d414f10 2024-05-21T03:24:36.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.349+0000 7f5f14033b80 1 -- 172.21.15.27:0/2709706193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f2d414f10 msgr2=0x558f2d415350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:36.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.349+0000 7f5f14033b80 1 --2- 172.21.15.27:0/2709706193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f2d414f10 0x558f2d415350 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5efc00a730 tx=0x7f5efc0078d0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.349+0000 7f5f14033b80 1 -- 172.21.15.27:0/2709706193 shutdown_connections 2024-05-21T03:24:36.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.349+0000 7f5f14033b80 1 --2- 172.21.15.27:0/2709706193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f2d414f10 0x558f2d415350 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.349+0000 7f5f14033b80 1 --2- 172.21.15.27:0/2709706193 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f2d410810 0x558f2d410bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.349+0000 7f5f14033b80 1 --2- 172.21.15.27:0/2709706193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f2d4111c0 0x558f2d4148f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.349+0000 7f5f14033b80 1 -- 172.21.15.27:0/2709706193 >> 172.21.15.27:0/2709706193 conn(0x558f2d33e030 msgr2=0x558f2d40ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:36.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.350+0000 7f5f14033b80 1 -- 172.21.15.27:0/2709706193 shutdown_connections 2024-05-21T03:24:36.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.350+0000 7f5f14033b80 1 -- 172.21.15.27:0/2709706193 wait complete. 2024-05-21T03:24:36.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.350+0000 7f5f14033b80 1 -- start start 2024-05-21T03:24:36.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.350+0000 7f5f14033b80 1 Processor -- start 2024-05-21T03:24:36.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.350+0000 7f5f14033b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f2d410810 0x558f2d41c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:36.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.350+0000 7f5f14033b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f2d4111c0 0x558f2d4195d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:36.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.350+0000 7f5f14033b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f2d414f10 0x558f2d419b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:36.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.350+0000 7f5f14033b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558f2d265f50 con 0x558f2d410810 2024-05-21T03:24:36.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.350+0000 7f5f14033b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558f2d301770 con 0x558f2d4111c0 2024-05-21T03:24:36.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.350+0000 7f5f14033b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558f2d41d070 con 0x558f2d414f10 2024-05-21T03:24:36.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.350+0000 7f5f14033b80 10 client.0 _ll_register_callbacks cb 0x558f2d2fa8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:24:36.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.350+0000 7f5f02ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f2d410810 0x558f2d41c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:36.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.350+0000 7f5f02ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f2d410810 0x558f2d41c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47554/0 (socket says 172.21.15.27:47554) 2024-05-21T03:24:36.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.350+0000 7f5f02ffd700 1 -- 172.21.15.27:0/3659276381 learned_addr learned my addr 172.21.15.27:0/3659276381 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:36.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.351+0000 7f5f037fe700 1 --2- 172.21.15.27:0/3659276381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f2d414f10 0x558f2d419b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:36.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.351+0000 7f5f027fc700 1 --2- 172.21.15.27:0/3659276381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f2d4111c0 0x558f2d4195d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:36.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.351+0000 7f5f02ffd700 1 -- 172.21.15.27:0/3659276381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f2d414f10 msgr2=0x558f2d419b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:36.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.351+0000 7f5f02ffd700 1 --2- 172.21.15.27:0/3659276381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f2d414f10 0x558f2d419b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.351+0000 7f5f02ffd700 1 -- 172.21.15.27:0/3659276381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f2d4111c0 msgr2=0x558f2d4195d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:36.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.351+0000 7f5f02ffd700 1 --2- 172.21.15.27:0/3659276381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f2d4111c0 0x558f2d4195d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.351+0000 7f5f02ffd700 1 -- 172.21.15.27:0/3659276381 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f2d2fe640 con 0x558f2d410810 2024-05-21T03:24:36.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.351+0000 7f5f027fc700 1 --2- 172.21.15.27:0/3659276381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f2d4111c0 0x558f2d4195d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:24:36.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.351+0000 7f5f037fe700 1 --2- 172.21.15.27:0/3659276381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f2d414f10 0x558f2d419b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:24:36.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.351+0000 7f5f02ffd700 1 --2- 172.21.15.27:0/3659276381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f2d410810 0x558f2d41c2d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4010ce0 tx=0x7f5ef40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7975dd966d1d7f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:36.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.351+0000 7f5ee6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:24:36.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.351+0000 7f5ee6ff5700 1 -- 172.21.15.27:0/3659276381 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4011be0 con 0x558f2d410810 2024-05-21T03:24:36.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.351+0000 7f5ee6ff5700 1 -- 172.21.15.27:0/3659276381 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f5ef4011d80 con 0x558f2d410810 2024-05-21T03:24:36.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.351+0000 7f5f14033b80 1 -- 172.21.15.27:0/3659276381 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5efc003ba0 con 0x558f2d410810 2024-05-21T03:24:36.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.351+0000 7f5ee6ff5700 1 -- 172.21.15.27:0/3659276381 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5ef4004020 con 0x558f2d410810 2024-05-21T03:24:36.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.351+0000 7f5f14033b80 10 client.15459 fetch_fsmap learned FSMap version 29 2024-05-21T03:24:36.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.352+0000 7f5f14033b80 1 -- 172.21.15.27:0/3659276381 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558f2d427750 con 0x558f2d410810 2024-05-21T03:24:36.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.352+0000 7f5ee6ff5700 1 -- 172.21.15.27:0/3659276381 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5efc003ba0 con 0x558f2d410810 2024-05-21T03:24:36.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.352+0000 7f5ee6ff5700 1 -- 172.21.15.27:0/3659276381 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5ef4038480 con 0x558f2d410810 2024-05-21T03:24:36.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.352+0000 7f5f14033b80 10 client.15459 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T03:24:36.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.352+0000 7f5f14033b80 10 client.15459 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-21T03:24:36.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.352+0000 7f5f14033b80 20 client.15459 populate_metadata read hostname 'smithi027' 2024-05-21T03:24:36.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.352+0000 7f5f14033b80 1 --2- 172.21.15.27:0/3659276381 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x558f2d429d50 0x558f2d42b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:36.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.352+0000 7f5f14033b80 4 client.15459 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:24:36.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.352+0000 7f5f14033b80 1 -- 172.21.15.27:0/3659276381 --> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f2d3dc990 con 0x558f2d429d50 2024-05-21T03:24:36.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.352+0000 7f5f027fc700 1 --2- 172.21.15.27:0/3659276381 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x558f2d429d50 0x558f2d42b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:36.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.353+0000 7f5f027fc700 1 --2- 172.21.15.27:0/3659276381 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x558f2d429d50 0x558f2d42b1c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=84fbee715bd9139d server_cookie=747a505c427c4ff6 in_seq=0 out_seq=0 2024-05-21T03:24:36.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.353+0000 7f5ee6ff5700 10 client.15459 ms_handle_connect on v2:172.21.15.165:6838/4158170814 2024-05-21T03:24:36.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.353+0000 7f5ee6ff5700 1 -- 172.21.15.27:0/3659276381 <== mds.1 v2:172.21.15.165:6838/4158170814 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558f2d3dc990 con 0x558f2d429d50 2024-05-21T03:24:36.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.353+0000 7f5ee6ff5700 10 client.15459 handle_command_reply: tid=0 2024-05-21T03:24:36.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.354+0000 7f5f14033b80 1 client.15459 shutdown 2024-05-21T03:24:36.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.354+0000 7f5f14033b80 1 -- 172.21.15.27:0/3659276381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f2d410810 msgr2=0x558f2d41c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:36.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.354+0000 7f5f14033b80 1 --2- 172.21.15.27:0/3659276381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f2d410810 0x558f2d41c2d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4010ce0 tx=0x7f5ef40047c0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.354+0000 7f5f14033b80 1 -- 172.21.15.27:0/3659276381 shutdown_connections 2024-05-21T03:24:36.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.354+0000 7f5f14033b80 1 --2- 172.21.15.27:0/3659276381 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x558f2d429d50 0x558f2d42b1c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.354+0000 7f5f14033b80 1 --2- 172.21.15.27:0/3659276381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f2d414f10 0x558f2d419b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.354+0000 7f5ee6ff5700 0 client.15459 ms_handle_reset on v2:172.21.15.165:6838/4158170814 2024-05-21T03:24:36.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.354+0000 7f5f14033b80 1 --2- 172.21.15.27:0/3659276381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f2d410810 0x558f2d41c2d0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.354+0000 7f5f14033b80 1 --2- 172.21.15.27:0/3659276381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f2d4111c0 0x558f2d4195d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.354+0000 7f5f14033b80 1 -- 172.21.15.27:0/3659276381 >> 172.21.15.27:0/3659276381 conn(0x558f2d33e030 msgr2=0x558f2d416300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:36.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.354+0000 7f5f14033b80 1 -- 172.21.15.27:0/3659276381 shutdown_connections 2024-05-21T03:24:36.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.354+0000 7f5f14033b80 1 -- 172.21.15.27:0/3659276381 wait complete. 2024-05-21T03:24:36.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.354+0000 7f5f14033b80 20 client.15459 trim_cache size 0 max 16384 2024-05-21T03:24:36.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.376+0000 7f5f14033b80 1 Processor -- start 2024-05-21T03:24:36.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.376+0000 7f5f14033b80 1 -- start start 2024-05-21T03:24:36.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.376+0000 7f5f14033b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f2d410810 0x558f2d40b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:36.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.376+0000 7f5f14033b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f2d4111c0 0x558f2d412860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:36.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.376+0000 7f5f14033b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f2d414f10 0x558f2d412da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:36.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.376+0000 7f5f14033b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558f2d429500 con 0x558f2d410810 2024-05-21T03:24:36.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.376+0000 7f5f14033b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558f2d429bd0 con 0x558f2d414f10 2024-05-21T03:24:36.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.376+0000 7f5f14033b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558f2d42a2a0 con 0x558f2d4111c0 2024-05-21T03:24:36.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.376+0000 7f5f02ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f2d410810 0x558f2d40b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:36.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.376+0000 7f5f02ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f2d410810 0x558f2d40b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47568/0 (socket says 172.21.15.27:47568) 2024-05-21T03:24:36.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.376+0000 7f5f02ffd700 1 -- 172.21.15.27:0/3863143335 learned_addr learned my addr 172.21.15.27:0/3863143335 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:36.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.376+0000 7f5f027fc700 1 --2- 172.21.15.27:0/3863143335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f2d4111c0 0x558f2d412860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:36.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.377+0000 7f5f02ffd700 1 -- 172.21.15.27:0/3863143335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f2d4111c0 msgr2=0x558f2d412860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:36.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.377+0000 7f5f02ffd700 1 --2- 172.21.15.27:0/3863143335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f2d4111c0 0x558f2d412860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.377+0000 7f5f02ffd700 1 -- 172.21.15.27:0/3863143335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f2d414f10 msgr2=0x558f2d412da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:36.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.377+0000 7f5f037fe700 1 --2- 172.21.15.27:0/3863143335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f2d414f10 0x558f2d412da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:36.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.377+0000 7f5f02ffd700 1 --2- 172.21.15.27:0/3863143335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f2d414f10 0x558f2d412da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.377+0000 7f5f02ffd700 1 -- 172.21.15.27:0/3863143335 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f2d41aa30 con 0x558f2d410810 2024-05-21T03:24:36.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.377+0000 7f5f037fe700 1 --2- 172.21.15.27:0/3863143335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f2d414f10 0x558f2d412da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:24:36.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.377+0000 7f5f02ffd700 1 --2- 172.21.15.27:0/3863143335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f2d410810 0x558f2d40b4e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4008010 tx=0x7f5ef4038690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03d095d2d720dd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:36.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.377+0000 7f5ee8ff9700 1 -- 172.21.15.27:0/3863143335 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4046070 con 0x558f2d410810 2024-05-21T03:24:36.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.377+0000 7f5ee8ff9700 1 -- 172.21.15.27:0/3863143335 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ef4038ac0 con 0x558f2d410810 2024-05-21T03:24:36.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.377+0000 7f5ee8ff9700 1 -- 172.21.15.27:0/3863143335 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef403e070 con 0x558f2d410810 2024-05-21T03:24:36.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.378+0000 7f5f14033b80 1 -- 172.21.15.27:0/3863143335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f2d410810 msgr2=0x558f2d40b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:36.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.378+0000 7f5f14033b80 1 --2- 172.21.15.27:0/3863143335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f2d410810 0x558f2d40b4e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4008010 tx=0x7f5ef4038690 comp rx=0 tx=0).stop 2024-05-21T03:24:36.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.378+0000 7f5f14033b80 1 -- 172.21.15.27:0/3863143335 shutdown_connections 2024-05-21T03:24:36.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.378+0000 7f5f14033b80 1 --2- 172.21.15.27:0/3863143335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f2d414f10 0x558f2d412da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.378+0000 7f5f14033b80 1 --2- 172.21.15.27:0/3863143335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f2d410810 0x558f2d40b4e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.378+0000 7f5f14033b80 1 --2- 172.21.15.27:0/3863143335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f2d4111c0 0x558f2d412860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.378+0000 7f5f14033b80 1 -- 172.21.15.27:0/3863143335 >> 172.21.15.27:0/3863143335 conn(0x558f2d33e030 msgr2=0x558f2d40add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:36.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.378+0000 7f5f14033b80 1 -- 172.21.15.27:0/3863143335 shutdown_connections 2024-05-21T03:24:36.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.378+0000 7f5f14033b80 1 -- 172.21.15.27:0/3863143335 wait complete. 2024-05-21T03:24:36.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.378+0000 7f5f14033b80 1 -- start start 2024-05-21T03:24:36.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.379+0000 7f5f14033b80 1 Processor -- start 2024-05-21T03:24:36.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.379+0000 7f5f14033b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f2d410810 0x558f2d434c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:36.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.379+0000 7f5f14033b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f2d4111c0 0x558f2d42f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:36.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.379+0000 7f5f14033b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f2d414f10 0x558f2d42fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:36.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.379+0000 7f5f14033b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558f2d42b040 con 0x558f2d4111c0 2024-05-21T03:24:36.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.379+0000 7f5f14033b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558f2d42a970 con 0x558f2d410810 2024-05-21T03:24:36.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.379+0000 7f5f14033b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558f2d42b710 con 0x558f2d414f10 2024-05-21T03:24:36.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.379+0000 7f5f14033b80 10 client.0 _ll_register_callbacks cb 0x558f2d2fa8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:24:36.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.379+0000 7f5f027fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f2d4111c0 0x558f2d42f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:36.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.379+0000 7f5f02ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f2d410810 0x558f2d434c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:36.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.379+0000 7f5f027fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f2d4111c0 0x558f2d42f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47576/0 (socket says 172.21.15.27:47576) 2024-05-21T03:24:36.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.379+0000 7f5f027fc700 1 -- 172.21.15.27:0/2804289158 learned_addr learned my addr 172.21.15.27:0/2804289158 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:36.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.379+0000 7f5f037fe700 1 --2- 172.21.15.27:0/2804289158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f2d414f10 0x558f2d42fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:36.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.379+0000 7f5f02ffd700 1 -- 172.21.15.27:0/2804289158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f2d414f10 msgr2=0x558f2d42fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:36.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.379+0000 7f5f02ffd700 1 --2- 172.21.15.27:0/2804289158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f2d414f10 0x558f2d42fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.379+0000 7f5f02ffd700 1 -- 172.21.15.27:0/2804289158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f2d4111c0 msgr2=0x558f2d42f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:36.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.379+0000 7f5f02ffd700 1 --2- 172.21.15.27:0/2804289158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f2d4111c0 0x558f2d42f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.379+0000 7f5f02ffd700 1 -- 172.21.15.27:0/2804289158 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f2d4303a0 con 0x558f2d410810 2024-05-21T03:24:36.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.379+0000 7f5f027fc700 1 --2- 172.21.15.27:0/2804289158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f2d4111c0 0x558f2d42f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:24:36.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.380+0000 7f5f02ffd700 1 --2- 172.21.15.27:0/2804289158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f2d410810 0x558f2d434c60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4005a80 tx=0x7f5ef4025c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=334d147f45c5e347 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:36.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.380+0000 7f5ee6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-21T03:24:36.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.380+0000 7f5ee6ff5700 1 -- 172.21.15.27:0/2804289158 <== mon.2 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4046070 con 0x558f2d410810 2024-05-21T03:24:36.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.380+0000 7f5f14033b80 1 -- 172.21.15.27:0/2804289158 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ef4038ac0 con 0x558f2d410810 2024-05-21T03:24:36.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.380+0000 7f5ee6ff5700 1 -- 172.21.15.27:0/2804289158 <== mon.2 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f5ef4003680 con 0x558f2d410810 2024-05-21T03:24:36.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.380+0000 7f5ee6ff5700 1 -- 172.21.15.27:0/2804289158 <== mon.2 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5ef403e070 con 0x558f2d410810 2024-05-21T03:24:36.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.380+0000 7f5f14033b80 10 client.25094 fetch_fsmap learned FSMap version 29 2024-05-21T03:24:36.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.380+0000 7f5f14033b80 1 -- 172.21.15.27:0/2804289158 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558f2d4963b0 con 0x558f2d410810 2024-05-21T03:24:36.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.380+0000 7f5ee6ff5700 1 -- 172.21.15.27:0/2804289158 <== mon.2 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ef4038ac0 con 0x558f2d410810 2024-05-21T03:24:36.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.381+0000 7f5ee6ff5700 1 -- 172.21.15.27:0/2804289158 <== mon.2 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5ef402fb40 con 0x558f2d410810 2024-05-21T03:24:36.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.381+0000 7f5f14033b80 10 client.25094 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T03:24:36.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.381+0000 7f5f14033b80 10 client.25094 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-21T03:24:36.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.381+0000 7f5f14033b80 20 client.25094 populate_metadata read hostname 'smithi027' 2024-05-21T03:24:36.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.381+0000 7f5f14033b80 1 --2- 172.21.15.27:0/2804289158 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x558f2d429d50 0x558f2d4998d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:36.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.381+0000 7f5f14033b80 4 client.25094 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T03:24:36.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.381+0000 7f5f14033b80 1 -- 172.21.15.27:0/2804289158 --> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] -- command(tid 0: {"prefix": "status"}) -- 0x558f2d371b90 con 0x558f2d429d50 2024-05-21T03:24:36.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.381+0000 7f5f027fc700 1 --2- 172.21.15.27:0/2804289158 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x558f2d429d50 0x558f2d4998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:36.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.382+0000 7f5f027fc700 1 --2- 172.21.15.27:0/2804289158 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x558f2d429d50 0x558f2d4998d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=48adac48615e379b server_cookie=1599d4f6f6724c35 in_seq=0 out_seq=0 2024-05-21T03:24:36.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.382+0000 7f5ee6ff5700 10 client.25094 ms_handle_connect on v2:172.21.15.165:6838/4158170814 2024-05-21T03:24:36.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.382+0000 7f5ee6ff5700 1 -- 172.21.15.27:0/2804289158 <== mds.1 v2:172.21.15.165:6838/4158170814 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558f2d371b90 con 0x558f2d429d50 2024-05-21T03:24:36.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.382+0000 7f5ee6ff5700 10 client.25094 handle_command_reply: tid=0 2024-05-21T03:24:36.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.382+0000 7f5f14033b80 1 client.25094 shutdown 2024-05-21T03:24:36.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.382+0000 7f5f14033b80 1 -- 172.21.15.27:0/2804289158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f2d410810 msgr2=0x558f2d434c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:36.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.382+0000 7f5f14033b80 1 --2- 172.21.15.27:0/2804289158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f2d410810 0x558f2d434c60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4005a80 tx=0x7f5ef4025c70 comp rx=0 tx=0).stop 2024-05-21T03:24:36.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.382+0000 7f5f14033b80 1 -- 172.21.15.27:0/2804289158 shutdown_connections 2024-05-21T03:24:36.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.382+0000 7f5f14033b80 1 --2- 172.21.15.27:0/2804289158 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x558f2d429d50 0x558f2d4998d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.382+0000 7f5f14033b80 1 --2- 172.21.15.27:0/2804289158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f2d414f10 0x558f2d42fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.382+0000 7f5f14033b80 1 --2- 172.21.15.27:0/2804289158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f2d410810 0x558f2d434c60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.382+0000 7f5ee6ff5700 0 client.25094 ms_handle_reset on v2:172.21.15.165:6838/4158170814 2024-05-21T03:24:36.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.382+0000 7f5f14033b80 1 --2- 172.21.15.27:0/2804289158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f2d4111c0 0x558f2d42f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.382+0000 7f5f14033b80 1 -- 172.21.15.27:0/2804289158 >> 172.21.15.27:0/2804289158 conn(0x558f2d33e030 msgr2=0x558f2d40feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:36.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.382+0000 7f5f14033b80 1 -- 172.21.15.27:0/2804289158 shutdown_connections 2024-05-21T03:24:36.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.382+0000 7f5f14033b80 1 -- 172.21.15.27:0/2804289158 wait complete. 2024-05-21T03:24:36.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.382+0000 7f5f14033b80 20 client.25094 trim_cache size 0 max 16384 2024-05-21T03:24:36.387 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:24:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "c9ca94be-171f-11ef-bc9a-c7b262605968", 2024-05-21T03:24:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 1, 2024-05-21T03:24:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24472, 2024-05-21T03:24:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-05-21T03:24:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-05-21T03:24:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-05-21T03:24:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 229.95954491200001, 2024-05-21T03:24:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 29, 2024-05-21T03:24:36.388 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 83, 2024-05-21T03:24:36.388 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 82, 2024-05-21T03:24:36.388 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 266.48119980600001 2024-05-21T03:24:36.388 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:24:36.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.383+0000 7f5f0979d700 1 -- 172.21.15.27:0/2516895584 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f5eec0b2a80 msgr2=0x7f5eec0b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:36.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.383+0000 7f5f0979d700 1 --2- 172.21.15.27:0/2516895584 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f5eec0b2a80 0x7f5eec0b4f40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4007b60 tx=0x7f5ef40073d0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.383+0000 7f5f0979d700 1 -- 172.21.15.27:0/2516895584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f0406c470 msgr2=0x7f5f04102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:36.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.383+0000 7f5f0979d700 1 --2- 172.21.15.27:0/2516895584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f0406c470 0x7f5f04102a40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800b5c0 tx=0x7f5ef80058e0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.383+0000 7f5f0979d700 1 -- 172.21.15.27:0/2516895584 shutdown_connections 2024-05-21T03:24:36.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.383+0000 7f5f0979d700 1 --2- 172.21.15.27:0/2516895584 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f5eec0b2a80 0x7f5eec0b4f40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.383+0000 7f5f0979d700 1 --2- 172.21.15.27:0/2516895584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f041095a0 0x7f5f04102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.383+0000 7f5f0979d700 1 --2- 172.21.15.27:0/2516895584 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f0406bab0 0x7f5f04102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.383+0000 7f5f0979d700 1 --2- 172.21.15.27:0/2516895584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f0406c470 0x7f5f04102a40 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:36.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.383+0000 7f5f0979d700 1 -- 172.21.15.27:0/2516895584 >> 172.21.15.27:0/2516895584 conn(0x7f5f040793b0 msgr2=0x7f5f041012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:36.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.383+0000 7f5f0979d700 1 -- 172.21.15.27:0/2516895584 shutdown_connections 2024-05-21T03:24:36.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:36.383+0000 7f5f0979d700 1 -- 172.21.15.27:0/2516895584 wait complete. 2024-05-21T03:24:36.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:36 smithi027 ceph-mon[32179]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-21T03:24:36.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:36 smithi039 ceph-mon[36429]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-21T03:24:36.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:36 smithi165 ceph-mon[36269]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-21T03:24:37.263 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell mds.b status 2024-05-21T03:24:37.516 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:24:38.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:38 smithi027 ceph-mon[32179]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-21T03:24:38.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:38 smithi039 ceph-mon[36429]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-21T03:24:38.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:38 smithi165 ceph-mon[36269]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-21T03:24:39.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.186+0000 7fdc250ca700 1 -- 172.21.15.27:0/3850588801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc2006c630 msgr2=0x7fdc20110420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:39.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.186+0000 7fdc250ca700 1 --2- 172.21.15.27:0/3850588801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc2006c630 0x7fdc20110420 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800b0d0 tx=0x7fdc1800b3e0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.187+0000 7fdc250ca700 1 -- 172.21.15.27:0/3850588801 shutdown_connections 2024-05-21T03:24:39.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.187+0000 7fdc250ca700 1 --2- 172.21.15.27:0/3850588801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc2006c630 0x7fdc20110420 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.187+0000 7fdc250ca700 1 --2- 172.21.15.27:0/3850588801 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc20109650 0x7fdc20109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.187+0000 7fdc250ca700 1 --2- 172.21.15.27:0/3850588801 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc2006bc70 0x7fdc2006c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.187+0000 7fdc250ca700 1 -- 172.21.15.27:0/3850588801 >> 172.21.15.27:0/3850588801 conn(0x7fdc200782b0 msgr2=0x7fdc200786c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:39.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.187+0000 7fdc250ca700 1 -- 172.21.15.27:0/3850588801 shutdown_connections 2024-05-21T03:24:39.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.187+0000 7fdc250ca700 1 -- 172.21.15.27:0/3850588801 wait complete. 2024-05-21T03:24:39.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.187+0000 7fdc250ca700 1 Processor -- start 2024-05-21T03:24:39.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.188+0000 7fdc250ca700 1 -- start start 2024-05-21T03:24:39.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.188+0000 7fdc250ca700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc2006bc70 0x7fdc2019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:39.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.188+0000 7fdc250ca700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc2006c630 0x7fdc2019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:39.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.188+0000 7fdc250ca700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc20109650 0x7fdc20199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:39.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.188+0000 7fdc250ca700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdc20113770 con 0x7fdc2006c630 2024-05-21T03:24:39.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.188+0000 7fdc250ca700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdc201135f0 con 0x7fdc20109650 2024-05-21T03:24:39.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.188+0000 7fdc250ca700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdc201138f0 con 0x7fdc2006bc70 2024-05-21T03:24:39.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.188+0000 7fdc1659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc2006c630 0x7fdc2019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:39.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.188+0000 7fdc1659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc2006c630 0x7fdc2019fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47594/0 (socket says 172.21.15.27:47594) 2024-05-21T03:24:39.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.188+0000 7fdc1659c700 1 -- 172.21.15.27:0/614420524 learned_addr learned my addr 172.21.15.27:0/614420524 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:39.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.188+0000 7fdc1ed9d700 1 --2- 172.21.15.27:0/614420524 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc2006bc70 0x7fdc2019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:39.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.188+0000 7fdc1f59e700 1 --2- 172.21.15.27:0/614420524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc20109650 0x7fdc20199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:39.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.188+0000 7fdc1659c700 1 -- 172.21.15.27:0/614420524 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc2006bc70 msgr2=0x7fdc2019f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:39.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.188+0000 7fdc1659c700 1 --2- 172.21.15.27:0/614420524 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc2006bc70 0x7fdc2019f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.188+0000 7fdc1659c700 1 -- 172.21.15.27:0/614420524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc20109650 msgr2=0x7fdc20199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:39.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.188+0000 7fdc1659c700 1 --2- 172.21.15.27:0/614420524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc20109650 0x7fdc20199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.188+0000 7fdc1659c700 1 -- 172.21.15.27:0/614420524 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc2019a090 con 0x7fdc2006c630 2024-05-21T03:24:39.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.189+0000 7fdc1659c700 1 --2- 172.21.15.27:0/614420524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc2006c630 0x7fdc2019fcb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800c970 tx=0x7fdc0800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:39.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.189+0000 7fdc1cd99700 1 -- 172.21.15.27:0/614420524 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc08007aa0 con 0x7fdc2006c630 2024-05-21T03:24:39.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.189+0000 7fdc250ca700 1 -- 172.21.15.27:0/614420524 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc2019a380 con 0x7fdc2006c630 2024-05-21T03:24:39.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.189+0000 7fdc1cd99700 1 -- 172.21.15.27:0/614420524 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc08004ce0 con 0x7fdc2006c630 2024-05-21T03:24:39.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.189+0000 7fdc1cd99700 1 -- 172.21.15.27:0/614420524 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc08005510 con 0x7fdc2006c630 2024-05-21T03:24:39.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.189+0000 7fdc250ca700 1 -- 172.21.15.27:0/614420524 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc201a6640 con 0x7fdc2006c630 2024-05-21T03:24:39.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.191+0000 7fdc1cd99700 1 -- 172.21.15.27:0/614420524 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fdc08007c40 con 0x7fdc2006c630 2024-05-21T03:24:39.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.192+0000 7fdc1cd99700 1 --2- 172.21.15.27:0/614420524 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fdc0c0b2a80 0x7fdc0c0b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:39.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.192+0000 7fdc1ed9d700 1 --2- 172.21.15.27:0/614420524 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fdc0c0b2a80 0x7fdc0c0b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:39.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.192+0000 7fdc1cd99700 1 -- 172.21.15.27:0/614420524 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fdc080e25c0 con 0x7fdc2006c630 2024-05-21T03:24:39.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.192+0000 7fdc2f960b80 1 Processor -- start 2024-05-21T03:24:39.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.193+0000 7fdc2f960b80 1 -- start start 2024-05-21T03:24:39.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.193+0000 7fdc1ed9d700 1 --2- 172.21.15.27:0/614420524 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fdc0c0b2a80 0x7fdc0c0b4f40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdc2019aa70 tx=0x7fdc1000a380 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:39.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.193+0000 7fdc2f960b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562fb57ce2d0 0x562fb57ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:39.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.193+0000 7fdc2f960b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb57c8850 0x562fb57c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:39.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.193+0000 7fdc2f960b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fb57c9210 0x562fb57d5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:39.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.193+0000 7fdc2f960b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562fb5726d20 con 0x562fb57c9210 2024-05-21T03:24:39.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.193+0000 7fdc2f960b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562fb57168c0 con 0x562fb57c8850 2024-05-21T03:24:39.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.193+0000 7fdc2f960b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562fb571a900 con 0x562fb57ce2d0 2024-05-21T03:24:39.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.193+0000 7fdc1f59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fb57c9210 0x562fb57d5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:39.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.193+0000 7fdc1f59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fb57c9210 0x562fb57d5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47604/0 (socket says 172.21.15.27:47604) 2024-05-21T03:24:39.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.193+0000 7fdc1f59e700 1 -- 172.21.15.27:0/2562032581 learned_addr learned my addr 172.21.15.27:0/2562032581 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:39.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.193+0000 7fdc1659c700 1 --2- 172.21.15.27:0/2562032581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb57c8850 0x562fb57c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:39.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.193+0000 7fdc1ed9d700 1 --2- 172.21.15.27:0/2562032581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562fb57ce2d0 0x562fb57ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:39.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.193+0000 7fdc1f59e700 1 -- 172.21.15.27:0/2562032581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562fb57ce2d0 msgr2=0x562fb57ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:39.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.193+0000 7fdc1f59e700 1 --2- 172.21.15.27:0/2562032581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562fb57ce2d0 0x562fb57ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.193+0000 7fdc1f59e700 1 -- 172.21.15.27:0/2562032581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb57c8850 msgr2=0x562fb57c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:39.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.193+0000 7fdc1f59e700 1 --2- 172.21.15.27:0/2562032581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb57c8850 0x562fb57c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.193+0000 7fdc1f59e700 1 -- 172.21.15.27:0/2562032581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fb56bff60 con 0x562fb57c9210 2024-05-21T03:24:39.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.193+0000 7fdc1ed9d700 1 --2- 172.21.15.27:0/2562032581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562fb57ce2d0 0x562fb57ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:24:39.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.194+0000 7fdc1f59e700 1 --2- 172.21.15.27:0/2562032581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fb57c9210 0x562fb57d5030 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdc18009dc0 tx=0x7fdc18009df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97cf71d8093ad7b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:39.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.194+0000 7fdc1559a700 1 -- 172.21.15.27:0/2562032581 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc18020030 con 0x562fb57c9210 2024-05-21T03:24:39.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.194+0000 7fdc1559a700 1 -- 172.21.15.27:0/2562032581 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc18009210 con 0x562fb57c9210 2024-05-21T03:24:39.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.194+0000 7fdc1559a700 1 -- 172.21.15.27:0/2562032581 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc18007d10 con 0x562fb57c9210 2024-05-21T03:24:39.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.195+0000 7fdc2f960b80 1 -- 172.21.15.27:0/2562032581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fb57c9210 msgr2=0x562fb57d5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.195+0000 7fdc2f960b80 1 --2- 172.21.15.27:0/2562032581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fb57c9210 0x562fb57d5030 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdc18009dc0 tx=0x7fdc18009df0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.195+0000 7fdc2f960b80 1 -- 172.21.15.27:0/2562032581 shutdown_connections 2024-05-21T03:24:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.195+0000 7fdc2f960b80 1 --2- 172.21.15.27:0/2562032581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fb57c9210 0x562fb57d5030 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.195+0000 7fdc2f960b80 1 --2- 172.21.15.27:0/2562032581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562fb57ce2d0 0x562fb57ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.195+0000 7fdc2f960b80 1 --2- 172.21.15.27:0/2562032581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb57c8850 0x562fb57c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.195+0000 7fdc2f960b80 1 -- 172.21.15.27:0/2562032581 >> 172.21.15.27:0/2562032581 conn(0x562fb56f41f0 msgr2=0x562fb57c3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.195+0000 7fdc2f960b80 1 -- 172.21.15.27:0/2562032581 shutdown_connections 2024-05-21T03:24:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.195+0000 7fdc2f960b80 1 -- 172.21.15.27:0/2562032581 wait complete. 2024-05-21T03:24:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.195+0000 7fdc2f960b80 1 -- start start 2024-05-21T03:24:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.195+0000 7fdc2f960b80 1 Processor -- start 2024-05-21T03:24:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.196+0000 7fdc2f960b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562fb57c8850 0x562fb57d4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.196+0000 7fdc2f960b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fb57c9210 0x562fb57d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.196+0000 7fdc2f960b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb57ce2d0 0x562fb57d24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:39.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.196+0000 7fdc2f960b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562fb561cf50 con 0x562fb57c9210 2024-05-21T03:24:39.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.196+0000 7fdc2f960b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562fb567ed50 con 0x562fb57ce2d0 2024-05-21T03:24:39.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.196+0000 7fdc2f960b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562fb57d5630 con 0x562fb57c8850 2024-05-21T03:24:39.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.196+0000 7fdc2f960b80 10 client.0 _ll_register_callbacks cb 0x562fb56b1900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:24:39.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.196+0000 7fdc1659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fb57c9210 0x562fb57d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:39.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.196+0000 7fdc1659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fb57c9210 0x562fb57d1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47608/0 (socket says 172.21.15.27:47608) 2024-05-21T03:24:39.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.196+0000 7fdc1659c700 1 -- 172.21.15.27:0/3175913697 learned_addr learned my addr 172.21.15.27:0/3175913697 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:39.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.196+0000 7fdc1ed9d700 1 --2- 172.21.15.27:0/3175913697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562fb57c8850 0x562fb57d4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:39.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.196+0000 7fdc1f59e700 1 --2- 172.21.15.27:0/3175913697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb57ce2d0 0x562fb57d24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:39.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.196+0000 7fdc1659c700 1 -- 172.21.15.27:0/3175913697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562fb57c8850 msgr2=0x562fb57d4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:39.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.196+0000 7fdc1659c700 1 --2- 172.21.15.27:0/3175913697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562fb57c8850 0x562fb57d4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.196+0000 7fdc1659c700 1 -- 172.21.15.27:0/3175913697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb57ce2d0 msgr2=0x562fb57d24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:39.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.196+0000 7fdc1659c700 1 --2- 172.21.15.27:0/3175913697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb57ce2d0 0x562fb57d24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.196+0000 7fdc1659c700 1 -- 172.21.15.27:0/3175913697 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fb5666880 con 0x562fb57c9210 2024-05-21T03:24:39.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.196+0000 7fdc1f59e700 1 --2- 172.21.15.27:0/3175913697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb57ce2d0 0x562fb57d24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:24:39.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.196+0000 7fdc1659c700 1 --2- 172.21.15.27:0/3175913697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fb57c9210 0x562fb57d1f60 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fdc080dfd30 tx=0x7fdc080a9d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d1894d87d5b787 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:39.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.196+0000 7fdc067fc700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:24:39.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.196+0000 7fdc067fc700 1 -- 172.21.15.27:0/3175913697 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc080b0e60 con 0x562fb57c9210 2024-05-21T03:24:39.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.196+0000 7fdc2f960b80 1 -- 172.21.15.27:0/3175913697 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc18009210 con 0x562fb57c9210 2024-05-21T03:24:39.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.196+0000 7fdc067fc700 1 -- 172.21.15.27:0/3175913697 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fdc080912f0 con 0x562fb57c9210 2024-05-21T03:24:39.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.197+0000 7fdc067fc700 1 -- 172.21.15.27:0/3175913697 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdc080a8f30 con 0x562fb57c9210 2024-05-21T03:24:39.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.197+0000 7fdc2f960b80 10 client.15477 fetch_fsmap learned FSMap version 29 2024-05-21T03:24:39.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.197+0000 7fdc2f960b80 1 -- 172.21.15.27:0/3175913697 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562fb56d0710 con 0x562fb57c9210 2024-05-21T03:24:39.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.197+0000 7fdc067fc700 1 -- 172.21.15.27:0/3175913697 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc18009210 con 0x562fb57c9210 2024-05-21T03:24:39.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.197+0000 7fdc067fc700 1 -- 172.21.15.27:0/3175913697 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdc0808fce0 con 0x562fb57c9210 2024-05-21T03:24:39.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.197+0000 7fdc2f960b80 10 client.15477 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T03:24:39.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.197+0000 7fdc2f960b80 10 client.15477 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-21T03:24:39.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.197+0000 7fdc2f960b80 20 client.15477 populate_metadata read hostname 'smithi027' 2024-05-21T03:24:39.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.197+0000 7fdc2f960b80 1 --2- 172.21.15.27:0/3175913697 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x562fb57e2180 0x562fb57e35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:39.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.197+0000 7fdc2f960b80 4 client.15477 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:24:39.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.197+0000 7fdc2f960b80 1 -- 172.21.15.27:0/3175913697 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fb5728c70 con 0x562fb57e2180 2024-05-21T03:24:39.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.198+0000 7fdc1ed9d700 1 --2- 172.21.15.27:0/3175913697 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x562fb57e2180 0x562fb57e35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:39.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.198+0000 7fdc1ed9d700 1 --2- 172.21.15.27:0/3175913697 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x562fb57e2180 0x562fb57e35f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a11027ad52fbb168 server_cookie=6e080e5dedaffa48 in_seq=0 out_seq=0 2024-05-21T03:24:39.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.198+0000 7fdc067fc700 10 client.15477 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:24:39.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.199+0000 7fdc067fc700 1 -- 172.21.15.27:0/3175913697 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562fb5728c70 con 0x562fb57e2180 2024-05-21T03:24:39.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.199+0000 7fdc067fc700 10 client.15477 handle_command_reply: tid=0 2024-05-21T03:24:39.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.199+0000 7fdc2f960b80 1 client.15477 shutdown 2024-05-21T03:24:39.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.199+0000 7fdc2f960b80 1 -- 172.21.15.27:0/3175913697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fb57c9210 msgr2=0x562fb57d1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:39.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.199+0000 7fdc2f960b80 1 --2- 172.21.15.27:0/3175913697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fb57c9210 0x562fb57d1f60 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fdc080dfd30 tx=0x7fdc080a9d00 comp rx=0 tx=0).stop 2024-05-21T03:24:39.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.199+0000 7fdc2f960b80 1 -- 172.21.15.27:0/3175913697 shutdown_connections 2024-05-21T03:24:39.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.199+0000 7fdc2f960b80 1 --2- 172.21.15.27:0/3175913697 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x562fb57e2180 0x562fb57e35f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.199+0000 7fdc2f960b80 1 --2- 172.21.15.27:0/3175913697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb57ce2d0 0x562fb57d24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.199+0000 7fdc2f960b80 1 --2- 172.21.15.27:0/3175913697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562fb57c8850 0x562fb57d4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.199+0000 7fdc2f960b80 1 --2- 172.21.15.27:0/3175913697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fb57c9210 0x562fb57d1f60 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.199+0000 7fdc067fc700 0 client.15477 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:24:39.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.199+0000 7fdc2f960b80 1 -- 172.21.15.27:0/3175913697 >> 172.21.15.27:0/3175913697 conn(0x562fb56f41f0 msgr2=0x562fb57c2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:39.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.199+0000 7fdc2f960b80 1 -- 172.21.15.27:0/3175913697 shutdown_connections 2024-05-21T03:24:39.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.199+0000 7fdc2f960b80 1 -- 172.21.15.27:0/3175913697 wait complete. 2024-05-21T03:24:39.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.199+0000 7fdc2f960b80 20 client.15477 trim_cache size 0 max 16384 2024-05-21T03:24:39.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.220+0000 7fdc2f960b80 1 Processor -- start 2024-05-21T03:24:39.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.220+0000 7fdc2f960b80 1 -- start start 2024-05-21T03:24:39.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.221+0000 7fdc2f960b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb57c8850 0x562fb57c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:39.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.221+0000 7fdc2f960b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562fb57c9210 0x562fb57cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:39.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.221+0000 7fdc2f960b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fb57ce2d0 0x562fb57c7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:39.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.221+0000 7fdc2f960b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562fb57e1930 con 0x562fb57ce2d0 2024-05-21T03:24:39.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.221+0000 7fdc2f960b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562fb57e2000 con 0x562fb57c8850 2024-05-21T03:24:39.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.221+0000 7fdc2f960b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562fb57e26d0 con 0x562fb57c9210 2024-05-21T03:24:39.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.221+0000 7fdc1f59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fb57ce2d0 0x562fb57c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:39.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.221+0000 7fdc1f59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fb57ce2d0 0x562fb57c7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47610/0 (socket says 172.21.15.27:47610) 2024-05-21T03:24:39.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.221+0000 7fdc1f59e700 1 -- 172.21.15.27:0/2663870909 learned_addr learned my addr 172.21.15.27:0/2663870909 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:39.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.221+0000 7fdc1659c700 1 --2- 172.21.15.27:0/2663870909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562fb57c9210 0x562fb57cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:39.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.221+0000 7fdc1f59e700 1 -- 172.21.15.27:0/2663870909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562fb57c9210 msgr2=0x562fb57cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:39.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.221+0000 7fdc1f59e700 1 --2- 172.21.15.27:0/2663870909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562fb57c9210 0x562fb57cd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.221+0000 7fdc1f59e700 1 -- 172.21.15.27:0/2663870909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb57c8850 msgr2=0x562fb57c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:39.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.221+0000 7fdc1f59e700 1 --2- 172.21.15.27:0/2663870909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb57c8850 0x562fb57c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.221+0000 7fdc1f59e700 1 -- 172.21.15.27:0/2663870909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fb57e3960 con 0x562fb57ce2d0 2024-05-21T03:24:39.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.221+0000 7fdc1659c700 1 --2- 172.21.15.27:0/2663870909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562fb57c9210 0x562fb57cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:24:39.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.221+0000 7fdc1f59e700 1 --2- 172.21.15.27:0/2663870909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fb57ce2d0 0x562fb57c7840 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdc18000f80 tx=0x7fdc180061b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6860975d99c3c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:39.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.222+0000 7fdc14d99700 1 -- 172.21.15.27:0/2663870909 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc18020040 con 0x562fb57ce2d0 2024-05-21T03:24:39.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.222+0000 7fdc14d99700 1 -- 172.21.15.27:0/2663870909 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc180074a0 con 0x562fb57ce2d0 2024-05-21T03:24:39.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.222+0000 7fdc14d99700 1 -- 172.21.15.27:0/2663870909 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1801c6a0 con 0x562fb57ce2d0 2024-05-21T03:24:39.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.222+0000 7fdc2f960b80 1 -- 172.21.15.27:0/2663870909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fb57ce2d0 msgr2=0x562fb57c7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:39.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.222+0000 7fdc2f960b80 1 --2- 172.21.15.27:0/2663870909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fb57ce2d0 0x562fb57c7840 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdc18000f80 tx=0x7fdc180061b0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.222+0000 7fdc2f960b80 1 -- 172.21.15.27:0/2663870909 shutdown_connections 2024-05-21T03:24:39.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.222+0000 7fdc2f960b80 1 --2- 172.21.15.27:0/2663870909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fb57ce2d0 0x562fb57c7840 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.222+0000 7fdc2f960b80 1 --2- 172.21.15.27:0/2663870909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb57c8850 0x562fb57c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.222+0000 7fdc2f960b80 1 --2- 172.21.15.27:0/2663870909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562fb57c9210 0x562fb57cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.223+0000 7fdc2f960b80 1 -- 172.21.15.27:0/2663870909 >> 172.21.15.27:0/2663870909 conn(0x562fb56f41f0 msgr2=0x562fb57cd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:39.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.223+0000 7fdc2f960b80 1 -- 172.21.15.27:0/2663870909 shutdown_connections 2024-05-21T03:24:39.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.223+0000 7fdc2f960b80 1 -- 172.21.15.27:0/2663870909 wait complete. 2024-05-21T03:24:39.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.223+0000 7fdc2f960b80 1 -- start start 2024-05-21T03:24:39.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.223+0000 7fdc2f960b80 1 Processor -- start 2024-05-21T03:24:39.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.223+0000 7fdc2f960b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562fb57c8850 0x562fb57c2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:39.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.223+0000 7fdc2f960b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fb57c9210 0x562fb57c3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:39.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.223+0000 7fdc2f960b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb57ce2d0 0x562fb5850930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:39.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.223+0000 7fdc2f960b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562fb57e3470 con 0x562fb57c9210 2024-05-21T03:24:39.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.223+0000 7fdc2f960b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562fb57e2da0 con 0x562fb57ce2d0 2024-05-21T03:24:39.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.223+0000 7fdc2f960b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562fb57e3b40 con 0x562fb57c8850 2024-05-21T03:24:39.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.223+0000 7fdc2f960b80 10 client.0 _ll_register_callbacks cb 0x562fb56b1900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:24:39.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.223+0000 7fdc1659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fb57c9210 0x562fb57c3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:39.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.223+0000 7fdc1659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fb57c9210 0x562fb57c3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47612/0 (socket says 172.21.15.27:47612) 2024-05-21T03:24:39.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.223+0000 7fdc1659c700 1 -- 172.21.15.27:0/2751363291 learned_addr learned my addr 172.21.15.27:0/2751363291 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:39.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.224+0000 7fdc1ed9d700 1 --2- 172.21.15.27:0/2751363291 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562fb57c8850 0x562fb57c2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:39.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.224+0000 7fdc1659c700 1 -- 172.21.15.27:0/2751363291 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562fb57c8850 msgr2=0x562fb57c2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:39.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.224+0000 7fdc1659c700 1 --2- 172.21.15.27:0/2751363291 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562fb57c8850 0x562fb57c2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.224+0000 7fdc1f59e700 1 --2- 172.21.15.27:0/2751363291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb57ce2d0 0x562fb5850930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:39.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.224+0000 7fdc1659c700 1 -- 172.21.15.27:0/2751363291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb57ce2d0 msgr2=0x562fb5850930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:39.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.224+0000 7fdc1659c700 1 --2- 172.21.15.27:0/2751363291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb57ce2d0 0x562fb5850930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.224+0000 7fdc1659c700 1 -- 172.21.15.27:0/2751363291 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fb5850f20 con 0x562fb57c9210 2024-05-21T03:24:39.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.224+0000 7fdc1ed9d700 1 --2- 172.21.15.27:0/2751363291 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562fb57c8850 0x562fb57c2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:24:39.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.224+0000 7fdc1f59e700 1 --2- 172.21.15.27:0/2751363291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb57ce2d0 0x562fb5850930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:24:39.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.224+0000 7fdc1659c700 1 --2- 172.21.15.27:0/2751363291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fb57c9210 0x562fb57c3070 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fdc080a9750 tx=0x7fdc080a96c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f748679f87f86264 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:39.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.224+0000 7fdc067fc700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:24:39.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.224+0000 7fdc067fc700 1 -- 172.21.15.27:0/2751363291 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc080b0e60 con 0x562fb57c9210 2024-05-21T03:24:39.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.224+0000 7fdc067fc700 1 -- 172.21.15.27:0/2751363291 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fdc080912f0 con 0x562fb57c9210 2024-05-21T03:24:39.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.224+0000 7fdc2f960b80 1 -- 172.21.15.27:0/2751363291 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc180074a0 con 0x562fb57c9210 2024-05-21T03:24:39.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.224+0000 7fdc067fc700 1 -- 172.21.15.27:0/2751363291 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdc08004750 con 0x562fb57c9210 2024-05-21T03:24:39.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.224+0000 7fdc2f960b80 10 client.15483 fetch_fsmap learned FSMap version 29 2024-05-21T03:24:39.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.224+0000 7fdc2f960b80 1 -- 172.21.15.27:0/2751363291 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562fb58520f0 con 0x562fb57c9210 2024-05-21T03:24:39.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.224+0000 7fdc067fc700 1 -- 172.21.15.27:0/2751363291 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc180074a0 con 0x562fb57c9210 2024-05-21T03:24:39.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.225+0000 7fdc067fc700 1 -- 172.21.15.27:0/2751363291 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdc0808fce0 con 0x562fb57c9210 2024-05-21T03:24:39.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.225+0000 7fdc2f960b80 10 client.15483 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T03:24:39.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.225+0000 7fdc2f960b80 10 client.15483 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-21T03:24:39.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.225+0000 7fdc2f960b80 20 client.15483 populate_metadata read hostname 'smithi027' 2024-05-21T03:24:39.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.225+0000 7fdc2f960b80 1 --2- 172.21.15.27:0/2751363291 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x562fb5856ed0 0x562fb5855be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:39.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.225+0000 7fdc2f960b80 4 client.15483 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T03:24:39.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.225+0000 7fdc2f960b80 1 -- 172.21.15.27:0/2751363291 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "status"}) -- 0x562fb5722450 con 0x562fb5856ed0 2024-05-21T03:24:39.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.225+0000 7fdc1ed9d700 1 --2- 172.21.15.27:0/2751363291 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x562fb5856ed0 0x562fb5855be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:39.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.225+0000 7fdc1ed9d700 1 --2- 172.21.15.27:0/2751363291 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x562fb5856ed0 0x562fb5855be0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a96cd615ab2accc server_cookie=389c74adecef81aa in_seq=0 out_seq=0 2024-05-21T03:24:39.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.225+0000 7fdc067fc700 10 client.15483 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:24:39.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.226+0000 7fdc067fc700 1 -- 172.21.15.27:0/2751363291 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562fb5722450 con 0x562fb5856ed0 2024-05-21T03:24:39.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.226+0000 7fdc067fc700 10 client.15483 handle_command_reply: tid=0 2024-05-21T03:24:39.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.226+0000 7fdc2f960b80 1 client.15483 shutdown 2024-05-21T03:24:39.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.226+0000 7fdc2f960b80 1 -- 172.21.15.27:0/2751363291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fb57c9210 msgr2=0x562fb57c3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:39.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.226+0000 7fdc2f960b80 1 --2- 172.21.15.27:0/2751363291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fb57c9210 0x562fb57c3070 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fdc080a9750 tx=0x7fdc080a96c0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.226+0000 7fdc2f960b80 1 -- 172.21.15.27:0/2751363291 shutdown_connections 2024-05-21T03:24:39.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.226+0000 7fdc2f960b80 1 --2- 172.21.15.27:0/2751363291 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x562fb5856ed0 0x562fb5855be0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.226+0000 7fdc2f960b80 1 --2- 172.21.15.27:0/2751363291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb57ce2d0 0x562fb5850930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.226+0000 7fdc2f960b80 1 --2- 172.21.15.27:0/2751363291 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562fb57c8850 0x562fb57c2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.226+0000 7fdc2f960b80 1 --2- 172.21.15.27:0/2751363291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562fb57c9210 0x562fb57c3070 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.226+0000 7fdc2f960b80 1 -- 172.21.15.27:0/2751363291 >> 172.21.15.27:0/2751363291 conn(0x562fb56f41f0 msgr2=0x562fb57cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:39.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.226+0000 7fdc067fc700 0 client.15483 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:24:39.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.226+0000 7fdc2f960b80 1 -- 172.21.15.27:0/2751363291 shutdown_connections 2024-05-21T03:24:39.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.226+0000 7fdc2f960b80 1 -- 172.21.15.27:0/2751363291 wait complete. 2024-05-21T03:24:39.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.226+0000 7fdc2f960b80 20 client.15483 trim_cache size 0 max 16384 2024-05-21T03:24:39.230 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:24:39.230 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "c9ca94be-171f-11ef-bc9a-c7b262605968", 2024-05-21T03:24:39.230 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-05-21T03:24:39.230 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24476, 2024-05-21T03:24:39.230 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-05-21T03:24:39.230 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-05-21T03:24:39.230 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-05-21T03:24:39.230 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 244.96244322300001, 2024-05-21T03:24:39.230 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 29, 2024-05-21T03:24:39.230 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 83, 2024-05-21T03:24:39.231 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 82, 2024-05-21T03:24:39.231 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 258.99030002500001 2024-05-21T03:24:39.231 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:24:39.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.227+0000 7fdc250ca700 1 -- 172.21.15.27:0/614420524 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fdc0c0b2a80 msgr2=0x7fdc0c0b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:39.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.227+0000 7fdc250ca700 1 --2- 172.21.15.27:0/614420524 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fdc0c0b2a80 0x7fdc0c0b4f40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdc2019aa70 tx=0x7fdc1000a380 comp rx=0 tx=0).stop 2024-05-21T03:24:39.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.227+0000 7fdc250ca700 1 -- 172.21.15.27:0/614420524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc2006c630 msgr2=0x7fdc2019fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:39.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.227+0000 7fdc250ca700 1 --2- 172.21.15.27:0/614420524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc2006c630 0x7fdc2019fcb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800c970 tx=0x7fdc0800cc80 comp rx=0 tx=0).stop 2024-05-21T03:24:39.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.227+0000 7fdc250ca700 1 -- 172.21.15.27:0/614420524 shutdown_connections 2024-05-21T03:24:39.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.227+0000 7fdc250ca700 1 --2- 172.21.15.27:0/614420524 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fdc0c0b2a80 0x7fdc0c0b4f40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.227+0000 7fdc250ca700 1 --2- 172.21.15.27:0/614420524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc20109650 0x7fdc20199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.227+0000 7fdc250ca700 1 --2- 172.21.15.27:0/614420524 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc2006bc70 0x7fdc2019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.227+0000 7fdc250ca700 1 --2- 172.21.15.27:0/614420524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc2006c630 0x7fdc2019fcb0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:39.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.227+0000 7fdc250ca700 1 -- 172.21.15.27:0/614420524 >> 172.21.15.27:0/614420524 conn(0x7fdc200782b0 msgr2=0x7fdc20101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:39.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.227+0000 7fdc250ca700 1 -- 172.21.15.27:0/614420524 shutdown_connections 2024-05-21T03:24:39.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:39.227+0000 7fdc250ca700 1 -- 172.21.15.27:0/614420524 wait complete. 2024-05-21T03:24:39.928 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-05-21T03:24:40.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:40 smithi039 ceph-mon[36429]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 938 B/s wr, 12 op/s 2024-05-21T03:24:40.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:40 smithi165 ceph-mon[36269]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 938 B/s wr, 12 op/s 2024-05-21T03:24:40.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:40 smithi027 ceph-mon[32179]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 938 B/s wr, 12 op/s 2024-05-21T03:24:42.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:42 smithi039 ceph-mon[36429]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:24:42.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:42 smithi165 ceph-mon[36269]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:24:42.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:42 smithi027 ceph-mon[32179]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:24:44.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:44 smithi039 ceph-mon[36429]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T03:24:44.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:44 smithi165 ceph-mon[36269]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T03:24:44.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:44 smithi027 ceph-mon[32179]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T03:24:46.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:46 smithi039 ceph-mon[36429]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:24:46.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:46 smithi165 ceph-mon[36269]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:24:46.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:46 smithi027 ceph-mon[32179]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:24:48.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:48 smithi039 ceph-mon[36429]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T03:24:48.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:48 smithi165 ceph-mon[36269]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T03:24:48.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:48 smithi027 ceph-mon[32179]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T03:24:50.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:50 smithi039 ceph-mon[36429]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-21T03:24:50.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:50 smithi165 ceph-mon[36269]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-21T03:24:50.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:50 smithi027 ceph-mon[32179]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-21T03:24:52.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:52 smithi039 ceph-mon[36429]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:24:52.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:52 smithi165 ceph-mon[36269]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:24:52.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:52 smithi027 ceph-mon[32179]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:24:54.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:54 smithi039 ceph-mon[36429]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T03:24:54.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:54 smithi165 ceph-mon[36269]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T03:24:54.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:54 smithi027 ceph-mon[32179]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T03:24:54.929 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-21T03:24:54.929 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-05-21T03:24:54.929 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:24:55.189 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:24:56.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:56 smithi027 ceph-mon[32179]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:24:56.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:56 smithi039 ceph-mon[36429]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:24:56.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:56 smithi165 ceph-mon[36269]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:24:56.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.937+0000 7f84b148b700 1 -- 172.21.15.27:0/14884379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84ac06bc70 msgr2=0x7f84ac06c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:56.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.937+0000 7f84b148b700 1 --2- 172.21.15.27:0/14884379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84ac06bc70 0x7f84ac06c0f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f84a0009990 tx=0x7f84a0009ca0 comp rx=0 tx=0).stop 2024-05-21T03:24:56.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.937+0000 7f84b148b700 1 -- 172.21.15.27:0/14884379 shutdown_connections 2024-05-21T03:24:56.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.937+0000 7f84b148b700 1 --2- 172.21.15.27:0/14884379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84ac06c630 0x7f84ac1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:56.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.937+0000 7f84b148b700 1 --2- 172.21.15.27:0/14884379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84ac1095b0 0x7f84ac109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:56.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.937+0000 7f84b148b700 1 --2- 172.21.15.27:0/14884379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84ac06bc70 0x7f84ac06c0f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:56.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.937+0000 7f84b148b700 1 -- 172.21.15.27:0/14884379 >> 172.21.15.27:0/14884379 conn(0x7f84ac0793f0 msgr2=0x7f84ac079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:56.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.937+0000 7f84b148b700 1 -- 172.21.15.27:0/14884379 shutdown_connections 2024-05-21T03:24:56.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.937+0000 7f84b148b700 1 -- 172.21.15.27:0/14884379 wait complete. 2024-05-21T03:24:56.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.937+0000 7f84b148b700 1 Processor -- start 2024-05-21T03:24:56.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.938+0000 7f84b148b700 1 -- start start 2024-05-21T03:24:56.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.938+0000 7f84b148b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84ac06bc70 0x7f84ac19be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:56.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.938+0000 7f84b148b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84ac06c630 0x7f84ac19c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:56.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.938+0000 7f84b148b700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84ac1095b0 0x7f84ac1a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:56.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.938+0000 7f84b148b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f84ac1136e0 con 0x7f84ac06c630 2024-05-21T03:24:56.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.938+0000 7f84b148b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f84ac113560 con 0x7f84ac06bc70 2024-05-21T03:24:56.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.938+0000 7f84b148b700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f84ac113860 con 0x7f84ac1095b0 2024-05-21T03:24:56.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.938+0000 7f84aa7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84ac06c630 0x7f84ac19c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:56.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.938+0000 7f84aa7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84ac06c630 0x7f84ac19c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37388/0 (socket says 172.21.15.27:37388) 2024-05-21T03:24:56.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.938+0000 7f84aa7fc700 1 -- 172.21.15.27:0/358841941 learned_addr learned my addr 172.21.15.27:0/358841941 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:56.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.938+0000 7f84aaffd700 1 --2- 172.21.15.27:0/358841941 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84ac06bc70 0x7f84ac19be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:56.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.938+0000 7f84ab7fe700 1 --2- 172.21.15.27:0/358841941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84ac1095b0 0x7f84ac1a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:56.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.938+0000 7f84aa7fc700 1 -- 172.21.15.27:0/358841941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84ac1095b0 msgr2=0x7f84ac1a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:56.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.938+0000 7f84aa7fc700 1 --2- 172.21.15.27:0/358841941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84ac1095b0 0x7f84ac1a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:56.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.938+0000 7f84aa7fc700 1 -- 172.21.15.27:0/358841941 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84ac06bc70 msgr2=0x7f84ac19be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:56.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.938+0000 7f84aa7fc700 1 --2- 172.21.15.27:0/358841941 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84ac06bc70 0x7f84ac19be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:56.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.938+0000 7f84aa7fc700 1 -- 172.21.15.27:0/358841941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84ac1a0ea0 con 0x7f84ac06c630 2024-05-21T03:24:56.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.938+0000 7f84ab7fe700 1 --2- 172.21.15.27:0/358841941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84ac1095b0 0x7f84ac1a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:24:56.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.938+0000 7f84aa7fc700 1 --2- 172.21.15.27:0/358841941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84ac06c630 0x7f84ac19c3d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f84a0000c00 tx=0x7f84a00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:56.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.939+0000 7f8493fff700 1 -- 172.21.15.27:0/358841941 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a0027070 con 0x7f84ac06c630 2024-05-21T03:24:56.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.939+0000 7f84b148b700 1 -- 172.21.15.27:0/358841941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84ac1a1130 con 0x7f84ac06c630 2024-05-21T03:24:56.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.939+0000 7f8493fff700 1 -- 172.21.15.27:0/358841941 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84a0003f20 con 0x7f84ac06c630 2024-05-21T03:24:56.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.939+0000 7f8493fff700 1 -- 172.21.15.27:0/358841941 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a0022cf0 con 0x7f84ac06c630 2024-05-21T03:24:56.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.939+0000 7f84b148b700 1 -- 172.21.15.27:0/358841941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84ac1a8a10 con 0x7f84ac06c630 2024-05-21T03:24:56.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.940+0000 7f84b148b700 1 -- 172.21.15.27:0/358841941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84ac06a230 con 0x7f84ac06c630 2024-05-21T03:24:56.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.942+0000 7f8493fff700 1 -- 172.21.15.27:0/358841941 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f84a002a070 con 0x7f84ac06c630 2024-05-21T03:24:56.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.943+0000 7f8493fff700 1 --2- 172.21.15.27:0/358841941 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f84940b2b00 0x7f84940b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:56.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.943+0000 7f84aaffd700 1 --2- 172.21.15.27:0/358841941 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f84940b2b00 0x7f84940b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:56.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.943+0000 7f8493fff700 1 -- 172.21.15.27:0/358841941 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f84a00e3860 con 0x7f84ac06c630 2024-05-21T03:24:56.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.943+0000 7f84aaffd700 1 --2- 172.21.15.27:0/358841941 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f84940b2b00 0x7f84940b4fc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f849c0045b0 tx=0x7f849c00b3c0 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:56.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:56.946+0000 7f8493fff700 1 -- 172.21.15.27:0/358841941 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f84a0013070 con 0x7f84ac06c630 2024-05-21T03:24:57.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:57.210+0000 7f84b148b700 1 -- 172.21.15.27:0/358841941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84ac1a8c40 con 0x7f84ac06c630 2024-05-21T03:24:57.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:57.210+0000 7f8493fff700 1 -- 172.21.15.27:0/358841941 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11924 (secure 0 0 0) 0x7f84a00ab100 con 0x7f84ac06c630 2024-05-21T03:24:57.211 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:24:57.211 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/148854869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":148854869},{"type":"v1","addr":"172.21.15.27:6835","nonce":148854869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/4145544730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":4145544730},{"type":"v1","addr":"172.21.15.27:6837","nonce":4145544730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T03:20:34.055985+0000","modified":"2024-05-21T03:23:58.657936+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24472,"mds_2":24457,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/745486764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":745486764},{"type":"v1","addr":"172.21.15.27:6839","nonce":745486764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6841/3982752351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3982752351},{"type":"v1","addr":"172.21.15.27:6841","nonce":3982752351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/3497521608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3497521608},{"type":"v1","addr":"172.21.15.39:6835","nonce":3497521608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.165:6835/3576910218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3576910218},{"type":"v1","addr":"172.21.15.165:6835","nonce":3576910218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/3673846852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3673846852},{"type":"v1","addr":"172.21.15.39:6837","nonce":3673846852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6837/691356139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":691356139},{"type":"v1","addr":"172.21.15.165:6837","nonce":691356139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6839/4158170814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4158170814},{"type":"v1","addr":"172.21.15.165:6839","nonce":4158170814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6839/287607819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":287607819},{"type":"v1","addr":"172.21.15.39:6839","nonce":287607819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24457,24452,24451,24472]},"id":1}]} 2024-05-21T03:24:57.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:57.213+0000 7f84b148b700 1 -- 172.21.15.27:0/358841941 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f84940b2b00 msgr2=0x7f84940b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:57.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:57.213+0000 7f84b148b700 1 --2- 172.21.15.27:0/358841941 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f84940b2b00 0x7f84940b4fc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f849c0045b0 tx=0x7f849c00b3c0 comp rx=0 tx=0).stop 2024-05-21T03:24:57.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:57.213+0000 7f84b148b700 1 -- 172.21.15.27:0/358841941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84ac06c630 msgr2=0x7f84ac19c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:57.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:57.213+0000 7f84b148b700 1 --2- 172.21.15.27:0/358841941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84ac06c630 0x7f84ac19c3d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f84a0000c00 tx=0x7f84a00047b0 comp rx=0 tx=0).stop 2024-05-21T03:24:57.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:57.213+0000 7f84b148b700 1 -- 172.21.15.27:0/358841941 shutdown_connections 2024-05-21T03:24:57.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:57.213+0000 7f84b148b700 1 --2- 172.21.15.27:0/358841941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84ac1095b0 0x7f84ac1a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:57.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:57.213+0000 7f84b148b700 1 --2- 172.21.15.27:0/358841941 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f84940b2b00 0x7f84940b4fc0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:57.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:57.213+0000 7f84b148b700 1 --2- 172.21.15.27:0/358841941 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84ac06bc70 0x7f84ac19be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:57.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:57.213+0000 7f84b148b700 1 --2- 172.21.15.27:0/358841941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84ac06c630 0x7f84ac19c3d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:57.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:57.213+0000 7f84b148b700 1 -- 172.21.15.27:0/358841941 >> 172.21.15.27:0/358841941 conn(0x7f84ac0793f0 msgr2=0x7f84ac10e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:57.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:57.213+0000 7f84b148b700 1 -- 172.21.15.27:0/358841941 shutdown_connections 2024-05-21T03:24:57.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:57.213+0000 7f84b148b700 1 -- 172.21.15.27:0/358841941 wait complete. 2024-05-21T03:24:57.215 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-05-21T03:24:57.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:57 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/358841941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:24:57.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:57 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/358841941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:24:57.832 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:57 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/358841941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:24:57.862 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-05-21T03:24:57.862 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-05-21T03:24:58.124 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:24:58.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:24:58 smithi039 ceph-mon[36429]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-21T03:24:58.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:24:58 smithi165 ceph-mon[36269]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-21T03:24:58.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:24:58 smithi027 ceph-mon[32179]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-21T03:24:59.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.885+0000 7efdab3fb700 1 -- 172.21.15.27:0/2485771199 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efda4102e00 msgr2=0x7efda410f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:59.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.885+0000 7efdab3fb700 1 --2- 172.21.15.27:0/2485771199 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efda4102e00 0x7efda410f300 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009990 tx=0x7efd9c009ca0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.885+0000 7efdab3fb700 1 -- 172.21.15.27:0/2485771199 shutdown_connections 2024-05-21T03:24:59.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.886+0000 7efdab3fb700 1 --2- 172.21.15.27:0/2485771199 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efda4102e00 0x7efda410f300 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.886+0000 7efdab3fb700 1 --2- 172.21.15.27:0/2485771199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efda41084c0 0x7efda41088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.886+0000 7efdab3fb700 1 --2- 172.21.15.27:0/2485771199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efda4102440 0x7efda41028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.886+0000 7efdab3fb700 1 -- 172.21.15.27:0/2485771199 >> 172.21.15.27:0/2485771199 conn(0x7efda40793e0 msgr2=0x7efda40797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:59.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.886+0000 7efdab3fb700 1 -- 172.21.15.27:0/2485771199 shutdown_connections 2024-05-21T03:24:59.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.886+0000 7efdab3fb700 1 -- 172.21.15.27:0/2485771199 wait complete. 2024-05-21T03:24:59.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.886+0000 7efdab3fb700 1 Processor -- start 2024-05-21T03:24:59.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.886+0000 7efdab3fb700 1 -- start start 2024-05-21T03:24:59.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.887+0000 7efdab3fb700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efda4102440 0x7efda419f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:59.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.887+0000 7efdab3fb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efda4102e00 0x7efda419fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:59.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.887+0000 7efdab3fb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efda41084c0 0x7efda4199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:59.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.887+0000 7efdab3fb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efda4112650 con 0x7efda4102e00 2024-05-21T03:24:59.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.887+0000 7efdab3fb700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efda41124d0 con 0x7efda41084c0 2024-05-21T03:24:59.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.887+0000 7efdab3fb700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efda41127d0 con 0x7efda4102440 2024-05-21T03:24:59.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.887+0000 7efda8996700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efda4102e00 0x7efda419fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:59.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.887+0000 7efda8996700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efda4102e00 0x7efda419fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37406/0 (socket says 172.21.15.27:37406) 2024-05-21T03:24:59.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.887+0000 7efda8996700 1 -- 172.21.15.27:0/3303811221 learned_addr learned my addr 172.21.15.27:0/3303811221 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:59.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.887+0000 7efda9998700 1 --2- 172.21.15.27:0/3303811221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efda41084c0 0x7efda4199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:59.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.887+0000 7efda9197700 1 --2- 172.21.15.27:0/3303811221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efda4102440 0x7efda419f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:59.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.887+0000 7efda8996700 1 -- 172.21.15.27:0/3303811221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efda4102440 msgr2=0x7efda419f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:59.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.887+0000 7efda8996700 1 --2- 172.21.15.27:0/3303811221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efda4102440 0x7efda419f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.887+0000 7efda8996700 1 -- 172.21.15.27:0/3303811221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efda41084c0 msgr2=0x7efda4199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:59.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.887+0000 7efda8996700 1 --2- 172.21.15.27:0/3303811221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efda41084c0 0x7efda4199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.887+0000 7efda8996700 1 -- 172.21.15.27:0/3303811221 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda419a1e0 con 0x7efda4102e00 2024-05-21T03:24:59.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.887+0000 7efda8996700 1 --2- 172.21.15.27:0/3303811221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efda4102e00 0x7efda419fdb0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7efd9400d950 tx=0x7efd9400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:59.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.888+0000 7efd9a7fc700 1 -- 172.21.15.27:0/3303811221 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd94016070 con 0x7efda4102e00 2024-05-21T03:24:59.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.888+0000 7efdab3fb700 1 -- 172.21.15.27:0/3303811221 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda419a4d0 con 0x7efda4102e00 2024-05-21T03:24:59.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.888+0000 7efd9a7fc700 1 -- 172.21.15.27:0/3303811221 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd94009df0 con 0x7efda4102e00 2024-05-21T03:24:59.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.888+0000 7efd9a7fc700 1 -- 172.21.15.27:0/3303811221 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9400f3c0 con 0x7efda4102e00 2024-05-21T03:24:59.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.888+0000 7efdab3fb700 1 -- 172.21.15.27:0/3303811221 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda41a6740 con 0x7efda4102e00 2024-05-21T03:24:59.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.890+0000 7efd9a7fc700 1 -- 172.21.15.27:0/3303811221 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7efd94019070 con 0x7efda4102e00 2024-05-21T03:24:59.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.891+0000 7efd9a7fc700 1 --2- 172.21.15.27:0/3303811221 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7efd900b2b20 0x7efd900b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:59.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.891+0000 7efd9a7fc700 1 -- 172.21.15.27:0/3303811221 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7efd940e23e0 con 0x7efda4102e00 2024-05-21T03:24:59.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.891+0000 7efda9197700 1 --2- 172.21.15.27:0/3303811221 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7efd900b2b20 0x7efd900b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:59.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.891+0000 7efda9197700 1 --2- 172.21.15.27:0/3303811221 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7efd900b2b20 0x7efd900b4fe0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efda4107930 tx=0x7efda00094f0 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:59.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.892+0000 7efdb5c91b80 1 Processor -- start 2024-05-21T03:24:59.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.892+0000 7efdb5c91b80 1 -- start start 2024-05-21T03:24:59.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.892+0000 7efdb5c91b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233618a7b0 0x56233618ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:59.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.892+0000 7efdb5c91b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336185c80 0x562336186100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:59.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.892+0000 7efdb5c91b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562336186640 0x5623361926b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:59.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.892+0000 7efdb5c91b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5623360e3f00 con 0x562336185c80 2024-05-21T03:24:59.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.892+0000 7efdb5c91b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5623360d3aa0 con 0x56233618a7b0 2024-05-21T03:24:59.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.892+0000 7efdb5c91b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5623360d7ae0 con 0x562336186640 2024-05-21T03:24:59.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.892+0000 7efda8996700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336185c80 0x562336186100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:59.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.892+0000 7efda8996700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336185c80 0x562336186100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37412/0 (socket says 172.21.15.27:37412) 2024-05-21T03:24:59.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.892+0000 7efda8996700 1 -- 172.21.15.27:0/2816974224 learned_addr learned my addr 172.21.15.27:0/2816974224 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:59.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.892+0000 7efda9197700 1 --2- 172.21.15.27:0/2816974224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233618a7b0 0x56233618ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:59.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.892+0000 7efda9998700 1 --2- 172.21.15.27:0/2816974224 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562336186640 0x5623361926b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:59.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.892+0000 7efda8996700 1 -- 172.21.15.27:0/2816974224 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562336186640 msgr2=0x5623361926b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:59.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.892+0000 7efda8996700 1 --2- 172.21.15.27:0/2816974224 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562336186640 0x5623361926b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.892+0000 7efda8996700 1 -- 172.21.15.27:0/2816974224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233618a7b0 msgr2=0x56233618ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:59.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.892+0000 7efda8996700 1 --2- 172.21.15.27:0/2816974224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233618a7b0 0x56233618ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.892+0000 7efda8996700 1 -- 172.21.15.27:0/2816974224 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56233607d140 con 0x562336185c80 2024-05-21T03:24:59.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.893+0000 7efda8996700 1 --2- 172.21.15.27:0/2816974224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336185c80 0x562336186100 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7efd940e0130 tx=0x7efd940a9c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e43b96727745196 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:59.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.893+0000 7efd8f7fe700 1 -- 172.21.15.27:0/2816974224 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd94020070 con 0x562336185c80 2024-05-21T03:24:59.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.893+0000 7efd8f7fe700 1 -- 172.21.15.27:0/2816974224 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd940ad900 con 0x562336185c80 2024-05-21T03:24:59.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.893+0000 7efd8f7fe700 1 -- 172.21.15.27:0/2816974224 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd940b1f20 con 0x562336185c80 2024-05-21T03:24:59.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.893+0000 7efdb5c91b80 1 -- 172.21.15.27:0/2816974224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336185c80 msgr2=0x562336186100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:59.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.893+0000 7efdb5c91b80 1 --2- 172.21.15.27:0/2816974224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336185c80 0x562336186100 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7efd940e0130 tx=0x7efd940a9c50 comp rx=0 tx=0).stop 2024-05-21T03:24:59.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.894+0000 7efdb5c91b80 1 -- 172.21.15.27:0/2816974224 shutdown_connections 2024-05-21T03:24:59.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.894+0000 7efdb5c91b80 1 --2- 172.21.15.27:0/2816974224 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562336186640 0x5623361926b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.894+0000 7efdb5c91b80 1 --2- 172.21.15.27:0/2816974224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233618a7b0 0x56233618ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.894+0000 7efdb5c91b80 1 --2- 172.21.15.27:0/2816974224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336185c80 0x562336186100 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.894+0000 7efdb5c91b80 1 -- 172.21.15.27:0/2816974224 >> 172.21.15.27:0/2816974224 conn(0x5623360b13d0 msgr2=0x562336180970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:59.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.894+0000 7efdb5c91b80 1 -- 172.21.15.27:0/2816974224 shutdown_connections 2024-05-21T03:24:59.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.894+0000 7efdb5c91b80 1 -- 172.21.15.27:0/2816974224 wait complete. 2024-05-21T03:24:59.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.894+0000 7efdb5c91b80 1 -- start start 2024-05-21T03:24:59.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.894+0000 7efdb5c91b80 1 Processor -- start 2024-05-21T03:24:59.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.895+0000 7efdb5c91b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562336185c80 0x562336192340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:59.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.895+0000 7efdb5c91b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336186640 0x56233618f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:59.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.895+0000 7efdb5c91b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56233618a7b0 0x56233618fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:59.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.895+0000 7efdb5c91b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562335fda130 con 0x562336186640 2024-05-21T03:24:59.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.895+0000 7efdb5c91b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56233603bf30 con 0x562336185c80 2024-05-21T03:24:59.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.895+0000 7efdb5c91b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562336192cb0 con 0x56233618a7b0 2024-05-21T03:24:59.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.895+0000 7efdb5c91b80 10 client.0 _ll_register_callbacks cb 0x56233606eae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:24:59.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.895+0000 7efda8996700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336186640 0x56233618f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:59.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.895+0000 7efda8996700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336186640 0x56233618f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37414/0 (socket says 172.21.15.27:37414) 2024-05-21T03:24:59.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.895+0000 7efda8996700 1 -- 172.21.15.27:0/1351654263 learned_addr learned my addr 172.21.15.27:0/1351654263 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:59.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.895+0000 7efda9998700 1 --2- 172.21.15.27:0/1351654263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56233618a7b0 0x56233618fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:59.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.895+0000 7efda9197700 1 --2- 172.21.15.27:0/1351654263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562336185c80 0x562336192340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:59.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.895+0000 7efda8996700 1 -- 172.21.15.27:0/1351654263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56233618a7b0 msgr2=0x56233618fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:59.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.895+0000 7efda8996700 1 --2- 172.21.15.27:0/1351654263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56233618a7b0 0x56233618fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.895+0000 7efda8996700 1 -- 172.21.15.27:0/1351654263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562336185c80 msgr2=0x562336192340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:59.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.895+0000 7efda8996700 1 --2- 172.21.15.27:0/1351654263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562336185c80 0x562336192340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.895+0000 7efda8996700 1 -- 172.21.15.27:0/1351654263 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562336023a60 con 0x562336186640 2024-05-21T03:24:59.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.895+0000 7efda8996700 1 --2- 172.21.15.27:0/1351654263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336186640 0x56233618f640 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7efd9400b2f0 tx=0x7efd940dfac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e0fcc54a659cda server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:59.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.895+0000 7efd8cff9700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:24:59.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.895+0000 7efd8cff9700 1 -- 172.21.15.27:0/1351654263 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd940c2070 con 0x562336186640 2024-05-21T03:24:59.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.895+0000 7efd8cff9700 1 -- 172.21.15.27:0/1351654263 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7efd940a97b0 con 0x562336186640 2024-05-21T03:24:59.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.895+0000 7efdb5c91b80 1 -- 172.21.15.27:0/1351654263 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd940ad900 con 0x562336186640 2024-05-21T03:24:59.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.896+0000 7efd8cff9700 1 -- 172.21.15.27:0/1351654263 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efd94020040 con 0x562336186640 2024-05-21T03:24:59.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.896+0000 7efdb5c91b80 10 client.15501 fetch_fsmap learned FSMap version 29 2024-05-21T03:24:59.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.896+0000 7efdb5c91b80 1 -- 172.21.15.27:0/1351654263 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56233608d8f0 con 0x562336186640 2024-05-21T03:24:59.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.896+0000 7efd8cff9700 1 -- 172.21.15.27:0/1351654263 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd940ad900 con 0x562336186640 2024-05-21T03:24:59.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.896+0000 7efd8cff9700 1 -- 172.21.15.27:0/1351654263 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7efd940a87b0 con 0x562336186640 2024-05-21T03:24:59.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.896+0000 7efdb5c91b80 10 client.15501 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T03:24:59.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.896+0000 7efdb5c91b80 10 client.15501 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-21T03:24:59.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.896+0000 7efdb5c91b80 20 client.15501 populate_metadata read hostname 'smithi027' 2024-05-21T03:24:59.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.896+0000 7efdb5c91b80 1 --2- 172.21.15.27:0/1351654263 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x56233619f6b0 0x5623361a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:59.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.896+0000 7efdb5c91b80 4 client.15501 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:24:59.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.896+0000 7efdb5c91b80 1 -- 172.21.15.27:0/1351654263 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623360e5e50 con 0x56233619f6b0 2024-05-21T03:24:59.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.896+0000 7efda9197700 1 --2- 172.21.15.27:0/1351654263 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x56233619f6b0 0x5623361a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:59.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.897+0000 7efda9197700 1 --2- 172.21.15.27:0/1351654263 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x56233619f6b0 0x5623361a1aa0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8284783b83fb030 server_cookie=868474e4aa1d0d13 in_seq=0 out_seq=0 2024-05-21T03:24:59.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.897+0000 7efd8cff9700 10 client.15501 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:24:59.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.897+0000 7efd8cff9700 1 -- 172.21.15.27:0/1351654263 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5623360e5e50 con 0x56233619f6b0 2024-05-21T03:24:59.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.897+0000 7efd8cff9700 10 client.15501 handle_command_reply: tid=0 2024-05-21T03:24:59.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.898+0000 7efdb5c91b80 1 client.15501 shutdown 2024-05-21T03:24:59.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.898+0000 7efdb5c91b80 1 -- 172.21.15.27:0/1351654263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336186640 msgr2=0x56233618f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:59.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.898+0000 7efdb5c91b80 1 --2- 172.21.15.27:0/1351654263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336186640 0x56233618f640 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7efd9400b2f0 tx=0x7efd940dfac0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.898+0000 7efdb5c91b80 1 -- 172.21.15.27:0/1351654263 shutdown_connections 2024-05-21T03:24:59.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.898+0000 7efdb5c91b80 1 --2- 172.21.15.27:0/1351654263 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x56233619f6b0 0x5623361a1aa0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.898+0000 7efdb5c91b80 1 --2- 172.21.15.27:0/1351654263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56233618a7b0 0x56233618fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.898+0000 7efdb5c91b80 1 --2- 172.21.15.27:0/1351654263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562336185c80 0x562336192340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.898+0000 7efdb5c91b80 1 --2- 172.21.15.27:0/1351654263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336186640 0x56233618f640 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.898+0000 7efdb5c91b80 1 -- 172.21.15.27:0/1351654263 >> 172.21.15.27:0/1351654263 conn(0x5623360b13d0 msgr2=0x562336184c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:59.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.898+0000 7efd8cff9700 0 client.15501 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:24:59.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.898+0000 7efdb5c91b80 1 -- 172.21.15.27:0/1351654263 shutdown_connections 2024-05-21T03:24:59.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.898+0000 7efdb5c91b80 1 -- 172.21.15.27:0/1351654263 wait complete. 2024-05-21T03:24:59.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.898+0000 7efdb5c91b80 20 client.15501 trim_cache size 0 max 16384 2024-05-21T03:24:59.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.918+0000 7efdb5c91b80 1 Processor -- start 2024-05-21T03:24:59.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.918+0000 7efdb5c91b80 1 -- start start 2024-05-21T03:24:59.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.919+0000 7efdb5c91b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562336185c80 0x562336189ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:59.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.919+0000 7efdb5c91b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336186640 0x56233617af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:59.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.919+0000 7efdb5c91b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233618a7b0 0x562336188390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:59.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.919+0000 7efdb5c91b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efda41affa0 con 0x562336186640 2024-05-21T03:24:59.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.919+0000 7efdb5c91b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efda41b1ae0 con 0x56233618a7b0 2024-05-21T03:24:59.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.919+0000 7efdb5c91b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efda41b0670 con 0x562336185c80 2024-05-21T03:24:59.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.919+0000 7efda8996700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336186640 0x56233617af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:59.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.919+0000 7efda8996700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336186640 0x56233617af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37426/0 (socket says 172.21.15.27:37426) 2024-05-21T03:24:59.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.919+0000 7efda8996700 1 -- 172.21.15.27:0/3342183245 learned_addr learned my addr 172.21.15.27:0/3342183245 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:59.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.919+0000 7efda9197700 1 --2- 172.21.15.27:0/3342183245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562336185c80 0x562336189ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:59.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.919+0000 7efda8996700 1 -- 172.21.15.27:0/3342183245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562336185c80 msgr2=0x562336189ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:59.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.919+0000 7efda8996700 1 --2- 172.21.15.27:0/3342183245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562336185c80 0x562336189ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.919+0000 7efda9998700 1 --2- 172.21.15.27:0/3342183245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233618a7b0 0x562336188390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:59.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.919+0000 7efda8996700 1 -- 172.21.15.27:0/3342183245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233618a7b0 msgr2=0x562336188390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:59.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.919+0000 7efda8996700 1 --2- 172.21.15.27:0/3342183245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233618a7b0 0x562336188390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.919+0000 7efda8996700 1 -- 172.21.15.27:0/3342183245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56233618dbb0 con 0x562336186640 2024-05-21T03:24:59.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.919+0000 7efda8996700 1 --2- 172.21.15.27:0/3342183245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336186640 0x56233617af00 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7efd94009520 tx=0x7efd94010ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a325a2096d1966 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:59.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.919+0000 7efd8effd700 1 -- 172.21.15.27:0/3342183245 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd940c2070 con 0x562336186640 2024-05-21T03:24:59.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.920+0000 7efd8effd700 1 -- 172.21.15.27:0/3342183245 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd940ca070 con 0x562336186640 2024-05-21T03:24:59.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.920+0000 7efd8effd700 1 -- 172.21.15.27:0/3342183245 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd940b2180 con 0x562336186640 2024-05-21T03:24:59.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.920+0000 7efdb5c91b80 1 -- 172.21.15.27:0/3342183245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336186640 msgr2=0x56233617af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:59.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.920+0000 7efdb5c91b80 1 --2- 172.21.15.27:0/3342183245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336186640 0x56233617af00 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7efd94009520 tx=0x7efd94010ae0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.920+0000 7efdb5c91b80 1 -- 172.21.15.27:0/3342183245 shutdown_connections 2024-05-21T03:24:59.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.920+0000 7efdb5c91b80 1 --2- 172.21.15.27:0/3342183245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233618a7b0 0x562336188390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.920+0000 7efdb5c91b80 1 --2- 172.21.15.27:0/3342183245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562336185c80 0x562336189ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.920+0000 7efdb5c91b80 1 --2- 172.21.15.27:0/3342183245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336186640 0x56233617af00 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.920+0000 7efdb5c91b80 1 -- 172.21.15.27:0/3342183245 >> 172.21.15.27:0/3342183245 conn(0x5623360b13d0 msgr2=0x5623361897e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:59.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.920+0000 7efdb5c91b80 1 -- 172.21.15.27:0/3342183245 shutdown_connections 2024-05-21T03:24:59.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.920+0000 7efdb5c91b80 1 -- 172.21.15.27:0/3342183245 wait complete. 2024-05-21T03:24:59.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.921+0000 7efdb5c91b80 1 -- start start 2024-05-21T03:24:59.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.921+0000 7efdb5c91b80 1 Processor -- start 2024-05-21T03:24:59.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.921+0000 7efdb5c91b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336185c80 0x5623361a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:59.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.921+0000 7efdb5c91b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562336186640 0x5623361a1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:59.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.921+0000 7efdb5c91b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56233618a7b0 0x5623361a1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:59.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.921+0000 7efdb5c91b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efda41b1410 con 0x562336185c80 2024-05-21T03:24:59.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.921+0000 7efdb5c91b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efda41b0d40 con 0x562336186640 2024-05-21T03:24:59.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.921+0000 7efdb5c91b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efda41adf40 con 0x56233618a7b0 2024-05-21T03:24:59.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.921+0000 7efdb5c91b80 10 client.0 _ll_register_callbacks cb 0x56233606eae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:24:59.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.921+0000 7efda9197700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336185c80 0x5623361a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:59.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.921+0000 7efda9197700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336185c80 0x5623361a4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37436/0 (socket says 172.21.15.27:37436) 2024-05-21T03:24:59.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.921+0000 7efda9197700 1 -- 172.21.15.27:0/727968701 learned_addr learned my addr 172.21.15.27:0/727968701 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:24:59.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.921+0000 7efda9998700 1 --2- 172.21.15.27:0/727968701 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56233618a7b0 0x5623361a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:59.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.921+0000 7efda8996700 1 --2- 172.21.15.27:0/727968701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562336186640 0x5623361a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:59.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.922+0000 7efda9197700 1 -- 172.21.15.27:0/727968701 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56233618a7b0 msgr2=0x5623361a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:59.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.922+0000 7efda9197700 1 --2- 172.21.15.27:0/727968701 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56233618a7b0 0x5623361a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.922+0000 7efda9197700 1 -- 172.21.15.27:0/727968701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562336186640 msgr2=0x5623361a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:24:59.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.922+0000 7efda9197700 1 --2- 172.21.15.27:0/727968701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562336186640 0x5623361a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.922+0000 7efda9197700 1 -- 172.21.15.27:0/727968701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623361a1e30 con 0x562336185c80 2024-05-21T03:24:59.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.922+0000 7efda9197700 1 --2- 172.21.15.27:0/727968701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336185c80 0x5623361a4f70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7efda0009d00 tx=0x7efda0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d8312db1aa826c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:24:59.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.922+0000 7efd8cff9700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:24:59.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.922+0000 7efd8cff9700 1 -- 172.21.15.27:0/727968701 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda0017e30 con 0x562336185c80 2024-05-21T03:24:59.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.922+0000 7efd8cff9700 1 -- 172.21.15.27:0/727968701 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7efda0025400 con 0x562336185c80 2024-05-21T03:24:59.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.922+0000 7efdb5c91b80 1 -- 172.21.15.27:0/727968701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd940ca070 con 0x562336185c80 2024-05-21T03:24:59.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.922+0000 7efd8cff9700 1 -- 172.21.15.27:0/727968701 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efda003d050 con 0x562336185c80 2024-05-21T03:24:59.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.922+0000 7efdb5c91b80 10 client.15507 fetch_fsmap learned FSMap version 29 2024-05-21T03:24:59.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.922+0000 7efdb5c91b80 1 -- 172.21.15.27:0/727968701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56233620acd0 con 0x562336185c80 2024-05-21T03:24:59.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.922+0000 7efd8cff9700 1 -- 172.21.15.27:0/727968701 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd940ca070 con 0x562336185c80 2024-05-21T03:24:59.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.923+0000 7efd8cff9700 1 -- 172.21.15.27:0/727968701 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7efda002f720 con 0x562336185c80 2024-05-21T03:24:59.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.923+0000 7efdb5c91b80 10 client.15507 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T03:24:59.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.923+0000 7efdb5c91b80 10 client.15507 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-21T03:24:59.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.923+0000 7efdb5c91b80 20 client.15507 populate_metadata read hostname 'smithi027' 2024-05-21T03:24:59.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.923+0000 7efdb5c91b80 1 --2- 172.21.15.27:0/727968701 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x56233620b450 0x56233620d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:24:59.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.923+0000 7efdb5c91b80 4 client.15507 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-05-21T03:24:59.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.923+0000 7efdb5c91b80 1 -- 172.21.15.27:0/727968701 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x5623360df630 con 0x56233620b450 2024-05-21T03:24:59.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.923+0000 7efda8996700 1 --2- 172.21.15.27:0/727968701 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x56233620b450 0x56233620d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:24:59.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.923+0000 7efda8996700 1 --2- 172.21.15.27:0/727968701 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x56233620b450 0x56233620d870 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5bdc87f267a8b8 server_cookie=fc4f575488a350de in_seq=0 out_seq=0 2024-05-21T03:24:59.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.923+0000 7efd8cff9700 10 client.15507 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:24:59.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.933+0000 7efd8cff9700 1 -- 172.21.15.27:0/727968701 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5623360df630 con 0x56233620b450 2024-05-21T03:24:59.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.933+0000 7efd8cff9700 10 client.15507 handle_command_reply: tid=0 2024-05-21T03:24:59.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.933+0000 7efdb5c91b80 1 client.15507 shutdown 2024-05-21T03:24:59.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.933+0000 7efdb5c91b80 1 -- 172.21.15.27:0/727968701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336185c80 msgr2=0x5623361a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:59.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.933+0000 7efdb5c91b80 1 --2- 172.21.15.27:0/727968701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336185c80 0x5623361a4f70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7efda0009d00 tx=0x7efda0004ad0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.933+0000 7efdb5c91b80 1 -- 172.21.15.27:0/727968701 shutdown_connections 2024-05-21T03:24:59.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.933+0000 7efdb5c91b80 1 --2- 172.21.15.27:0/727968701 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x56233620b450 0x56233620d870 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.933+0000 7efdb5c91b80 1 --2- 172.21.15.27:0/727968701 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56233618a7b0 0x5623361a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.933+0000 7efdb5c91b80 1 --2- 172.21.15.27:0/727968701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562336185c80 0x5623361a4f70 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.933+0000 7efdb5c91b80 1 --2- 172.21.15.27:0/727968701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562336186640 0x5623361a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.933+0000 7efdb5c91b80 1 -- 172.21.15.27:0/727968701 >> 172.21.15.27:0/727968701 conn(0x5623360b13d0 msgr2=0x56233618b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:59.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.933+0000 7efd8cff9700 0 client.15507 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:24:59.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.933+0000 7efdb5c91b80 1 -- 172.21.15.27:0/727968701 shutdown_connections 2024-05-21T03:24:59.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.933+0000 7efdb5c91b80 1 -- 172.21.15.27:0/727968701 wait complete. 2024-05-21T03:24:59.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.933+0000 7efdb5c91b80 20 client.15507 trim_cache size 0 max 16384 2024-05-21T03:24:59.935 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:24:59.935 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 29, 2024-05-21T03:24:59.936 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24476, 2024-05-21T03:24:59.936 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 2, 2024-05-21T03:24:59.936 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-05-21T03:24:59.936 INFO:teuthology.orchestra.run.smithi027.stdout: "454d9796": { 2024-05-21T03:24:59.936 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 2, 2024-05-21T03:24:59.936 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-05-21T03:24:59.936 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-05-21T03:24:59.936 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-05-21T03:24:59.936 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-05-21T03:24:59.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:24:59.936 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-05-21T03:24:59.936 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 150.0, 2024-05-21T03:24:59.936 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-05-21T03:24:59.936 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-05-21T03:24:59.937 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-05-21T03:24:59.937 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-05-21T03:24:59.937 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-05-21T03:24:59.937 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-05-21T03:24:59.937 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:24:59.937 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:24:59.937 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:24:59.937 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:24:59.937 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:24:59.937 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:24:59.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.934+0000 7efdab3fb700 1 -- 172.21.15.27:0/3303811221 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7efd900b2b20 msgr2=0x7efd900b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:59.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.934+0000 7efdab3fb700 1 --2- 172.21.15.27:0/3303811221 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7efd900b2b20 0x7efd900b4fe0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efda4107930 tx=0x7efda00094f0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.934+0000 7efdab3fb700 1 -- 172.21.15.27:0/3303811221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efda4102e00 msgr2=0x7efda419fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:24:59.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.934+0000 7efdab3fb700 1 --2- 172.21.15.27:0/3303811221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efda4102e00 0x7efda419fdb0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7efd9400d950 tx=0x7efd9400dc60 comp rx=0 tx=0).stop 2024-05-21T03:24:59.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.934+0000 7efdab3fb700 1 -- 172.21.15.27:0/3303811221 shutdown_connections 2024-05-21T03:24:59.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.934+0000 7efdab3fb700 1 --2- 172.21.15.27:0/3303811221 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7efd900b2b20 0x7efd900b4fe0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.934+0000 7efdab3fb700 1 --2- 172.21.15.27:0/3303811221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efda41084c0 0x7efda4199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.934+0000 7efdab3fb700 1 --2- 172.21.15.27:0/3303811221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efda4102440 0x7efda419f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.934+0000 7efdab3fb700 1 --2- 172.21.15.27:0/3303811221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efda4102e00 0x7efda419fdb0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:24:59.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.934+0000 7efdab3fb700 1 -- 172.21.15.27:0/3303811221 >> 172.21.15.27:0/3303811221 conn(0x7efda40793e0 msgr2=0x7efda406b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:24:59.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.934+0000 7efdab3fb700 1 -- 172.21.15.27:0/3303811221 shutdown_connections 2024-05-21T03:24:59.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:24:59.934+0000 7efdab3fb700 1 -- 172.21.15.27:0/3303811221 wait complete. 2024-05-21T03:25:00.597 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:00 smithi027 ceph-mon[32179]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:25:00.629 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '454d9796', quiesce took 5.7 seconds. Will release after: 24.3 2024-05-21T03:25:00.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:00 smithi039 ceph-mon[36429]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:25:00.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:00 smithi165 ceph-mon[36269]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T03:25:01.177 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:25:00 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:25:00.907+0000 7f0404727700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T03:25:01.177 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:25:00 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:25:00.907+0000 7f0404727700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T03:25:01.177 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:25:00 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:25:00.907+0000 7f0404727700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T03:25:01.177 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:25:00 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:25:00.907+0000 7f0404727700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T03:25:01.177 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:25:00 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:25:00.907+0000 7f0404727700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T03:25:02.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:02 smithi027 ceph-mon[32179]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:25:02.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:02 smithi027 ceph-mon[32179]: mgrmap e25: x(active, since 60s), standbys: z, y 2024-05-21T03:25:03.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:02 smithi039 ceph-mon[36429]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:25:03.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:02 smithi039 ceph-mon[36429]: mgrmap e25: x(active, since 60s), standbys: z, y 2024-05-21T03:25:03.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:02 smithi165 ceph-mon[36269]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:25:03.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:02 smithi165 ceph-mon[36269]: mgrmap e25: x(active, since 60s), standbys: z, y 2024-05-21T03:25:04.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:04 smithi027 ceph-mon[32179]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-05-21T03:25:05.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:04 smithi039 ceph-mon[36429]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-05-21T03:25:05.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:04 smithi165 ceph-mon[36269]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-05-21T03:25:06.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:06 smithi027 ceph-mon[32179]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-21T03:25:07.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:06 smithi039 ceph-mon[36429]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-21T03:25:07.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:06 smithi165 ceph-mon[36269]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-21T03:25:07.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:07 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:25:07.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:07 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:25:07.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:07 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:25:07.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:07 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:25:08.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:07 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:25:08.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:07 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:25:08.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:07 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:25:08.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:07 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:25:08.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:07 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:25:08.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:07 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:25:08.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:07 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:25:08.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:07 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:25:08.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:08 smithi027 ceph-mon[32179]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-05-21T03:25:09.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:08 smithi039 ceph-mon[36429]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-05-21T03:25:09.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:08 smithi165 ceph-mon[36269]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-05-21T03:25:10.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:10 smithi027 ceph-mon[32179]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-05-21T03:25:11.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:10 smithi039 ceph-mon[36429]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-05-21T03:25:11.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:10 smithi165 ceph-mon[36269]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-05-21T03:25:12.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:12 smithi027 ceph-mon[32179]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-21T03:25:13.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:12 smithi039 ceph-mon[36429]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-21T03:25:13.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:12 smithi165 ceph-mon[36269]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-21T03:25:14.808 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:14 smithi027 ceph-mon[32179]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-05-21T03:25:15.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:14 smithi039 ceph-mon[36429]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-05-21T03:25:15.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:14 smithi165 ceph-mon[36269]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-05-21T03:25:16.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:16 smithi027 ceph-mon[32179]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-21T03:25:17.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:16 smithi039 ceph-mon[36429]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-21T03:25:17.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:16 smithi165 ceph-mon[36269]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-21T03:25:18.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:18 smithi027 ceph-mon[32179]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-21T03:25:19.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:18 smithi039 ceph-mon[36429]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-21T03:25:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:18 smithi165 ceph-mon[36269]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-21T03:25:20.930 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:20 smithi027 ceph-mon[32179]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-21T03:25:21.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:20 smithi039 ceph-mon[36429]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-21T03:25:21.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:20 smithi165 ceph-mon[36269]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-21T03:25:22.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:22 smithi027 ceph-mon[32179]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T03:25:23.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:22 smithi039 ceph-mon[36429]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T03:25:23.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:22 smithi165 ceph-mon[36269]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T03:25:24.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:24 smithi027 ceph-mon[32179]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2024-05-21T03:25:24.930 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:25:25.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:24 smithi039 ceph-mon[36429]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2024-05-21T03:25:25.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:24 smithi165 ceph-mon[36269]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2024-05-21T03:25:25.311 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:25:26.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:26 smithi027 ceph-mon[32179]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:25:27.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:26 smithi039 ceph-mon[36429]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:25:27.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:26 smithi165 ceph-mon[36269]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:25:27.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.218+0000 7fd0f5b26700 1 -- 172.21.15.27:0/1693053225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0f01095b0 msgr2=0x7fd0f0109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:25:27.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.218+0000 7fd0f5b26700 1 --2- 172.21.15.27:0/1693053225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0f01095b0 0x7fd0f0109990 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0009990 tx=0x7fd0e0009ca0 comp rx=0 tx=0).stop 2024-05-21T03:25:27.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.218+0000 7fd0f5b26700 1 -- 172.21.15.27:0/1693053225 shutdown_connections 2024-05-21T03:25:27.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.218+0000 7fd0f5b26700 1 --2- 172.21.15.27:0/1693053225 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0f006bbe0 0x7fd0f01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:27.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.218+0000 7fd0f5b26700 1 --2- 172.21.15.27:0/1693053225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0f01095b0 0x7fd0f0109990 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:27.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.218+0000 7fd0f5b26700 1 --2- 172.21.15.27:0/1693053225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0f006b220 0x7fd0f006b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:27.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.218+0000 7fd0f5b26700 1 -- 172.21.15.27:0/1693053225 >> 172.21.15.27:0/1693053225 conn(0x7fd0f00793f0 msgr2=0x7fd0f0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:25:27.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.218+0000 7fd0f5b26700 1 -- 172.21.15.27:0/1693053225 shutdown_connections 2024-05-21T03:25:27.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.218+0000 7fd0f5b26700 1 -- 172.21.15.27:0/1693053225 wait complete. 2024-05-21T03:25:27.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.220+0000 7fd0f5b26700 1 Processor -- start 2024-05-21T03:25:27.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.220+0000 7fd0f5b26700 1 -- start start 2024-05-21T03:25:27.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.220+0000 7fd0f5b26700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0f006b220 0x7fd0f01a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:27.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.220+0000 7fd0f5b26700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0f006bbe0 0x7fd0f01a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:27.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.220+0000 7fd0f5b26700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0f01095b0 0x7fd0f010e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:27.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.220+0000 7fd0f5b26700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f01136e0 con 0x7fd0f01095b0 2024-05-21T03:25:27.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.220+0000 7fd0f5b26700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f0113560 con 0x7fd0f006bbe0 2024-05-21T03:25:27.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.220+0000 7fd0f5b26700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f0113860 con 0x7fd0f006b220 2024-05-21T03:25:27.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.220+0000 7fd0effff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0f01095b0 0x7fd0f010e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:27.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.220+0000 7fd0effff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0f01095b0 0x7fd0f010e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46076/0 (socket says 172.21.15.27:46076) 2024-05-21T03:25:27.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.220+0000 7fd0effff700 1 -- 172.21.15.27:0/2797417690 learned_addr learned my addr 172.21.15.27:0/2797417690 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:25:27.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.221+0000 7fd0effff700 1 -- 172.21.15.27:0/2797417690 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0f006b220 msgr2=0x7fd0f01a4e70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-21T03:25:27.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.221+0000 7fd0effff700 1 --2- 172.21.15.27:0/2797417690 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0f006b220 0x7fd0f01a4e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:27.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.221+0000 7fd0effff700 1 -- 172.21.15.27:0/2797417690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0f006bbe0 msgr2=0x7fd0f01a53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:25:27.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.221+0000 7fd0eeffd700 1 --2- 172.21.15.27:0/2797417690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0f006bbe0 0x7fd0f01a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:27.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.221+0000 7fd0effff700 1 --2- 172.21.15.27:0/2797417690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0f006bbe0 0x7fd0f01a53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:27.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.221+0000 7fd0effff700 1 -- 172.21.15.27:0/2797417690 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f010eb80 con 0x7fd0f01095b0 2024-05-21T03:25:27.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.221+0000 7fd0effff700 1 --2- 172.21.15.27:0/2797417690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0f01095b0 0x7fd0f010e3f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8007ee0 tx=0x7fd0e800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:25:27.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.221+0000 7fd0ecff9700 1 -- 172.21.15.27:0/2797417690 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e800dbd0 con 0x7fd0f01095b0 2024-05-21T03:25:27.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.221+0000 7fd0f5b26700 1 -- 172.21.15.27:0/2797417690 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f010ee70 con 0x7fd0f01095b0 2024-05-21T03:25:27.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.222+0000 7fd0f5b26700 1 -- 172.21.15.27:0/2797417690 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f01acda0 con 0x7fd0f01095b0 2024-05-21T03:25:27.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.222+0000 7fd0eeffd700 1 --2- 172.21.15.27:0/2797417690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0f006bbe0 0x7fd0f01a53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:25:27.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.222+0000 7fd0ecff9700 1 -- 172.21.15.27:0/2797417690 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0e800dd70 con 0x7fd0f01095b0 2024-05-21T03:25:27.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.222+0000 7fd0ecff9700 1 -- 172.21.15.27:0/2797417690 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8005a30 con 0x7fd0f01095b0 2024-05-21T03:25:27.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.223+0000 7fd0f5b26700 1 -- 172.21.15.27:0/2797417690 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0dc005330 con 0x7fd0f01095b0 2024-05-21T03:25:27.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.225+0000 7fd0ecff9700 1 -- 172.21.15.27:0/2797417690 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7fd0e801a020 con 0x7fd0f01095b0 2024-05-21T03:25:27.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.226+0000 7fd0ecff9700 1 --2- 172.21.15.27:0/2797417690 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fd0d80b2b90 0x7fd0d80b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:27.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.227+0000 7fd0ef7fe700 1 --2- 172.21.15.27:0/2797417690 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fd0d80b2b90 0x7fd0d80b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:27.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.227+0000 7fd0ecff9700 1 -- 172.21.15.27:0/2797417690 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fd0e80e26d0 con 0x7fd0f01095b0 2024-05-21T03:25:27.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.227+0000 7fd0ef7fe700 1 --2- 172.21.15.27:0/2797417690 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fd0d80b2b90 0x7fd0d80b5050 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd0e00098e0 tx=0x7fd0e0005ab0 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:25:27.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.233+0000 7fd0ecff9700 1 -- 172.21.15.27:0/2797417690 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd0e8010070 con 0x7fd0f01095b0 2024-05-21T03:25:27.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.510+0000 7fd0f5b26700 1 -- 172.21.15.27:0/2797417690 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0dc005620 con 0x7fd0f01095b0 2024-05-21T03:25:27.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.511+0000 7fd0ecff9700 1 -- 172.21.15.27:0/2797417690 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11924 (secure 0 0 0) 0x7fd0e80a3e20 con 0x7fd0f01095b0 2024-05-21T03:25:27.510 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:25:27.510 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/148854869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":148854869},{"type":"v1","addr":"172.21.15.27:6835","nonce":148854869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/4145544730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":4145544730},{"type":"v1","addr":"172.21.15.27:6837","nonce":4145544730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T03:20:34.055985+0000","modified":"2024-05-21T03:23:58.657936+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24472,"mds_2":24457,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/745486764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":745486764},{"type":"v1","addr":"172.21.15.27:6839","nonce":745486764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6841/3982752351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3982752351},{"type":"v1","addr":"172.21.15.27:6841","nonce":3982752351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/3497521608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3497521608},{"type":"v1","addr":"172.21.15.39:6835","nonce":3497521608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.165:6835/3576910218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3576910218},{"type":"v1","addr":"172.21.15.165:6835","nonce":3576910218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/3673846852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3673846852},{"type":"v1","addr":"172.21.15.39:6837","nonce":3673846852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6837/691356139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":691356139},{"type":"v1","addr":"172.21.15.165:6837","nonce":691356139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6839/4158170814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4158170814},{"type":"v1","addr":"172.21.15.165:6839","nonce":4158170814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6839/287607819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":287607819},{"type":"v1","addr":"172.21.15.39:6839","nonce":287607819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24457,24452,24451,24472]},"id":1}]} 2024-05-21T03:25:27.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.513+0000 7fd0f5b26700 1 -- 172.21.15.27:0/2797417690 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fd0d80b2b90 msgr2=0x7fd0d80b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:25:27.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.513+0000 7fd0f5b26700 1 --2- 172.21.15.27:0/2797417690 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fd0d80b2b90 0x7fd0d80b5050 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd0e00098e0 tx=0x7fd0e0005ab0 comp rx=0 tx=0).stop 2024-05-21T03:25:27.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.513+0000 7fd0f5b26700 1 -- 172.21.15.27:0/2797417690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0f01095b0 msgr2=0x7fd0f010e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:25:27.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.513+0000 7fd0f5b26700 1 --2- 172.21.15.27:0/2797417690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0f01095b0 0x7fd0f010e3f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8007ee0 tx=0x7fd0e800d3b0 comp rx=0 tx=0).stop 2024-05-21T03:25:27.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.513+0000 7fd0f5b26700 1 -- 172.21.15.27:0/2797417690 shutdown_connections 2024-05-21T03:25:27.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.513+0000 7fd0f5b26700 1 --2- 172.21.15.27:0/2797417690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0f01095b0 0x7fd0f010e3f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:27.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.513+0000 7fd0f5b26700 1 --2- 172.21.15.27:0/2797417690 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0f006b220 0x7fd0f01a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:27.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.513+0000 7fd0f5b26700 1 --2- 172.21.15.27:0/2797417690 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fd0d80b2b90 0x7fd0d80b5050 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:27.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.513+0000 7fd0f5b26700 1 --2- 172.21.15.27:0/2797417690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0f006bbe0 0x7fd0f01a53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:27.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.514+0000 7fd0f5b26700 1 -- 172.21.15.27:0/2797417690 >> 172.21.15.27:0/2797417690 conn(0x7fd0f00793f0 msgr2=0x7fd0f0102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:25:27.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.514+0000 7fd0f5b26700 1 -- 172.21.15.27:0/2797417690 shutdown_connections 2024-05-21T03:25:27.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:27.514+0000 7fd0f5b26700 1 -- 172.21.15.27:0/2797417690 wait complete. 2024-05-21T03:25:27.514 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-05-21T03:25:27.830 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:27 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2797417690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:25:28.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:27 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/2797417690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:25:28.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:27 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/2797417690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:25:28.152 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 454d9796 --release --await' 2024-05-21T03:25:28.152 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 454d9796 --release --await 2024-05-21T03:25:28.446 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:25:28.849 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:28 smithi027 ceph-mon[32179]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-21T03:25:28.920 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:28 smithi039 ceph-mon[36429]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-21T03:25:28.922 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:28 smithi165 ceph-mon[36269]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-21T03:25:30.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.308+0000 7f21a1780700 1 -- 172.21.15.27:0/931448587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f219c103850 msgr2=0x7f219c103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:25:30.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.308+0000 7f21a1780700 1 --2- 172.21.15.27:0/931448587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f219c103850 0x7f219c103c30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f218c0099c0 tx=0x7f218c009cd0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.308+0000 7f21a1780700 1 -- 172.21.15.27:0/931448587 shutdown_connections 2024-05-21T03:25:30.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.308+0000 7f21a1780700 1 --2- 172.21.15.27:0/931448587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f219c108920 0x7f219c1104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.308+0000 7f21a1780700 1 --2- 172.21.15.27:0/931448587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f219c103850 0x7f219c103c30 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.308+0000 7f21a1780700 1 --2- 172.21.15.27:0/931448587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f219c104200 0x7f219c1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.308+0000 7f21a1780700 1 -- 172.21.15.27:0/931448587 >> 172.21.15.27:0/931448587 conn(0x7f219c07a620 msgr2=0x7f219c07aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:25:30.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.309+0000 7f21a1780700 1 -- 172.21.15.27:0/931448587 shutdown_connections 2024-05-21T03:25:30.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.309+0000 7f21a1780700 1 -- 172.21.15.27:0/931448587 wait complete. 2024-05-21T03:25:30.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.310+0000 7f21a1780700 1 Processor -- start 2024-05-21T03:25:30.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.310+0000 7f21a1780700 1 -- start start 2024-05-21T03:25:30.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.310+0000 7f21a1780700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f219c103850 0x7f219c10fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:30.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.310+0000 7f21a1780700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f219c104200 0x7f219c110420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:30.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.310+0000 7f21a1780700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f219c108920 0x7f219c10c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:30.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.310+0000 7f21a1780700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f219c06f770 con 0x7f219c108920 2024-05-21T03:25:30.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.310+0000 7f21a1780700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f219c06f5f0 con 0x7f219c103850 2024-05-21T03:25:30.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.310+0000 7f21a1780700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f219c06f8f0 con 0x7f219c104200 2024-05-21T03:25:30.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.311+0000 7f219b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f219c108920 0x7f219c10c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:30.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.311+0000 7f219affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f219c103850 0x7f219c10fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:30.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.311+0000 7f219affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f219c103850 0x7f219c10fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.27:33934/0 (socket says 172.21.15.27:33934) 2024-05-21T03:25:30.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.311+0000 7f219affd700 1 -- 172.21.15.27:0/2479509335 learned_addr learned my addr 172.21.15.27:0/2479509335 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:25:30.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.311+0000 7f219a7fc700 1 --2- 172.21.15.27:0/2479509335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f219c104200 0x7f219c110420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:30.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.311+0000 7f219affd700 1 -- 172.21.15.27:0/2479509335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f219c104200 msgr2=0x7f219c110420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:25:30.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.311+0000 7f219affd700 1 --2- 172.21.15.27:0/2479509335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f219c104200 0x7f219c110420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.311+0000 7f219affd700 1 -- 172.21.15.27:0/2479509335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f219c108920 msgr2=0x7f219c10c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:25:30.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.311+0000 7f219affd700 1 --2- 172.21.15.27:0/2479509335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f219c108920 0x7f219c10c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.311+0000 7f219affd700 1 -- 172.21.15.27:0/2479509335 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219c10ca60 con 0x7f219c103850 2024-05-21T03:25:30.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.311+0000 7f219a7fc700 1 --2- 172.21.15.27:0/2479509335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f219c104200 0x7f219c110420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:25:30.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.311+0000 7f219affd700 1 --2- 172.21.15.27:0/2479509335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f219c103850 0x7f219c10fee0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f218c0098e0 tx=0x7f218c0038b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:25:30.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.312+0000 7f2183fff700 1 -- 172.21.15.27:0/2479509335 <== mon.2 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c027070 con 0x7f219c103850 2024-05-21T03:25:30.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.312+0000 7f21a1780700 1 -- 172.21.15.27:0/2479509335 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f219c10ccf0 con 0x7f219c103850 2024-05-21T03:25:30.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.312+0000 7f2183fff700 1 -- 172.21.15.27:0/2479509335 <== mon.2 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f218c004020 con 0x7f219c103850 2024-05-21T03:25:30.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.312+0000 7f2183fff700 1 -- 172.21.15.27:0/2479509335 <== mon.2 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c022c40 con 0x7f219c103850 2024-05-21T03:25:30.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.312+0000 7f21a1780700 1 -- 172.21.15.27:0/2479509335 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f219c075770 con 0x7f219c103850 2024-05-21T03:25:30.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.316+0000 7f21ac016b80 1 Processor -- start 2024-05-21T03:25:30.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.316+0000 7f2183fff700 1 -- 172.21.15.27:0/2479509335 <== mon.2 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f218c02a070 con 0x7f219c103850 2024-05-21T03:25:30.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.316+0000 7f21ac016b80 1 -- start start 2024-05-21T03:25:30.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.316+0000 7f21ac016b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55777d2adf30 0x55777d2ae310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:30.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.316+0000 7f21ac016b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55777d2a9400 0x55777d2a9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:30.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.316+0000 7f21ac016b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55777d2a9dc0 0x55777d2b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:30.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.316+0000 7f21ac016b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55777d1fee70 con 0x55777d2a9dc0 2024-05-21T03:25:30.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.316+0000 7f21ac016b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55777d1eea10 con 0x55777d2a9400 2024-05-21T03:25:30.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.316+0000 7f21ac016b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55777d1f2a50 con 0x55777d2adf30 2024-05-21T03:25:30.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.317+0000 7f219a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55777d2a9dc0 0x55777d2b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:30.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.317+0000 7f219a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55777d2a9dc0 0x55777d2b5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46088/0 (socket says 172.21.15.27:46088) 2024-05-21T03:25:30.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.317+0000 7f219a7fc700 1 -- 172.21.15.27:0/4080477375 learned_addr learned my addr 172.21.15.27:0/4080477375 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:25:30.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.317+0000 7f219b7fe700 1 --2- 172.21.15.27:0/4080477375 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55777d2adf30 0x55777d2ae310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.317+0000 7f219affd700 1 --2- 172.21.15.27:0/4080477375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55777d2a9400 0x55777d2a9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.317+0000 7f219a7fc700 1 -- 172.21.15.27:0/4080477375 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55777d2adf30 msgr2=0x55777d2ae310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:25:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.317+0000 7f219a7fc700 1 --2- 172.21.15.27:0/4080477375 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55777d2adf30 0x55777d2ae310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.317+0000 7f219a7fc700 1 -- 172.21.15.27:0/4080477375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55777d2a9400 msgr2=0x55777d2a9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:25:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.317+0000 7f219a7fc700 1 --2- 172.21.15.27:0/4080477375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55777d2a9400 0x55777d2a9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.317+0000 7f219a7fc700 1 -- 172.21.15.27:0/4080477375 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55777d1980b0 con 0x55777d2a9dc0 2024-05-21T03:25:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.317+0000 7f2183fff700 1 --2- 172.21.15.27:0/2479509335 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f21840c3bb0 0x7f21840c6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.317+0000 7f219a7fc700 1 --2- 172.21.15.27:0/4080477375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55777d2a9dc0 0x55777d2b5e30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f219000be00 tx=0x7f219000f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c4fd03a26aa8fd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:25:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.318+0000 7f219b7fe700 1 --2- 172.21.15.27:0/4080477375 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55777d2adf30 0x55777d2ae310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:25:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.318+0000 7f21817fa700 1 -- 172.21.15.27:0/4080477375 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219000fe50 con 0x55777d2a9dc0 2024-05-21T03:25:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.318+0000 7f21817fa700 1 -- 172.21.15.27:0/4080477375 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21900044e0 con 0x55777d2a9dc0 2024-05-21T03:25:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.318+0000 7f2183fff700 1 -- 172.21.15.27:0/2479509335 <== mon.2 v2:172.21.15.39:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f218c0e3550 con 0x7f219c103850 2024-05-21T03:25:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.318+0000 7f219b7fe700 1 --2- 172.21.15.27:0/2479509335 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f21840c3bb0 0x7f21840c6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.318+0000 7f21817fa700 1 -- 172.21.15.27:0/4080477375 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21900116a0 con 0x55777d2a9dc0 2024-05-21T03:25:30.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.319+0000 7f21ac016b80 1 -- 172.21.15.27:0/4080477375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55777d2a9dc0 msgr2=0x55777d2b5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:25:30.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.319+0000 7f21ac016b80 1 --2- 172.21.15.27:0/4080477375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55777d2a9dc0 0x55777d2b5e30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f219000be00 tx=0x7f219000f640 comp rx=0 tx=0).stop 2024-05-21T03:25:30.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.319+0000 7f219b7fe700 1 --2- 172.21.15.27:0/2479509335 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f21840c3bb0 0x7f21840c6070 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x55777d29d580 tx=0x7f219400a590 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:25:30.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.319+0000 7f21ac016b80 1 -- 172.21.15.27:0/4080477375 shutdown_connections 2024-05-21T03:25:30.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.319+0000 7f21ac016b80 1 --2- 172.21.15.27:0/4080477375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55777d2a9dc0 0x55777d2b5e30 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.319+0000 7f21ac016b80 1 --2- 172.21.15.27:0/4080477375 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55777d2adf30 0x55777d2ae310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.319+0000 7f21ac016b80 1 --2- 172.21.15.27:0/4080477375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55777d2a9400 0x55777d2a9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.319+0000 7f21ac016b80 1 -- 172.21.15.27:0/4080477375 >> 172.21.15.27:0/4080477375 conn(0x55777d1cc340 msgr2=0x55777d29b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:25:30.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.319+0000 7f21ac016b80 1 -- 172.21.15.27:0/4080477375 shutdown_connections 2024-05-21T03:25:30.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.320+0000 7f21ac016b80 1 -- 172.21.15.27:0/4080477375 wait complete. 2024-05-21T03:25:30.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.320+0000 7f21ac016b80 1 -- start start 2024-05-21T03:25:30.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.320+0000 7f21ac016b80 1 Processor -- start 2024-05-21T03:25:30.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.320+0000 7f21ac016b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55777d2a9400 0x55777d2b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:30.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.320+0000 7f21ac016b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55777d2a9dc0 0x55777d2b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:30.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.320+0000 7f21ac016b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55777d2adf30 0x55777d2b3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:30.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.320+0000 7f21ac016b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55777d0f50a0 con 0x55777d2adf30 2024-05-21T03:25:30.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.320+0000 7f21ac016b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55777d156ea0 con 0x55777d2a9400 2024-05-21T03:25:30.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.320+0000 7f21ac016b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55777d2b64e0 con 0x55777d2a9dc0 2024-05-21T03:25:30.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.320+0000 7f21ac016b80 10 client.0 _ll_register_callbacks cb 0x55777d189a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:25:30.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.321+0000 7f219affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55777d2a9400 0x55777d2b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:30.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.321+0000 7f219affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55777d2a9400 0x55777d2b59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.27:33954/0 (socket says 172.21.15.27:33954) 2024-05-21T03:25:30.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.321+0000 7f219affd700 1 -- 172.21.15.27:0/718891375 learned_addr learned my addr 172.21.15.27:0/718891375 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:25:30.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.321+0000 7f219a7fc700 1 --2- 172.21.15.27:0/718891375 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55777d2a9dc0 0x55777d2b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:30.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.321+0000 7f219b7fe700 1 --2- 172.21.15.27:0/718891375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55777d2adf30 0x55777d2b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:30.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.321+0000 7f219affd700 1 -- 172.21.15.27:0/718891375 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55777d2a9dc0 msgr2=0x55777d2b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:25:30.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.321+0000 7f219affd700 1 --2- 172.21.15.27:0/718891375 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55777d2a9dc0 0x55777d2b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.321+0000 7f219affd700 1 -- 172.21.15.27:0/718891375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55777d2adf30 msgr2=0x55777d2b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:25:30.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.321+0000 7f219affd700 1 --2- 172.21.15.27:0/718891375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55777d2adf30 0x55777d2b3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.321+0000 7f219affd700 1 -- 172.21.15.27:0/718891375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55777d13e9d0 con 0x55777d2a9400 2024-05-21T03:25:30.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.321+0000 7f219b7fe700 1 --2- 172.21.15.27:0/718891375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55777d2adf30 0x55777d2b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:25:30.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.322+0000 7f219affd700 1 --2- 172.21.15.27:0/718891375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55777d2a9400 0x55777d2b59e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f218c00c010 tx=0x7f218c009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ab2e47fe2a43c19 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:25:30.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.322+0000 7f217eff5700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-21T03:25:30.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.322+0000 7f217eff5700 1 -- 172.21.15.27:0/718891375 <== mon.2 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c0aa3e0 con 0x55777d2a9400 2024-05-21T03:25:30.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.322+0000 7f21ac016b80 1 -- 172.21.15.27:0/718891375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21900044e0 con 0x55777d2a9400 2024-05-21T03:25:30.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.322+0000 7f217eff5700 1 -- 172.21.15.27:0/718891375 <== mon.2 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f218c0afb50 con 0x55777d2a9400 2024-05-21T03:25:30.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.322+0000 7f217eff5700 1 -- 172.21.15.27:0/718891375 <== mon.2 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f218c0a9d80 con 0x55777d2a9400 2024-05-21T03:25:30.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.322+0000 7f21ac016b80 10 client.25142 fetch_fsmap learned FSMap version 29 2024-05-21T03:25:30.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.322+0000 7f21ac016b80 1 -- 172.21.15.27:0/718891375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55777d1a8860 con 0x55777d2a9400 2024-05-21T03:25:30.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.322+0000 7f217eff5700 1 -- 172.21.15.27:0/718891375 <== mon.2 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21900044e0 con 0x55777d2a9400 2024-05-21T03:25:30.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.323+0000 7f217eff5700 1 -- 172.21.15.27:0/718891375 <== mon.2 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f218c0e0da0 con 0x55777d2a9400 2024-05-21T03:25:30.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.323+0000 7f21ac016b80 10 client.25142 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T03:25:30.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.323+0000 7f21ac016b80 10 client.25142 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-21T03:25:30.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.323+0000 7f21ac016b80 20 client.25142 populate_metadata read hostname 'smithi027' 2024-05-21T03:25:30.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.323+0000 7f21ac016b80 1 --2- 172.21.15.27:0/718891375 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55777d2c2e90 0x55777d2c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:30.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.323+0000 7f21ac016b80 4 client.25142 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:25:30.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.323+0000 7f21ac016b80 1 -- 172.21.15.27:0/718891375 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55777d200dc0 con 0x55777d2c2e90 2024-05-21T03:25:30.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.323+0000 7f219a7fc700 1 --2- 172.21.15.27:0/718891375 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55777d2c2e90 0x55777d2c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:30.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.323+0000 7f217eff5700 10 client.25142 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:25:30.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.323+0000 7f219a7fc700 1 --2- 172.21.15.27:0/718891375 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55777d2c2e90 0x55777d2c52b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=817486b18c9e1424 server_cookie=2d77cbfc7203de6d in_seq=0 out_seq=0 2024-05-21T03:25:30.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.324+0000 7f217eff5700 1 -- 172.21.15.27:0/718891375 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55777d200dc0 con 0x55777d2c2e90 2024-05-21T03:25:30.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.324+0000 7f217eff5700 10 client.25142 handle_command_reply: tid=0 2024-05-21T03:25:30.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.324+0000 7f21ac016b80 1 client.25142 shutdown 2024-05-21T03:25:30.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.325+0000 7f21ac016b80 1 -- 172.21.15.27:0/718891375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55777d2a9400 msgr2=0x55777d2b59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:25:30.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.325+0000 7f21ac016b80 1 --2- 172.21.15.27:0/718891375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55777d2a9400 0x55777d2b59e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f218c00c010 tx=0x7f218c009cd0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.325+0000 7f21ac016b80 1 -- 172.21.15.27:0/718891375 shutdown_connections 2024-05-21T03:25:30.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.325+0000 7f21ac016b80 1 --2- 172.21.15.27:0/718891375 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55777d2c2e90 0x55777d2c52b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.325+0000 7f21ac016b80 1 --2- 172.21.15.27:0/718891375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55777d2adf30 0x55777d2b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.325+0000 7f21ac016b80 1 --2- 172.21.15.27:0/718891375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55777d2a9400 0x55777d2b59e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.325+0000 7f217eff5700 0 client.25142 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:25:30.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.325+0000 7f21ac016b80 1 --2- 172.21.15.27:0/718891375 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55777d2a9dc0 0x55777d2b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.325+0000 7f21ac016b80 1 -- 172.21.15.27:0/718891375 >> 172.21.15.27:0/718891375 conn(0x55777d1cc340 msgr2=0x55777d2a83f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:25:30.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.325+0000 7f21ac016b80 1 -- 172.21.15.27:0/718891375 shutdown_connections 2024-05-21T03:25:30.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.325+0000 7f21ac016b80 1 -- 172.21.15.27:0/718891375 wait complete. 2024-05-21T03:25:30.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.325+0000 7f21ac016b80 20 client.25142 trim_cache size 0 max 16384 2024-05-21T03:25:30.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.366+0000 7f21ac016b80 1 Processor -- start 2024-05-21T03:25:30.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.366+0000 7f21ac016b80 1 -- start start 2024-05-21T03:25:30.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.366+0000 7f21ac016b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55777d2a9400 0x55777d295e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.366+0000 7f21ac016b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55777d2a9dc0 0x55777d2aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.366+0000 7f21ac016b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55777d2adf30 0x55777d2af110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.366+0000 7f21ac016b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f219c1a3300 con 0x55777d2adf30 2024-05-21T03:25:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.366+0000 7f21ac016b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f219c1a4e40 con 0x55777d2a9400 2024-05-21T03:25:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.366+0000 7f21ac016b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f219c1a39d0 con 0x55777d2a9dc0 2024-05-21T03:25:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.366+0000 7f219a7fc700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55777d2a9dc0 0x55777d2aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.366+0000 7f219a7fc700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55777d2a9dc0 0x55777d2aebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.27:49928/0 (socket says 172.21.15.27:49928) 2024-05-21T03:25:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.366+0000 7f219a7fc700 1 -- 172.21.15.27:0/969870072 learned_addr learned my addr 172.21.15.27:0/969870072 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:25:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.367+0000 7f219affd700 1 --2- 172.21.15.27:0/969870072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55777d2a9400 0x55777d295e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.367+0000 7f219b7fe700 1 --2- 172.21.15.27:0/969870072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55777d2adf30 0x55777d2af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.367+0000 7f219a7fc700 1 -- 172.21.15.27:0/969870072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55777d2a9400 msgr2=0x55777d295e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:25:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.367+0000 7f219a7fc700 1 --2- 172.21.15.27:0/969870072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55777d2a9400 0x55777d295e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.367+0000 7f219a7fc700 1 -- 172.21.15.27:0/969870072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55777d2adf30 msgr2=0x55777d2af110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:25:30.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.367+0000 7f219a7fc700 1 --2- 172.21.15.27:0/969870072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55777d2adf30 0x55777d2af110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.367+0000 7f219a7fc700 1 -- 172.21.15.27:0/969870072 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55777d2b1210 con 0x55777d2a9dc0 2024-05-21T03:25:30.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.367+0000 7f219b7fe700 1 --2- 172.21.15.27:0/969870072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55777d2adf30 0x55777d2af110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:25:30.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.367+0000 7f219a7fc700 1 --2- 172.21.15.27:0/969870072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55777d2a9dc0 0x55777d2aebd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2190003ca0 tx=0x7f2190003d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee55cb934bfd19bf server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:25:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.367+0000 7f2180ff9700 1 -- 172.21.15.27:0/969870072 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2190003880 con 0x55777d2a9dc0 2024-05-21T03:25:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.368+0000 7f2180ff9700 1 -- 172.21.15.27:0/969870072 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2190003a20 con 0x55777d2a9dc0 2024-05-21T03:25:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.368+0000 7f2180ff9700 1 -- 172.21.15.27:0/969870072 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2190004dc0 con 0x55777d2a9dc0 2024-05-21T03:25:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.368+0000 7f21ac016b80 1 -- 172.21.15.27:0/969870072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55777d2a9dc0 msgr2=0x55777d2aebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:25:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.368+0000 7f21ac016b80 1 --2- 172.21.15.27:0/969870072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55777d2a9dc0 0x55777d2aebd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2190003ca0 tx=0x7f2190003d00 comp rx=0 tx=0).stop 2024-05-21T03:25:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.369+0000 7f21ac016b80 1 -- 172.21.15.27:0/969870072 shutdown_connections 2024-05-21T03:25:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.369+0000 7f21ac016b80 1 --2- 172.21.15.27:0/969870072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55777d2adf30 0x55777d2af110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.369+0000 7f21ac016b80 1 --2- 172.21.15.27:0/969870072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55777d2a9400 0x55777d295e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.369+0000 7f21ac016b80 1 --2- 172.21.15.27:0/969870072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55777d2a9dc0 0x55777d2aebd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.369+0000 7f21ac016b80 1 -- 172.21.15.27:0/969870072 >> 172.21.15.27:0/969870072 conn(0x55777d1cc340 msgr2=0x55777d2a8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:25:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.369+0000 7f21ac016b80 1 -- 172.21.15.27:0/969870072 shutdown_connections 2024-05-21T03:25:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.369+0000 7f21ac016b80 1 -- 172.21.15.27:0/969870072 wait complete. 2024-05-21T03:25:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.369+0000 7f21ac016b80 1 -- start start 2024-05-21T03:25:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.369+0000 7f21ac016b80 1 Processor -- start 2024-05-21T03:25:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.370+0000 7f21ac016b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55777d2a9400 0x55777d2bbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.370+0000 7f21ac016b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55777d2a9dc0 0x55777d2aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.370+0000 7f21ac016b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55777d2adf30 0x55777d2aafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.370+0000 7f21ac016b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f219c1a4770 con 0x55777d2a9dc0 2024-05-21T03:25:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.370+0000 7f21ac016b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f219c1a40a0 con 0x55777d2a9400 2024-05-21T03:25:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.370+0000 7f21ac016b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f219c1a12a0 con 0x55777d2adf30 2024-05-21T03:25:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.370+0000 7f219affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55777d2a9400 0x55777d2bbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.370+0000 7f219affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55777d2a9400 0x55777d2bbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.27:33970/0 (socket says 172.21.15.27:33970) 2024-05-21T03:25:30.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.370+0000 7f219affd700 1 -- 172.21.15.27:0/2319138344 learned_addr learned my addr 172.21.15.27:0/2319138344 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:25:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.370+0000 7f21ac016b80 10 client.0 _ll_register_callbacks cb 0x55777d189a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:25:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.370+0000 7f219b7fe700 1 --2- 172.21.15.27:0/2319138344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55777d2adf30 0x55777d2aafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.370+0000 7f219affd700 1 -- 172.21.15.27:0/2319138344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55777d2adf30 msgr2=0x55777d2aafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:25:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.370+0000 7f219affd700 1 --2- 172.21.15.27:0/2319138344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55777d2adf30 0x55777d2aafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.370+0000 7f219affd700 1 -- 172.21.15.27:0/2319138344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55777d2a9dc0 msgr2=0x55777d2aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:25:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.370+0000 7f219affd700 1 --2- 172.21.15.27:0/2319138344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55777d2a9dc0 0x55777d2aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.370+0000 7f219affd700 1 -- 172.21.15.27:0/2319138344 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55777d2ab5a0 con 0x55777d2a9400 2024-05-21T03:25:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.370+0000 7f219affd700 1 --2- 172.21.15.27:0/2319138344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55777d2a9400 0x55777d2bbcb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f218c0e11c0 tx=0x7f218c0afe10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e39cdaad0f45516 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:25:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.370+0000 7f217eff5700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-21T03:25:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.371+0000 7f217eff5700 1 -- 172.21.15.27:0/2319138344 <== mon.2 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c0a9f60 con 0x55777d2a9400 2024-05-21T03:25:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.371+0000 7f21ac016b80 1 -- 172.21.15.27:0/2319138344 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2190003a20 con 0x55777d2a9400 2024-05-21T03:25:30.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.371+0000 7f217eff5700 1 -- 172.21.15.27:0/2319138344 <== mon.2 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f218c003d80 con 0x55777d2a9400 2024-05-21T03:25:30.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.371+0000 7f217eff5700 1 -- 172.21.15.27:0/2319138344 <== mon.2 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f218c0aa3e0 con 0x55777d2a9400 2024-05-21T03:25:30.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.371+0000 7f21ac016b80 10 client.25148 fetch_fsmap learned FSMap version 29 2024-05-21T03:25:30.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.371+0000 7f21ac016b80 1 -- 172.21.15.27:0/2319138344 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55777d2ac810 con 0x55777d2a9400 2024-05-21T03:25:30.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.372+0000 7f217eff5700 1 -- 172.21.15.27:0/2319138344 <== mon.2 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f218c0e0a00 con 0x55777d2a9400 2024-05-21T03:25:30.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.372+0000 7f217eff5700 1 -- 172.21.15.27:0/2319138344 <== mon.2 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f218c0a9870 con 0x55777d2a9400 2024-05-21T03:25:30.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.372+0000 7f21ac016b80 10 client.25148 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T03:25:30.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.372+0000 7f21ac016b80 10 client.25148 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-21T03:25:30.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.372+0000 7f21ac016b80 20 client.25148 populate_metadata read hostname 'smithi027' 2024-05-21T03:25:30.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.372+0000 7f21ac016b80 1 --2- 172.21.15.27:0/2319138344 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55777d2c2e90 0x55777d330fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:30.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.372+0000 7f21ac016b80 4 client.25148 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "454d9796", "release": true, "await": true}] 2024-05-21T03:25:30.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.372+0000 7f21ac016b80 1 -- 172.21.15.27:0/2319138344 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "quiesce db", "set_id": "454d9796", "release": true, "await": true}) -- 0x7f2190003a20 con 0x55777d2c2e90 2024-05-21T03:25:30.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.372+0000 7f219a7fc700 1 --2- 172.21.15.27:0/2319138344 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55777d2c2e90 0x55777d330fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:30.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.373+0000 7f217eff5700 10 client.25148 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:25:30.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.373+0000 7f219a7fc700 1 --2- 172.21.15.27:0/2319138344 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55777d2c2e90 0x55777d330fe0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9f9b9df2c503720 server_cookie=4c2bd388f1eb9555 in_seq=0 out_seq=0 2024-05-21T03:25:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.375+0000 7f217eff5700 1 -- 172.21.15.27:0/2319138344 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7f2190003a20 con 0x55777d2c2e90 2024-05-21T03:25:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.375+0000 7f217eff5700 10 client.25148 handle_command_reply: tid=0 2024-05-21T03:25:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.375+0000 7f21ac016b80 1 client.25148 shutdown 2024-05-21T03:25:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.375+0000 7f21ac016b80 1 -- 172.21.15.27:0/2319138344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55777d2a9400 msgr2=0x55777d2bbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:25:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.375+0000 7f21ac016b80 1 --2- 172.21.15.27:0/2319138344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55777d2a9400 0x55777d2bbcb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f218c0e11c0 tx=0x7f218c0afe10 comp rx=0 tx=0).stop 2024-05-21T03:25:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.375+0000 7f21ac016b80 1 -- 172.21.15.27:0/2319138344 shutdown_connections 2024-05-21T03:25:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.375+0000 7f21ac016b80 1 --2- 172.21.15.27:0/2319138344 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55777d2c2e90 0x55777d330fe0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.375+0000 7f21ac016b80 1 --2- 172.21.15.27:0/2319138344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55777d2adf30 0x55777d2aafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.375+0000 7f21ac016b80 1 --2- 172.21.15.27:0/2319138344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55777d2a9400 0x55777d2bbcb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.375+0000 7f21ac016b80 1 --2- 172.21.15.27:0/2319138344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55777d2a9dc0 0x55777d2aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.375+0000 7f217eff5700 0 client.25148 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:25:30.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.375+0000 7f21ac016b80 1 -- 172.21.15.27:0/2319138344 >> 172.21.15.27:0/2319138344 conn(0x55777d1cc340 msgr2=0x55777d2afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:25:30.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.375+0000 7f21ac016b80 1 -- 172.21.15.27:0/2319138344 shutdown_connections 2024-05-21T03:25:30.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.375+0000 7f21ac016b80 1 -- 172.21.15.27:0/2319138344 wait complete. 2024-05-21T03:25:30.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.375+0000 7f21ac016b80 20 client.25148 trim_cache size 0 max 16384 2024-05-21T03:25:30.377 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:25:30.377 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 29, 2024-05-21T03:25:30.377 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24476, 2024-05-21T03:25:30.377 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 4, 2024-05-21T03:25:30.377 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-05-21T03:25:30.377 INFO:teuthology.orchestra.run.smithi027.stdout: "454d9796": { 2024-05-21T03:25:30.377 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 4, 2024-05-21T03:25:30.377 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-05-21T03:25:30.377 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-05-21T03:25:30.378 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-05-21T03:25:30.378 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-05-21T03:25:30.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:25:30.378 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-05-21T03:25:30.378 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 150.0, 2024-05-21T03:25:30.378 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-05-21T03:25:30.378 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-05-21T03:25:30.378 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-05-21T03:25:30.378 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-05-21T03:25:30.378 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-05-21T03:25:30.378 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-05-21T03:25:30.378 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:25:30.378 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:25:30.379 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:25:30.379 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:25:30.379 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:25:30.379 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:25:30.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.376+0000 7f21a1780700 1 -- 172.21.15.27:0/2479509335 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f21840c3bb0 msgr2=0x7f21840c6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:25:30.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.376+0000 7f21a1780700 1 --2- 172.21.15.27:0/2479509335 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f21840c3bb0 0x7f21840c6070 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x55777d29d580 tx=0x7f219400a590 comp rx=0 tx=0).stop 2024-05-21T03:25:30.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.376+0000 7f21a1780700 1 -- 172.21.15.27:0/2479509335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f219c103850 msgr2=0x7f219c10fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:25:30.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.376+0000 7f21a1780700 1 --2- 172.21.15.27:0/2479509335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f219c103850 0x7f219c10fee0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f218c0098e0 tx=0x7f218c0038b0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.377+0000 7f21a1780700 1 -- 172.21.15.27:0/2479509335 shutdown_connections 2024-05-21T03:25:30.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.377+0000 7f21a1780700 1 --2- 172.21.15.27:0/2479509335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f219c108920 0x7f219c10c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.377+0000 7f21a1780700 1 --2- 172.21.15.27:0/2479509335 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f21840c3bb0 0x7f21840c6070 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.377+0000 7f21a1780700 1 --2- 172.21.15.27:0/2479509335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f219c103850 0x7f219c10fee0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.377+0000 7f21a1780700 1 --2- 172.21.15.27:0/2479509335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f219c104200 0x7f219c110420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:30.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.377+0000 7f21a1780700 1 -- 172.21.15.27:0/2479509335 >> 172.21.15.27:0/2479509335 conn(0x7f219c07a620 msgr2=0x7f219c109a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:25:30.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.377+0000 7f21a1780700 1 -- 172.21.15.27:0/2479509335 shutdown_connections 2024-05-21T03:25:30.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:30.377+0000 7f21a1780700 1 -- 172.21.15.27:0/2479509335 wait complete. 2024-05-21T03:25:30.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:30 smithi027 ceph-mon[32179]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:25:31.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:30 smithi039 ceph-mon[36429]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:25:31.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:30 smithi165 ceph-mon[36269]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:25:31.130 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '454d9796', total seconds elapsed: 36.2 2024-05-21T03:25:31.131 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-05-21T03:25:31.131 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:25:31.409 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:25:32.845 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:32 smithi027 ceph-mon[32179]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:25:33.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:32 smithi039 ceph-mon[36429]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:25:33.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:32 smithi165 ceph-mon[36269]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:25:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.240+0000 7f420b6f6700 1 -- 172.21.15.27:0/2480048171 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4204108520 msgr2=0x7f4204108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:25:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.240+0000 7f420b6f6700 1 --2- 172.21.15.27:0/2480048171 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4204108520 0x7f4204108900 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4200009990 tx=0x7f4200009ca0 comp rx=0 tx=0).stop 2024-05-21T03:25:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.241+0000 7f420b6f6700 1 -- 172.21.15.27:0/2480048171 shutdown_connections 2024-05-21T03:25:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.241+0000 7f420b6f6700 1 --2- 172.21.15.27:0/2480048171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4204102e00 0x7f420410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.241+0000 7f420b6f6700 1 --2- 172.21.15.27:0/2480048171 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4204108520 0x7f4204108900 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.241+0000 7f420b6f6700 1 --2- 172.21.15.27:0/2480048171 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4204102440 0x7f42041028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.241+0000 7f420b6f6700 1 -- 172.21.15.27:0/2480048171 >> 172.21.15.27:0/2480048171 conn(0x7f42040793f0 msgr2=0x7f4204079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:25:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.241+0000 7f420b6f6700 1 -- 172.21.15.27:0/2480048171 shutdown_connections 2024-05-21T03:25:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.241+0000 7f420b6f6700 1 -- 172.21.15.27:0/2480048171 wait complete. 2024-05-21T03:25:33.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.242+0000 7f420b6f6700 1 Processor -- start 2024-05-21T03:25:33.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.242+0000 7f420b6f6700 1 -- start start 2024-05-21T03:25:33.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.243+0000 7f420b6f6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4204102440 0x7f4204198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:33.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.243+0000 7f420b6f6700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4204102e00 0x7f4204198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:33.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.243+0000 7f4209492700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4204102440 0x7f4204198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:33.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.243+0000 7f4209492700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4204102440 0x7f4204198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53546/0 (socket says 172.21.15.27:53546) 2024-05-21T03:25:33.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.243+0000 7f4209492700 1 -- 172.21.15.27:0/3857841031 learned_addr learned my addr 172.21.15.27:0/3857841031 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:25:33.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.243+0000 7f420b6f6700 1 --2- 172.21.15.27:0/3857841031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4204108520 0x7f42041992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:33.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.243+0000 7f420b6f6700 1 -- 172.21.15.27:0/3857841031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4204112620 con 0x7f4204102440 2024-05-21T03:25:33.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.243+0000 7f420b6f6700 1 -- 172.21.15.27:0/3857841031 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f42041124a0 con 0x7f4204108520 2024-05-21T03:25:33.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.243+0000 7f420b6f6700 1 -- 172.21.15.27:0/3857841031 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f42041127a0 con 0x7f4204102e00 2024-05-21T03:25:33.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.243+0000 7f4209c93700 1 --2- 172.21.15.27:0/3857841031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4204108520 0x7f42041992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:33.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.243+0000 7f4209492700 1 -- 172.21.15.27:0/3857841031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4204102e00 msgr2=0x7f4204198d90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-21T03:25:33.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.243+0000 7f4209492700 1 --2- 172.21.15.27:0/3857841031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4204102e00 0x7f4204198d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:33.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.243+0000 7f4209492700 1 -- 172.21.15.27:0/3857841031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4204108520 msgr2=0x7f42041992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:25:33.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.243+0000 7f4209492700 1 --2- 172.21.15.27:0/3857841031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4204108520 0x7f42041992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:33.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.243+0000 7f4209492700 1 -- 172.21.15.27:0/3857841031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42041a80c0 con 0x7f4204102440 2024-05-21T03:25:33.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.244+0000 7f4209492700 1 --2- 172.21.15.27:0/3857841031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4204102440 0x7f4204198850 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4200005910 tx=0x7f42000047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:25:33.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.244+0000 7f41fa7fc700 1 -- 172.21.15.27:0/3857841031 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4200027070 con 0x7f4204102440 2024-05-21T03:25:33.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.244+0000 7f41fa7fc700 1 -- 172.21.15.27:0/3857841031 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4200003f20 con 0x7f4204102440 2024-05-21T03:25:33.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.244+0000 7f41fa7fc700 1 -- 172.21.15.27:0/3857841031 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4200022c70 con 0x7f4204102440 2024-05-21T03:25:33.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.244+0000 7f420b6f6700 1 -- 172.21.15.27:0/3857841031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42041a8350 con 0x7f4204102440 2024-05-21T03:25:33.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.245+0000 7f420b6f6700 1 -- 172.21.15.27:0/3857841031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42041a8750 con 0x7f4204102440 2024-05-21T03:25:33.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.246+0000 7f420b6f6700 1 -- 172.21.15.27:0/3857841031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f420410a9b0 con 0x7f4204102440 2024-05-21T03:25:33.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.248+0000 7f41fa7fc700 1 -- 172.21.15.27:0/3857841031 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f420002a070 con 0x7f4204102440 2024-05-21T03:25:33.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.249+0000 7f41fa7fc700 1 --2- 172.21.15.27:0/3857841031 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f41f00b2c10 0x7f41f00b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:33.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.250+0000 7f4208c91700 1 --2- 172.21.15.27:0/3857841031 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f41f00b2c10 0x7f41f00b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:33.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.250+0000 7f41fa7fc700 1 -- 172.21.15.27:0/3857841031 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f42000e3700 con 0x7f4204102440 2024-05-21T03:25:33.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.250+0000 7f4208c91700 1 --2- 172.21.15.27:0/3857841031 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f41f00b2c10 0x7f41f00b50d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f420419a9b0 tx=0x7f41f4005c90 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:25:33.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.259+0000 7f41fa7fc700 1 -- 172.21.15.27:0/3857841031 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4200013070 con 0x7f4204102440 2024-05-21T03:25:33.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.541+0000 7f420b6f6700 1 -- 172.21.15.27:0/3857841031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f420406b810 con 0x7f4204102440 2024-05-21T03:25:33.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.542+0000 7f41fa7fc700 1 -- 172.21.15.27:0/3857841031 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11924 (secure 0 0 0) 0x7f42000aaec0 con 0x7f4204102440 2024-05-21T03:25:33.543 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:25:33.543 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/148854869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":148854869},{"type":"v1","addr":"172.21.15.27:6835","nonce":148854869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/4145544730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":4145544730},{"type":"v1","addr":"172.21.15.27:6837","nonce":4145544730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T03:20:34.055985+0000","modified":"2024-05-21T03:23:58.657936+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24472,"mds_2":24457,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/745486764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":745486764},{"type":"v1","addr":"172.21.15.27:6839","nonce":745486764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6841/3982752351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3982752351},{"type":"v1","addr":"172.21.15.27:6841","nonce":3982752351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/3497521608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3497521608},{"type":"v1","addr":"172.21.15.39:6835","nonce":3497521608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.165:6835/3576910218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3576910218},{"type":"v1","addr":"172.21.15.165:6835","nonce":3576910218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/3673846852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3673846852},{"type":"v1","addr":"172.21.15.39:6837","nonce":3673846852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6837/691356139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":691356139},{"type":"v1","addr":"172.21.15.165:6837","nonce":691356139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6839/4158170814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4158170814},{"type":"v1","addr":"172.21.15.165:6839","nonce":4158170814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6839/287607819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":287607819},{"type":"v1","addr":"172.21.15.39:6839","nonce":287607819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24457,24452,24451,24472]},"id":1}]} 2024-05-21T03:25:33.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.544+0000 7f420b6f6700 1 -- 172.21.15.27:0/3857841031 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f41f00b2c10 msgr2=0x7f41f00b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:25:33.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.544+0000 7f420b6f6700 1 --2- 172.21.15.27:0/3857841031 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f41f00b2c10 0x7f41f00b50d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f420419a9b0 tx=0x7f41f4005c90 comp rx=0 tx=0).stop 2024-05-21T03:25:33.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.544+0000 7f420b6f6700 1 -- 172.21.15.27:0/3857841031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4204102440 msgr2=0x7f4204198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:25:33.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.544+0000 7f420b6f6700 1 --2- 172.21.15.27:0/3857841031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4204102440 0x7f4204198850 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4200005910 tx=0x7f42000047b0 comp rx=0 tx=0).stop 2024-05-21T03:25:33.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.545+0000 7f420b6f6700 1 -- 172.21.15.27:0/3857841031 shutdown_connections 2024-05-21T03:25:33.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.545+0000 7f420b6f6700 1 --2- 172.21.15.27:0/3857841031 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f41f00b2c10 0x7f41f00b50d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:33.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.545+0000 7f420b6f6700 1 --2- 172.21.15.27:0/3857841031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4204108520 0x7f42041992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:33.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.545+0000 7f420b6f6700 1 --2- 172.21.15.27:0/3857841031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4204102440 0x7f4204198850 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:33.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.545+0000 7f420b6f6700 1 --2- 172.21.15.27:0/3857841031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4204102e00 0x7f4204198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:33.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.545+0000 7f420b6f6700 1 -- 172.21.15.27:0/3857841031 >> 172.21.15.27:0/3857841031 conn(0x7f42040793f0 msgr2=0x7f4204101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:25:33.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.545+0000 7f420b6f6700 1 -- 172.21.15.27:0/3857841031 shutdown_connections 2024-05-21T03:25:33.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:33.545+0000 7f420b6f6700 1 -- 172.21.15.27:0/3857841031 wait complete. 2024-05-21T03:25:33.546 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-05-21T03:25:33.868 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:33 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3857841031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:25:34.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:33 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/3857841031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:25:34.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:33 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/3857841031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:25:34.215 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-05-21T03:25:34.216 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-05-21T03:25:34.477 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:25:34.586 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:34 smithi027 ceph-mon[32179]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 853 B/s wr, 13 op/s 2024-05-21T03:25:34.586 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:34 smithi027 ceph-mon[32179]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-05-21T03:25:35.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:34 smithi039 ceph-mon[36429]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 853 B/s wr, 13 op/s 2024-05-21T03:25:35.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:34 smithi039 ceph-mon[36429]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-05-21T03:25:35.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:34 smithi165 ceph-mon[36269]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 853 B/s wr, 13 op/s 2024-05-21T03:25:35.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:34 smithi165 ceph-mon[36269]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-05-21T03:25:36.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.317+0000 7f3a177fc700 1 -- 172.21.15.27:0/2780676249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a1006c4d0 msgr2=0x7f3a101103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:25:36.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.317+0000 7f3a177fc700 1 --2- 172.21.15.27:0/2780676249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a1006c4d0 0x7f3a101103f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3a0800b0a0 tx=0x7f3a0800b3b0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.317+0000 7f3a177fc700 1 -- 172.21.15.27:0/2780676249 shutdown_connections 2024-05-21T03:25:36.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.317+0000 7f3a177fc700 1 --2- 172.21.15.27:0/2780676249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a1006c4d0 0x7f3a101103f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.317+0000 7f3a177fc700 1 --2- 172.21.15.27:0/2780676249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a101095d0 0x7f3a101099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.317+0000 7f3a177fc700 1 --2- 172.21.15.27:0/2780676249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a1006bb10 0x7f3a1006bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.317+0000 7f3a177fc700 1 -- 172.21.15.27:0/2780676249 >> 172.21.15.27:0/2780676249 conn(0x7f3a10079480 msgr2=0x7f3a10079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:25:36.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.319+0000 7f3a177fc700 1 -- 172.21.15.27:0/2780676249 shutdown_connections 2024-05-21T03:25:36.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.319+0000 7f3a177fc700 1 -- 172.21.15.27:0/2780676249 wait complete. 2024-05-21T03:25:36.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.320+0000 7f3a177fc700 1 Processor -- start 2024-05-21T03:25:36.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.320+0000 7f3a177fc700 1 -- start start 2024-05-21T03:25:36.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.320+0000 7f3a177fc700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a1006bb10 0x7f3a101a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:36.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.320+0000 7f3a177fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a1006c4d0 0x7f3a101a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:36.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.320+0000 7f3a177fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a101095d0 0x7f3a1019bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:36.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.320+0000 7f3a177fc700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3a10113700 con 0x7f3a101095d0 2024-05-21T03:25:36.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.320+0000 7f3a177fc700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3a10113580 con 0x7f3a1006c4d0 2024-05-21T03:25:36.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.320+0000 7f3a177fc700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3a10113880 con 0x7f3a1006bb10 2024-05-21T03:25:36.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.320+0000 7f3a15598700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a1006bb10 0x7f3a101a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:36.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.320+0000 7f3a15598700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a1006bb10 0x7f3a101a1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.27:42780/0 (socket says 172.21.15.27:42780) 2024-05-21T03:25:36.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.320+0000 7f3a15598700 1 -- 172.21.15.27:0/3675722378 learned_addr learned my addr 172.21.15.27:0/3675722378 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:25:36.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.320+0000 7f3a14d97700 1 --2- 172.21.15.27:0/3675722378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a1006c4d0 0x7f3a101a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:36.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.320+0000 7f3a15d99700 1 --2- 172.21.15.27:0/3675722378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a101095d0 0x7f3a1019bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:36.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.321+0000 7f3a14d97700 1 -- 172.21.15.27:0/3675722378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a1006bb10 msgr2=0x7f3a101a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:25:36.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.321+0000 7f3a14d97700 1 --2- 172.21.15.27:0/3675722378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a1006bb10 0x7f3a101a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.321+0000 7f3a14d97700 1 -- 172.21.15.27:0/3675722378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a101095d0 msgr2=0x7f3a1019bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:25:36.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.321+0000 7f3a14d97700 1 --2- 172.21.15.27:0/3675722378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a101095d0 0x7f3a1019bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.321+0000 7f3a14d97700 1 -- 172.21.15.27:0/3675722378 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a1019c260 con 0x7f3a1006c4d0 2024-05-21T03:25:36.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.321+0000 7f3a14d97700 1 --2- 172.21.15.27:0/3675722378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a1006c4d0 0x7f3a101a1e80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3a0000d950 tx=0x7f3a0000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:25:36.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.321+0000 7f3a067fc700 1 -- 172.21.15.27:0/3675722378 <== mon.2 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a00016070 con 0x7f3a1006c4d0 2024-05-21T03:25:36.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.321+0000 7f3a067fc700 1 -- 172.21.15.27:0/3675722378 <== mon.2 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a0000fd40 con 0x7f3a1006c4d0 2024-05-21T03:25:36.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.321+0000 7f3a177fc700 1 -- 172.21.15.27:0/3675722378 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a1019c550 con 0x7f3a1006c4d0 2024-05-21T03:25:36.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.321+0000 7f3a067fc700 1 -- 172.21.15.27:0/3675722378 <== mon.2 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0000e3b0 con 0x7f3a1006c4d0 2024-05-21T03:25:36.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.322+0000 7f3a177fc700 1 -- 172.21.15.27:0/3675722378 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a101a8810 con 0x7f3a1006c4d0 2024-05-21T03:25:36.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.325+0000 7f3a067fc700 1 -- 172.21.15.27:0/3675722378 <== mon.2 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3a00019070 con 0x7f3a1006c4d0 2024-05-21T03:25:36.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.326+0000 7f3a22092b80 1 Processor -- start 2024-05-21T03:25:36.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.326+0000 7f3a22092b80 1 -- start start 2024-05-21T03:25:36.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.327+0000 7f3a067fc700 1 --2- 172.21.15.27:0/3675722378 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f39fc0b2ba0 0x7f39fc0b5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:36.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.327+0000 7f3a22092b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be26d330e0 0x55be26d334c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:36.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.327+0000 7f3a22092b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be26d2d490 0x55be26d2d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:36.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.327+0000 7f3a22092b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be26d2de50 0x55be26d39ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:36.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.327+0000 7f3a22092b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55be26c82f00 con 0x55be26d330e0 2024-05-21T03:25:36.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.327+0000 7f3a22092b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55be26c72aa0 con 0x55be26d2de50 2024-05-21T03:25:36.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.327+0000 7f3a22092b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55be26c76ae0 con 0x55be26d2d490 2024-05-21T03:25:36.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.327+0000 7f3a067fc700 1 -- 172.21.15.27:0/3675722378 <== mon.2 v2:172.21.15.39:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f3a000e1a30 con 0x7f3a1006c4d0 2024-05-21T03:25:36.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.327+0000 7f3a15598700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be26d330e0 0x55be26d334c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:36.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.327+0000 7f3a15598700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be26d330e0 0x55be26d334c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53564/0 (socket says 172.21.15.27:53564) 2024-05-21T03:25:36.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.327+0000 7f3a15598700 1 -- 172.21.15.27:0/2543725677 learned_addr learned my addr 172.21.15.27:0/2543725677 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:25:36.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.327+0000 7f3a15d99700 1 --2- 172.21.15.27:0/3675722378 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f39fc0b2ba0 0x7f39fc0b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:36.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.327+0000 7f3a14d97700 1 --2- 172.21.15.27:0/2543725677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be26d2d490 0x55be26d2d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:36.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.328+0000 7f3a15598700 1 -- 172.21.15.27:0/2543725677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be26d2d490 msgr2=0x55be26d2d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:25:36.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.328+0000 7f3a15598700 1 --2- 172.21.15.27:0/2543725677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be26d2d490 0x55be26d2d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.328+0000 7f3a15598700 1 -- 172.21.15.27:0/2543725677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be26d2de50 msgr2=0x55be26d39ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T03:25:36.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.328+0000 7f3a15598700 1 --2- 172.21.15.27:0/2543725677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be26d2de50 0x55be26d39ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.328+0000 7f3a15598700 1 -- 172.21.15.27:0/2543725677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be26c1c140 con 0x55be26d330e0 2024-05-21T03:25:36.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.328+0000 7f3a15d99700 1 --2- 172.21.15.27:0/3675722378 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f39fc0b2ba0 0x7f39fc0b5060 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009fd0 tx=0x7f3a0800b7d0 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:25:36.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.328+0000 7f3a15598700 1 --2- 172.21.15.27:0/2543725677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be26d330e0 0x55be26d334c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00be20 tx=0x7f3a0c0106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1932ceea48bcbc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:25:36.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.328+0000 7f39fb7fe700 1 -- 172.21.15.27:0/2543725677 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c004830 con 0x55be26d330e0 2024-05-21T03:25:36.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.328+0000 7f39fb7fe700 1 -- 172.21.15.27:0/2543725677 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a0c005020 con 0x55be26d330e0 2024-05-21T03:25:36.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.328+0000 7f39fb7fe700 1 -- 172.21.15.27:0/2543725677 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c0116a0 con 0x55be26d330e0 2024-05-21T03:25:36.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.329+0000 7f3a22092b80 1 -- 172.21.15.27:0/2543725677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be26d330e0 msgr2=0x55be26d334c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:25:36.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.329+0000 7f3a22092b80 1 --2- 172.21.15.27:0/2543725677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be26d330e0 0x55be26d334c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00be20 tx=0x7f3a0c0106b0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.329+0000 7f3a22092b80 1 -- 172.21.15.27:0/2543725677 shutdown_connections 2024-05-21T03:25:36.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.329+0000 7f3a22092b80 1 --2- 172.21.15.27:0/2543725677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be26d2de50 0x55be26d39ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.329+0000 7f3a22092b80 1 --2- 172.21.15.27:0/2543725677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be26d330e0 0x55be26d334c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.329+0000 7f3a22092b80 1 --2- 172.21.15.27:0/2543725677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be26d2d490 0x55be26d2d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.329+0000 7f3a22092b80 1 -- 172.21.15.27:0/2543725677 >> 172.21.15.27:0/2543725677 conn(0x55be26c503d0 msgr2=0x55be26d1f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:25:36.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.329+0000 7f3a22092b80 1 -- 172.21.15.27:0/2543725677 shutdown_connections 2024-05-21T03:25:36.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.329+0000 7f3a22092b80 1 -- 172.21.15.27:0/2543725677 wait complete. 2024-05-21T03:25:36.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.330+0000 7f3a22092b80 1 -- start start 2024-05-21T03:25:36.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.330+0000 7f3a22092b80 1 Processor -- start 2024-05-21T03:25:36.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.330+0000 7f3a22092b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be26d2d490 0x55be26d39bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:36.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.330+0000 7f3a22092b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be26d2de50 0x55be26d36ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:36.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.330+0000 7f3a22092b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be26d330e0 0x55be26d37410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:36.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.331+0000 7f3a22092b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55be26b79130 con 0x55be26d2de50 2024-05-21T03:25:36.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.331+0000 7f3a22092b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55be26bdaf30 con 0x55be26d330e0 2024-05-21T03:25:36.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.331+0000 7f3a22092b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55be26d3a540 con 0x55be26d2d490 2024-05-21T03:25:36.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.331+0000 7f3a22092b80 10 client.0 _ll_register_callbacks cb 0x55be26c0dae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:25:36.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.331+0000 7f3a14d97700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be26d2de50 0x55be26d36ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:36.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.331+0000 7f3a14d97700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be26d2de50 0x55be26d36ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53580/0 (socket says 172.21.15.27:53580) 2024-05-21T03:25:36.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.331+0000 7f3a14d97700 1 -- 172.21.15.27:0/454691212 learned_addr learned my addr 172.21.15.27:0/454691212 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:25:36.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.331+0000 7f3a15598700 1 --2- 172.21.15.27:0/454691212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be26d2d490 0x55be26d39bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:36.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.331+0000 7f3a15d99700 1 --2- 172.21.15.27:0/454691212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be26d330e0 0x55be26d37410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:36.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.331+0000 7f3a14d97700 1 -- 172.21.15.27:0/454691212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be26d2d490 msgr2=0x55be26d39bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:25:36.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.331+0000 7f3a14d97700 1 --2- 172.21.15.27:0/454691212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be26d2d490 0x55be26d39bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.331+0000 7f3a14d97700 1 -- 172.21.15.27:0/454691212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be26d330e0 msgr2=0x55be26d37410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:25:36.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.331+0000 7f3a14d97700 1 --2- 172.21.15.27:0/454691212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be26d330e0 0x55be26d37410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.331+0000 7f3a14d97700 1 -- 172.21.15.27:0/454691212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be26bc2a60 con 0x55be26d2de50 2024-05-21T03:25:36.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.331+0000 7f3a14d97700 1 --2- 172.21.15.27:0/454691212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be26d2de50 0x55be26d36ed0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3a00005950 tx=0x7f3a000df680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfffcdf1a657d436 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:25:36.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.331+0000 7f39f8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:25:36.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.332+0000 7f39f8ff9700 1 -- 172.21.15.27:0/454691212 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a00020030 con 0x55be26d2de50 2024-05-21T03:25:36.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.332+0000 7f39f8ff9700 1 -- 172.21.15.27:0/454691212 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f3a000a52a0 con 0x55be26d2de50 2024-05-21T03:25:36.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.332+0000 7f3a22092b80 1 -- 172.21.15.27:0/454691212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a0c005020 con 0x55be26d2de50 2024-05-21T03:25:36.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.332+0000 7f39f8ff9700 1 -- 172.21.15.27:0/454691212 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3a000a4440 con 0x55be26d2de50 2024-05-21T03:25:36.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.332+0000 7f3a22092b80 10 client.15549 fetch_fsmap learned FSMap version 29 2024-05-21T03:25:36.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.332+0000 7f3a22092b80 1 -- 172.21.15.27:0/454691212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55be26c2c8f0 con 0x55be26d2de50 2024-05-21T03:25:36.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.332+0000 7f39f8ff9700 1 -- 172.21.15.27:0/454691212 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a0c005020 con 0x55be26d2de50 2024-05-21T03:25:36.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.333+0000 7f39f8ff9700 1 -- 172.21.15.27:0/454691212 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3a000b1680 con 0x55be26d2de50 2024-05-21T03:25:36.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.333+0000 7f3a22092b80 10 client.15549 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T03:25:36.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.333+0000 7f3a22092b80 10 client.15549 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-21T03:25:36.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.333+0000 7f3a22092b80 20 client.15549 populate_metadata read hostname 'smithi027' 2024-05-21T03:25:36.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.333+0000 7f3a22092b80 1 --2- 172.21.15.27:0/454691212 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55be26d46f60 0x55be26d49380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:36.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.333+0000 7f3a22092b80 4 client.15549 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:25:36.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.333+0000 7f3a22092b80 1 -- 172.21.15.27:0/454691212 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be26c84e50 con 0x55be26d46f60 2024-05-21T03:25:36.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.333+0000 7f3a15598700 1 --2- 172.21.15.27:0/454691212 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55be26d46f60 0x55be26d49380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:36.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.334+0000 7f3a15598700 1 --2- 172.21.15.27:0/454691212 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55be26d46f60 0x55be26d49380 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d252b2f71dcc78 server_cookie=7b9df3e68f793b2c in_seq=0 out_seq=0 2024-05-21T03:25:36.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.334+0000 7f39f8ff9700 10 client.15549 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:25:36.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.335+0000 7f39f8ff9700 1 -- 172.21.15.27:0/454691212 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55be26c84e50 con 0x55be26d46f60 2024-05-21T03:25:36.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.335+0000 7f39f8ff9700 10 client.15549 handle_command_reply: tid=0 2024-05-21T03:25:36.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.335+0000 7f3a22092b80 1 client.15549 shutdown 2024-05-21T03:25:36.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.335+0000 7f3a22092b80 1 -- 172.21.15.27:0/454691212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be26d2de50 msgr2=0x55be26d36ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:25:36.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.335+0000 7f3a22092b80 1 --2- 172.21.15.27:0/454691212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be26d2de50 0x55be26d36ed0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3a00005950 tx=0x7f3a000df680 comp rx=0 tx=0).stop 2024-05-21T03:25:36.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.335+0000 7f3a22092b80 1 -- 172.21.15.27:0/454691212 shutdown_connections 2024-05-21T03:25:36.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.335+0000 7f3a22092b80 1 --2- 172.21.15.27:0/454691212 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55be26d46f60 0x55be26d49380 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.335+0000 7f39f8ff9700 0 client.15549 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:25:36.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.335+0000 7f3a22092b80 1 --2- 172.21.15.27:0/454691212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be26d330e0 0x55be26d37410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.335+0000 7f3a22092b80 1 --2- 172.21.15.27:0/454691212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be26d2d490 0x55be26d39bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.335+0000 7f3a22092b80 1 --2- 172.21.15.27:0/454691212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be26d2de50 0x55be26d36ed0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.335+0000 7f3a22092b80 1 -- 172.21.15.27:0/454691212 >> 172.21.15.27:0/454691212 conn(0x55be26c503d0 msgr2=0x55be26d23c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:25:36.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.335+0000 7f3a22092b80 1 -- 172.21.15.27:0/454691212 shutdown_connections 2024-05-21T03:25:36.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.335+0000 7f3a22092b80 1 -- 172.21.15.27:0/454691212 wait complete. 2024-05-21T03:25:36.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.335+0000 7f3a22092b80 20 client.15549 trim_cache size 0 max 16384 2024-05-21T03:25:36.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.377+0000 7f3a22092b80 1 Processor -- start 2024-05-21T03:25:36.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.377+0000 7f3a22092b80 1 -- start start 2024-05-21T03:25:36.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.378+0000 7f3a22092b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be26d2d490 0x55be26d19f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:36.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.378+0000 7f3a22092b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be26d2de50 0x55be26d33dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:36.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.378+0000 7f3a22092b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be26d330e0 0x55be26d34310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:36.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.378+0000 7f3a22092b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3a10072260 con 0x55be26d2d490 2024-05-21T03:25:36.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.378+0000 7f3a22092b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3a10070050 con 0x55be26d330e0 2024-05-21T03:25:36.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.378+0000 7f3a22092b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3a10071b90 con 0x55be26d2de50 2024-05-21T03:25:36.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.378+0000 7f3a15598700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be26d2d490 0x55be26d19f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:36.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.378+0000 7f3a15598700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be26d2d490 0x55be26d19f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53594/0 (socket says 172.21.15.27:53594) 2024-05-21T03:25:36.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.378+0000 7f3a15598700 1 -- 172.21.15.27:0/2499747533 learned_addr learned my addr 172.21.15.27:0/2499747533 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:25:36.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.378+0000 7f3a14d97700 1 --2- 172.21.15.27:0/2499747533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be26d2de50 0x55be26d33dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:36.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.378+0000 7f3a15598700 1 -- 172.21.15.27:0/2499747533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be26d2de50 msgr2=0x55be26d33dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:25:36.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.378+0000 7f3a15598700 1 --2- 172.21.15.27:0/2499747533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be26d2de50 0x55be26d33dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.378+0000 7f3a15598700 1 -- 172.21.15.27:0/2499747533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be26d330e0 msgr2=0x55be26d34310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:25:36.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.378+0000 7f3a15598700 1 --2- 172.21.15.27:0/2499747533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be26d330e0 0x55be26d34310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.378+0000 7f3a15598700 1 -- 172.21.15.27:0/2499747533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be26d37790 con 0x55be26d2d490 2024-05-21T03:25:36.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.379+0000 7f3a15598700 1 --2- 172.21.15.27:0/2499747533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be26d2d490 0x55be26d19f00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00b750 tx=0x7f3a0c010be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b18fff3a036432b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:25:36.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.379+0000 7f39faffd700 1 -- 172.21.15.27:0/2499747533 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c049440 con 0x55be26d2d490 2024-05-21T03:25:36.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.379+0000 7f39faffd700 1 -- 172.21.15.27:0/2499747533 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a0c04e030 con 0x55be26d2d490 2024-05-21T03:25:36.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.379+0000 7f39faffd700 1 -- 172.21.15.27:0/2499747533 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c011730 con 0x55be26d2d490 2024-05-21T03:25:36.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.380+0000 7f3a22092b80 1 -- 172.21.15.27:0/2499747533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be26d2d490 msgr2=0x55be26d19f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:25:36.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.380+0000 7f3a22092b80 1 --2- 172.21.15.27:0/2499747533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be26d2d490 0x55be26d19f00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00b750 tx=0x7f3a0c010be0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.380+0000 7f3a22092b80 1 -- 172.21.15.27:0/2499747533 shutdown_connections 2024-05-21T03:25:36.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.380+0000 7f3a22092b80 1 --2- 172.21.15.27:0/2499747533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be26d330e0 0x55be26d34310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.380+0000 7f3a22092b80 1 --2- 172.21.15.27:0/2499747533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be26d2d490 0x55be26d19f00 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.380+0000 7f3a22092b80 1 --2- 172.21.15.27:0/2499747533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be26d2de50 0x55be26d33dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.380+0000 7f3a22092b80 1 -- 172.21.15.27:0/2499747533 >> 172.21.15.27:0/2499747533 conn(0x55be26c503d0 msgr2=0x55be26d31410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:25:36.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.380+0000 7f3a22092b80 1 -- 172.21.15.27:0/2499747533 shutdown_connections 2024-05-21T03:25:36.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.380+0000 7f3a22092b80 1 -- 172.21.15.27:0/2499747533 wait complete. 2024-05-21T03:25:36.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.381+0000 7f3a22092b80 1 -- start start 2024-05-21T03:25:36.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.381+0000 7f3a22092b80 1 Processor -- start 2024-05-21T03:25:36.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.381+0000 7f3a22092b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be26d2d490 0x55be26d25fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:36.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.381+0000 7f3a22092b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be26d2de50 0x55be26d26520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:36.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.381+0000 7f3a22092b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be26d330e0 0x55be26dca910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:36.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.381+0000 7f3a22092b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3a10070df0 con 0x55be26d330e0 2024-05-21T03:25:36.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.381+0000 7f3a22092b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3a10070720 con 0x55be26d2d490 2024-05-21T03:25:36.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.381+0000 7f3a22092b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3a100714c0 con 0x55be26d2de50 2024-05-21T03:25:36.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.382+0000 7f3a22092b80 10 client.0 _ll_register_callbacks cb 0x55be26c0dae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:25:36.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.382+0000 7f3a15d99700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be26d330e0 0x55be26dca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:36.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.382+0000 7f3a15598700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be26d2d490 0x55be26d25fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:36.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.382+0000 7f3a15d99700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be26d330e0 0x55be26dca910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53610/0 (socket says 172.21.15.27:53610) 2024-05-21T03:25:36.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.382+0000 7f3a15598700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be26d2d490 0x55be26d25fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.27:42812/0 (socket says 172.21.15.27:42812) 2024-05-21T03:25:36.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.382+0000 7f3a15d99700 1 -- 172.21.15.27:0/1599069201 learned_addr learned my addr 172.21.15.27:0/1599069201 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:25:36.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.382+0000 7f3a14d97700 1 --2- 172.21.15.27:0/1599069201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be26d2de50 0x55be26d26520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:36.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.382+0000 7f3a15d99700 1 -- 172.21.15.27:0/1599069201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be26d2de50 msgr2=0x55be26d26520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:25:36.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.382+0000 7f3a15d99700 1 --2- 172.21.15.27:0/1599069201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be26d2de50 0x55be26d26520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.382+0000 7f3a15d99700 1 -- 172.21.15.27:0/1599069201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be26d2d490 msgr2=0x55be26d25fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:25:36.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.382+0000 7f3a15d99700 1 --2- 172.21.15.27:0/1599069201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be26d2d490 0x55be26d25fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.382+0000 7f3a15d99700 1 -- 172.21.15.27:0/1599069201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be26dcaf00 con 0x55be26d330e0 2024-05-21T03:25:36.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.382+0000 7f3a15d99700 1 --2- 172.21.15.27:0/1599069201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be26d330e0 0x55be26dca910 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3a0800b0a0 tx=0x7f3a08007760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99327cf41e8ba4dd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:25:36.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.382+0000 7f39f8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:25:36.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.383+0000 7f39f8ff9700 1 -- 172.21.15.27:0/1599069201 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a08007c60 con 0x55be26d330e0 2024-05-21T03:25:36.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.383+0000 7f39f8ff9700 1 -- 172.21.15.27:0/1599069201 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f3a08007e00 con 0x55be26d330e0 2024-05-21T03:25:36.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.383+0000 7f3a22092b80 1 -- 172.21.15.27:0/1599069201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a0c04e030 con 0x55be26d330e0 2024-05-21T03:25:36.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.383+0000 7f39f8ff9700 1 -- 172.21.15.27:0/1599069201 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3a080050b0 con 0x55be26d330e0 2024-05-21T03:25:36.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.383+0000 7f3a22092b80 10 client.15555 fetch_fsmap learned FSMap version 29 2024-05-21T03:25:36.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.383+0000 7f3a22092b80 1 -- 172.21.15.27:0/1599069201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55be26dcc1c0 con 0x55be26d330e0 2024-05-21T03:25:36.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.383+0000 7f39f8ff9700 1 -- 172.21.15.27:0/1599069201 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a0c04e030 con 0x55be26d330e0 2024-05-21T03:25:36.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.383+0000 7f39f8ff9700 1 -- 172.21.15.27:0/1599069201 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3a08011070 con 0x55be26d330e0 2024-05-21T03:25:36.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.384+0000 7f3a22092b80 10 client.15555 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T03:25:36.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.384+0000 7f3a22092b80 10 client.15555 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-21T03:25:36.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.384+0000 7f3a22092b80 20 client.15555 populate_metadata read hostname 'smithi027' 2024-05-21T03:25:36.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.384+0000 7f3a22092b80 1 --2- 172.21.15.27:0/1599069201 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55be26d46f60 0x55be26dce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:25:36.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.384+0000 7f3a22092b80 4 client.15555 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-05-21T03:25:36.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.384+0000 7f3a22092b80 1 -- 172.21.15.27:0/1599069201 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55be26c7e630 con 0x55be26d46f60 2024-05-21T03:25:36.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.384+0000 7f3a15598700 1 --2- 172.21.15.27:0/1599069201 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55be26d46f60 0x55be26dce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:25:36.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.384+0000 7f39f8ff9700 10 client.15555 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:25:36.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.385+0000 7f3a15598700 1 --2- 172.21.15.27:0/1599069201 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55be26d46f60 0x55be26dce950 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9438297f95b4958 server_cookie=c1bb262ac4a41efe in_seq=0 out_seq=0 2024-05-21T03:25:36.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.394+0000 7f39f8ff9700 1 -- 172.21.15.27:0/1599069201 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55be26c7e630 con 0x55be26d46f60 2024-05-21T03:25:36.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.394+0000 7f39f8ff9700 10 client.15555 handle_command_reply: tid=0 2024-05-21T03:25:36.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.394+0000 7f3a22092b80 1 client.15555 shutdown 2024-05-21T03:25:36.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.395+0000 7f3a22092b80 1 -- 172.21.15.27:0/1599069201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be26d330e0 msgr2=0x55be26dca910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:25:36.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.395+0000 7f3a22092b80 1 --2- 172.21.15.27:0/1599069201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be26d330e0 0x55be26dca910 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3a0800b0a0 tx=0x7f3a08007760 comp rx=0 tx=0).stop 2024-05-21T03:25:36.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.395+0000 7f3a22092b80 1 -- 172.21.15.27:0/1599069201 shutdown_connections 2024-05-21T03:25:36.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.395+0000 7f3a22092b80 1 --2- 172.21.15.27:0/1599069201 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55be26d46f60 0x55be26dce950 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.395+0000 7f3a22092b80 1 --2- 172.21.15.27:0/1599069201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55be26d330e0 0x55be26dca910 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.395+0000 7f39f8ff9700 0 client.15555 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:25:36.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.395+0000 7f3a22092b80 1 --2- 172.21.15.27:0/1599069201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be26d2d490 0x55be26d25fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.397 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:25:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 29, 2024-05-21T03:25:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24476, 2024-05-21T03:25:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 6, 2024-05-21T03:25:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-05-21T03:25:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: "3ba64728": { 2024-05-21T03:25:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 6, 2024-05-21T03:25:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-05-21T03:25:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-05-21T03:25:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-05-21T03:25:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-05-21T03:25:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:25:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-05-21T03:25:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 150.0, 2024-05-21T03:25:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-05-21T03:25:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-05-21T03:25:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-05-21T03:25:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-05-21T03:25:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-05-21T03:25:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-05-21T03:25:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:25:36.399 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:25:36.399 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:25:36.399 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:25:36.399 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:25:36.399 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:25:36.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.395+0000 7f3a22092b80 1 --2- 172.21.15.27:0/1599069201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be26d2de50 0x55be26d26520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.395+0000 7f3a22092b80 1 -- 172.21.15.27:0/1599069201 >> 172.21.15.27:0/1599069201 conn(0x55be26c503d0 msgr2=0x55be26d38700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:25:36.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.395+0000 7f3a22092b80 1 -- 172.21.15.27:0/1599069201 shutdown_connections 2024-05-21T03:25:36.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.395+0000 7f3a22092b80 1 -- 172.21.15.27:0/1599069201 wait complete. 2024-05-21T03:25:36.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.395+0000 7f3a22092b80 20 client.15555 trim_cache size 0 max 16384 2024-05-21T03:25:36.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.397+0000 7f3a177fc700 1 -- 172.21.15.27:0/3675722378 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f39fc0b2ba0 msgr2=0x7f39fc0b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:25:36.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.397+0000 7f3a177fc700 1 --2- 172.21.15.27:0/3675722378 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f39fc0b2ba0 0x7f39fc0b5060 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009fd0 tx=0x7f3a0800b7d0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.397+0000 7f3a177fc700 1 -- 172.21.15.27:0/3675722378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a1006c4d0 msgr2=0x7f3a101a1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:25:36.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.397+0000 7f3a177fc700 1 --2- 172.21.15.27:0/3675722378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a1006c4d0 0x7f3a101a1e80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3a0000d950 tx=0x7f3a0000dc60 comp rx=0 tx=0).stop 2024-05-21T03:25:36.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.397+0000 7f3a177fc700 1 -- 172.21.15.27:0/3675722378 shutdown_connections 2024-05-21T03:25:36.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.397+0000 7f3a177fc700 1 --2- 172.21.15.27:0/3675722378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a101095d0 0x7f3a1019bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.397+0000 7f3a177fc700 1 --2- 172.21.15.27:0/3675722378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a1006bb10 0x7f3a101a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.397+0000 7f3a177fc700 1 --2- 172.21.15.27:0/3675722378 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f39fc0b2ba0 0x7f39fc0b5060 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.397+0000 7f3a177fc700 1 --2- 172.21.15.27:0/3675722378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a1006c4d0 0x7f3a101a1e80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:25:36.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.397+0000 7f3a177fc700 1 -- 172.21.15.27:0/3675722378 >> 172.21.15.27:0/3675722378 conn(0x7f3a10079480 msgr2=0x7f3a1010e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:25:36.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.397+0000 7f3a177fc700 1 -- 172.21.15.27:0/3675722378 shutdown_connections 2024-05-21T03:25:36.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:25:36.397+0000 7f3a177fc700 1 -- 172.21.15.27:0/3675722378 wait complete. 2024-05-21T03:25:36.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:36 smithi027 ceph-mon[32179]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 853 B/s wr, 9 op/s 2024-05-21T03:25:37.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:36 smithi039 ceph-mon[36429]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 853 B/s wr, 9 op/s 2024-05-21T03:25:37.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:36 smithi165 ceph-mon[36269]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 853 B/s wr, 9 op/s 2024-05-21T03:25:37.116 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3ba64728', quiesce took 6.0 seconds. Will release after: 24.0 2024-05-21T03:25:38.930 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:38 smithi027 ceph-mon[32179]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 14 op/s 2024-05-21T03:25:39.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:38 smithi039 ceph-mon[36429]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 14 op/s 2024-05-21T03:25:39.086 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:38 smithi165 ceph-mon[36269]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 14 op/s 2024-05-21T03:25:40.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:40 smithi027 ceph-mon[32179]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 938 B/s wr, 11 op/s 2024-05-21T03:25:41.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:40 smithi039 ceph-mon[36429]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 938 B/s wr, 11 op/s 2024-05-21T03:25:41.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:40 smithi165 ceph-mon[36269]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 938 B/s wr, 11 op/s 2024-05-21T03:25:42.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:42 smithi027 ceph-mon[32179]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-21T03:25:43.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:42 smithi039 ceph-mon[36429]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-21T03:25:43.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:42 smithi165 ceph-mon[36269]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-21T03:25:44.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:44 smithi027 ceph-mon[32179]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 14 op/s 2024-05-21T03:25:45.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:44 smithi039 ceph-mon[36429]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 14 op/s 2024-05-21T03:25:45.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:44 smithi165 ceph-mon[36269]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 14 op/s 2024-05-21T03:25:46.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:46 smithi027 ceph-mon[32179]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T03:25:47.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:46 smithi039 ceph-mon[36429]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T03:25:47.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:46 smithi165 ceph-mon[36269]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T03:25:48.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:48 smithi027 ceph-mon[32179]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T03:25:49.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:48 smithi039 ceph-mon[36429]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T03:25:49.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:48 smithi165 ceph-mon[36269]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T03:25:50.085 INFO:tasks.workunit.client.0.smithi027.stderr:Updating files: 93% (11448/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-21T03:25:50.464 INFO:tasks.workunit.client.0.smithi027.stderr:Note: switching to '3b748ea08891006c1152513d25ab4a03f7c9a995'. 2024-05-21T03:25:50.464 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-05-21T03:25:50.464 INFO:tasks.workunit.client.0.smithi027.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-21T03:25:50.464 INFO:tasks.workunit.client.0.smithi027.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-21T03:25:50.464 INFO:tasks.workunit.client.0.smithi027.stderr:state without impacting any branches by switching back to a branch. 2024-05-21T03:25:50.464 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-05-21T03:25:50.464 INFO:tasks.workunit.client.0.smithi027.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-21T03:25:50.464 INFO:tasks.workunit.client.0.smithi027.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-21T03:25:50.464 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-05-21T03:25:50.464 INFO:tasks.workunit.client.0.smithi027.stderr: git switch -c 2024-05-21T03:25:50.464 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-05-21T03:25:50.465 INFO:tasks.workunit.client.0.smithi027.stderr:Or undo this operation with: 2024-05-21T03:25:50.465 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-05-21T03:25:50.465 INFO:tasks.workunit.client.0.smithi027.stderr: git switch - 2024-05-21T03:25:50.465 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-05-21T03:25:50.465 INFO:tasks.workunit.client.0.smithi027.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-21T03:25:50.465 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-05-21T03:25:50.465 INFO:tasks.workunit.client.0.smithi027.stderr:HEAD is now at 3b748ea0889 mds/quiesce: overdrive exporting that is still freezing 2024-05-21T03:25:50.470 DEBUG:teuthology.orchestra.run.smithi027:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-21T03:25:50.504 INFO:tasks.workunit.client.0.smithi027.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-21T03:25:50.506 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-21T03:25:50.506 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-21T03:25:50.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:50 smithi027 ceph-mon[32179]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:25:51.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:50 smithi039 ceph-mon[36429]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:25:51.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:50 smithi165 ceph-mon[36269]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:25:51.160 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-21T03:25:51.219 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-21T03:25:51.262 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-21T03:25:51.264 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-21T03:25:51.264 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-21T03:25:51.305 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-21T03:25:51.308 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-05-21T03:25:51.308 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-21T03:25:51.366 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2024-05-21T03:25:51.367 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2024-05-21T03:25:51.367 DEBUG:teuthology.orchestra.run.smithi027:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3b748ea08891006c1152513d25ab4a03f7c9a995 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-05-21T03:25:52.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:52 smithi027 ceph-mon[32179]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:25:53.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:52 smithi039 ceph-mon[36429]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:25:53.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:52 smithi165 ceph-mon[36269]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:25:54.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:54 smithi027 ceph-mon[32179]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T03:25:55.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:54 smithi039 ceph-mon[36429]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T03:25:55.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:54 smithi165 ceph-mon[36269]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T03:25:56.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:56 smithi027 ceph-mon[32179]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T03:25:57.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:56 smithi039 ceph-mon[36429]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T03:25:57.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:56 smithi165 ceph-mon[36269]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T03:25:58.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:25:58 smithi027 ceph-mon[32179]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T03:25:59.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:25:58 smithi039 ceph-mon[36429]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T03:25:59.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:25:58 smithi165 ceph-mon[36269]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T03:26:00.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:00 smithi027 ceph-mon[32179]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T03:26:01.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:00 smithi039 ceph-mon[36429]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T03:26:01.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:00 smithi165 ceph-mon[36269]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T03:26:01.117 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:26:01.374 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:26:02.793 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:02 smithi027 ceph-mon[32179]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T03:26:03.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:02 smithi039 ceph-mon[36429]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T03:26:03.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:02 smithi165 ceph-mon[36269]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T03:26:03.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.124+0000 7f08a3c08700 1 -- 172.21.15.27:0/1535077091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f089c06bd10 msgr2=0x7f089c06c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:26:03.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.124+0000 7f08a3c08700 1 --2- 172.21.15.27:0/1535077091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f089c06bd10 0x7f089c06c190 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f088c0099c0 tx=0x7f088c009cd0 comp rx=0 tx=0).stop 2024-05-21T03:26:03.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.124+0000 7f08a3c08700 1 -- 172.21.15.27:0/1535077091 shutdown_connections 2024-05-21T03:26:03.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.124+0000 7f08a3c08700 1 --2- 172.21.15.27:0/1535077091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f089c06c6d0 0x7f089c110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.125+0000 7f08a3c08700 1 --2- 172.21.15.27:0/1535077091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f089c109630 0x7f089c109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.125+0000 7f08a3c08700 1 --2- 172.21.15.27:0/1535077091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f089c06bd10 0x7f089c06c190 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.125+0000 7f08a3c08700 1 -- 172.21.15.27:0/1535077091 >> 172.21.15.27:0/1535077091 conn(0x7f089c0794e0 msgr2=0x7f089c0798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:26:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.125+0000 7f08a3c08700 1 -- 172.21.15.27:0/1535077091 shutdown_connections 2024-05-21T03:26:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.125+0000 7f08a3c08700 1 -- 172.21.15.27:0/1535077091 wait complete. 2024-05-21T03:26:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.125+0000 7f08a3c08700 1 Processor -- start 2024-05-21T03:26:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.125+0000 7f08a3c08700 1 -- start start 2024-05-21T03:26:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.126+0000 7f08a3c08700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f089c06bd10 0x7f089c1040a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:26:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.126+0000 7f08a3c08700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f089c06c6d0 0x7f089c1024b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:26:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.126+0000 7f08a3c08700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f089c109630 0x7f089c102b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:26:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.126+0000 7f08a3c08700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f089c113790 con 0x7f089c109630 2024-05-21T03:26:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.126+0000 7f08a3c08700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f089c113610 con 0x7f089c06bd10 2024-05-21T03:26:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.126+0000 7f08a3c08700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f089c113910 con 0x7f089c06c6d0 2024-05-21T03:26:03.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.126+0000 7f08a21a5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f089c109630 0x7f089c102b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:26:03.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.126+0000 7f08a19a4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f089c06bd10 0x7f089c1040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:26:03.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.126+0000 7f08a11a3700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f089c06c6d0 0x7f089c1024b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:26:03.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.126+0000 7f08a19a4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f089c06bd10 0x7f089c1040a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.27:34148/0 (socket says 172.21.15.27:34148) 2024-05-21T03:26:03.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.126+0000 7f08a21a5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f089c109630 0x7f089c102b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51452/0 (socket says 172.21.15.27:51452) 2024-05-21T03:26:03.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.126+0000 7f08a19a4700 1 -- 172.21.15.27:0/1978357767 learned_addr learned my addr 172.21.15.27:0/1978357767 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:26:03.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.126+0000 7f08a21a5700 1 -- 172.21.15.27:0/1978357767 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f089c06c6d0 msgr2=0x7f089c1024b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:26:03.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.126+0000 7f08a21a5700 1 --2- 172.21.15.27:0/1978357767 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f089c06c6d0 0x7f089c1024b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:03.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.126+0000 7f08a21a5700 1 -- 172.21.15.27:0/1978357767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f089c06bd10 msgr2=0x7f089c1040a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:26:03.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.126+0000 7f08a21a5700 1 --2- 172.21.15.27:0/1978357767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f089c06bd10 0x7f089c1040a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:03.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.126+0000 7f08a21a5700 1 -- 172.21.15.27:0/1978357767 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f089c1ac650 con 0x7f089c109630 2024-05-21T03:26:03.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.127+0000 7f08a21a5700 1 --2- 172.21.15.27:0/1978357767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f089c109630 0x7f089c102b10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0894007f10 tx=0x7f089400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:26:03.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.127+0000 7f0892ffd700 1 -- 172.21.15.27:0/1978357767 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0894004db0 con 0x7f089c109630 2024-05-21T03:26:03.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.127+0000 7f08a3c08700 1 -- 172.21.15.27:0/1978357767 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f089c1ac8e0 con 0x7f089c109630 2024-05-21T03:26:03.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.127+0000 7f0892ffd700 1 -- 172.21.15.27:0/1978357767 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0894009e20 con 0x7f089c109630 2024-05-21T03:26:03.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.127+0000 7f0892ffd700 1 -- 172.21.15.27:0/1978357767 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0894015400 con 0x7f089c109630 2024-05-21T03:26:03.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.127+0000 7f08a3c08700 1 -- 172.21.15.27:0/1978357767 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f089c1acda0 con 0x7f089c109630 2024-05-21T03:26:03.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.128+0000 7f08a3c08700 1 -- 172.21.15.27:0/1978357767 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f089c10bac0 con 0x7f089c109630 2024-05-21T03:26:03.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.130+0000 7f0892ffd700 1 -- 172.21.15.27:0/1978357767 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0894005260 con 0x7f089c109630 2024-05-21T03:26:03.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.131+0000 7f0892ffd700 1 --2- 172.21.15.27:0/1978357767 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f08880b29f0 0x7f08880b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:26:03.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.131+0000 7f08a19a4700 1 --2- 172.21.15.27:0/1978357767 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f08880b29f0 0x7f08880b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:26:03.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.131+0000 7f0892ffd700 1 -- 172.21.15.27:0/1978357767 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f08940e2480 con 0x7f089c109630 2024-05-21T03:26:03.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.132+0000 7f08a19a4700 1 --2- 172.21.15.27:0/1978357767 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f08880b29f0 0x7f08880b4eb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0898009ad0 tx=0x7f0898009290 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:26:03.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.135+0000 7f0892ffd700 1 -- 172.21.15.27:0/1978357767 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0894010070 con 0x7f089c109630 2024-05-21T03:26:03.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.397+0000 7f08a3c08700 1 -- 172.21.15.27:0/1978357767 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f089c103990 con 0x7f089c109630 2024-05-21T03:26:03.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.398+0000 7f0892ffd700 1 -- 172.21.15.27:0/1978357767 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11924 (secure 0 0 0) 0x7f08940a9e60 con 0x7f089c109630 2024-05-21T03:26:03.398 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:26:03.398 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/148854869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":148854869},{"type":"v1","addr":"172.21.15.27:6835","nonce":148854869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/4145544730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":4145544730},{"type":"v1","addr":"172.21.15.27:6837","nonce":4145544730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T03:20:34.055985+0000","modified":"2024-05-21T03:23:58.657936+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24472,"mds_2":24457,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/745486764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":745486764},{"type":"v1","addr":"172.21.15.27:6839","nonce":745486764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6841/3982752351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3982752351},{"type":"v1","addr":"172.21.15.27:6841","nonce":3982752351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/3497521608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3497521608},{"type":"v1","addr":"172.21.15.39:6835","nonce":3497521608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.165:6835/3576910218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3576910218},{"type":"v1","addr":"172.21.15.165:6835","nonce":3576910218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/3673846852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3673846852},{"type":"v1","addr":"172.21.15.39:6837","nonce":3673846852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6837/691356139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":691356139},{"type":"v1","addr":"172.21.15.165:6837","nonce":691356139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6839/4158170814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4158170814},{"type":"v1","addr":"172.21.15.165:6839","nonce":4158170814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6839/287607819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":287607819},{"type":"v1","addr":"172.21.15.39:6839","nonce":287607819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24457,24452,24451,24472]},"id":1}]} 2024-05-21T03:26:03.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.400+0000 7f08a3c08700 1 -- 172.21.15.27:0/1978357767 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f08880b29f0 msgr2=0x7f08880b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:26:03.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.400+0000 7f08a3c08700 1 --2- 172.21.15.27:0/1978357767 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f08880b29f0 0x7f08880b4eb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0898009ad0 tx=0x7f0898009290 comp rx=0 tx=0).stop 2024-05-21T03:26:03.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.400+0000 7f08a3c08700 1 -- 172.21.15.27:0/1978357767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f089c109630 msgr2=0x7f089c102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:26:03.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.400+0000 7f08a3c08700 1 --2- 172.21.15.27:0/1978357767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f089c109630 0x7f089c102b10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0894007f10 tx=0x7f089400d3b0 comp rx=0 tx=0).stop 2024-05-21T03:26:03.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.400+0000 7f08a3c08700 1 -- 172.21.15.27:0/1978357767 shutdown_connections 2024-05-21T03:26:03.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.400+0000 7f08a3c08700 1 --2- 172.21.15.27:0/1978357767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f089c109630 0x7f089c102b10 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:03.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.401+0000 7f08a3c08700 1 --2- 172.21.15.27:0/1978357767 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f08880b29f0 0x7f08880b4eb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:03.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.401+0000 7f08a3c08700 1 --2- 172.21.15.27:0/1978357767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f089c06bd10 0x7f089c1040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:03.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.401+0000 7f08a3c08700 1 --2- 172.21.15.27:0/1978357767 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f089c06c6d0 0x7f089c1024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:03.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.401+0000 7f08a3c08700 1 -- 172.21.15.27:0/1978357767 >> 172.21.15.27:0/1978357767 conn(0x7f089c0794e0 msgr2=0x7f089c1010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:26:03.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.401+0000 7f08a3c08700 1 -- 172.21.15.27:0/1978357767 shutdown_connections 2024-05-21T03:26:03.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:03.401+0000 7f08a3c08700 1 -- 172.21.15.27:0/1978357767 wait complete. 2024-05-21T03:26:03.401 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-05-21T03:26:03.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:03 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/1978357767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:26:04.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:03 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/1978357767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:26:04.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:03 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/1978357767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:26:04.066 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 3ba64728 --release --await' 2024-05-21T03:26:04.067 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 3ba64728 --release --await 2024-05-21T03:26:04.334 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:26:04.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:04 smithi027 ceph-mon[32179]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T03:26:05.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:04 smithi039 ceph-mon[36429]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T03:26:05.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:04 smithi165 ceph-mon[36269]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T03:26:06.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.047+0000 7f43dc929700 1 -- 172.21.15.27:0/323341186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d806bd10 msgr2=0x7f43d806c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:26:06.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.047+0000 7f43dc929700 1 --2- 172.21.15.27:0/323341186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d806bd10 0x7f43d806c190 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0099c0 tx=0x7f43cc009cd0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.047+0000 7f43dc929700 1 -- 172.21.15.27:0/323341186 shutdown_connections 2024-05-21T03:26:06.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.047+0000 7f43dc929700 1 --2- 172.21.15.27:0/323341186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43d806c6d0 0x7f43d8110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.047+0000 7f43dc929700 1 --2- 172.21.15.27:0/323341186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43d8109630 0x7f43d8109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.047+0000 7f43dc929700 1 --2- 172.21.15.27:0/323341186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d806bd10 0x7f43d806c190 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.048+0000 7f43dc929700 1 -- 172.21.15.27:0/323341186 >> 172.21.15.27:0/323341186 conn(0x7f43d80794e0 msgr2=0x7f43d80798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:26:06.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.048+0000 7f43dc929700 1 -- 172.21.15.27:0/323341186 shutdown_connections 2024-05-21T03:26:06.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.048+0000 7f43dc929700 1 -- 172.21.15.27:0/323341186 wait complete. 2024-05-21T03:26:06.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.048+0000 7f43dc929700 1 Processor -- start 2024-05-21T03:26:06.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.048+0000 7f43dc929700 1 -- start start 2024-05-21T03:26:06.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.049+0000 7f43dc929700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43d806bd10 0x7f43d819f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:26:06.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.049+0000 7f43dc929700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43d806c6d0 0x7f43d819fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:26:06.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.049+0000 7f43dc929700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d8109630 0x7f43d8199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:26:06.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.049+0000 7f43dc929700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f43d8113790 con 0x7f43d8109630 2024-05-21T03:26:06.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.049+0000 7f43dc929700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f43d8113610 con 0x7f43d806bd10 2024-05-21T03:26:06.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.049+0000 7f43dc929700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f43d8113910 con 0x7f43d806c6d0 2024-05-21T03:26:06.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.049+0000 7f43d6d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d8109630 0x7f43d8199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:26:06.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.049+0000 7f43d6d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d8109630 0x7f43d8199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51474/0 (socket says 172.21.15.27:51474) 2024-05-21T03:26:06.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.049+0000 7f43d6d9d700 1 -- 172.21.15.27:0/437195422 learned_addr learned my addr 172.21.15.27:0/437195422 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:26:06.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.049+0000 7f43d659c700 1 --2- 172.21.15.27:0/437195422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43d806bd10 0x7f43d819f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:26:06.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.049+0000 7f43d5d9b700 1 --2- 172.21.15.27:0/437195422 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43d806c6d0 0x7f43d819fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:26:06.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.049+0000 7f43d6d9d700 1 -- 172.21.15.27:0/437195422 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43d806c6d0 msgr2=0x7f43d819fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:26:06.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.049+0000 7f43d6d9d700 1 --2- 172.21.15.27:0/437195422 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43d806c6d0 0x7f43d819fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.049+0000 7f43d6d9d700 1 -- 172.21.15.27:0/437195422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43d806bd10 msgr2=0x7f43d819f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:26:06.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.049+0000 7f43d6d9d700 1 --2- 172.21.15.27:0/437195422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43d806bd10 0x7f43d819f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.049+0000 7f43d6d9d700 1 -- 172.21.15.27:0/437195422 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d819a250 con 0x7f43d8109630 2024-05-21T03:26:06.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.049+0000 7f43d5d9b700 1 --2- 172.21.15.27:0/437195422 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43d806c6d0 0x7f43d819fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:26:06.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.049+0000 7f43d6d9d700 1 --2- 172.21.15.27:0/437195422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d8109630 0x7f43d8199af0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f43d0007d70 tx=0x7f43d000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:26:06.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.050+0000 7f43c77fe700 1 -- 172.21.15.27:0/437195422 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d0016020 con 0x7f43d8109630 2024-05-21T03:26:06.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.050+0000 7f43dc929700 1 -- 172.21.15.27:0/437195422 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d819a540 con 0x7f43d8109630 2024-05-21T03:26:06.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.050+0000 7f43c77fe700 1 -- 172.21.15.27:0/437195422 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43d000ad40 con 0x7f43d8109630 2024-05-21T03:26:06.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.050+0000 7f43c77fe700 1 -- 172.21.15.27:0/437195422 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d00115e0 con 0x7f43d8109630 2024-05-21T03:26:06.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.050+0000 7f43dc929700 1 -- 172.21.15.27:0/437195422 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d81a67b0 con 0x7f43d8109630 2024-05-21T03:26:06.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.052+0000 7f43c77fe700 1 -- 172.21.15.27:0/437195422 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f43d0019020 con 0x7f43d8109630 2024-05-21T03:26:06.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.053+0000 7f43c77fe700 1 --2- 172.21.15.27:0/437195422 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f43c00b29f0 0x7f43c00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:26:06.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.053+0000 7f43d659c700 1 --2- 172.21.15.27:0/437195422 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f43c00b29f0 0x7f43c00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:26:06.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.053+0000 7f43c77fe700 1 -- 172.21.15.27:0/437195422 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f43d00e24b0 con 0x7f43d8109630 2024-05-21T03:26:06.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.053+0000 7f43e71bfb80 1 Processor -- start 2024-05-21T03:26:06.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.053+0000 7f43d659c700 1 --2- 172.21.15.27:0/437195422 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f43c00b29f0 0x7f43c00b4eb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f43d8109e70 tx=0x7f43c8009190 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:26:06.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.053+0000 7f43e71bfb80 1 -- start start 2024-05-21T03:26:06.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.054+0000 7f43e71bfb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559031bab2b0 0x559031bab690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:26:06.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.054+0000 7f43e71bfb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559031ba58e0 0x559031ba5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:26:06.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.054+0000 7f43e71bfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559031ba62a0 0x559031bb20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:26:06.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.054+0000 7f43e71bfb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559031b04d90 con 0x559031ba58e0 2024-05-21T03:26:06.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.054+0000 7f43e71bfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559031af4930 con 0x559031ba62a0 2024-05-21T03:26:06.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.054+0000 7f43e71bfb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559031af8970 con 0x559031bab2b0 2024-05-21T03:26:06.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.054+0000 7f43d5d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559031ba58e0 0x559031ba5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:26:06.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.054+0000 7f43d5d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559031ba58e0 0x559031ba5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51488/0 (socket says 172.21.15.27:51488) 2024-05-21T03:26:06.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.054+0000 7f43d5d9b700 1 -- 172.21.15.27:0/1077496441 learned_addr learned my addr 172.21.15.27:0/1077496441 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:26:06.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.054+0000 7f43d6d9d700 1 --2- 172.21.15.27:0/1077496441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559031ba62a0 0x559031bb20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:26:06.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.054+0000 7f43d659c700 1 --2- 172.21.15.27:0/1077496441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559031bab2b0 0x559031bab690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:26:06.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.054+0000 7f43d5d9b700 1 -- 172.21.15.27:0/1077496441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559031bab2b0 msgr2=0x559031bab690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:26:06.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.054+0000 7f43d5d9b700 1 --2- 172.21.15.27:0/1077496441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559031bab2b0 0x559031bab690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.054+0000 7f43d5d9b700 1 -- 172.21.15.27:0/1077496441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559031ba62a0 msgr2=0x559031bb20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:26:06.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.054+0000 7f43d5d9b700 1 --2- 172.21.15.27:0/1077496441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559031ba62a0 0x559031bb20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.054+0000 7f43d5d9b700 1 -- 172.21.15.27:0/1077496441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559031a449d0 con 0x559031ba58e0 2024-05-21T03:26:06.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.054+0000 7f43d5d9b700 1 --2- 172.21.15.27:0/1077496441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559031ba58e0 0x559031ba5d60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0099c0 tx=0x7f43cc003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2497de56caad3156 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:26:06.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.055+0000 7f43c4ff9700 1 -- 172.21.15.27:0/1077496441 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc027070 con 0x559031ba58e0 2024-05-21T03:26:06.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.055+0000 7f43c4ff9700 1 -- 172.21.15.27:0/1077496441 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43cc0050d0 con 0x559031ba58e0 2024-05-21T03:26:06.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.055+0000 7f43c4ff9700 1 -- 172.21.15.27:0/1077496441 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc00bc30 con 0x559031ba58e0 2024-05-21T03:26:06.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.055+0000 7f43e71bfb80 1 -- 172.21.15.27:0/1077496441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559031ba58e0 msgr2=0x559031ba5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:26:06.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.055+0000 7f43e71bfb80 1 --2- 172.21.15.27:0/1077496441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559031ba58e0 0x559031ba5d60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0099c0 tx=0x7f43cc003680 comp rx=0 tx=0).stop 2024-05-21T03:26:06.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.055+0000 7f43e71bfb80 1 -- 172.21.15.27:0/1077496441 shutdown_connections 2024-05-21T03:26:06.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.055+0000 7f43e71bfb80 1 --2- 172.21.15.27:0/1077496441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559031ba62a0 0x559031bb20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.056+0000 7f43e71bfb80 1 --2- 172.21.15.27:0/1077496441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559031bab2b0 0x559031bab690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.056+0000 7f43e71bfb80 1 --2- 172.21.15.27:0/1077496441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559031ba58e0 0x559031ba5d60 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.056+0000 7f43e71bfb80 1 -- 172.21.15.27:0/1077496441 >> 172.21.15.27:0/1077496441 conn(0x559031ad3180 msgr2=0x559031ba01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:26:06.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.056+0000 7f43e71bfb80 1 -- 172.21.15.27:0/1077496441 shutdown_connections 2024-05-21T03:26:06.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.056+0000 7f43e71bfb80 1 -- 172.21.15.27:0/1077496441 wait complete. 2024-05-21T03:26:06.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.056+0000 7f43e71bfb80 1 -- start start 2024-05-21T03:26:06.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.056+0000 7f43e71bfb80 1 Processor -- start 2024-05-21T03:26:06.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.056+0000 7f43e71bfb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559031ba58e0 0x559031bb1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:26:06.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.056+0000 7f43e71bfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559031ba62a0 0x559031baf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:26:06.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.057+0000 7f43e71bfb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559031bab2b0 0x559031baf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:26:06.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.057+0000 7f43e71bfb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5590319fb0a0 con 0x559031ba58e0 2024-05-21T03:26:06.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.057+0000 7f43e71bfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559031a968c0 con 0x559031ba62a0 2024-05-21T03:26:06.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.057+0000 7f43e71bfb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559031bb2680 con 0x559031bab2b0 2024-05-21T03:26:06.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.057+0000 7f43e71bfb80 10 client.0 _ll_register_callbacks cb 0x559031a8fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:26:06.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.057+0000 7f43d659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559031ba58e0 0x559031bb1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:26:06.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.057+0000 7f43d659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559031ba58e0 0x559031bb1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51504/0 (socket says 172.21.15.27:51504) 2024-05-21T03:26:06.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.057+0000 7f43d659c700 1 -- 172.21.15.27:0/4101164182 learned_addr learned my addr 172.21.15.27:0/4101164182 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:26:06.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.057+0000 7f43d5d9b700 1 --2- 172.21.15.27:0/4101164182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559031ba62a0 0x559031baf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:26:06.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.057+0000 7f43d6d9d700 1 --2- 172.21.15.27:0/4101164182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559031bab2b0 0x559031baf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:26:06.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.057+0000 7f43d659c700 1 -- 172.21.15.27:0/4101164182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559031bab2b0 msgr2=0x559031baf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:26:06.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.057+0000 7f43d659c700 1 --2- 172.21.15.27:0/4101164182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559031bab2b0 0x559031baf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.057+0000 7f43d659c700 1 -- 172.21.15.27:0/4101164182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559031ba62a0 msgr2=0x559031baf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:26:06.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.057+0000 7f43d659c700 1 --2- 172.21.15.27:0/4101164182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559031ba62a0 0x559031baf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.057+0000 7f43d659c700 1 -- 172.21.15.27:0/4101164182 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559031a93790 con 0x559031ba58e0 2024-05-21T03:26:06.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.057+0000 7f43d5d9b700 1 --2- 172.21.15.27:0/4101164182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559031ba62a0 0x559031baf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:26:06.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.057+0000 7f43d6d9d700 1 --2- 172.21.15.27:0/4101164182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559031bab2b0 0x559031baf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:26:06.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.057+0000 7f43d659c700 1 --2- 172.21.15.27:0/4101164182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559031ba58e0 0x559031bb1d80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f43c8015700 tx=0x7f43c8004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c6b6a842ec867e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:26:06.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.057+0000 7f43bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:26:06.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.057+0000 7f43bdffb700 1 -- 172.21.15.27:0/4101164182 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c8025020 con 0x559031ba58e0 2024-05-21T03:26:06.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.057+0000 7f43bdffb700 1 -- 172.21.15.27:0/4101164182 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f43c8015d20 con 0x559031ba58e0 2024-05-21T03:26:06.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.057+0000 7f43e71bfb80 1 -- 172.21.15.27:0/4101164182 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43cc0050d0 con 0x559031ba58e0 2024-05-21T03:26:06.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.058+0000 7f43bdffb700 1 -- 172.21.15.27:0/4101164182 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f43c8030050 con 0x559031ba58e0 2024-05-21T03:26:06.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.058+0000 7f43e71bfb80 10 client.15573 fetch_fsmap learned FSMap version 29 2024-05-21T03:26:06.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.058+0000 7f43e71bfb80 1 -- 172.21.15.27:0/4101164182 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559031bbcdf0 con 0x559031ba58e0 2024-05-21T03:26:06.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.058+0000 7f43bdffb700 1 -- 172.21.15.27:0/4101164182 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43cc0050d0 con 0x559031ba58e0 2024-05-21T03:26:06.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.058+0000 7f43bdffb700 1 -- 172.21.15.27:0/4101164182 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f43c8013450 con 0x559031ba58e0 2024-05-21T03:26:06.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.058+0000 7f43e71bfb80 10 client.15573 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T03:26:06.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.058+0000 7f43e71bfb80 10 client.15573 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-21T03:26:06.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.058+0000 7f43e71bfb80 20 client.15573 populate_metadata read hostname 'smithi027' 2024-05-21T03:26:06.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.058+0000 7f43e71bfb80 1 --2- 172.21.15.27:0/4101164182 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x559031bbf480 0x559031bc18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:26:06.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.058+0000 7f43e71bfb80 4 client.15573 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:26:06.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.058+0000 7f43e71bfb80 1 -- 172.21.15.27:0/4101164182 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559031b71ae0 con 0x559031bbf480 2024-05-21T03:26:06.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.059+0000 7f43d5d9b700 1 --2- 172.21.15.27:0/4101164182 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x559031bbf480 0x559031bc18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:26:06.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.059+0000 7f43d5d9b700 1 --2- 172.21.15.27:0/4101164182 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x559031bbf480 0x559031bc18c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b509c3de5633d server_cookie=23a22413c5550ffc in_seq=0 out_seq=0 2024-05-21T03:26:06.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.059+0000 7f43bdffb700 10 client.15573 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:26:06.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.060+0000 7f43bdffb700 1 -- 172.21.15.27:0/4101164182 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559031b71ae0 con 0x559031bbf480 2024-05-21T03:26:06.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.060+0000 7f43bdffb700 10 client.15573 handle_command_reply: tid=0 2024-05-21T03:26:06.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.060+0000 7f43e71bfb80 1 client.15573 shutdown 2024-05-21T03:26:06.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.060+0000 7f43e71bfb80 1 -- 172.21.15.27:0/4101164182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559031ba58e0 msgr2=0x559031bb1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:26:06.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.060+0000 7f43e71bfb80 1 --2- 172.21.15.27:0/4101164182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559031ba58e0 0x559031bb1d80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f43c8015700 tx=0x7f43c8004860 comp rx=0 tx=0).stop 2024-05-21T03:26:06.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.060+0000 7f43e71bfb80 1 -- 172.21.15.27:0/4101164182 shutdown_connections 2024-05-21T03:26:06.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.060+0000 7f43e71bfb80 1 --2- 172.21.15.27:0/4101164182 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x559031bbf480 0x559031bc18c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.060+0000 7f43e71bfb80 1 --2- 172.21.15.27:0/4101164182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559031bab2b0 0x559031baf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.060+0000 7f43e71bfb80 1 --2- 172.21.15.27:0/4101164182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559031ba58e0 0x559031bb1d80 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.060+0000 7f43bdffb700 0 client.15573 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:26:06.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.060+0000 7f43e71bfb80 1 --2- 172.21.15.27:0/4101164182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559031ba62a0 0x559031baf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.060+0000 7f43e71bfb80 1 -- 172.21.15.27:0/4101164182 >> 172.21.15.27:0/4101164182 conn(0x559031ad3180 msgr2=0x559031b9aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:26:06.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.060+0000 7f43e71bfb80 1 -- 172.21.15.27:0/4101164182 shutdown_connections 2024-05-21T03:26:06.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.060+0000 7f43e71bfb80 1 -- 172.21.15.27:0/4101164182 wait complete. 2024-05-21T03:26:06.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.060+0000 7f43e71bfb80 20 client.15573 trim_cache size 0 max 16384 2024-05-21T03:26:06.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.080+0000 7f43e71bfb80 1 Processor -- start 2024-05-21T03:26:06.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.080+0000 7f43e71bfb80 1 -- start start 2024-05-21T03:26:06.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.081+0000 7f43e71bfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559031ba62a0 0x559031bb0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:26:06.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.081+0000 7f43e71bfb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559031bab2b0 0x559031ba7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:26:06.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.081+0000 7f43e71bfb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559031bbb720 0x559031ba82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:26:06.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.081+0000 7f43e71bfb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f43d81add90 con 0x559031bab2b0 2024-05-21T03:26:06.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.081+0000 7f43e71bfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f43d81af8d0 con 0x559031ba62a0 2024-05-21T03:26:06.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.081+0000 7f43e71bfb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f43d81ae460 con 0x559031bbb720 2024-05-21T03:26:06.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.081+0000 7f43d5d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559031bab2b0 0x559031ba7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:26:06.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.081+0000 7f43d5d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559031bab2b0 0x559031ba7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51516/0 (socket says 172.21.15.27:51516) 2024-05-21T03:26:06.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.081+0000 7f43d5d9b700 1 -- 172.21.15.27:0/3872302466 learned_addr learned my addr 172.21.15.27:0/3872302466 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:26:06.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.081+0000 7f43d659c700 1 --2- 172.21.15.27:0/3872302466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559031ba62a0 0x559031bb0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:26:06.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.081+0000 7f43d6d9d700 1 --2- 172.21.15.27:0/3872302466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559031bbb720 0x559031ba82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:26:06.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.081+0000 7f43d5d9b700 1 -- 172.21.15.27:0/3872302466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559031bbb720 msgr2=0x559031ba82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:26:06.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.081+0000 7f43d5d9b700 1 --2- 172.21.15.27:0/3872302466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559031bbb720 0x559031ba82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.081+0000 7f43d5d9b700 1 -- 172.21.15.27:0/3872302466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559031ba62a0 msgr2=0x559031bb0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:26:06.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.081+0000 7f43d5d9b700 1 --2- 172.21.15.27:0/3872302466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559031ba62a0 0x559031bb0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.081+0000 7f43d5d9b700 1 -- 172.21.15.27:0/3872302466 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559031bc1f40 con 0x559031bab2b0 2024-05-21T03:26:06.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.081+0000 7f43d659c700 1 --2- 172.21.15.27:0/3872302466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559031ba62a0 0x559031bb0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:26:06.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.081+0000 7f43d5d9b700 1 --2- 172.21.15.27:0/3872302466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559031bab2b0 0x559031ba7d90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f43cc00b600 tx=0x7f43cc009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c23f7b86a54264 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:26:06.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.082+0000 7f43bffff700 1 -- 172.21.15.27:0/3872302466 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc027040 con 0x559031bab2b0 2024-05-21T03:26:06.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.082+0000 7f43bffff700 1 -- 172.21.15.27:0/3872302466 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43cc00b890 con 0x559031bab2b0 2024-05-21T03:26:06.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.082+0000 7f43bffff700 1 -- 172.21.15.27:0/3872302466 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc02e6e0 con 0x559031bab2b0 2024-05-21T03:26:06.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.082+0000 7f43e71bfb80 1 -- 172.21.15.27:0/3872302466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559031bab2b0 msgr2=0x559031ba7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:26:06.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.082+0000 7f43e71bfb80 1 --2- 172.21.15.27:0/3872302466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559031bab2b0 0x559031ba7d90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f43cc00b600 tx=0x7f43cc009f80 comp rx=0 tx=0).stop 2024-05-21T03:26:06.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.082+0000 7f43e71bfb80 1 -- 172.21.15.27:0/3872302466 shutdown_connections 2024-05-21T03:26:06.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.082+0000 7f43e71bfb80 1 --2- 172.21.15.27:0/3872302466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559031bbb720 0x559031ba82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.082+0000 7f43e71bfb80 1 --2- 172.21.15.27:0/3872302466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559031ba62a0 0x559031bb0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.082+0000 7f43e71bfb80 1 --2- 172.21.15.27:0/3872302466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559031bab2b0 0x559031ba7d90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.082+0000 7f43e71bfb80 1 -- 172.21.15.27:0/3872302466 >> 172.21.15.27:0/3872302466 conn(0x559031ad3180 msgr2=0x559031bb08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:26:06.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.082+0000 7f43e71bfb80 1 -- 172.21.15.27:0/3872302466 shutdown_connections 2024-05-21T03:26:06.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.082+0000 7f43e71bfb80 1 -- 172.21.15.27:0/3872302466 wait complete. 2024-05-21T03:26:06.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.083+0000 7f43e71bfb80 1 -- start start 2024-05-21T03:26:06.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.083+0000 7f43e71bfb80 1 Processor -- start 2024-05-21T03:26:06.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.083+0000 7f43e71bfb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559031ba62a0 0x559031bb7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:26:06.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.083+0000 7f43e71bfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559031bab2b0 0x559031bb7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:26:06.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.083+0000 7f43e71bfb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559031bbb720 0x559031c27050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:26:06.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.083+0000 7f43e71bfb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f43d81af200 con 0x559031bbb720 2024-05-21T03:26:06.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.083+0000 7f43e71bfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f43d81aeb30 con 0x559031bab2b0 2024-05-21T03:26:06.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.083+0000 7f43e71bfb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f43d81abd30 con 0x559031ba62a0 2024-05-21T03:26:06.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.083+0000 7f43e71bfb80 10 client.0 _ll_register_callbacks cb 0x559031a8fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:26:06.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.083+0000 7f43d6d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559031bbb720 0x559031c27050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:26:06.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.083+0000 7f43d6d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559031bbb720 0x559031c27050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51520/0 (socket says 172.21.15.27:51520) 2024-05-21T03:26:06.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.083+0000 7f43d659c700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559031ba62a0 0x559031bb7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:26:06.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.083+0000 7f43d6d9d700 1 -- 172.21.15.27:0/3695290189 learned_addr learned my addr 172.21.15.27:0/3695290189 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:26:06.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.083+0000 7f43d5d9b700 1 --2- 172.21.15.27:0/3695290189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559031bab2b0 0x559031bb7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:26:06.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.083+0000 7f43d6d9d700 1 -- 172.21.15.27:0/3695290189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559031ba62a0 msgr2=0x559031bb7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:26:06.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.083+0000 7f43d6d9d700 1 --2- 172.21.15.27:0/3695290189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559031ba62a0 0x559031bb7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.083+0000 7f43d6d9d700 1 -- 172.21.15.27:0/3695290189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559031bab2b0 msgr2=0x559031bb7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:26:06.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.083+0000 7f43d6d9d700 1 --2- 172.21.15.27:0/3695290189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559031bab2b0 0x559031bb7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.084+0000 7f43d6d9d700 1 -- 172.21.15.27:0/3695290189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559031bc7d10 con 0x559031bbb720 2024-05-21T03:26:06.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.084+0000 7f43d6d9d700 1 --2- 172.21.15.27:0/3695290189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559031bbb720 0x559031c27050 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f43d000a740 tx=0x7f43d0007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c94c477a3a622e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:26:06.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.084+0000 7f43bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:26:06.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.084+0000 7f43bdffb700 1 -- 172.21.15.27:0/3695290189 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d0019520 con 0x559031bbb720 2024-05-21T03:26:06.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.084+0000 7f43bdffb700 1 -- 172.21.15.27:0/3695290189 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f43d00a9b00 con 0x559031bbb720 2024-05-21T03:26:06.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.084+0000 7f43e71bfb80 1 -- 172.21.15.27:0/3695290189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43cc00b890 con 0x559031bbb720 2024-05-21T03:26:06.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.084+0000 7f43bdffb700 1 -- 172.21.15.27:0/3695290189 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f43d00a98a0 con 0x559031bbb720 2024-05-21T03:26:06.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.084+0000 7f43e71bfb80 10 client.15579 fetch_fsmap learned FSMap version 29 2024-05-21T03:26:06.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.084+0000 7f43e71bfb80 1 -- 172.21.15.27:0/3695290189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559031bc6810 con 0x559031bbb720 2024-05-21T03:26:06.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.084+0000 7f43bdffb700 1 -- 172.21.15.27:0/3695290189 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43cc00b890 con 0x559031bbb720 2024-05-21T03:26:06.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.084+0000 7f43bdffb700 1 -- 172.21.15.27:0/3695290189 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f43d00aabd0 con 0x559031bbb720 2024-05-21T03:26:06.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.084+0000 7f43e71bfb80 10 client.15579 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T03:26:06.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.084+0000 7f43e71bfb80 10 client.15579 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-21T03:26:06.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.084+0000 7f43e71bfb80 20 client.15579 populate_metadata read hostname 'smithi027' 2024-05-21T03:26:06.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.084+0000 7f43e71bfb80 1 --2- 172.21.15.27:0/3695290189 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x559031c30480 0x559031c32870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:26:06.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.084+0000 7f43e71bfb80 4 client.15579 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3ba64728", "release": true, "await": true}] 2024-05-21T03:26:06.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.084+0000 7f43e71bfb80 1 -- 172.21.15.27:0/3695290189 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3ba64728", "release": true, "await": true}) -- 0x559031b06ce0 con 0x559031c30480 2024-05-21T03:26:06.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.085+0000 7f43d659c700 1 --2- 172.21.15.27:0/3695290189 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x559031c30480 0x559031c32870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:26:06.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.085+0000 7f43bdffb700 10 client.15579 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:26:06.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.085+0000 7f43d659c700 1 --2- 172.21.15.27:0/3695290189 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x559031c30480 0x559031c32870 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=500f73640c84dfa0 server_cookie=ed0974345b3639f4 in_seq=0 out_seq=0 2024-05-21T03:26:06.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.089+0000 7f43bdffb700 1 -- 172.21.15.27:0/3695290189 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x559031b06ce0 con 0x559031c30480 2024-05-21T03:26:06.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.089+0000 7f43bdffb700 10 client.15579 handle_command_reply: tid=0 2024-05-21T03:26:06.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.089+0000 7f43e71bfb80 1 client.15579 shutdown 2024-05-21T03:26:06.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.089+0000 7f43e71bfb80 1 -- 172.21.15.27:0/3695290189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559031bbb720 msgr2=0x559031c27050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:26:06.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.089+0000 7f43e71bfb80 1 --2- 172.21.15.27:0/3695290189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559031bbb720 0x559031c27050 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f43d000a740 tx=0x7f43d0007f50 comp rx=0 tx=0).stop 2024-05-21T03:26:06.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.089+0000 7f43e71bfb80 1 -- 172.21.15.27:0/3695290189 shutdown_connections 2024-05-21T03:26:06.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.089+0000 7f43e71bfb80 1 --2- 172.21.15.27:0/3695290189 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x559031c30480 0x559031c32870 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.089+0000 7f43e71bfb80 1 --2- 172.21.15.27:0/3695290189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559031bbb720 0x559031c27050 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.089+0000 7f43e71bfb80 1 --2- 172.21.15.27:0/3695290189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559031ba62a0 0x559031bb7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.090+0000 7f43e71bfb80 1 --2- 172.21.15.27:0/3695290189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559031bab2b0 0x559031bb7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.090+0000 7f43bdffb700 0 client.15579 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:26:06.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.090+0000 7f43e71bfb80 1 -- 172.21.15.27:0/3695290189 >> 172.21.15.27:0/3695290189 conn(0x559031ad3180 msgr2=0x559031bbafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:26:06.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.090+0000 7f43e71bfb80 1 -- 172.21.15.27:0/3695290189 shutdown_connections 2024-05-21T03:26:06.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.090+0000 7f43e71bfb80 1 -- 172.21.15.27:0/3695290189 wait complete. 2024-05-21T03:26:06.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.090+0000 7f43e71bfb80 20 client.15579 trim_cache size 0 max 16384 2024-05-21T03:26:06.091 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:26:06.091 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 29, 2024-05-21T03:26:06.091 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24476, 2024-05-21T03:26:06.092 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 8, 2024-05-21T03:26:06.092 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-05-21T03:26:06.092 INFO:teuthology.orchestra.run.smithi027.stdout: "3ba64728": { 2024-05-21T03:26:06.092 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 8, 2024-05-21T03:26:06.092 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-05-21T03:26:06.092 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-05-21T03:26:06.092 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-05-21T03:26:06.092 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-05-21T03:26:06.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:26:06.092 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-05-21T03:26:06.092 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 150.0, 2024-05-21T03:26:06.092 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-05-21T03:26:06.092 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-05-21T03:26:06.093 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-05-21T03:26:06.093 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-05-21T03:26:06.093 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-05-21T03:26:06.093 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-05-21T03:26:06.093 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:26:06.093 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:26:06.093 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:26:06.093 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:26:06.093 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:26:06.093 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:26:06.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.090+0000 7f43dc929700 1 -- 172.21.15.27:0/437195422 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f43c00b29f0 msgr2=0x7f43c00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:26:06.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.090+0000 7f43dc929700 1 --2- 172.21.15.27:0/437195422 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f43c00b29f0 0x7f43c00b4eb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f43d8109e70 tx=0x7f43c8009190 comp rx=0 tx=0).stop 2024-05-21T03:26:06.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.091+0000 7f43dc929700 1 -- 172.21.15.27:0/437195422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d8109630 msgr2=0x7f43d8199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:26:06.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.091+0000 7f43dc929700 1 --2- 172.21.15.27:0/437195422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d8109630 0x7f43d8199af0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f43d0007d70 tx=0x7f43d000a340 comp rx=0 tx=0).stop 2024-05-21T03:26:06.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.091+0000 7f43dc929700 1 -- 172.21.15.27:0/437195422 shutdown_connections 2024-05-21T03:26:06.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.091+0000 7f43dc929700 1 --2- 172.21.15.27:0/437195422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d8109630 0x7f43d8199af0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.091+0000 7f43dc929700 1 --2- 172.21.15.27:0/437195422 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f43c00b29f0 0x7f43c00b4eb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.091+0000 7f43dc929700 1 --2- 172.21.15.27:0/437195422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43d806bd10 0x7f43d819f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.091+0000 7f43dc929700 1 --2- 172.21.15.27:0/437195422 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43d806c6d0 0x7f43d819fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:26:06.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.091+0000 7f43dc929700 1 -- 172.21.15.27:0/437195422 >> 172.21.15.27:0/437195422 conn(0x7f43d80794e0 msgr2=0x7f43d8100ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:26:06.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.091+0000 7f43dc929700 1 -- 172.21.15.27:0/437195422 shutdown_connections 2024-05-21T03:26:06.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:26:06.091+0000 7f43dc929700 1 -- 172.21.15.27:0/437195422 wait complete. 2024-05-21T03:26:06.095 INFO:tasks.workunit.client.0.smithi027.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-05-21T03:26:06.096 INFO:tasks.workunit.client.0.smithi027.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/suites 2024-05-21T03:26:06.096 INFO:tasks.workunit.client.0.smithi027.stderr:+ trap -- retry EXIT 2024-05-21T03:26:06.096 INFO:tasks.workunit.client.0.smithi027.stderr:+ rm -rf ffsb 2024-05-21T03:26:06.097 INFO:tasks.workunit.client.0.smithi027.stderr:+ timeout 1800 git clone https://git.ceph.com/ffsb.git --depth 1 2024-05-21T03:26:06.100 INFO:tasks.workunit.client.0.smithi027.stderr:Cloning into 'ffsb'... 2024-05-21T03:26:06.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:06 smithi027 ceph-mon[32179]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T03:26:06.739 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3ba64728', total seconds elapsed: 35.6 2024-05-21T03:26:07.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:06 smithi039 ceph-mon[36429]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T03:26:07.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:06 smithi165 ceph-mon[36269]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T03:26:07.098 INFO:tasks.workunit.client.0.smithi027.stderr:+ trap - EXIT 2024-05-21T03:26:07.098 INFO:tasks.workunit.client.0.smithi027.stderr:+ cd ffsb 2024-05-21T03:26:07.098 INFO:tasks.workunit.client.0.smithi027.stderr:+ ./configure 2024-05-21T03:26:07.235 INFO:tasks.workunit.client.0.smithi027.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-05-21T03:26:07.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:07 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:26:08.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:07 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:26:08.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:07 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:26:08.243 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether build environment is sane... yes 2024-05-21T03:26:08.252 INFO:tasks.workunit.client.0.smithi027.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-05-21T03:26:08.252 INFO:tasks.workunit.client.0.smithi027.stdout:checking for gawk... gawk 2024-05-21T03:26:08.267 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether make sets $(MAKE)... yes 2024-05-21T03:26:08.296 INFO:tasks.workunit.client.0.smithi027.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-05-21T03:26:08.296 INFO:tasks.workunit.client.0.smithi027.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-05-21T03:26:08.307 INFO:tasks.workunit.client.0.smithi027.stdout:checking for style of include used by make... GNU 2024-05-21T03:26:08.312 INFO:tasks.workunit.client.0.smithi027.stdout:checking for gcc... gcc 2024-05-21T03:26:08.363 INFO:tasks.workunit.client.0.smithi027.stdout:checking for C compiler default output file name... a.out 2024-05-21T03:26:08.365 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether the C compiler works... yes 2024-05-21T03:26:08.367 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether we are cross compiling... no 2024-05-21T03:26:08.403 INFO:tasks.workunit.client.0.smithi027.stdout:checking for suffix of executables... 2024-05-21T03:26:08.437 INFO:tasks.workunit.client.0.smithi027.stdout:checking for suffix of object files... o 2024-05-21T03:26:08.467 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether we are using the GNU C compiler... yes 2024-05-21T03:26:08.499 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether gcc accepts -g... yes 2024-05-21T03:26:08.579 INFO:tasks.workunit.client.0.smithi027.stdout:checking for gcc option to accept ISO C89... none needed 2024-05-21T03:26:08.691 INFO:tasks.workunit.client.0.smithi027.stdout:checking dependency style of gcc... gcc3 2024-05-21T03:26:08.774 INFO:tasks.workunit.client.0.smithi027.stdout:checking how to run the C preprocessor... gcc -E 2024-05-21T03:26:08.817 INFO:tasks.workunit.client.0.smithi027.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-05-21T03:26:08.819 INFO:tasks.workunit.client.0.smithi027.stdout:checking for egrep... /usr/bin/grep -E 2024-05-21T03:26:08.828 INFO:tasks.workunit.client.0.smithi027.stdout:checking for AIX... no 2024-05-21T03:26:08.849 INFO:tasks.workunit.client.0.smithi027.stdout:checking for gcc... (cached) gcc 2024-05-21T03:26:08.857 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-05-21T03:26:08.858 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether gcc accepts -g... (cached) yes 2024-05-21T03:26:08.858 INFO:tasks.workunit.client.0.smithi027.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-05-21T03:26:08.859 INFO:tasks.workunit.client.0.smithi027.stdout:checking dependency style of gcc... (cached) gcc3 2024-05-21T03:26:08.918 INFO:tasks.workunit.client.0.smithi027.stdout:checking for main in -lm... yes 2024-05-21T03:26:08.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:08 smithi027 ceph-mon[32179]: pgmap v66: 97 pgs: 97 active+clean; 2.8 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 149 KiB/s wr, 34 op/s 2024-05-21T03:26:08.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:08 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:26:08.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:08 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:26:08.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:08 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:26:08.982 INFO:tasks.workunit.client.0.smithi027.stdout:checking for main in -lpthread... yes 2024-05-21T03:26:09.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:08 smithi039 ceph-mon[36429]: pgmap v66: 97 pgs: 97 active+clean; 2.8 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 149 KiB/s wr, 34 op/s 2024-05-21T03:26:09.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:08 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:26:09.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:08 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:26:09.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:08 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:26:09.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:08 smithi165 ceph-mon[36269]: pgmap v66: 97 pgs: 97 active+clean; 2.8 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 149 KiB/s wr, 34 op/s 2024-05-21T03:26:09.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:08 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:26:09.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:08 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:26:09.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:08 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:26:09.154 INFO:tasks.workunit.client.0.smithi027.stdout:checking for ANSI C header files... yes 2024-05-21T03:26:09.221 INFO:tasks.workunit.client.0.smithi027.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-05-21T03:26:09.270 INFO:tasks.workunit.client.0.smithi027.stdout:checking for sys/types.h... yes 2024-05-21T03:26:09.318 INFO:tasks.workunit.client.0.smithi027.stdout:checking for sys/stat.h... yes 2024-05-21T03:26:09.366 INFO:tasks.workunit.client.0.smithi027.stdout:checking for stdlib.h... yes 2024-05-21T03:26:09.415 INFO:tasks.workunit.client.0.smithi027.stdout:checking for string.h... yes 2024-05-21T03:26:09.465 INFO:tasks.workunit.client.0.smithi027.stdout:checking for memory.h... yes 2024-05-21T03:26:09.514 INFO:tasks.workunit.client.0.smithi027.stdout:checking for strings.h... yes 2024-05-21T03:26:09.571 INFO:tasks.workunit.client.0.smithi027.stdout:checking for inttypes.h... yes 2024-05-21T03:26:09.629 INFO:tasks.workunit.client.0.smithi027.stdout:checking for stdint.h... yes 2024-05-21T03:26:09.681 INFO:tasks.workunit.client.0.smithi027.stdout:checking for unistd.h... yes 2024-05-21T03:26:09.746 INFO:tasks.workunit.client.0.smithi027.stdout:checking pthread.h usability... yes 2024-05-21T03:26:09.774 INFO:tasks.workunit.client.0.smithi027.stdout:checking pthread.h presence... yes 2024-05-21T03:26:09.774 INFO:tasks.workunit.client.0.smithi027.stdout:checking for pthread.h... yes 2024-05-21T03:26:09.829 INFO:tasks.workunit.client.0.smithi027.stdout:checking fcntl.h usability... yes 2024-05-21T03:26:09.850 INFO:tasks.workunit.client.0.smithi027.stdout:checking fcntl.h presence... yes 2024-05-21T03:26:09.850 INFO:tasks.workunit.client.0.smithi027.stdout:checking for fcntl.h... yes 2024-05-21T03:26:09.905 INFO:tasks.workunit.client.0.smithi027.stdout:checking limits.h usability... yes 2024-05-21T03:26:09.926 INFO:tasks.workunit.client.0.smithi027.stdout:checking limits.h presence... yes 2024-05-21T03:26:09.926 INFO:tasks.workunit.client.0.smithi027.stdout:checking for limits.h... yes 2024-05-21T03:26:09.942 INFO:tasks.workunit.client.0.smithi027.stdout:checking for stdint.h... (cached) yes 2024-05-21T03:26:09.996 INFO:tasks.workunit.client.0.smithi027.stdout:checking sys/time.h usability... yes 2024-05-21T03:26:10.020 INFO:tasks.workunit.client.0.smithi027.stdout:checking sys/time.h presence... yes 2024-05-21T03:26:10.020 INFO:tasks.workunit.client.0.smithi027.stdout:checking for sys/time.h... yes 2024-05-21T03:26:10.037 INFO:tasks.workunit.client.0.smithi027.stdout:checking for unistd.h... (cached) yes 2024-05-21T03:26:10.086 INFO:tasks.workunit.client.0.smithi027.stdout:checking sys/vfs.h usability... yes 2024-05-21T03:26:10.112 INFO:tasks.workunit.client.0.smithi027.stdout:checking sys/vfs.h presence... yes 2024-05-21T03:26:10.112 INFO:tasks.workunit.client.0.smithi027.stdout:checking for sys/vfs.h... yes 2024-05-21T03:26:10.162 INFO:tasks.workunit.client.0.smithi027.stdout:checking sys/limits.h usability... no 2024-05-21T03:26:10.184 INFO:tasks.workunit.client.0.smithi027.stdout:checking sys/limits.h presence... no 2024-05-21T03:26:10.185 INFO:tasks.workunit.client.0.smithi027.stdout:checking for sys/limits.h... no 2024-05-21T03:26:10.219 INFO:tasks.workunit.client.0.smithi027.stdout:checking for an ANSI C-conforming const... yes 2024-05-21T03:26:10.263 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-05-21T03:26:10.264 INFO:tasks.workunit.client.0.smithi027.stdout:checking for function prototypes... yes 2024-05-21T03:26:10.299 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether setvbuf arguments are reversed... no 2024-05-21T03:26:10.366 INFO:tasks.workunit.client.0.smithi027.stdout:checking for system... yes 2024-05-21T03:26:10.436 INFO:tasks.workunit.client.0.smithi027.stdout:checking for gettimeofday... yes 2024-05-21T03:26:10.515 INFO:tasks.workunit.client.0.smithi027.stdout:checking for mkdir... yes 2024-05-21T03:26:10.579 INFO:tasks.workunit.client.0.smithi027.stdout:checking for strerror... yes 2024-05-21T03:26:10.644 INFO:tasks.workunit.client.0.smithi027.stdout:checking for open64... yes 2024-05-21T03:26:10.708 INFO:tasks.workunit.client.0.smithi027.stdout:checking for stat64... yes 2024-05-21T03:26:10.775 INFO:tasks.workunit.client.0.smithi027.stdout:checking for fseeko64... yes 2024-05-21T03:26:10.839 INFO:tasks.workunit.client.0.smithi027.stdout:checking for lrand48_r... yes 2024-05-21T03:26:10.902 INFO:tasks.workunit.client.0.smithi027.stdout:checking for srand48_r... yes 2024-05-21T03:26:10.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:10 smithi027 ceph-mon[32179]: pgmap v67: 97 pgs: 97 active+clean; 2.8 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 149 KiB/s wr, 29 op/s 2024-05-21T03:26:10.927 INFO:tasks.workunit.client.0.smithi027.stdout:configure: creating ./config.status 2024-05-21T03:26:11.018 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:10 smithi039 ceph-mon[36429]: pgmap v67: 97 pgs: 97 active+clean; 2.8 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 149 KiB/s wr, 29 op/s 2024-05-21T03:26:11.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:10 smithi165 ceph-mon[36269]: pgmap v67: 97 pgs: 97 active+clean; 2.8 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 149 KiB/s wr, 29 op/s 2024-05-21T03:26:11.027 INFO:tasks.workunit.client.0.smithi027.stdout:config.status: creating Makefile 2024-05-21T03:26:11.041 INFO:tasks.workunit.client.0.smithi027.stdout:config.status: creating config.h 2024-05-21T03:26:11.058 INFO:tasks.workunit.client.0.smithi027.stdout:config.status: executing depfiles commands 2024-05-21T03:26:11.109 INFO:tasks.workunit.client.0.smithi027.stdout:config.status: executing default commands 2024-05-21T03:26:11.164 INFO:tasks.workunit.client.0.smithi027.stderr:+ make 2024-05-21T03:26:11.172 INFO:tasks.workunit.client.0.smithi027.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run aclocal-1.10 2024-05-21T03:26:11.174 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2024-05-21T03:26:11.175 INFO:tasks.workunit.client.0.smithi027.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2024-05-21T03:26:11.175 INFO:tasks.workunit.client.0.smithi027.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2024-05-21T03:26:11.175 INFO:tasks.workunit.client.0.smithi027.stderr: to install the `Automake' and `Perl' packages. Grab them from 2024-05-21T03:26:11.175 INFO:tasks.workunit.client.0.smithi027.stderr: any GNU archive site. 2024-05-21T03:26:11.177 INFO:tasks.workunit.client.0.smithi027.stdout: cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run automake-1.10 --gnu 2024-05-21T03:26:11.178 INFO:tasks.workunit.client.0.smithi027.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: automake-1.10: command not found 2024-05-21T03:26:11.179 INFO:tasks.workunit.client.0.smithi027.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2024-05-21T03:26:11.179 INFO:tasks.workunit.client.0.smithi027.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2024-05-21T03:26:11.179 INFO:tasks.workunit.client.0.smithi027.stderr: You might want to install the `Automake' and `Perl' packages. 2024-05-21T03:26:11.179 INFO:tasks.workunit.client.0.smithi027.stderr: Grab them from any GNU archive site. 2024-05-21T03:26:11.403 INFO:tasks.workunit.client.0.smithi027.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoconf 2024-05-21T03:26:11.708 INFO:tasks.workunit.client.0.smithi027.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-05-21T03:26:11.708 INFO:tasks.workunit.client.0.smithi027.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-05-21T03:26:11.708 INFO:tasks.workunit.client.0.smithi027.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-05-21T03:26:11.708 INFO:tasks.workunit.client.0.smithi027.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-05-21T03:26:11.841 INFO:tasks.workunit.client.0.smithi027.stdout:/bin/sh ./config.status --recheck 2024-05-21T03:26:11.895 INFO:tasks.workunit.client.0.smithi027.stdout:running CONFIG_SHELL=/bin/sh /bin/sh ./configure --no-create --no-recursion 2024-05-21T03:26:12.016 INFO:tasks.workunit.client.0.smithi027.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-05-21T03:26:12.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:12 smithi027 ceph-mon[32179]: pgmap v68: 97 pgs: 97 active+clean; 2.8 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 149 KiB/s wr, 29 op/s 2024-05-21T03:26:13.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:12 smithi039 ceph-mon[36429]: pgmap v68: 97 pgs: 97 active+clean; 2.8 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 149 KiB/s wr, 29 op/s 2024-05-21T03:26:13.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:12 smithi165 ceph-mon[36269]: pgmap v68: 97 pgs: 97 active+clean; 2.8 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 149 KiB/s wr, 29 op/s 2024-05-21T03:26:13.026 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether build environment is sane... yes 2024-05-21T03:26:13.035 INFO:tasks.workunit.client.0.smithi027.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-05-21T03:26:13.035 INFO:tasks.workunit.client.0.smithi027.stdout:checking for gawk... gawk 2024-05-21T03:26:13.059 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether make sets $(MAKE)... yes 2024-05-21T03:26:13.095 INFO:tasks.workunit.client.0.smithi027.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-05-21T03:26:13.096 INFO:tasks.workunit.client.0.smithi027.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-05-21T03:26:13.112 INFO:tasks.workunit.client.0.smithi027.stdout:checking for style of include used by make... GNU 2024-05-21T03:26:13.118 INFO:tasks.workunit.client.0.smithi027.stdout:checking for gcc... gcc 2024-05-21T03:26:13.216 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether the C compiler works... yes 2024-05-21T03:26:13.217 INFO:tasks.workunit.client.0.smithi027.stdout:checking for C compiler default output file name... a.out 2024-05-21T03:26:13.259 INFO:tasks.workunit.client.0.smithi027.stdout:checking for suffix of executables... 2024-05-21T03:26:13.301 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether we are cross compiling... no 2024-05-21T03:26:13.343 INFO:tasks.workunit.client.0.smithi027.stdout:checking for suffix of object files... o 2024-05-21T03:26:13.370 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether we are using the GNU C compiler... yes 2024-05-21T03:26:13.395 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether gcc accepts -g... yes 2024-05-21T03:26:13.445 INFO:tasks.workunit.client.0.smithi027.stdout:checking for gcc option to accept ISO C89... none needed 2024-05-21T03:26:13.580 INFO:tasks.workunit.client.0.smithi027.stdout:checking dependency style of gcc... gcc3 2024-05-21T03:26:13.630 INFO:tasks.workunit.client.0.smithi027.stdout:checking how to run the C preprocessor... gcc -E 2024-05-21T03:26:13.681 INFO:tasks.workunit.client.0.smithi027.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-05-21T03:26:13.682 INFO:tasks.workunit.client.0.smithi027.stdout:checking for egrep... /usr/bin/grep -E 2024-05-21T03:26:13.814 INFO:tasks.workunit.client.0.smithi027.stdout:checking for ANSI C header files... yes 2024-05-21T03:26:13.850 INFO:tasks.workunit.client.0.smithi027.stdout:checking for sys/types.h... yes 2024-05-21T03:26:13.889 INFO:tasks.workunit.client.0.smithi027.stdout:checking for sys/stat.h... yes 2024-05-21T03:26:13.929 INFO:tasks.workunit.client.0.smithi027.stdout:checking for stdlib.h... yes 2024-05-21T03:26:13.969 INFO:tasks.workunit.client.0.smithi027.stdout:checking for string.h... yes 2024-05-21T03:26:14.011 INFO:tasks.workunit.client.0.smithi027.stdout:checking for memory.h... yes 2024-05-21T03:26:14.052 INFO:tasks.workunit.client.0.smithi027.stdout:checking for strings.h... yes 2024-05-21T03:26:14.094 INFO:tasks.workunit.client.0.smithi027.stdout:checking for inttypes.h... yes 2024-05-21T03:26:14.138 INFO:tasks.workunit.client.0.smithi027.stdout:checking for stdint.h... yes 2024-05-21T03:26:14.185 INFO:tasks.workunit.client.0.smithi027.stdout:checking for unistd.h... yes 2024-05-21T03:26:14.237 INFO:tasks.workunit.client.0.smithi027.stdout:checking minix/config.h usability... no 2024-05-21T03:26:14.274 INFO:tasks.workunit.client.0.smithi027.stdout:checking minix/config.h presence... no 2024-05-21T03:26:14.275 INFO:tasks.workunit.client.0.smithi027.stdout:checking for minix/config.h... no 2024-05-21T03:26:14.316 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2024-05-21T03:26:14.316 INFO:tasks.workunit.client.0.smithi027.stdout:checking for gcc... (cached) gcc 2024-05-21T03:26:14.365 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-05-21T03:26:14.365 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether gcc accepts -g... (cached) yes 2024-05-21T03:26:14.365 INFO:tasks.workunit.client.0.smithi027.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-05-21T03:26:14.366 INFO:tasks.workunit.client.0.smithi027.stdout:checking dependency style of gcc... (cached) gcc3 2024-05-21T03:26:14.413 INFO:tasks.workunit.client.0.smithi027.stdout:checking for main in -lm... yes 2024-05-21T03:26:14.465 INFO:tasks.workunit.client.0.smithi027.stdout:checking for main in -lpthread... yes 2024-05-21T03:26:14.466 INFO:tasks.workunit.client.0.smithi027.stdout:checking for ANSI C header files... (cached) yes 2024-05-21T03:26:14.505 INFO:tasks.workunit.client.0.smithi027.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-05-21T03:26:14.576 INFO:tasks.workunit.client.0.smithi027.stdout:checking pthread.h usability... yes 2024-05-21T03:26:14.597 INFO:tasks.workunit.client.0.smithi027.stdout:checking pthread.h presence... yes 2024-05-21T03:26:14.597 INFO:tasks.workunit.client.0.smithi027.stdout:checking for pthread.h... yes 2024-05-21T03:26:14.646 INFO:tasks.workunit.client.0.smithi027.stdout:checking fcntl.h usability... yes 2024-05-21T03:26:14.664 INFO:tasks.workunit.client.0.smithi027.stdout:checking fcntl.h presence... yes 2024-05-21T03:26:14.664 INFO:tasks.workunit.client.0.smithi027.stdout:checking for fcntl.h... yes 2024-05-21T03:26:14.714 INFO:tasks.workunit.client.0.smithi027.stdout:checking limits.h usability... yes 2024-05-21T03:26:14.731 INFO:tasks.workunit.client.0.smithi027.stdout:checking limits.h presence... yes 2024-05-21T03:26:14.732 INFO:tasks.workunit.client.0.smithi027.stdout:checking for limits.h... yes 2024-05-21T03:26:14.746 INFO:tasks.workunit.client.0.smithi027.stdout:checking for stdint.h... (cached) yes 2024-05-21T03:26:14.787 INFO:tasks.workunit.client.0.smithi027.stdout:checking sys/time.h usability... yes 2024-05-21T03:26:14.806 INFO:tasks.workunit.client.0.smithi027.stdout:checking sys/time.h presence... yes 2024-05-21T03:26:14.806 INFO:tasks.workunit.client.0.smithi027.stdout:checking for sys/time.h... yes 2024-05-21T03:26:14.821 INFO:tasks.workunit.client.0.smithi027.stdout:checking for unistd.h... (cached) yes 2024-05-21T03:26:14.858 INFO:tasks.workunit.client.0.smithi027.stdout:checking sys/vfs.h usability... yes 2024-05-21T03:26:14.876 INFO:tasks.workunit.client.0.smithi027.stdout:checking sys/vfs.h presence... yes 2024-05-21T03:26:14.876 INFO:tasks.workunit.client.0.smithi027.stdout:checking for sys/vfs.h... yes 2024-05-21T03:26:14.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:14 smithi027 ceph-mon[32179]: pgmap v69: 97 pgs: 97 active+clean; 5.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 453 KiB/s wr, 76 op/s 2024-05-21T03:26:14.929 INFO:tasks.workunit.client.0.smithi027.stdout:checking sys/limits.h usability... no 2024-05-21T03:26:14.959 INFO:tasks.workunit.client.0.smithi027.stdout:checking sys/limits.h presence... no 2024-05-21T03:26:14.959 INFO:tasks.workunit.client.0.smithi027.stdout:checking for sys/limits.h... no 2024-05-21T03:26:14.985 INFO:tasks.workunit.client.0.smithi027.stdout:checking for an ANSI C-conforming const... yes 2024-05-21T03:26:15.015 INFO:tasks.workunit.client.0.smithi027.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-05-21T03:26:15.018 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:14 smithi039 ceph-mon[36429]: pgmap v69: 97 pgs: 97 active+clean; 5.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 453 KiB/s wr, 76 op/s 2024-05-21T03:26:15.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:14 smithi165 ceph-mon[36269]: pgmap v69: 97 pgs: 97 active+clean; 5.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 453 KiB/s wr, 76 op/s 2024-05-21T03:26:15.076 INFO:tasks.workunit.client.0.smithi027.stdout:checking for system... yes 2024-05-21T03:26:15.132 INFO:tasks.workunit.client.0.smithi027.stdout:checking for gettimeofday... yes 2024-05-21T03:26:15.194 INFO:tasks.workunit.client.0.smithi027.stdout:checking for mkdir... yes 2024-05-21T03:26:15.262 INFO:tasks.workunit.client.0.smithi027.stdout:checking for strerror... yes 2024-05-21T03:26:15.322 INFO:tasks.workunit.client.0.smithi027.stdout:checking for open64... yes 2024-05-21T03:26:15.385 INFO:tasks.workunit.client.0.smithi027.stdout:checking for stat64... yes 2024-05-21T03:26:15.449 INFO:tasks.workunit.client.0.smithi027.stdout:checking for fseeko64... yes 2024-05-21T03:26:15.511 INFO:tasks.workunit.client.0.smithi027.stdout:checking for lrand48_r... yes 2024-05-21T03:26:15.567 INFO:tasks.workunit.client.0.smithi027.stdout:checking for srand48_r... yes 2024-05-21T03:26:15.595 INFO:tasks.workunit.client.0.smithi027.stdout:configure: creating ./config.status 2024-05-21T03:26:15.661 INFO:tasks.workunit.client.0.smithi027.stdout: /bin/sh ./config.status 2024-05-21T03:26:15.714 INFO:tasks.workunit.client.0.smithi027.stdout:config.status: creating Makefile 2024-05-21T03:26:15.727 INFO:tasks.workunit.client.0.smithi027.stdout:config.status: creating config.h 2024-05-21T03:26:15.756 INFO:tasks.workunit.client.0.smithi027.stdout:config.status: executing depfiles commands 2024-05-21T03:26:15.767 INFO:tasks.workunit.client.0.smithi027.stdout:config.status: executing default commands 2024-05-21T03:26:15.794 INFO:tasks.workunit.client.0.smithi027.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoheader 2024-05-21T03:26:15.942 INFO:tasks.workunit.client.0.smithi027.stdout:rm -f stamp-h1 2024-05-21T03:26:15.946 INFO:tasks.workunit.client.0.smithi027.stdout:touch config.h.in 2024-05-21T03:26:15.947 INFO:tasks.workunit.client.0.smithi027.stdout:cd . && /bin/sh ./config.status config.h 2024-05-21T03:26:15.989 INFO:tasks.workunit.client.0.smithi027.stdout:config.status: creating config.h 2024-05-21T03:26:16.018 INFO:tasks.workunit.client.0.smithi027.stdout:make all-am 2024-05-21T03:26:16.024 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-05-21T03:26:16.024 INFO:tasks.workunit.client.0.smithi027.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2024-05-21T03:26:16.190 INFO:tasks.workunit.client.0.smithi027.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2024-05-21T03:26:16.195 INFO:tasks.workunit.client.0.smithi027.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2024-05-21T03:26:16.296 INFO:tasks.workunit.client.0.smithi027.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2024-05-21T03:26:16.299 INFO:tasks.workunit.client.0.smithi027.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2024-05-21T03:26:16.385 INFO:tasks.workunit.client.0.smithi027.stdout:mv -f .deps/main.Tpo .deps/main.Po 2024-05-21T03:26:16.388 INFO:tasks.workunit.client.0.smithi027.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2024-05-21T03:26:16.479 INFO:tasks.workunit.client.0.smithi027.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2024-05-21T03:26:16.482 INFO:tasks.workunit.client.0.smithi027.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2024-05-21T03:26:16.598 INFO:tasks.workunit.client.0.smithi027.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2024-05-21T03:26:16.602 INFO:tasks.workunit.client.0.smithi027.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2024-05-21T03:26:16.650 INFO:tasks.workunit.client.0.smithi027.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2024-05-21T03:26:16.653 INFO:tasks.workunit.client.0.smithi027.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2024-05-21T03:26:16.695 INFO:tasks.workunit.client.0.smithi027.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2024-05-21T03:26:16.698 INFO:tasks.workunit.client.0.smithi027.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2024-05-21T03:26:16.745 INFO:tasks.workunit.client.0.smithi027.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2024-05-21T03:26:16.748 INFO:tasks.workunit.client.0.smithi027.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2024-05-21T03:26:16.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:16 smithi027 ceph-mon[32179]: pgmap v70: 97 pgs: 97 active+clean; 5.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 431 KiB/s wr, 57 op/s 2024-05-21T03:26:16.954 INFO:tasks.workunit.client.0.smithi027.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2024-05-21T03:26:16.958 INFO:tasks.workunit.client.0.smithi027.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2024-05-21T03:26:17.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:16 smithi039 ceph-mon[36429]: pgmap v70: 97 pgs: 97 active+clean; 5.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 431 KiB/s wr, 57 op/s 2024-05-21T03:26:17.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:16 smithi165 ceph-mon[36269]: pgmap v70: 97 pgs: 97 active+clean; 5.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 431 KiB/s wr, 57 op/s 2024-05-21T03:26:17.088 INFO:tasks.workunit.client.0.smithi027.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2024-05-21T03:26:17.092 INFO:tasks.workunit.client.0.smithi027.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2024-05-21T03:26:17.242 INFO:tasks.workunit.client.0.smithi027.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2024-05-21T03:26:17.245 INFO:tasks.workunit.client.0.smithi027.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2024-05-21T03:26:17.312 INFO:tasks.workunit.client.0.smithi027.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2024-05-21T03:26:17.315 INFO:tasks.workunit.client.0.smithi027.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2024-05-21T03:26:17.387 INFO:tasks.workunit.client.0.smithi027.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2024-05-21T03:26:17.390 INFO:tasks.workunit.client.0.smithi027.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2024-05-21T03:26:17.494 INFO:tasks.workunit.client.0.smithi027.stdout:mv -f .deps/util.Tpo .deps/util.Po 2024-05-21T03:26:17.497 INFO:tasks.workunit.client.0.smithi027.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2024-05-21T03:26:17.527 INFO:tasks.workunit.client.0.smithi027.stderr:In file included from parser.h:22, 2024-05-21T03:26:17.527 INFO:tasks.workunit.client.0.smithi027.stderr: from parser.c:25: 2024-05-21T03:26:17.527 INFO:tasks.workunit.client.0.smithi027.stderr:parser.c: In function ‘init_filesys’: 2024-05-21T03:26:17.527 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-21T03:26:17.527 INFO:tasks.workunit.client.0.smithi027.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-21T03:26:17.527 INFO:tasks.workunit.client.0.smithi027.stderr: | ^ 2024-05-21T03:26:17.528 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-21T03:26:17.528 INFO:tasks.workunit.client.0.smithi027.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-21T03:26:17.528 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~ 2024-05-21T03:26:17.528 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-21T03:26:17.528 INFO:tasks.workunit.client.0.smithi027.stderr: 36 | container_of(ptr, type, member) 2024-05-21T03:26:17.528 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~~~~~ 2024-05-21T03:26:17.528 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-05-21T03:26:17.528 INFO:tasks.workunit.client.0.smithi027.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-05-21T03:26:17.528 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~~~ 2024-05-21T03:26:17.528 INFO:tasks.workunit.client.0.smithi027.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-05-21T03:26:17.528 INFO:tasks.workunit.client.0.smithi027.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-05-21T03:26:17.529 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-21T03:26:17.529 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-21T03:26:17.529 INFO:tasks.workunit.client.0.smithi027.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-21T03:26:17.529 INFO:tasks.workunit.client.0.smithi027.stderr: | ^ 2024-05-21T03:26:17.529 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-21T03:26:17.529 INFO:tasks.workunit.client.0.smithi027.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-21T03:26:17.529 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~ 2024-05-21T03:26:17.529 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-21T03:26:17.529 INFO:tasks.workunit.client.0.smithi027.stderr: 36 | container_of(ptr, type, member) 2024-05-21T03:26:17.529 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~~~~~ 2024-05-21T03:26:17.529 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-05-21T03:26:17.529 INFO:tasks.workunit.client.0.smithi027.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-05-21T03:26:17.529 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~~~ 2024-05-21T03:26:17.530 INFO:tasks.workunit.client.0.smithi027.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-05-21T03:26:17.530 INFO:tasks.workunit.client.0.smithi027.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-05-21T03:26:17.530 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-21T03:26:17.530 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-21T03:26:17.530 INFO:tasks.workunit.client.0.smithi027.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-21T03:26:17.530 INFO:tasks.workunit.client.0.smithi027.stderr: | ^ 2024-05-21T03:26:17.530 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-21T03:26:17.530 INFO:tasks.workunit.client.0.smithi027.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-21T03:26:17.530 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~ 2024-05-21T03:26:17.530 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-21T03:26:17.530 INFO:tasks.workunit.client.0.smithi027.stderr: 36 | container_of(ptr, type, member) 2024-05-21T03:26:17.531 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~~~~~ 2024-05-21T03:26:17.531 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-05-21T03:26:17.531 INFO:tasks.workunit.client.0.smithi027.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-05-21T03:26:17.531 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~~~ 2024-05-21T03:26:17.531 INFO:tasks.workunit.client.0.smithi027.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-05-21T03:26:17.531 INFO:tasks.workunit.client.0.smithi027.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-05-21T03:26:17.531 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-21T03:26:17.531 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-21T03:26:17.531 INFO:tasks.workunit.client.0.smithi027.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-21T03:26:17.531 INFO:tasks.workunit.client.0.smithi027.stderr: | ^ 2024-05-21T03:26:17.531 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-21T03:26:17.531 INFO:tasks.workunit.client.0.smithi027.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-21T03:26:17.532 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~ 2024-05-21T03:26:17.532 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-21T03:26:17.532 INFO:tasks.workunit.client.0.smithi027.stderr: 36 | container_of(ptr, type, member) 2024-05-21T03:26:17.532 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~~~~~ 2024-05-21T03:26:17.532 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-05-21T03:26:17.532 INFO:tasks.workunit.client.0.smithi027.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-05-21T03:26:17.532 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~~~ 2024-05-21T03:26:17.532 INFO:tasks.workunit.client.0.smithi027.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-05-21T03:26:17.532 INFO:tasks.workunit.client.0.smithi027.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-05-21T03:26:17.532 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-21T03:26:17.532 INFO:tasks.workunit.client.0.smithi027.stderr:parser.c: In function ‘init_tg_stats’: 2024-05-21T03:26:17.532 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-21T03:26:17.532 INFO:tasks.workunit.client.0.smithi027.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-21T03:26:17.533 INFO:tasks.workunit.client.0.smithi027.stderr: | ^ 2024-05-21T03:26:17.533 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-21T03:26:17.533 INFO:tasks.workunit.client.0.smithi027.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-21T03:26:17.533 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~ 2024-05-21T03:26:17.533 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-21T03:26:17.533 INFO:tasks.workunit.client.0.smithi027.stderr: 36 | container_of(ptr, type, member) 2024-05-21T03:26:17.533 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~~~~~ 2024-05-21T03:26:17.533 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-05-21T03:26:17.533 INFO:tasks.workunit.client.0.smithi027.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-05-21T03:26:17.533 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~~~ 2024-05-21T03:26:17.533 INFO:tasks.workunit.client.0.smithi027.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-05-21T03:26:17.533 INFO:tasks.workunit.client.0.smithi027.stderr: 971 | list_for_each_entry(tmp_list, 2024-05-21T03:26:17.533 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-21T03:26:17.533 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-21T03:26:17.534 INFO:tasks.workunit.client.0.smithi027.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-21T03:26:17.534 INFO:tasks.workunit.client.0.smithi027.stderr: | ^ 2024-05-21T03:26:17.534 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-21T03:26:17.534 INFO:tasks.workunit.client.0.smithi027.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-21T03:26:17.534 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~ 2024-05-21T03:26:17.534 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-21T03:26:17.534 INFO:tasks.workunit.client.0.smithi027.stderr: 36 | container_of(ptr, type, member) 2024-05-21T03:26:17.534 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~~~~~ 2024-05-21T03:26:17.534 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-05-21T03:26:17.534 INFO:tasks.workunit.client.0.smithi027.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-05-21T03:26:17.534 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~~~ 2024-05-21T03:26:17.534 INFO:tasks.workunit.client.0.smithi027.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-05-21T03:26:17.535 INFO:tasks.workunit.client.0.smithi027.stderr: 971 | list_for_each_entry(tmp_list, 2024-05-21T03:26:17.535 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-21T03:26:17.535 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-21T03:26:17.535 INFO:tasks.workunit.client.0.smithi027.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-21T03:26:17.535 INFO:tasks.workunit.client.0.smithi027.stderr: | ^ 2024-05-21T03:26:17.535 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-21T03:26:17.535 INFO:tasks.workunit.client.0.smithi027.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-21T03:26:17.535 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~ 2024-05-21T03:26:17.535 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-21T03:26:17.535 INFO:tasks.workunit.client.0.smithi027.stderr: 36 | container_of(ptr, type, member) 2024-05-21T03:26:17.535 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~~~~~ 2024-05-21T03:26:17.535 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-05-21T03:26:17.536 INFO:tasks.workunit.client.0.smithi027.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-05-21T03:26:17.536 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~~~ 2024-05-21T03:26:17.536 INFO:tasks.workunit.client.0.smithi027.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-05-21T03:26:17.536 INFO:tasks.workunit.client.0.smithi027.stderr: 980 | list_for_each_entry(tmp_list, 2024-05-21T03:26:17.536 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-21T03:26:17.536 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-21T03:26:17.536 INFO:tasks.workunit.client.0.smithi027.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-21T03:26:17.536 INFO:tasks.workunit.client.0.smithi027.stderr: | ^ 2024-05-21T03:26:17.536 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-21T03:26:17.536 INFO:tasks.workunit.client.0.smithi027.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-21T03:26:17.536 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~ 2024-05-21T03:26:17.536 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-21T03:26:17.536 INFO:tasks.workunit.client.0.smithi027.stderr: 36 | container_of(ptr, type, member) 2024-05-21T03:26:17.537 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~~~~~ 2024-05-21T03:26:17.537 INFO:tasks.workunit.client.0.smithi027.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-05-21T03:26:17.537 INFO:tasks.workunit.client.0.smithi027.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-05-21T03:26:17.537 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~~~ 2024-05-21T03:26:17.537 INFO:tasks.workunit.client.0.smithi027.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-05-21T03:26:17.537 INFO:tasks.workunit.client.0.smithi027.stderr: 980 | list_for_each_entry(tmp_list, 2024-05-21T03:26:17.537 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-21T03:26:17.572 INFO:tasks.workunit.client.0.smithi027.stderr:parser.c: In function ‘get_optstr’: 2024-05-21T03:26:17.572 INFO:tasks.workunit.client.0.smithi027.stderr:parser.c:206:23: warning: ‘strnlen’ specified bound 4095 exceeds source size 1024 [-Wstringop-overread] 2024-05-21T03:26:17.573 INFO:tasks.workunit.client.0.smithi027.stderr: 206 | len = strnlen(temp, 4095) + 1; 2024-05-21T03:26:17.573 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-21T03:26:17.573 INFO:tasks.workunit.client.0.smithi027.stderr:parser.c:200:14: note: source object declared here 2024-05-21T03:26:17.573 INFO:tasks.workunit.client.0.smithi027.stderr: 200 | char temp[BUFSIZE]; 2024-05-21T03:26:17.573 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~ 2024-05-21T03:26:17.579 INFO:tasks.workunit.client.0.smithi027.stderr:parser.c: In function ‘size64_convert’: 2024-05-21T03:26:17.579 INFO:tasks.workunit.client.0.smithi027.stderr:parser.c:119:9: warning: ‘memcpy’ reading 3 bytes from a region of size 2 [-Wstringop-overread] 2024-05-21T03:26:17.579 INFO:tasks.workunit.client.0.smithi027.stderr: 119 | memcpy(unit, "\0", 3); 2024-05-21T03:26:17.579 INFO:tasks.workunit.client.0.smithi027.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-05-21T03:26:17.867 INFO:tasks.workunit.client.0.smithi027.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2024-05-21T03:26:17.870 INFO:tasks.workunit.client.0.smithi027.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2024-05-21T03:26:17.927 INFO:tasks.workunit.client.0.smithi027.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2024-05-21T03:26:17.931 INFO:tasks.workunit.client.0.smithi027.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2024-05-21T03:26:18.022 INFO:tasks.workunit.client.0.smithi027.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2024-05-21T03:26:18.026 INFO:tasks.workunit.client.0.smithi027.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2024-05-21T03:26:18.056 INFO:tasks.workunit.client.0.smithi027.stdout:mv -f .deps/list.Tpo .deps/list.Po 2024-05-21T03:26:18.061 INFO:tasks.workunit.client.0.smithi027.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2024-05-21T03:26:18.095 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-05-21T03:26:18.095 INFO:tasks.workunit.client.0.smithi027.stderr:+ cd .. 2024-05-21T03:26:18.095 INFO:tasks.workunit.client.0.smithi027.stderr:+ mkdir tmp 2024-05-21T03:26:18.098 INFO:tasks.workunit.client.0.smithi027.stderr:+ cd tmp 2024-05-21T03:26:18.098 INFO:tasks.workunit.client.0.smithi027.stderr:+ for f in $mydir/*.ffsb 2024-05-21T03:26:18.098 INFO:tasks.workunit.client.0.smithi027.stderr:+ ../ffsb/ffsb /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/random_write.32.ffsb 2024-05-21T03:26:18.098 INFO:tasks.workunit.client.0.smithi027.stdout:FFSB version 6.0-RC2 started 2024-05-21T03:26:18.099 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-05-21T03:26:18.101 INFO:tasks.workunit.client.0.smithi027.stdout:benchmark time = 300 2024-05-21T03:26:18.101 INFO:tasks.workunit.client.0.smithi027.stdout:ThreadGroup 0 2024-05-21T03:26:18.101 INFO:tasks.workunit.client.0.smithi027.stdout:================ 2024-05-21T03:26:18.101 INFO:tasks.workunit.client.0.smithi027.stdout: num_threads = 32 2024-05-21T03:26:18.101 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-05-21T03:26:18.101 INFO:tasks.workunit.client.0.smithi027.stdout: read_random = off 2024-05-21T03:26:18.101 INFO:tasks.workunit.client.0.smithi027.stdout: read_size = 0 (0B) 2024-05-21T03:26:18.102 INFO:tasks.workunit.client.0.smithi027.stdout: read_blocksize = 0 (0B) 2024-05-21T03:26:18.102 INFO:tasks.workunit.client.0.smithi027.stdout: read_skip = off 2024-05-21T03:26:18.102 INFO:tasks.workunit.client.0.smithi027.stdout: read_skipsize = 0 (0B) 2024-05-21T03:26:18.102 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-05-21T03:26:18.102 INFO:tasks.workunit.client.0.smithi027.stdout: write_random = on 2024-05-21T03:26:18.102 INFO:tasks.workunit.client.0.smithi027.stdout: write_size = 5242880 (5MB) 2024-05-21T03:26:18.102 INFO:tasks.workunit.client.0.smithi027.stdout: fsync_file = 0 2024-05-21T03:26:18.102 INFO:tasks.workunit.client.0.smithi027.stdout: write_blocksize = 4096 (4KB) 2024-05-21T03:26:18.102 INFO:tasks.workunit.client.0.smithi027.stdout: wait time = 0 2024-05-21T03:26:18.102 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-05-21T03:26:18.102 INFO:tasks.workunit.client.0.smithi027.stdout: op weights 2024-05-21T03:26:18.102 INFO:tasks.workunit.client.0.smithi027.stdout: read = 0 (0.00%) 2024-05-21T03:26:18.102 INFO:tasks.workunit.client.0.smithi027.stdout: readall = 0 (0.00%) 2024-05-21T03:26:18.103 INFO:tasks.workunit.client.0.smithi027.stdout: write = 1 (100.00%) 2024-05-21T03:26:18.103 INFO:tasks.workunit.client.0.smithi027.stdout: create = 0 (0.00%) 2024-05-21T03:26:18.103 INFO:tasks.workunit.client.0.smithi027.stdout: append = 0 (0.00%) 2024-05-21T03:26:18.103 INFO:tasks.workunit.client.0.smithi027.stdout: delete = 0 (0.00%) 2024-05-21T03:26:18.103 INFO:tasks.workunit.client.0.smithi027.stdout: metaop = 0 (0.00%) 2024-05-21T03:26:18.103 INFO:tasks.workunit.client.0.smithi027.stdout: createdir = 0 (0.00%) 2024-05-21T03:26:18.103 INFO:tasks.workunit.client.0.smithi027.stdout: stat = 0 (0.00%) 2024-05-21T03:26:18.103 INFO:tasks.workunit.client.0.smithi027.stdout: writeall = 0 (0.00%) 2024-05-21T03:26:18.103 INFO:tasks.workunit.client.0.smithi027.stdout: writeall_fsync = 0 (0.00%) 2024-05-21T03:26:18.103 INFO:tasks.workunit.client.0.smithi027.stdout: open_close = 0 (0.00%) 2024-05-21T03:26:18.103 INFO:tasks.workunit.client.0.smithi027.stdout: write_fsync = 0 (0.00%) 2024-05-21T03:26:18.103 INFO:tasks.workunit.client.0.smithi027.stdout: create_fsync = 0 (0.00%) 2024-05-21T03:26:18.103 INFO:tasks.workunit.client.0.smithi027.stdout: append_fsync = 0 (0.00%) 2024-05-21T03:26:18.103 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-05-21T03:26:18.104 INFO:tasks.workunit.client.0.smithi027.stdout:FileSystem . 2024-05-21T03:26:18.104 INFO:tasks.workunit.client.0.smithi027.stdout:========== 2024-05-21T03:26:18.104 INFO:tasks.workunit.client.0.smithi027.stdout: num_dirs = 0 2024-05-21T03:26:18.104 INFO:tasks.workunit.client.0.smithi027.stdout: starting files = 128 2024-05-21T03:26:18.104 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-05-21T03:26:18.104 INFO:tasks.workunit.client.0.smithi027.stdout: min file size = 104857600 (100MB) 2024-05-21T03:26:18.104 INFO:tasks.workunit.client.0.smithi027.stdout: max file size = 104857600 (100MB) 2024-05-21T03:26:18.104 INFO:tasks.workunit.client.0.smithi027.stdout: directio = off 2024-05-21T03:26:18.104 INFO:tasks.workunit.client.0.smithi027.stdout: alignedio = on 2024-05-21T03:26:18.104 INFO:tasks.workunit.client.0.smithi027.stdout: bufferedio = off 2024-05-21T03:26:18.104 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-05-21T03:26:18.104 INFO:tasks.workunit.client.0.smithi027.stdout: aging is off 2024-05-21T03:26:18.105 INFO:tasks.workunit.client.0.smithi027.stdout: current utilization = 0.00% 2024-05-21T03:26:18.105 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-05-21T03:26:18.105 INFO:tasks.workunit.client.0.smithi027.stdout:checking existing fs: . 2024-05-21T03:26:18.105 INFO:tasks.workunit.client.0.smithi027.stdout:recreating new fileset 2024-05-21T03:26:18.105 INFO:tasks.workunit.client.0.smithi027.stderr:opendir: No such file or directory 2024-05-21T03:26:18.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:18 smithi027 ceph-mon[32179]: pgmap v71: 97 pgs: 97 active+clean; 7.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 575 KiB/s wr, 70 op/s 2024-05-21T03:26:19.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:18 smithi039 ceph-mon[36429]: pgmap v71: 97 pgs: 97 active+clean; 7.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 575 KiB/s wr, 70 op/s 2024-05-21T03:26:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:18 smithi165 ceph-mon[36269]: pgmap v71: 97 pgs: 97 active+clean; 7.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 575 KiB/s wr, 70 op/s 2024-05-21T03:26:21.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:20 smithi039 ceph-mon[36429]: pgmap v72: 97 pgs: 97 active+clean; 7.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 448 KiB/s wr, 59 op/s 2024-05-21T03:26:21.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:20 smithi165 ceph-mon[36269]: pgmap v72: 97 pgs: 97 active+clean; 7.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 448 KiB/s wr, 59 op/s 2024-05-21T03:26:21.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:20 smithi027 ceph-mon[32179]: pgmap v72: 97 pgs: 97 active+clean; 7.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 448 KiB/s wr, 59 op/s 2024-05-21T03:26:23.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:22 smithi039 ceph-mon[36429]: pgmap v73: 97 pgs: 97 active+clean; 7.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 448 KiB/s wr, 59 op/s 2024-05-21T03:26:23.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:22 smithi165 ceph-mon[36269]: pgmap v73: 97 pgs: 97 active+clean; 7.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 448 KiB/s wr, 59 op/s 2024-05-21T03:26:23.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:22 smithi027 ceph-mon[32179]: pgmap v73: 97 pgs: 97 active+clean; 7.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 448 KiB/s wr, 59 op/s 2024-05-21T03:26:25.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:24 smithi039 ceph-mon[36429]: pgmap v74: 97 pgs: 97 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 127 MiB/s wr, 165 op/s 2024-05-21T03:26:25.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:24 smithi165 ceph-mon[36269]: pgmap v74: 97 pgs: 97 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 127 MiB/s wr, 165 op/s 2024-05-21T03:26:25.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:24 smithi027 ceph-mon[32179]: pgmap v74: 97 pgs: 97 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 127 MiB/s wr, 165 op/s 2024-05-21T03:26:26.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:25 smithi027 ceph-mon[32179]: pgmap v75: 97 pgs: 97 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 127 MiB/s wr, 118 op/s 2024-05-21T03:26:26.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:25 smithi039 ceph-mon[36429]: pgmap v75: 97 pgs: 97 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 127 MiB/s wr, 118 op/s 2024-05-21T03:26:26.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:25 smithi165 ceph-mon[36269]: pgmap v75: 97 pgs: 97 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 127 MiB/s wr, 118 op/s 2024-05-21T03:26:28.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:28 smithi027 ceph-mon[32179]: pgmap v76: 97 pgs: 97 active+clean; 4.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 417 MiB/s wr, 223 op/s 2024-05-21T03:26:28.970 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:28 smithi039 ceph-mon[36429]: pgmap v76: 97 pgs: 97 active+clean; 4.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 417 MiB/s wr, 223 op/s 2024-05-21T03:26:28.972 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:28 smithi165 ceph-mon[36269]: pgmap v76: 97 pgs: 97 active+clean; 4.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 417 MiB/s wr, 223 op/s 2024-05-21T03:26:30.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:30 smithi027 ceph-mon[32179]: pgmap v77: 97 pgs: 97 active+clean; 4.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 417 MiB/s wr, 210 op/s 2024-05-21T03:26:31.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:30 smithi039 ceph-mon[36429]: pgmap v77: 97 pgs: 97 active+clean; 4.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 417 MiB/s wr, 210 op/s 2024-05-21T03:26:31.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:30 smithi165 ceph-mon[36269]: pgmap v77: 97 pgs: 97 active+clean; 4.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 417 MiB/s wr, 210 op/s 2024-05-21T03:26:31.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:31 smithi027 ceph-mon[32179]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:26:32.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:31 smithi039 ceph-mon[36429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:26:32.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:31 smithi165 ceph-mon[36269]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:26:32.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:32 smithi027 ceph-mon[32179]: pgmap v78: 97 pgs: 97 active+clean; 4.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 417 MiB/s wr, 210 op/s 2024-05-21T03:26:33.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:32 smithi039 ceph-mon[36429]: pgmap v78: 97 pgs: 97 active+clean; 4.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 417 MiB/s wr, 210 op/s 2024-05-21T03:26:33.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:32 smithi165 ceph-mon[36269]: pgmap v78: 97 pgs: 97 active+clean; 4.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 417 MiB/s wr, 210 op/s 2024-05-21T03:26:34.637 INFO:tasks.workunit.client.0.smithi027.stdout:fs setup took 16 secs 2024-05-21T03:26:35.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:34 smithi039 ceph-mon[36429]: pgmap v79: 97 pgs: 97 active+clean; 8.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 703 MiB/s wr, 319 op/s 2024-05-21T03:26:35.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:34 smithi165 ceph-mon[36269]: pgmap v79: 97 pgs: 97 active+clean; 8.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 703 MiB/s wr, 319 op/s 2024-05-21T03:26:35.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:34 smithi027 ceph-mon[32179]: pgmap v79: 97 pgs: 97 active+clean; 8.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 703 MiB/s wr, 319 op/s 2024-05-21T03:26:36.627 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:36 smithi027 ceph-mon[32179]: pgmap v80: 97 pgs: 97 active+clean; 8.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 576 MiB/s wr, 214 op/s 2024-05-21T03:26:36.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:36 smithi039 ceph-mon[36429]: pgmap v80: 97 pgs: 97 active+clean; 8.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 576 MiB/s wr, 214 op/s 2024-05-21T03:26:36.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:36 smithi165 ceph-mon[36269]: pgmap v80: 97 pgs: 97 active+clean; 8.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 576 MiB/s wr, 214 op/s 2024-05-21T03:26:37.671 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:37 smithi027 ceph-mon[32179]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:26:37.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:37 smithi039 ceph-mon[36429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:26:37.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:37 smithi165 ceph-mon[36269]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:26:38.180 INFO:tasks.workunit.client.0.smithi027.stdout:Syncing()...3 sec 2024-05-21T03:26:38.181 INFO:tasks.workunit.client.0.smithi027.stdout:Starting Actual Benchmark At: Tue May 21 03:26:38 2024 2024-05-21T03:26:38.181 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-05-21T03:26:38.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:38 smithi027 ceph-mon[32179]: pgmap v81: 97 pgs: 97 active+clean; 12 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 891 MiB/s wr, 312 op/s 2024-05-21T03:26:38.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:38 smithi039 ceph-mon[36429]: pgmap v81: 97 pgs: 97 active+clean; 12 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 891 MiB/s wr, 312 op/s 2024-05-21T03:26:38.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:38 smithi165 ceph-mon[36269]: pgmap v81: 97 pgs: 97 active+clean; 12 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 891 MiB/s wr, 312 op/s 2024-05-21T03:26:40.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:40 smithi039 ceph-mon[36429]: pgmap v82: 97 pgs: 97 active+clean; 12 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 600 MiB/s wr, 207 op/s 2024-05-21T03:26:40.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:40 smithi165 ceph-mon[36269]: pgmap v82: 97 pgs: 97 active+clean; 12 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 600 MiB/s wr, 207 op/s 2024-05-21T03:26:40.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:40 smithi027 ceph-mon[32179]: pgmap v82: 97 pgs: 97 active+clean; 12 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 600 MiB/s wr, 207 op/s 2024-05-21T03:26:42.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:42 smithi039 ceph-mon[36429]: pgmap v83: 97 pgs: 97 active+clean; 12 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 600 MiB/s wr, 207 op/s 2024-05-21T03:26:42.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:42 smithi165 ceph-mon[36269]: pgmap v83: 97 pgs: 97 active+clean; 12 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 600 MiB/s wr, 207 op/s 2024-05-21T03:26:42.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:42 smithi027 ceph-mon[32179]: pgmap v83: 97 pgs: 97 active+clean; 12 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 600 MiB/s wr, 207 op/s 2024-05-21T03:26:44.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:44 smithi039 ceph-mon[36429]: pgmap v84: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 665 MiB/s wr, 2.87k op/s 2024-05-21T03:26:44.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:44 smithi165 ceph-mon[36269]: pgmap v84: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 665 MiB/s wr, 2.87k op/s 2024-05-21T03:26:44.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:44 smithi027 ceph-mon[32179]: pgmap v84: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 665 MiB/s wr, 2.87k op/s 2024-05-21T03:26:46.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:46 smithi039 ceph-mon[36429]: pgmap v85: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 379 MiB/s wr, 2.77k op/s 2024-05-21T03:26:46.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:46 smithi165 ceph-mon[36269]: pgmap v85: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 379 MiB/s wr, 2.77k op/s 2024-05-21T03:26:46.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:46 smithi027 ceph-mon[32179]: pgmap v85: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 379 MiB/s wr, 2.77k op/s 2024-05-21T03:26:48.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:48 smithi039 ceph-mon[36429]: pgmap v86: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 398 MiB/s wr, 5.80k op/s 2024-05-21T03:26:48.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:48 smithi165 ceph-mon[36269]: pgmap v86: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 398 MiB/s wr, 5.80k op/s 2024-05-21T03:26:48.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:48 smithi027 ceph-mon[32179]: pgmap v86: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 398 MiB/s wr, 5.80k op/s 2024-05-21T03:26:50.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:50 smithi027 ceph-mon[32179]: pgmap v87: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 84 MiB/s wr, 5.70k op/s 2024-05-21T03:26:51.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:50 smithi039 ceph-mon[36429]: pgmap v87: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 84 MiB/s wr, 5.70k op/s 2024-05-21T03:26:51.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:50 smithi165 ceph-mon[36269]: pgmap v87: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 84 MiB/s wr, 5.70k op/s 2024-05-21T03:26:52.783 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:52 smithi027 ceph-mon[32179]: pgmap v88: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 84 MiB/s wr, 5.70k op/s 2024-05-21T03:26:53.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:52 smithi039 ceph-mon[36429]: pgmap v88: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 84 MiB/s wr, 5.70k op/s 2024-05-21T03:26:53.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:52 smithi165 ceph-mon[36269]: pgmap v88: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 84 MiB/s wr, 5.70k op/s 2024-05-21T03:26:54.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:54 smithi027 ceph-mon[32179]: pgmap v89: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 105 MiB/s wr, 8.34k op/s 2024-05-21T03:26:55.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:54 smithi039 ceph-mon[36429]: pgmap v89: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 105 MiB/s wr, 8.34k op/s 2024-05-21T03:26:55.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:54 smithi165 ceph-mon[36269]: pgmap v89: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 105 MiB/s wr, 8.34k op/s 2024-05-21T03:26:56.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:56 smithi027 ceph-mon[32179]: pgmap v90: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 40 MiB/s wr, 5.68k op/s 2024-05-21T03:26:57.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:56 smithi039 ceph-mon[36429]: pgmap v90: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 40 MiB/s wr, 5.68k op/s 2024-05-21T03:26:57.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:56 smithi165 ceph-mon[36269]: pgmap v90: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 40 MiB/s wr, 5.68k op/s 2024-05-21T03:26:58.821 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:26:58 smithi039 ceph-mon[36429]: pgmap v91: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 66 MiB/s wr, 8.79k op/s 2024-05-21T03:26:58.822 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:26:58 smithi165 ceph-mon[36269]: pgmap v91: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 66 MiB/s wr, 8.79k op/s 2024-05-21T03:26:58.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:26:58 smithi027 ceph-mon[32179]: pgmap v91: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 66 MiB/s wr, 8.79k op/s 2024-05-21T03:27:00.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:00 smithi027 ceph-mon[32179]: pgmap v92: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 46 MiB/s wr, 5.75k op/s 2024-05-21T03:27:01.033 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:00 smithi165 ceph-mon[36269]: pgmap v92: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 46 MiB/s wr, 5.75k op/s 2024-05-21T03:27:01.033 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:00 smithi039 ceph-mon[36429]: pgmap v92: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 46 MiB/s wr, 5.75k op/s 2024-05-21T03:27:02.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:02 smithi027 ceph-mon[32179]: pgmap v93: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 46 MiB/s wr, 5.75k op/s 2024-05-21T03:27:03.018 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:02 smithi039 ceph-mon[36429]: pgmap v93: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 46 MiB/s wr, 5.75k op/s 2024-05-21T03:27:03.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:02 smithi165 ceph-mon[36269]: pgmap v93: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 46 MiB/s wr, 5.75k op/s 2024-05-21T03:27:04.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:04 smithi027 ceph-mon[32179]: pgmap v94: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 71 MiB/s wr, 8.47k op/s 2024-05-21T03:27:05.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:04 smithi039 ceph-mon[36429]: pgmap v94: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 71 MiB/s wr, 8.47k op/s 2024-05-21T03:27:05.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:04 smithi165 ceph-mon[36269]: pgmap v94: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 71 MiB/s wr, 8.47k op/s 2024-05-21T03:27:05.927 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:27:05 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:27:05.572+0000 7f04142c6700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T03:27:05.927 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:27:05 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:27:05.572+0000 7f04142c6700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T03:27:05.927 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:27:05 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:27:05.572+0000 7f04142c6700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T03:27:05.927 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:27:05 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:27:05.572+0000 7f04142c6700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T03:27:05.927 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:27:05 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:27:05.572+0000 7f04142c6700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T03:27:06.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:06 smithi027 ceph-mon[32179]: pgmap v95: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 51 MiB/s wr, 5.83k op/s 2024-05-21T03:27:07.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:06 smithi039 ceph-mon[36429]: pgmap v95: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 51 MiB/s wr, 5.83k op/s 2024-05-21T03:27:07.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:06 smithi165 ceph-mon[36269]: pgmap v95: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 51 MiB/s wr, 5.83k op/s 2024-05-21T03:27:08.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:08 smithi027 ceph-mon[32179]: pgmap v96: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 74 MiB/s wr, 8.80k op/s 2024-05-21T03:27:08.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:08 smithi027 ceph-mon[32179]: mgrmap e27: x(active, since 3m), standbys: z, y 2024-05-21T03:27:08.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:08 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:27:09.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:08 smithi039 ceph-mon[36429]: pgmap v96: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 74 MiB/s wr, 8.80k op/s 2024-05-21T03:27:09.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:08 smithi039 ceph-mon[36429]: mgrmap e27: x(active, since 3m), standbys: z, y 2024-05-21T03:27:09.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:08 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:27:09.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:08 smithi165 ceph-mon[36269]: pgmap v96: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 74 MiB/s wr, 8.80k op/s 2024-05-21T03:27:09.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:08 smithi165 ceph-mon[36269]: mgrmap e27: x(active, since 3m), standbys: z, y 2024-05-21T03:27:09.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:08 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:27:09.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:09 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:27:09.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:09 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:27:09.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:09 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:27:10.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:09 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:27:10.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:09 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:27:10.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:09 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:27:10.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:09 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:27:10.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:09 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:27:10.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:09 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:27:10.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:10 smithi027 ceph-mon[32179]: pgmap v97: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 49 MiB/s wr, 5.69k op/s 2024-05-21T03:27:10.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:10 smithi027 ceph-mon[32179]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:27:11.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:10 smithi039 ceph-mon[36429]: pgmap v97: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 49 MiB/s wr, 5.69k op/s 2024-05-21T03:27:11.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:10 smithi039 ceph-mon[36429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:27:11.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:10 smithi165 ceph-mon[36269]: pgmap v97: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 49 MiB/s wr, 5.69k op/s 2024-05-21T03:27:11.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:10 smithi165 ceph-mon[36269]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:27:12.929 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:12 smithi027 ceph-mon[32179]: pgmap v98: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 49 MiB/s wr, 5.69k op/s 2024-05-21T03:27:13.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:12 smithi039 ceph-mon[36429]: pgmap v98: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 49 MiB/s wr, 5.69k op/s 2024-05-21T03:27:13.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:12 smithi165 ceph-mon[36269]: pgmap v98: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 49 MiB/s wr, 5.69k op/s 2024-05-21T03:27:14.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:14 smithi027 ceph-mon[32179]: pgmap v99: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 70 MiB/s wr, 8.08k op/s 2024-05-21T03:27:15.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:14 smithi039 ceph-mon[36429]: pgmap v99: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 70 MiB/s wr, 8.08k op/s 2024-05-21T03:27:15.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:14 smithi165 ceph-mon[36269]: pgmap v99: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 70 MiB/s wr, 8.08k op/s 2024-05-21T03:27:16.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:16 smithi027 ceph-mon[32179]: pgmap v100: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 45 MiB/s wr, 5.36k op/s 2024-05-21T03:27:16.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:16 smithi027 ceph-mon[32179]: mgrmap e28: x(active, since 3m), standbys: z, y 2024-05-21T03:27:17.018 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:16 smithi039 ceph-mon[36429]: pgmap v100: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 45 MiB/s wr, 5.36k op/s 2024-05-21T03:27:17.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:16 smithi039 ceph-mon[36429]: mgrmap e28: x(active, since 3m), standbys: z, y 2024-05-21T03:27:17.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:16 smithi165 ceph-mon[36269]: pgmap v100: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 45 MiB/s wr, 5.36k op/s 2024-05-21T03:27:17.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:16 smithi165 ceph-mon[36269]: mgrmap e28: x(active, since 3m), standbys: z, y 2024-05-21T03:27:18.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:18 smithi027 ceph-mon[32179]: pgmap v101: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 71 MiB/s wr, 8.12k op/s 2024-05-21T03:27:19.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:18 smithi039 ceph-mon[36429]: pgmap v101: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 71 MiB/s wr, 8.12k op/s 2024-05-21T03:27:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:18 smithi165 ceph-mon[36269]: pgmap v101: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 71 MiB/s wr, 8.12k op/s 2024-05-21T03:27:20.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:20 smithi027 ceph-mon[32179]: pgmap v102: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 47 MiB/s wr, 5.14k op/s 2024-05-21T03:27:21.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:20 smithi039 ceph-mon[36429]: pgmap v102: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 47 MiB/s wr, 5.14k op/s 2024-05-21T03:27:21.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:20 smithi165 ceph-mon[36269]: pgmap v102: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 47 MiB/s wr, 5.14k op/s 2024-05-21T03:27:22.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:22 smithi027 ceph-mon[32179]: pgmap v103: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 47 MiB/s wr, 5.14k op/s 2024-05-21T03:27:23.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:22 smithi039 ceph-mon[36429]: pgmap v103: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 47 MiB/s wr, 5.14k op/s 2024-05-21T03:27:23.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:22 smithi165 ceph-mon[36269]: pgmap v103: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 47 MiB/s wr, 5.14k op/s 2024-05-21T03:27:24.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:24 smithi027 ceph-mon[32179]: pgmap v104: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 71 MiB/s wr, 7.84k op/s 2024-05-21T03:27:25.018 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:24 smithi039 ceph-mon[36429]: pgmap v104: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 71 MiB/s wr, 7.84k op/s 2024-05-21T03:27:25.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:24 smithi165 ceph-mon[36269]: pgmap v104: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 71 MiB/s wr, 7.84k op/s 2024-05-21T03:27:26.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:26 smithi027 ceph-mon[32179]: pgmap v105: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 50 MiB/s wr, 5.45k op/s 2024-05-21T03:27:27.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:26 smithi039 ceph-mon[36429]: pgmap v105: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 50 MiB/s wr, 5.45k op/s 2024-05-21T03:27:27.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:26 smithi165 ceph-mon[36269]: pgmap v105: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 50 MiB/s wr, 5.45k op/s 2024-05-21T03:27:28.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:28 smithi027 ceph-mon[32179]: pgmap v106: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 74 MiB/s wr, 8.47k op/s 2024-05-21T03:27:29.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:28 smithi039 ceph-mon[36429]: pgmap v106: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 74 MiB/s wr, 8.47k op/s 2024-05-21T03:27:29.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:28 smithi165 ceph-mon[36269]: pgmap v106: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 74 MiB/s wr, 8.47k op/s 2024-05-21T03:27:30.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:30 smithi027 ceph-mon[32179]: pgmap v107: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 49 MiB/s wr, 5.71k op/s 2024-05-21T03:27:31.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:30 smithi039 ceph-mon[36429]: pgmap v107: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 49 MiB/s wr, 5.71k op/s 2024-05-21T03:27:31.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:30 smithi165 ceph-mon[36269]: pgmap v107: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 49 MiB/s wr, 5.71k op/s 2024-05-21T03:27:32.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:32 smithi027 ceph-mon[32179]: pgmap v108: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 49 MiB/s wr, 5.71k op/s 2024-05-21T03:27:33.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:32 smithi039 ceph-mon[36429]: pgmap v108: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 49 MiB/s wr, 5.71k op/s 2024-05-21T03:27:33.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:32 smithi165 ceph-mon[36269]: pgmap v108: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 49 MiB/s wr, 5.71k op/s 2024-05-21T03:27:34.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:34 smithi027 ceph-mon[32179]: pgmap v109: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 76 MiB/s wr, 9.03k op/s 2024-05-21T03:27:35.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:34 smithi039 ceph-mon[36429]: pgmap v109: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 76 MiB/s wr, 9.03k op/s 2024-05-21T03:27:35.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:34 smithi165 ceph-mon[36269]: pgmap v109: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 76 MiB/s wr, 9.03k op/s 2024-05-21T03:27:36.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:36 smithi027 ceph-mon[32179]: pgmap v110: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 52 MiB/s wr, 6.33k op/s 2024-05-21T03:27:37.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:36 smithi039 ceph-mon[36429]: pgmap v110: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 52 MiB/s wr, 6.33k op/s 2024-05-21T03:27:37.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:36 smithi165 ceph-mon[36269]: pgmap v110: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 52 MiB/s wr, 6.33k op/s 2024-05-21T03:27:39.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:38 smithi039 ceph-mon[36429]: pgmap v111: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 72 MiB/s wr, 9.09k op/s 2024-05-21T03:27:39.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:38 smithi165 ceph-mon[36269]: pgmap v111: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 72 MiB/s wr, 9.09k op/s 2024-05-21T03:27:39.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:38 smithi027 ceph-mon[32179]: pgmap v111: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 72 MiB/s wr, 9.09k op/s 2024-05-21T03:27:41.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:40 smithi039 ceph-mon[36429]: pgmap v112: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 58 MiB/s wr, 7.51k op/s 2024-05-21T03:27:41.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:40 smithi165 ceph-mon[36269]: pgmap v112: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 58 MiB/s wr, 7.51k op/s 2024-05-21T03:27:41.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:40 smithi027 ceph-mon[32179]: pgmap v112: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 58 MiB/s wr, 7.51k op/s 2024-05-21T03:27:43.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:42 smithi039 ceph-mon[36429]: pgmap v113: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 58 MiB/s wr, 7.51k op/s 2024-05-21T03:27:43.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:42 smithi165 ceph-mon[36269]: pgmap v113: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 58 MiB/s wr, 7.51k op/s 2024-05-21T03:27:43.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:42 smithi027 ceph-mon[32179]: pgmap v113: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 58 MiB/s wr, 7.51k op/s 2024-05-21T03:27:45.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:44 smithi039 ceph-mon[36429]: pgmap v114: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 78 MiB/s wr, 10.09k op/s 2024-05-21T03:27:45.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:44 smithi165 ceph-mon[36269]: pgmap v114: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 78 MiB/s wr, 10.09k op/s 2024-05-21T03:27:45.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:44 smithi027 ceph-mon[32179]: pgmap v114: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 78 MiB/s wr, 10.09k op/s 2024-05-21T03:27:47.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:46 smithi039 ceph-mon[36429]: pgmap v115: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 50 MiB/s wr, 6.78k op/s 2024-05-21T03:27:47.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:46 smithi165 ceph-mon[36269]: pgmap v115: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 50 MiB/s wr, 6.78k op/s 2024-05-21T03:27:47.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:46 smithi027 ceph-mon[32179]: pgmap v115: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 50 MiB/s wr, 6.78k op/s 2024-05-21T03:27:49.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:48 smithi039 ceph-mon[36429]: pgmap v116: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 63 MiB/s wr, 8.48k op/s 2024-05-21T03:27:49.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:48 smithi039 ceph-mon[36429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:27:49.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:48 smithi165 ceph-mon[36269]: pgmap v116: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 63 MiB/s wr, 8.48k op/s 2024-05-21T03:27:49.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:48 smithi165 ceph-mon[36269]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:27:49.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:48 smithi027 ceph-mon[32179]: pgmap v116: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 63 MiB/s wr, 8.48k op/s 2024-05-21T03:27:49.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:48 smithi027 ceph-mon[32179]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:27:51.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:50 smithi039 ceph-mon[36429]: pgmap v117: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 46 MiB/s wr, 6.18k op/s 2024-05-21T03:27:51.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:50 smithi165 ceph-mon[36269]: pgmap v117: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 46 MiB/s wr, 6.18k op/s 2024-05-21T03:27:51.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:50 smithi027 ceph-mon[32179]: pgmap v117: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 46 MiB/s wr, 6.18k op/s 2024-05-21T03:27:53.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:52 smithi039 ceph-mon[36429]: pgmap v118: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 35 MiB/s wr, 4.74k op/s 2024-05-21T03:27:53.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:52 smithi165 ceph-mon[36269]: pgmap v118: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 35 MiB/s wr, 4.74k op/s 2024-05-21T03:27:53.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:52 smithi027 ceph-mon[32179]: pgmap v118: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 35 MiB/s wr, 4.74k op/s 2024-05-21T03:27:55.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:54 smithi039 ceph-mon[36429]: pgmap v119: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 48 MiB/s wr, 6.49k op/s 2024-05-21T03:27:55.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:54 smithi165 ceph-mon[36269]: pgmap v119: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 48 MiB/s wr, 6.49k op/s 2024-05-21T03:27:55.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:54 smithi027 ceph-mon[32179]: pgmap v119: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 48 MiB/s wr, 6.49k op/s 2024-05-21T03:27:57.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:56 smithi039 ceph-mon[36429]: pgmap v120: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 3.91k op/s 2024-05-21T03:27:57.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:56 smithi165 ceph-mon[36269]: pgmap v120: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 3.91k op/s 2024-05-21T03:27:57.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:56 smithi027 ceph-mon[32179]: pgmap v120: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 3.91k op/s 2024-05-21T03:27:58.169 INFO:tasks.quiescer.fs.[cephfs]:Total duration (42.9) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-21T03:27:58.169 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.45 2024-05-21T03:27:58.170 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:27:58.470 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:27:58.925 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:27:58 smithi027 ceph-mon[32179]: pgmap v121: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 40 MiB/s wr, 5.38k op/s 2024-05-21T03:27:59.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:27:58 smithi039 ceph-mon[36429]: pgmap v121: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 40 MiB/s wr, 5.38k op/s 2024-05-21T03:27:59.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:27:58 smithi165 ceph-mon[36269]: pgmap v121: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 40 MiB/s wr, 5.38k op/s 2024-05-21T03:28:00.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.364+0000 7f1349d93700 1 -- 172.21.15.27:0/4279239624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1344108500 msgr2=0x7f13441088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:00.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.364+0000 7f1349d93700 1 --2- 172.21.15.27:0/4279239624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1344108500 0x7f13441088e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1334009990 tx=0x7f1334009ca0 comp rx=0 tx=0).stop 2024-05-21T03:28:00.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.364+0000 7f1349d93700 1 -- 172.21.15.27:0/4279239624 shutdown_connections 2024-05-21T03:28:00.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.364+0000 7f1349d93700 1 --2- 172.21.15.27:0/4279239624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1344103f90 0x7f1344110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:00.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.364+0000 7f1349d93700 1 --2- 172.21.15.27:0/4279239624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1344108500 0x7f13441088e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:00.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.364+0000 7f1349d93700 1 --2- 172.21.15.27:0/4279239624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13441035f0 0x7f1344103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:00.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.364+0000 7f1349d93700 1 -- 172.21.15.27:0/4279239624 >> 172.21.15.27:0/4279239624 conn(0x7f134407a6a0 msgr2=0x7f134407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:28:00.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.364+0000 7f1349d93700 1 -- 172.21.15.27:0/4279239624 shutdown_connections 2024-05-21T03:28:00.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.364+0000 7f1349d93700 1 -- 172.21.15.27:0/4279239624 wait complete. 2024-05-21T03:28:00.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.364+0000 7f1349d93700 1 Processor -- start 2024-05-21T03:28:00.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.365+0000 7f1349d93700 1 -- start start 2024-05-21T03:28:00.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.365+0000 7f1349d93700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13441035f0 0x7f1344075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:00.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.365+0000 7f1349d93700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1344103f90 0x7f1344075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:00.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.365+0000 7f1349d93700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1344108500 0x7f13440762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:00.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.365+0000 7f1349d93700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1344113770 con 0x7f1344103f90 2024-05-21T03:28:00.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.365+0000 7f1349d93700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f13441135f0 con 0x7f13441035f0 2024-05-21T03:28:00.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.365+0000 7f1349d93700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f13441138f0 con 0x7f1344108500 2024-05-21T03:28:00.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.365+0000 7f13437fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13441035f0 0x7f1344075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:00.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.365+0000 7f1342ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1344103f90 0x7f1344075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:00.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.365+0000 7f13437fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13441035f0 0x7f1344075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.27:45918/0 (socket says 172.21.15.27:45918) 2024-05-21T03:28:00.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.365+0000 7f1342ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1344103f90 0x7f1344075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40594/0 (socket says 172.21.15.27:40594) 2024-05-21T03:28:00.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.365+0000 7f13437fe700 1 -- 172.21.15.27:0/3859952107 learned_addr learned my addr 172.21.15.27:0/3859952107 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:28:00.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.365+0000 7f1343fff700 1 --2- 172.21.15.27:0/3859952107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1344108500 0x7f13440762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:00.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.365+0000 7f13437fe700 1 -- 172.21.15.27:0/3859952107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1344108500 msgr2=0x7f13440762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:00.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.365+0000 7f13437fe700 1 --2- 172.21.15.27:0/3859952107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1344108500 0x7f13440762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:00.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.365+0000 7f13437fe700 1 -- 172.21.15.27:0/3859952107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1344103f90 msgr2=0x7f1344075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:00.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.365+0000 7f13437fe700 1 --2- 172.21.15.27:0/3859952107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1344103f90 0x7f1344075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:00.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.365+0000 7f13437fe700 1 -- 172.21.15.27:0/3859952107 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13441a8140 con 0x7f13441035f0 2024-05-21T03:28:00.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.365+0000 7f1342ffd700 1 --2- 172.21.15.27:0/3859952107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1344103f90 0x7f1344075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:28:00.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.366+0000 7f13437fe700 1 --2- 172.21.15.27:0/3859952107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13441035f0 0x7f1344075840 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1334000c00 tx=0x7f13340047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:28:00.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.366+0000 7f1340ff9700 1 -- 172.21.15.27:0/3859952107 <== mon.2 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1334026070 con 0x7f13441035f0 2024-05-21T03:28:00.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.366+0000 7f1340ff9700 1 -- 172.21.15.27:0/3859952107 <== mon.2 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1334021410 con 0x7f13441035f0 2024-05-21T03:28:00.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.366+0000 7f1349d93700 1 -- 172.21.15.27:0/3859952107 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13441a83d0 con 0x7f13441035f0 2024-05-21T03:28:00.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.366+0000 7f1340ff9700 1 -- 172.21.15.27:0/3859952107 <== mon.2 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1334017650 con 0x7f13441035f0 2024-05-21T03:28:00.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.366+0000 7f1349d93700 1 -- 172.21.15.27:0/3859952107 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13441a8880 con 0x7f13441035f0 2024-05-21T03:28:00.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.367+0000 7f1349d93700 1 -- 172.21.15.27:0/3859952107 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f134410bb30 con 0x7f13441035f0 2024-05-21T03:28:00.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.369+0000 7f1340ff9700 1 -- 172.21.15.27:0/3859952107 <== mon.2 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7f1334029070 con 0x7f13441035f0 2024-05-21T03:28:00.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.370+0000 7f1340ff9700 1 --2- 172.21.15.27:0/3859952107 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f132c0b2a70 0x7f132c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:00.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.370+0000 7f1342ffd700 1 --2- 172.21.15.27:0/3859952107 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f132c0b2a70 0x7f132c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:00.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.370+0000 7f1340ff9700 1 -- 172.21.15.27:0/3859952107 <== mon.2 v2:172.21.15.39:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f13340e3130 con 0x7f13441035f0 2024-05-21T03:28:00.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.370+0000 7f1342ffd700 1 --2- 172.21.15.27:0/3859952107 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f132c0b2a70 0x7f132c0b4f30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f134407ad10 tx=0x7f1338009260 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:28:00.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.374+0000 7f1340ff9700 1 -- 172.21.15.27:0/3859952107 <== mon.2 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1334013070 con 0x7f13441035f0 2024-05-21T03:28:00.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.652+0000 7f1349d93700 1 -- 172.21.15.27:0/3859952107 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1344076d20 con 0x7f13441035f0 2024-05-21T03:28:00.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.653+0000 7f1340ff9700 1 -- 172.21.15.27:0/3859952107 <== mon.2 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11933 (secure 0 0 0) 0x7f13340aaba0 con 0x7f13441035f0 2024-05-21T03:28:00.652 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:28:00.653 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/148854869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":148854869},{"type":"v1","addr":"172.21.15.27:6835","nonce":148854869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/4145544730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":4145544730},{"type":"v1","addr":"172.21.15.27:6837","nonce":4145544730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T03:20:34.055985+0000","modified":"2024-05-21T03:27:47.260575+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24472,"mds_2":24457,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/745486764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":745486764},{"type":"v1","addr":"172.21.15.27:6839","nonce":745486764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6841/3982752351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3982752351},{"type":"v1","addr":"172.21.15.27:6841","nonce":3982752351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/3497521608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3497521608},{"type":"v1","addr":"172.21.15.39:6835","nonce":3497521608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.165:6835/3576910218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3576910218},{"type":"v1","addr":"172.21.15.165:6835","nonce":3576910218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/3673846852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3673846852},{"type":"v1","addr":"172.21.15.39:6837","nonce":3673846852}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6837/691356139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":691356139},{"type":"v1","addr":"172.21.15.165:6837","nonce":691356139}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6839/4158170814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4158170814},{"type":"v1","addr":"172.21.15.165:6839","nonce":4158170814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6839/287607819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":287607819},{"type":"v1","addr":"172.21.15.39:6839","nonce":287607819}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24457,24452,24451,24472]},"id":1}]} 2024-05-21T03:28:00.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.655+0000 7f1349d93700 1 -- 172.21.15.27:0/3859952107 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f132c0b2a70 msgr2=0x7f132c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:00.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.655+0000 7f1349d93700 1 --2- 172.21.15.27:0/3859952107 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f132c0b2a70 0x7f132c0b4f30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f134407ad10 tx=0x7f1338009260 comp rx=0 tx=0).stop 2024-05-21T03:28:00.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.655+0000 7f1349d93700 1 -- 172.21.15.27:0/3859952107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13441035f0 msgr2=0x7f1344075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:00.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.655+0000 7f1349d93700 1 --2- 172.21.15.27:0/3859952107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13441035f0 0x7f1344075840 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1334000c00 tx=0x7f13340047b0 comp rx=0 tx=0).stop 2024-05-21T03:28:00.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.655+0000 7f1349d93700 1 -- 172.21.15.27:0/3859952107 shutdown_connections 2024-05-21T03:28:00.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.655+0000 7f1349d93700 1 --2- 172.21.15.27:0/3859952107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1344108500 0x7f13440762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:00.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.655+0000 7f1349d93700 1 --2- 172.21.15.27:0/3859952107 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f132c0b2a70 0x7f132c0b4f30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:00.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.655+0000 7f1349d93700 1 --2- 172.21.15.27:0/3859952107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13441035f0 0x7f1344075840 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:00.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.655+0000 7f1349d93700 1 --2- 172.21.15.27:0/3859952107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1344103f90 0x7f1344075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:00.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.655+0000 7f1349d93700 1 -- 172.21.15.27:0/3859952107 >> 172.21.15.27:0/3859952107 conn(0x7f134407a6a0 msgr2=0x7f13441016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:28:00.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.655+0000 7f1349d93700 1 -- 172.21.15.27:0/3859952107 shutdown_connections 2024-05-21T03:28:00.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:00.655+0000 7f1349d93700 1 -- 172.21.15.27:0/3859952107 wait complete. 2024-05-21T03:28:00.656 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 33 2024-05-21T03:28:00.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:00 smithi027 ceph-mon[32179]: pgmap v122: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 43 MiB/s wr, 5.67k op/s 2024-05-21T03:28:00.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:00 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3859952107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:28:01.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:00 smithi039 ceph-mon[36429]: pgmap v122: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 43 MiB/s wr, 5.67k op/s 2024-05-21T03:28:01.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:00 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/3859952107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:28:01.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:00 smithi165 ceph-mon[36269]: pgmap v122: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 43 MiB/s wr, 5.67k op/s 2024-05-21T03:28:01.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:00 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/3859952107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:28:01.447 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 141.45 --await-for 45.0' 2024-05-21T03:28:01.447 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 141.45 --await-for 45.0 2024-05-21T03:28:01.999 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:28:03.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:02 smithi039 ceph-mon[36429]: pgmap v123: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 40 MiB/s wr, 5.21k op/s 2024-05-21T03:28:03.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:02 smithi165 ceph-mon[36269]: pgmap v123: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 40 MiB/s wr, 5.21k op/s 2024-05-21T03:28:03.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:02 smithi027 ceph-mon[32179]: pgmap v123: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 40 MiB/s wr, 5.21k op/s 2024-05-21T03:28:04.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.148+0000 7feb80ab8700 1 -- 172.21.15.27:0/811658985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb7c102e30 msgr2=0x7feb7c10f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:04.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.148+0000 7feb80ab8700 1 --2- 172.21.15.27:0/811658985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb7c102e30 0x7feb7c10f2d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7feb7400b0a0 tx=0x7feb7400b3b0 comp rx=0 tx=0).stop 2024-05-21T03:28:04.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.148+0000 7feb80ab8700 1 -- 172.21.15.27:0/811658985 shutdown_connections 2024-05-21T03:28:04.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.148+0000 7feb80ab8700 1 --2- 172.21.15.27:0/811658985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb7c102e30 0x7feb7c10f2d0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:04.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.148+0000 7feb80ab8700 1 --2- 172.21.15.27:0/811658985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7c1073d0 0x7feb7c1077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:04.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.148+0000 7feb80ab8700 1 --2- 172.21.15.27:0/811658985 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb7c102470 0x7feb7c1028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:04.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.148+0000 7feb80ab8700 1 -- 172.21.15.27:0/811658985 >> 172.21.15.27:0/811658985 conn(0x7feb7c079480 msgr2=0x7feb7c079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:28:04.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.148+0000 7feb80ab8700 1 -- 172.21.15.27:0/811658985 shutdown_connections 2024-05-21T03:28:04.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.148+0000 7feb80ab8700 1 -- 172.21.15.27:0/811658985 wait complete. 2024-05-21T03:28:04.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.149+0000 7feb80ab8700 1 Processor -- start 2024-05-21T03:28:04.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.149+0000 7feb80ab8700 1 -- start start 2024-05-21T03:28:04.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.149+0000 7feb80ab8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb7c102470 0x7feb7c19bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:04.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.149+0000 7feb80ab8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7c102e30 0x7feb7c19c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:04.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.149+0000 7feb80ab8700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb7c1073d0 0x7feb7c1a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:04.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.149+0000 7feb80ab8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c112630 con 0x7feb7c102470 2024-05-21T03:28:04.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.149+0000 7feb80ab8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c1124b0 con 0x7feb7c102e30 2024-05-21T03:28:04.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.149+0000 7feb80ab8700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c1127b0 con 0x7feb7c1073d0 2024-05-21T03:28:04.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.150+0000 7feb7a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb7c102470 0x7feb7c19bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:04.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.150+0000 7feb7a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb7c102470 0x7feb7c19bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47292/0 (socket says 172.21.15.27:47292) 2024-05-21T03:28:04.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.150+0000 7feb7a59c700 1 -- 172.21.15.27:0/2877170052 learned_addr learned my addr 172.21.15.27:0/2877170052 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:28:04.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.150+0000 7feb79d9b700 1 --2- 172.21.15.27:0/2877170052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7c102e30 0x7feb7c19c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:04.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.150+0000 7feb7ad9d700 1 --2- 172.21.15.27:0/2877170052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb7c1073d0 0x7feb7c1a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:04.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.150+0000 7feb7a59c700 1 -- 172.21.15.27:0/2877170052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb7c1073d0 msgr2=0x7feb7c1a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:04.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.150+0000 7feb7a59c700 1 --2- 172.21.15.27:0/2877170052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb7c1073d0 0x7feb7c1a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:04.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.150+0000 7feb7a59c700 1 -- 172.21.15.27:0/2877170052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7c102e30 msgr2=0x7feb7c19c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:04.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.150+0000 7feb7a59c700 1 --2- 172.21.15.27:0/2877170052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7c102e30 0x7feb7c19c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:04.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.150+0000 7feb7a59c700 1 -- 172.21.15.27:0/2877170052 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7c1a0d20 con 0x7feb7c102470 2024-05-21T03:28:04.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.150+0000 7feb79d9b700 1 --2- 172.21.15.27:0/2877170052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7c102e30 0x7feb7c19c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:28:04.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.151+0000 7feb7a59c700 1 --2- 172.21.15.27:0/2877170052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb7c102470 0x7feb7c19bd10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7feb6c00e9a0 tx=0x7feb6c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:28:04.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.151+0000 7feb6b7fe700 1 -- 172.21.15.27:0/2877170052 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c00ca10 con 0x7feb7c102470 2024-05-21T03:28:04.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.151+0000 7feb80ab8700 1 -- 172.21.15.27:0/2877170052 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb7c1a1010 con 0x7feb7c102470 2024-05-21T03:28:04.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.151+0000 7feb80ab8700 1 -- 172.21.15.27:0/2877170052 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb7c1a8890 con 0x7feb7c102470 2024-05-21T03:28:04.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.151+0000 7feb6b7fe700 1 -- 172.21.15.27:0/2877170052 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb6c00cbb0 con 0x7feb7c102470 2024-05-21T03:28:04.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.151+0000 7feb6b7fe700 1 -- 172.21.15.27:0/2877170052 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c018680 con 0x7feb7c102470 2024-05-21T03:28:04.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.155+0000 7feb6b7fe700 1 -- 172.21.15.27:0/2877170052 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7feb6c018820 con 0x7feb7c102470 2024-05-21T03:28:04.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.156+0000 7feb6b7fe700 1 --2- 172.21.15.27:0/2877170052 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7feb640b2aa0 0x7feb640b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:04.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.156+0000 7feb6b7fe700 1 -- 172.21.15.27:0/2877170052 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7feb6c0e2780 con 0x7feb7c102470 2024-05-21T03:28:04.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.156+0000 7feb79d9b700 1 --2- 172.21.15.27:0/2877170052 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7feb640b2aa0 0x7feb640b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:04.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.156+0000 7feb8b34eb80 1 Processor -- start 2024-05-21T03:28:04.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.157+0000 7feb8b34eb80 1 -- start start 2024-05-21T03:28:04.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.157+0000 7feb79d9b700 1 --2- 172.21.15.27:0/2877170052 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7feb640b2aa0 0x7feb640b4f60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feb7c10f680 tx=0x7feb70009260 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:28:04.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.157+0000 7feb8b34eb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9f8b4c7c0 0x55d9f8b4cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:04.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.157+0000 7feb8b34eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9f8b47c90 0x55d9f8b48110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:04.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.157+0000 7feb8b34eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9f8b48650 0x55d9f8b546c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:04.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.157+0000 7feb8b34eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d9f8aa5f10 con 0x55d9f8b48650 2024-05-21T03:28:04.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.157+0000 7feb79d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9f8b47c90 0x55d9f8b48110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:04.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.157+0000 7feb7a59c700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9f8b4c7c0 0x55d9f8b4cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:04.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.157+0000 7feb79d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9f8b47c90 0x55d9f8b48110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.27:39782/0 (socket says 172.21.15.27:39782) 2024-05-21T03:28:04.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.157+0000 7feb7a59c700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9f8b4c7c0 0x55d9f8b4cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.27:36324/0 (socket says 172.21.15.27:36324) 2024-05-21T03:28:04.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.157+0000 7feb79d9b700 1 -- 172.21.15.27:0/4289435454 learned_addr learned my addr 172.21.15.27:0/4289435454 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:28:04.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.157+0000 7feb8b34eb80 1 -- 172.21.15.27:0/4289435454 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d9f8a95ab0 con 0x55d9f8b47c90 2024-05-21T03:28:04.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.157+0000 7feb8b34eb80 1 -- 172.21.15.27:0/4289435454 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d9f8a99af0 con 0x55d9f8b4c7c0 2024-05-21T03:28:04.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.157+0000 7feb7ad9d700 1 --2- 172.21.15.27:0/4289435454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9f8b48650 0x55d9f8b546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:04.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.158+0000 7feb79d9b700 1 -- 172.21.15.27:0/4289435454 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9f8b4c7c0 msgr2=0x55d9f8b4cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:28:04.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.158+0000 7feb79d9b700 1 --2- 172.21.15.27:0/4289435454 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9f8b4c7c0 0x55d9f8b4cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:04.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.158+0000 7feb79d9b700 1 -- 172.21.15.27:0/4289435454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9f8b48650 msgr2=0x55d9f8b546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:28:04.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.158+0000 7feb79d9b700 1 --2- 172.21.15.27:0/4289435454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9f8b48650 0x55d9f8b546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:04.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.158+0000 7feb79d9b700 1 -- 172.21.15.27:0/4289435454 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9f8a3f150 con 0x55d9f8b47c90 2024-05-21T03:28:04.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.158+0000 7feb79d9b700 1 --2- 172.21.15.27:0/4289435454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9f8b47c90 0x55d9f8b48110 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7feb700043b0 tx=0x7feb70014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47b2641d82cd592b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:28:04.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.158+0000 7feb68ff9700 1 -- 172.21.15.27:0/4289435454 <== mon.2 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb70003d10 con 0x55d9f8b47c90 2024-05-21T03:28:04.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.158+0000 7feb68ff9700 1 -- 172.21.15.27:0/4289435454 <== mon.2 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb7001e8a0 con 0x55d9f8b47c90 2024-05-21T03:28:04.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.159+0000 7feb68ff9700 1 -- 172.21.15.27:0/4289435454 <== mon.2 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7000e810 con 0x55d9f8b47c90 2024-05-21T03:28:04.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.159+0000 7feb8b34eb80 1 -- 172.21.15.27:0/4289435454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9f8b47c90 msgr2=0x55d9f8b48110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:04.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.159+0000 7feb8b34eb80 1 --2- 172.21.15.27:0/4289435454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9f8b47c90 0x55d9f8b48110 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7feb700043b0 tx=0x7feb70014000 comp rx=0 tx=0).stop 2024-05-21T03:28:04.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.160+0000 7feb8b34eb80 1 -- 172.21.15.27:0/4289435454 shutdown_connections 2024-05-21T03:28:04.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.160+0000 7feb8b34eb80 1 --2- 172.21.15.27:0/4289435454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9f8b48650 0x55d9f8b546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:04.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.160+0000 7feb8b34eb80 1 --2- 172.21.15.27:0/4289435454 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9f8b4c7c0 0x55d9f8b4cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:04.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.160+0000 7feb8b34eb80 1 --2- 172.21.15.27:0/4289435454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9f8b47c90 0x55d9f8b48110 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:04.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.160+0000 7feb8b34eb80 1 -- 172.21.15.27:0/4289435454 >> 172.21.15.27:0/4289435454 conn(0x55d9f8a733e0 msgr2=0x55d9f8b42980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:28:04.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.160+0000 7feb8b34eb80 1 -- 172.21.15.27:0/4289435454 shutdown_connections 2024-05-21T03:28:04.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.160+0000 7feb8b34eb80 1 -- 172.21.15.27:0/4289435454 wait complete. 2024-05-21T03:28:04.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.161+0000 7feb8b34eb80 1 -- start start 2024-05-21T03:28:04.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.161+0000 7feb8b34eb80 1 Processor -- start 2024-05-21T03:28:04.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.161+0000 7feb8b34eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9f8b47c90 0x55d9f8b542e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:04.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.161+0000 7feb8b34eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9f8b48650 0x55d9f8b515e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:04.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.161+0000 7feb8b34eb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9f8b4c7c0 0x55d9f8b51b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:04.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.161+0000 7feb79d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9f8b48650 0x55d9f8b515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:04.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.161+0000 7feb8b34eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d9f899c140 con 0x55d9f8b47c90 2024-05-21T03:28:04.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.161+0000 7feb8b34eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d9f89fdf40 con 0x55d9f8b48650 2024-05-21T03:28:04.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.161+0000 7feb8b34eb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d9f8b54d70 con 0x55d9f8b4c7c0 2024-05-21T03:28:04.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.161+0000 7feb79d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9f8b48650 0x55d9f8b515e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.27:39784/0 (socket says 172.21.15.27:39784) 2024-05-21T03:28:04.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.161+0000 7feb79d9b700 1 -- 172.21.15.27:0/2645801518 learned_addr learned my addr 172.21.15.27:0/2645801518 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:28:04.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.161+0000 7feb8b34eb80 10 client.0 _ll_register_callbacks cb 0x55d9f8a30af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:28:04.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.161+0000 7feb7a59c700 1 --2- 172.21.15.27:0/2645801518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9f8b47c90 0x55d9f8b542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:04.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.161+0000 7feb7ad9d700 1 --2- 172.21.15.27:0/2645801518 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9f8b4c7c0 0x55d9f8b51b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:04.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.162+0000 7feb79d9b700 1 -- 172.21.15.27:0/2645801518 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9f8b4c7c0 msgr2=0x55d9f8b51b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:28:04.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.162+0000 7feb79d9b700 1 --2- 172.21.15.27:0/2645801518 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9f8b4c7c0 0x55d9f8b51b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:04.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.162+0000 7feb79d9b700 1 -- 172.21.15.27:0/2645801518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9f8b47c90 msgr2=0x55d9f8b542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:28:04.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.162+0000 7feb79d9b700 1 --2- 172.21.15.27:0/2645801518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9f8b47c90 0x55d9f8b542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:04.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.162+0000 7feb79d9b700 1 -- 172.21.15.27:0/2645801518 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9f89e5a70 con 0x55d9f8b48650 2024-05-21T03:28:04.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.163+0000 7feb79d9b700 1 --2- 172.21.15.27:0/2645801518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9f8b48650 0x55d9f8b515e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7feb7001ec80 tx=0x7feb7001ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8710e86731d238dc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:28:04.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.163+0000 7feb61ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-21T03:28:04.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.163+0000 7feb61ffb700 1 -- 172.21.15.27:0/2645801518 <== mon.2 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb70005250 con 0x55d9f8b48650 2024-05-21T03:28:04.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.163+0000 7feb8b34eb80 1 -- 172.21.15.27:0/2645801518 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb7001e8a0 con 0x55d9f8b48650 2024-05-21T03:28:04.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.163+0000 7feb61ffb700 1 -- 172.21.15.27:0/2645801518 <== mon.2 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7feb700053f0 con 0x55d9f8b48650 2024-05-21T03:28:04.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.163+0000 7feb61ffb700 1 -- 172.21.15.27:0/2645801518 <== mon.2 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7feb700055f0 con 0x55d9f8b48650 2024-05-21T03:28:04.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.164+0000 7feb8b34eb80 10 client.25211 fetch_fsmap learned FSMap version 33 2024-05-21T03:28:04.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.164+0000 7feb8b34eb80 1 -- 172.21.15.27:0/2645801518 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d9f8a4f900 con 0x55d9f8b48650 2024-05-21T03:28:04.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.164+0000 7feb61ffb700 1 -- 172.21.15.27:0/2645801518 <== mon.2 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb70004b70 con 0x55d9f8b48650 2024-05-21T03:28:04.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.164+0000 7feb61ffb700 1 -- 172.21.15.27:0/2645801518 <== mon.2 v2:172.21.15.39:3300/0 5 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7feb7003a780 con 0x55d9f8b48650 2024-05-21T03:28:04.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.164+0000 7feb8b34eb80 10 client.25211 fetch_fsmap finished waiting for FSMap version 33 2024-05-21T03:28:04.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.164+0000 7feb8b34eb80 10 client.25211 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-21T03:28:04.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.164+0000 7feb8b34eb80 20 client.25211 populate_metadata read hostname 'smithi027' 2024-05-21T03:28:04.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.165+0000 7feb8b34eb80 1 --2- 172.21.15.27:0/2645801518 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55d9f8b69d50 0x55d9f8b6c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:04.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.165+0000 7feb8b34eb80 4 client.25211 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:28:04.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.165+0000 7feb8b34eb80 1 -- 172.21.15.27:0/2645801518 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7feb7001e8a0 con 0x55d9f8b69d50 2024-05-21T03:28:04.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.165+0000 7feb7a59c700 1 --2- 172.21.15.27:0/2645801518 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55d9f8b69d50 0x55d9f8b6c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:04.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.166+0000 7feb61ffb700 10 client.25211 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:28:04.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.166+0000 7feb7a59c700 1 --2- 172.21.15.27:0/2645801518 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55d9f8b69d50 0x55d9f8b6c140 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfd6302b2e93531b server_cookie=19ef454cefa213d in_seq=0 out_seq=0 2024-05-21T03:28:04.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.167+0000 7feb61ffb700 1 -- 172.21.15.27:0/2645801518 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7feb7001e8a0 con 0x55d9f8b69d50 2024-05-21T03:28:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.167+0000 7feb61ffb700 10 client.25211 handle_command_reply: tid=0 2024-05-21T03:28:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.167+0000 7feb8b34eb80 1 client.25211 shutdown 2024-05-21T03:28:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.167+0000 7feb8b34eb80 1 -- 172.21.15.27:0/2645801518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9f8b48650 msgr2=0x55d9f8b515e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.167+0000 7feb8b34eb80 1 --2- 172.21.15.27:0/2645801518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9f8b48650 0x55d9f8b515e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7feb7001ec80 tx=0x7feb7001ecb0 comp rx=0 tx=0).stop 2024-05-21T03:28:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.167+0000 7feb8b34eb80 1 -- 172.21.15.27:0/2645801518 shutdown_connections 2024-05-21T03:28:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.167+0000 7feb8b34eb80 1 --2- 172.21.15.27:0/2645801518 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55d9f8b69d50 0x55d9f8b6c140 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.167+0000 7feb61ffb700 0 client.25211 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:28:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.167+0000 7feb8b34eb80 1 --2- 172.21.15.27:0/2645801518 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9f8b4c7c0 0x55d9f8b51b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.167+0000 7feb8b34eb80 1 --2- 172.21.15.27:0/2645801518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9f8b47c90 0x55d9f8b542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.167+0000 7feb8b34eb80 1 --2- 172.21.15.27:0/2645801518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9f8b48650 0x55d9f8b515e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.167+0000 7feb8b34eb80 1 -- 172.21.15.27:0/2645801518 >> 172.21.15.27:0/2645801518 conn(0x55d9f8a733e0 msgr2=0x55d9f8b46c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:28:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.168+0000 7feb8b34eb80 1 -- 172.21.15.27:0/2645801518 shutdown_connections 2024-05-21T03:28:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.168+0000 7feb8b34eb80 1 -- 172.21.15.27:0/2645801518 wait complete. 2024-05-21T03:28:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.168+0000 7feb8b34eb80 20 client.25211 trim_cache size 0 max 16384 2024-05-21T03:28:04.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.200+0000 7feb8b34eb80 1 Processor -- start 2024-05-21T03:28:04.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.200+0000 7feb8b34eb80 1 -- start start 2024-05-21T03:28:04.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.200+0000 7feb8b34eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9f8b47c90 0x55d9f8b3cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:04.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.200+0000 7feb8b34eb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9f8b48650 0x55d9f8b4ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:04.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.200+0000 7feb8b34eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9f8b4c7c0 0x55d9f8b71800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:04.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.200+0000 7feb8b34eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feb5c00d610 con 0x55d9f8b47c90 2024-05-21T03:28:04.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.200+0000 7feb8b34eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7feb5c00f240 con 0x55d9f8b4c7c0 2024-05-21T03:28:04.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.200+0000 7feb8b34eb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7feb5c00dce0 con 0x55d9f8b48650 2024-05-21T03:28:04.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.200+0000 7feb7a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9f8b47c90 0x55d9f8b3cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:04.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.200+0000 7feb7a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9f8b47c90 0x55d9f8b3cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47306/0 (socket says 172.21.15.27:47306) 2024-05-21T03:28:04.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.200+0000 7feb7a59c700 1 -- 172.21.15.27:0/2016199542 learned_addr learned my addr 172.21.15.27:0/2016199542 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:28:04.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.200+0000 7feb79d9b700 1 --2- 172.21.15.27:0/2016199542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9f8b48650 0x55d9f8b4ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:04.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.200+0000 7feb7ad9d700 1 --2- 172.21.15.27:0/2016199542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9f8b4c7c0 0x55d9f8b71800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:04.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.201+0000 7feb7a59c700 1 -- 172.21.15.27:0/2016199542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9f8b48650 msgr2=0x55d9f8b4ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:28:04.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.201+0000 7feb7a59c700 1 --2- 172.21.15.27:0/2016199542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9f8b48650 0x55d9f8b4ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:04.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.201+0000 7feb7a59c700 1 -- 172.21.15.27:0/2016199542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9f8b4c7c0 msgr2=0x55d9f8b71800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:28:04.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.201+0000 7feb7a59c700 1 --2- 172.21.15.27:0/2016199542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9f8b4c7c0 0x55d9f8b71800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:04.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.201+0000 7feb7a59c700 1 -- 172.21.15.27:0/2016199542 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9f8b54650 con 0x55d9f8b47c90 2024-05-21T03:28:04.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.201+0000 7feb7ad9d700 1 --2- 172.21.15.27:0/2016199542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9f8b4c7c0 0x55d9f8b71800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:28:04.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.201+0000 7feb7a59c700 1 --2- 172.21.15.27:0/2016199542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9f8b47c90 0x55d9f8b3cf10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0a6cd0 tx=0x7feb6c009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5300814fc052acdd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:28:04.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.201+0000 7feb63fff700 1 -- 172.21.15.27:0/2016199542 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c0a9a60 con 0x55d9f8b47c90 2024-05-21T03:28:04.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.201+0000 7feb63fff700 1 -- 172.21.15.27:0/2016199542 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb6c0a5040 con 0x55d9f8b47c90 2024-05-21T03:28:04.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.201+0000 7feb63fff700 1 -- 172.21.15.27:0/2016199542 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c0b2430 con 0x55d9f8b47c90 2024-05-21T03:28:04.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.202+0000 7feb8b34eb80 1 -- 172.21.15.27:0/2016199542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9f8b47c90 msgr2=0x55d9f8b3cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:04.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.202+0000 7feb8b34eb80 1 --2- 172.21.15.27:0/2016199542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9f8b47c90 0x55d9f8b3cf10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0a6cd0 tx=0x7feb6c009790 comp rx=0 tx=0).stop 2024-05-21T03:28:04.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.202+0000 7feb8b34eb80 1 -- 172.21.15.27:0/2016199542 shutdown_connections 2024-05-21T03:28:04.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.202+0000 7feb8b34eb80 1 --2- 172.21.15.27:0/2016199542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9f8b4c7c0 0x55d9f8b71800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:04.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.202+0000 7feb8b34eb80 1 --2- 172.21.15.27:0/2016199542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9f8b47c90 0x55d9f8b3cf10 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:04.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.202+0000 7feb8b34eb80 1 --2- 172.21.15.27:0/2016199542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9f8b48650 0x55d9f8b4ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:04.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.202+0000 7feb8b34eb80 1 -- 172.21.15.27:0/2016199542 >> 172.21.15.27:0/2016199542 conn(0x55d9f8a733e0 msgr2=0x55d9f8b4e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:28:04.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.202+0000 7feb8b34eb80 1 -- 172.21.15.27:0/2016199542 shutdown_connections 2024-05-21T03:28:04.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.202+0000 7feb8b34eb80 1 -- 172.21.15.27:0/2016199542 wait complete. 2024-05-21T03:28:04.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.203+0000 7feb8b34eb80 1 -- start start 2024-05-21T03:28:04.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.203+0000 7feb8b34eb80 1 Processor -- start 2024-05-21T03:28:04.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.204+0000 7feb8b34eb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9f8b47c90 0x55d9f8b61600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:04.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.204+0000 7feb8b34eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9f8b48650 0x55d9f8b61b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:04.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.204+0000 7feb8b34eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9f8b4c7c0 0x55d9f8b62080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:04.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.204+0000 7feb8b34eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feb5c00eab0 con 0x55d9f8b48650 2024-05-21T03:28:04.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.204+0000 7feb8b34eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7feb5c00e3e0 con 0x55d9f8b4c7c0 2024-05-21T03:28:04.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.204+0000 7feb8b34eb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7feb5c00b5b0 con 0x55d9f8b47c90 2024-05-21T03:28:04.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.204+0000 7feb8b34eb80 10 client.0 _ll_register_callbacks cb 0x55d9f8a30af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:28:04.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.204+0000 7feb7a59c700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9f8b47c90 0x55d9f8b61600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:04.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.204+0000 7feb79d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9f8b48650 0x55d9f8b61b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:04.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.204+0000 7feb79d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9f8b48650 0x55d9f8b61b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47320/0 (socket says 172.21.15.27:47320) 2024-05-21T03:28:04.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.204+0000 7feb79d9b700 1 -- 172.21.15.27:0/2691121664 learned_addr learned my addr 172.21.15.27:0/2691121664 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:28:04.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.204+0000 7feb7ad9d700 1 --2- 172.21.15.27:0/2691121664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9f8b4c7c0 0x55d9f8b62080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:04.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.205+0000 7feb79d9b700 1 -- 172.21.15.27:0/2691121664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9f8b47c90 msgr2=0x55d9f8b61600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:28:04.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.205+0000 7feb79d9b700 1 --2- 172.21.15.27:0/2691121664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9f8b47c90 0x55d9f8b61600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:04.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.205+0000 7feb79d9b700 1 -- 172.21.15.27:0/2691121664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9f8b4c7c0 msgr2=0x55d9f8b62080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:28:04.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.205+0000 7feb79d9b700 1 --2- 172.21.15.27:0/2691121664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9f8b4c7c0 0x55d9f8b62080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:04.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.205+0000 7feb79d9b700 1 -- 172.21.15.27:0/2691121664 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9f8bd46f0 con 0x55d9f8b48650 2024-05-21T03:28:04.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.205+0000 7feb79d9b700 1 --2- 172.21.15.27:0/2691121664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9f8b48650 0x55d9f8b61b40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7feb700040b0 tx=0x7feb70033270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c8c210db5b1f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:28:04.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.205+0000 7feb61ffb700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:28:04.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.206+0000 7feb61ffb700 1 -- 172.21.15.27:0/2691121664 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb70033460 con 0x55d9f8b48650 2024-05-21T03:28:04.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.206+0000 7feb61ffb700 1 -- 172.21.15.27:0/2691121664 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7feb7003a690 con 0x55d9f8b48650 2024-05-21T03:28:04.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.206+0000 7feb8b34eb80 1 -- 172.21.15.27:0/2691121664 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb6c0a5040 con 0x55d9f8b48650 2024-05-21T03:28:04.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.206+0000 7feb61ffb700 1 -- 172.21.15.27:0/2691121664 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7feb7003a890 con 0x55d9f8b48650 2024-05-21T03:28:04.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.206+0000 7feb8b34eb80 10 client.15609 fetch_fsmap learned FSMap version 33 2024-05-21T03:28:04.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.206+0000 7feb8b34eb80 1 -- 172.21.15.27:0/2691121664 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d9f8bd5980 con 0x55d9f8b48650 2024-05-21T03:28:04.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.207+0000 7feb61ffb700 1 -- 172.21.15.27:0/2691121664 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb70033ba0 con 0x55d9f8b48650 2024-05-21T03:28:04.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.207+0000 7feb61ffb700 1 -- 172.21.15.27:0/2691121664 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7feb70004840 con 0x55d9f8b48650 2024-05-21T03:28:04.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.207+0000 7feb8b34eb80 10 client.15609 fetch_fsmap finished waiting for FSMap version 33 2024-05-21T03:28:04.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.207+0000 7feb8b34eb80 10 client.15609 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-21T03:28:04.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.207+0000 7feb8b34eb80 20 client.15609 populate_metadata read hostname 'smithi027' 2024-05-21T03:28:04.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.207+0000 7feb8b34eb80 1 --2- 172.21.15.27:0/2691121664 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55d9f8b69d50 0x55d9f8bd8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:04.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.207+0000 7feb8b34eb80 4 client.15609 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.45, "await_for": 45.0}] 2024-05-21T03:28:04.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.207+0000 7feb8b34eb80 1 -- 172.21.15.27:0/2691121664 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.45, "await_for": 45.0}) -- 0x7feb6c0a5040 con 0x55d9f8b69d50 2024-05-21T03:28:04.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.208+0000 7feb7a59c700 1 --2- 172.21.15.27:0/2691121664 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55d9f8b69d50 0x55d9f8bd8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:04.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.208+0000 7feb61ffb700 10 client.15609 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:28:04.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:04.208+0000 7feb7a59c700 1 --2- 172.21.15.27:0/2691121664 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55d9f8b69d50 0x55d9f8bd8170 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11bf0da0e533a765 server_cookie=75e2c2bb1d33bc38 in_seq=0 out_seq=0 2024-05-21T03:28:05.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:04 smithi039 ceph-mon[36429]: pgmap v124: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 70 MiB/s wr, 8.84k op/s 2024-05-21T03:28:05.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:04 smithi165 ceph-mon[36269]: pgmap v124: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 70 MiB/s wr, 8.84k op/s 2024-05-21T03:28:05.112 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:04 smithi027 ceph-mon[32179]: pgmap v124: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 70 MiB/s wr, 8.84k op/s 2024-05-21T03:28:07.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:06 smithi039 ceph-mon[36429]: pgmap v125: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 MiB/s wr, 7.09k op/s 2024-05-21T03:28:07.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:06 smithi165 ceph-mon[36269]: pgmap v125: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 MiB/s wr, 7.09k op/s 2024-05-21T03:28:07.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:06 smithi027 ceph-mon[32179]: pgmap v125: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 MiB/s wr, 7.09k op/s 2024-05-21T03:28:09.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:08 smithi039 ceph-mon[36429]: pgmap v126: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 MiB/s wr, 7.34k op/s 2024-05-21T03:28:09.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:08 smithi165 ceph-mon[36269]: pgmap v126: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 MiB/s wr, 7.34k op/s 2024-05-21T03:28:09.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:08 smithi027 ceph-mon[32179]: pgmap v126: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 MiB/s wr, 7.34k op/s 2024-05-21T03:28:10.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:09 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:28:10.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:09 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:28:10.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:09 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:28:11.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:10 smithi039 ceph-mon[36429]: pgmap v127: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 81 MiB/s wr, 10.10k op/s 2024-05-21T03:28:11.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:10 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:28:11.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:10 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:28:11.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:10 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:28:11.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:10 smithi165 ceph-mon[36269]: pgmap v127: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 81 MiB/s wr, 10.10k op/s 2024-05-21T03:28:11.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:10 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:28:11.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:10 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:28:11.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:10 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:28:11.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:10 smithi027 ceph-mon[32179]: pgmap v127: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 81 MiB/s wr, 10.10k op/s 2024-05-21T03:28:11.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:10 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:28:11.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:10 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:28:11.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:10 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:28:12.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:11 smithi027 ceph-mon[32179]: pgmap v128: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 65 MiB/s wr, 8.11k op/s 2024-05-21T03:28:12.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:11 smithi039 ceph-mon[36429]: pgmap v128: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 65 MiB/s wr, 8.11k op/s 2024-05-21T03:28:12.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:11 smithi165 ceph-mon[36269]: pgmap v128: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 65 MiB/s wr, 8.11k op/s 2024-05-21T03:28:14.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:14 smithi039 ceph-mon[36429]: pgmap v129: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 87 MiB/s wr, 10.86k op/s 2024-05-21T03:28:14.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:14 smithi165 ceph-mon[36269]: pgmap v129: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 87 MiB/s wr, 10.86k op/s 2024-05-21T03:28:14.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:14 smithi027 ceph-mon[32179]: pgmap v129: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 87 MiB/s wr, 10.86k op/s 2024-05-21T03:28:16.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:16 smithi027 ceph-mon[32179]: pgmap v130: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 57 MiB/s wr, 7.23k op/s 2024-05-21T03:28:17.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:16 smithi039 ceph-mon[36429]: pgmap v130: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 57 MiB/s wr, 7.23k op/s 2024-05-21T03:28:17.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:16 smithi165 ceph-mon[36269]: pgmap v130: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 57 MiB/s wr, 7.23k op/s 2024-05-21T03:28:18.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:18 smithi027 ceph-mon[32179]: pgmap v131: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 59 MiB/s wr, 7.43k op/s 2024-05-21T03:28:19.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:18 smithi039 ceph-mon[36429]: pgmap v131: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 59 MiB/s wr, 7.43k op/s 2024-05-21T03:28:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:18 smithi165 ceph-mon[36269]: pgmap v131: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 59 MiB/s wr, 7.43k op/s 2024-05-21T03:28:20.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:20 smithi027 ceph-mon[32179]: pgmap v132: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 76 MiB/s wr, 9.57k op/s 2024-05-21T03:28:21.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:20 smithi039 ceph-mon[36429]: pgmap v132: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 76 MiB/s wr, 9.57k op/s 2024-05-21T03:28:21.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:20 smithi165 ceph-mon[36269]: pgmap v132: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 76 MiB/s wr, 9.57k op/s 2024-05-21T03:28:22.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:22 smithi027 ceph-mon[32179]: pgmap v133: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 43 MiB/s wr, 5.34k op/s 2024-05-21T03:28:23.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:22 smithi039 ceph-mon[36429]: pgmap v133: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 43 MiB/s wr, 5.34k op/s 2024-05-21T03:28:23.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:22 smithi165 ceph-mon[36269]: pgmap v133: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 43 MiB/s wr, 5.34k op/s 2024-05-21T03:28:24.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:24 smithi027 ceph-mon[32179]: pgmap v134: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 62 MiB/s wr, 7.83k op/s 2024-05-21T03:28:25.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:24 smithi039 ceph-mon[36429]: pgmap v134: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 62 MiB/s wr, 7.83k op/s 2024-05-21T03:28:25.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:24 smithi165 ceph-mon[36269]: pgmap v134: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 62 MiB/s wr, 7.83k op/s 2024-05-21T03:28:26.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:26 smithi027 ceph-mon[32179]: pgmap v135: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 41 MiB/s wr, 5.08k op/s 2024-05-21T03:28:27.018 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:26 smithi039 ceph-mon[36429]: pgmap v135: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 41 MiB/s wr, 5.08k op/s 2024-05-21T03:28:27.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:26 smithi165 ceph-mon[36269]: pgmap v135: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 41 MiB/s wr, 5.08k op/s 2024-05-21T03:28:28.820 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:28 smithi039 ceph-mon[36429]: pgmap v136: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 43 MiB/s wr, 5.45k op/s 2024-05-21T03:28:28.821 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:28 smithi165 ceph-mon[36269]: pgmap v136: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 43 MiB/s wr, 5.45k op/s 2024-05-21T03:28:28.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:28 smithi027 ceph-mon[32179]: pgmap v136: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 43 MiB/s wr, 5.45k op/s 2024-05-21T03:28:30.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:30 smithi027 ceph-mon[32179]: pgmap v137: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 61 MiB/s wr, 7.91k op/s 2024-05-21T03:28:31.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:30 smithi039 ceph-mon[36429]: pgmap v137: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 61 MiB/s wr, 7.91k op/s 2024-05-21T03:28:31.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:30 smithi165 ceph-mon[36269]: pgmap v137: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 61 MiB/s wr, 7.91k op/s 2024-05-21T03:28:32.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:32 smithi027 ceph-mon[32179]: pgmap v138: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 42 MiB/s wr, 5.52k op/s 2024-05-21T03:28:33.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:32 smithi039 ceph-mon[36429]: pgmap v138: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 42 MiB/s wr, 5.52k op/s 2024-05-21T03:28:33.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:32 smithi165 ceph-mon[36269]: pgmap v138: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 42 MiB/s wr, 5.52k op/s 2024-05-21T03:28:34.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:34 smithi027 ceph-mon[32179]: pgmap v139: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 62 MiB/s wr, 8.46k op/s 2024-05-21T03:28:35.018 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:34 smithi039 ceph-mon[36429]: pgmap v139: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 62 MiB/s wr, 8.46k op/s 2024-05-21T03:28:35.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:34 smithi165 ceph-mon[36269]: pgmap v139: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 62 MiB/s wr, 8.46k op/s 2024-05-21T03:28:36.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:36 smithi027 ceph-mon[32179]: pgmap v140: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 43 MiB/s wr, 5.97k op/s 2024-05-21T03:28:37.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:36 smithi039 ceph-mon[36429]: pgmap v140: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 43 MiB/s wr, 5.97k op/s 2024-05-21T03:28:37.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:36 smithi165 ceph-mon[36269]: pgmap v140: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 43 MiB/s wr, 5.97k op/s 2024-05-21T03:28:38.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:38 smithi027 ceph-mon[32179]: pgmap v141: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 44 MiB/s wr, 6.15k op/s 2024-05-21T03:28:39.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:38 smithi039 ceph-mon[36429]: pgmap v141: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 44 MiB/s wr, 6.15k op/s 2024-05-21T03:28:39.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:38 smithi165 ceph-mon[36269]: pgmap v141: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 44 MiB/s wr, 6.15k op/s 2024-05-21T03:28:40.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:40 smithi027 ceph-mon[32179]: pgmap v142: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 64 MiB/s wr, 8.84k op/s 2024-05-21T03:28:41.018 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:40 smithi039 ceph-mon[36429]: pgmap v142: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 64 MiB/s wr, 8.84k op/s 2024-05-21T03:28:41.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:40 smithi165 ceph-mon[36269]: pgmap v142: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 64 MiB/s wr, 8.84k op/s 2024-05-21T03:28:42.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:42 smithi027 ceph-mon[32179]: pgmap v143: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 MiB/s wr, 6.17k op/s 2024-05-21T03:28:43.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:42 smithi039 ceph-mon[36429]: pgmap v143: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 MiB/s wr, 6.17k op/s 2024-05-21T03:28:43.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:42 smithi165 ceph-mon[36269]: pgmap v143: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 MiB/s wr, 6.17k op/s 2024-05-21T03:28:44.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:44 smithi027 ceph-mon[32179]: pgmap v144: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 63 MiB/s wr, 8.89k op/s 2024-05-21T03:28:45.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:44 smithi039 ceph-mon[36429]: pgmap v144: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 63 MiB/s wr, 8.89k op/s 2024-05-21T03:28:45.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:44 smithi165 ceph-mon[36269]: pgmap v144: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 63 MiB/s wr, 8.89k op/s 2024-05-21T03:28:46.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:46 smithi027 ceph-mon[32179]: pgmap v145: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 43 MiB/s wr, 5.95k op/s 2024-05-21T03:28:47.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:46 smithi039 ceph-mon[36429]: pgmap v145: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 43 MiB/s wr, 5.95k op/s 2024-05-21T03:28:47.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:46 smithi165 ceph-mon[36269]: pgmap v145: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 43 MiB/s wr, 5.95k op/s 2024-05-21T03:28:48.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:48 smithi027 ceph-mon[32179]: pgmap v146: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 44 MiB/s wr, 6.01k op/s 2024-05-21T03:28:49.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:48 smithi039 ceph-mon[36429]: pgmap v146: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 44 MiB/s wr, 6.01k op/s 2024-05-21T03:28:49.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:48 smithi165 ceph-mon[36269]: pgmap v146: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 44 MiB/s wr, 6.01k op/s 2024-05-21T03:28:49.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:49.209+0000 7feb61ffb700 1 -- 172.21.15.27:0/2691121664 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: -115 ) ==== 8+0+593 (crc 0 0 0) 0x7feb6c0a5040 con 0x55d9f8b69d50 2024-05-21T03:28:49.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:49.209+0000 7feb61ffb700 10 client.15609 handle_command_reply: tid=0 2024-05-21T03:28:49.211 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:28:49.212 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 33, 2024-05-21T03:28:49.212 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24476, 2024-05-21T03:28:49.212 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 9, 2024-05-21T03:28:49.212 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-05-21T03:28:49.212 INFO:teuthology.orchestra.run.smithi027.stdout: "72cf81f5": { 2024-05-21T03:28:49.212 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 9, 2024-05-21T03:28:49.212 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-05-21T03:28:49.212 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-05-21T03:28:49.212 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCING", 2024-05-21T03:28:49.212 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 45.0 2024-05-21T03:28:49.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:28:49.212 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-05-21T03:28:49.212 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 141.4, 2024-05-21T03:28:49.213 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-05-21T03:28:49.213 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-05-21T03:28:49.213 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-05-21T03:28:49.213 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-05-21T03:28:49.213 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCING", 2024-05-21T03:28:49.213 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 45.0 2024-05-21T03:28:49.213 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:28:49.213 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:28:49.213 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:28:49.213 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:28:49.213 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:28:49.213 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:28:49.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:49.209+0000 7feb8b34eb80 1 client.15609 shutdown 2024-05-21T03:28:49.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:49.210+0000 7feb8b34eb80 1 -- 172.21.15.27:0/2691121664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9f8b48650 msgr2=0x55d9f8b61b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:49.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:49.210+0000 7feb8b34eb80 1 --2- 172.21.15.27:0/2691121664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9f8b48650 0x55d9f8b61b40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7feb700040b0 tx=0x7feb70033270 comp rx=0 tx=0).stop 2024-05-21T03:28:49.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:49.210+0000 7feb8b34eb80 1 -- 172.21.15.27:0/2691121664 shutdown_connections 2024-05-21T03:28:49.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:49.210+0000 7feb8b34eb80 1 --2- 172.21.15.27:0/2691121664 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55d9f8b69d50 0x55d9f8bd8170 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:49.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:49.210+0000 7feb8b34eb80 1 --2- 172.21.15.27:0/2691121664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9f8b4c7c0 0x55d9f8b62080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:49.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:49.210+0000 7feb8b34eb80 1 --2- 172.21.15.27:0/2691121664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9f8b47c90 0x55d9f8b61600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:49.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:49.210+0000 7feb61ffb700 0 client.15609 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:28:49.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:49.210+0000 7feb8b34eb80 1 --2- 172.21.15.27:0/2691121664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d9f8b48650 0x55d9f8b61b40 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:49.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:49.210+0000 7feb8b34eb80 1 -- 172.21.15.27:0/2691121664 >> 172.21.15.27:0/2691121664 conn(0x55d9f8a733e0 msgr2=0x55d9f8b4eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:28:49.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:49.210+0000 7feb8b34eb80 1 -- 172.21.15.27:0/2691121664 shutdown_connections 2024-05-21T03:28:49.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:49.210+0000 7feb8b34eb80 1 -- 172.21.15.27:0/2691121664 wait complete. 2024-05-21T03:28:49.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:49.210+0000 7feb8b34eb80 20 client.15609 trim_cache size 0 max 16384 2024-05-21T03:28:49.215 INFO:teuthology.orchestra.run.smithi027.stderr:Error EINPROGRESS: 2024-05-21T03:28:50.084 DEBUG:teuthology.orchestra.run:got remote process result: 115 2024-05-21T03:28:50.085 WARNING:tasks.quiescer.fs.[cephfs]:Set '72cf81f5' hasn't quiesced after 51.9 seconds (timeout: 90). Dumping ops with locks from all ranks. 2024-05-21T03:28:50.085 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:28:50.381 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:28:50.767 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:50 smithi027 ceph-mon[32179]: pgmap v147: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 60 MiB/s wr, 8.32k op/s 2024-05-21T03:28:51.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:50 smithi039 ceph-mon[36429]: pgmap v147: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 60 MiB/s wr, 8.32k op/s 2024-05-21T03:28:51.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:50 smithi165 ceph-mon[36269]: pgmap v147: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 60 MiB/s wr, 8.32k op/s 2024-05-21T03:28:52.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.465+0000 7f217b33e700 1 -- 172.21.15.27:0/1940963480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21741120a0 msgr2=0x7f217410aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:52.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.465+0000 7f217b33e700 1 --2- 172.21.15.27:0/1940963480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21741120a0 0x7f217410aa80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2164009990 tx=0x7f2164009ca0 comp rx=0 tx=0).stop 2024-05-21T03:28:52.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.465+0000 7f217b33e700 1 -- 172.21.15.27:0/1940963480 shutdown_connections 2024-05-21T03:28:52.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.465+0000 7f217b33e700 1 --2- 172.21.15.27:0/1940963480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f217411cd20 0x7f217411f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:52.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.465+0000 7f217b33e700 1 --2- 172.21.15.27:0/1940963480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2174111c30 0x7f217410a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:52.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.465+0000 7f217b33e700 1 --2- 172.21.15.27:0/1940963480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21741120a0 0x7f217410aa80 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:52.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.465+0000 7f217b33e700 1 -- 172.21.15.27:0/1940963480 >> 172.21.15.27:0/1940963480 conn(0x7f217406e520 msgr2=0x7f217406e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:28:52.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.472+0000 7f217b33e700 1 -- 172.21.15.27:0/1940963480 shutdown_connections 2024-05-21T03:28:52.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.472+0000 7f217b33e700 1 -- 172.21.15.27:0/1940963480 wait complete. 2024-05-21T03:28:52.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.473+0000 7f217b33e700 1 Processor -- start 2024-05-21T03:28:52.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.473+0000 7f217b33e700 1 -- start start 2024-05-21T03:28:52.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.473+0000 7f217b33e700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2174111c30 0x7f217411a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:52.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.473+0000 7f217b33e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21741120a0 0x7f217411a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:52.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.473+0000 7f217b33e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f217411cd20 0x7f2174113520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:52.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.473+0000 7f217b33e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f21741225c0 con 0x7f217411cd20 2024-05-21T03:28:52.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.473+0000 7f217b33e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2174122440 con 0x7f21741120a0 2024-05-21T03:28:52.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.473+0000 7f217b33e700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2174122740 con 0x7f2174111c30 2024-05-21T03:28:52.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.475+0000 7f217a33c700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2174111c30 0x7f217411a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:52.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.475+0000 7f217a33c700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2174111c30 0x7f217411a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.27:33840/0 (socket says 172.21.15.27:33840) 2024-05-21T03:28:52.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.475+0000 7f217a33c700 1 -- 172.21.15.27:0/3348374661 learned_addr learned my addr 172.21.15.27:0/3348374661 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:28:52.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.475+0000 7f217ab3d700 1 --2- 172.21.15.27:0/3348374661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f217411cd20 0x7f2174113520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:52.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.475+0000 7f2179b3b700 1 --2- 172.21.15.27:0/3348374661 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21741120a0 0x7f217411a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:52.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.476+0000 7f217a33c700 1 -- 172.21.15.27:0/3348374661 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21741120a0 msgr2=0x7f217411a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:52.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.476+0000 7f217a33c700 1 --2- 172.21.15.27:0/3348374661 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21741120a0 0x7f217411a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:52.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.476+0000 7f217a33c700 1 -- 172.21.15.27:0/3348374661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f217411cd20 msgr2=0x7f2174113520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:52.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.476+0000 7f217a33c700 1 --2- 172.21.15.27:0/3348374661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f217411cd20 0x7f2174113520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:52.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.476+0000 7f217a33c700 1 -- 172.21.15.27:0/3348374661 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2174113e20 con 0x7f2174111c30 2024-05-21T03:28:52.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.479+0000 7f217a33c700 1 --2- 172.21.15.27:0/3348374661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2174111c30 0x7f217411a240 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f216c00ba50 tx=0x7f216c00be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:28:52.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.479+0000 7f216b7fe700 1 -- 172.21.15.27:0/3348374661 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216c019070 con 0x7f2174111c30 2024-05-21T03:28:52.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.479+0000 7f217b33e700 1 -- 172.21.15.27:0/3348374661 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2174114110 con 0x7f2174111c30 2024-05-21T03:28:52.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.479+0000 7f217b33e700 1 -- 172.21.15.27:0/3348374661 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2174118050 con 0x7f2174111c30 2024-05-21T03:28:52.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.479+0000 7f216b7fe700 1 -- 172.21.15.27:0/3348374661 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f216c014440 con 0x7f2174111c30 2024-05-21T03:28:52.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.479+0000 7f216b7fe700 1 -- 172.21.15.27:0/3348374661 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216c007590 con 0x7f2174111c30 2024-05-21T03:28:52.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.483+0000 7f217b33e700 1 -- 172.21.15.27:0/3348374661 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2158005330 con 0x7f2174111c30 2024-05-21T03:28:52.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.486+0000 7f216b7fe700 1 -- 172.21.15.27:0/3348374661 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7f216c01c070 con 0x7f2174111c30 2024-05-21T03:28:52.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.487+0000 7f216b7fe700 1 --2- 172.21.15.27:0/3348374661 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f21600b2c00 0x7f21600b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:52.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.488+0000 7f2179b3b700 1 --2- 172.21.15.27:0/3348374661 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f21600b2c00 0x7f21600b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:52.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.488+0000 7f216b7fe700 1 -- 172.21.15.27:0/3348374661 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f216c0e18c0 con 0x7f2174111c30 2024-05-21T03:28:52.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.488+0000 7f2179b3b700 1 --2- 172.21.15.27:0/3348374661 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f21600b2c00 0x7f21600b50c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2164006010 tx=0x7f216401a040 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:28:52.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.490+0000 7f216b7fe700 1 -- 172.21.15.27:0/3348374661 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f216c014b80 con 0x7f2174111c30 2024-05-21T03:28:52.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.871+0000 7f217b33e700 1 -- 172.21.15.27:0/3348374661 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2158005620 con 0x7f2174111c30 2024-05-21T03:28:52.872 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:52 smithi027 ceph-mon[32179]: pgmap v148: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 MiB/s wr, 5.26k op/s 2024-05-21T03:28:52.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.872+0000 7f216b7fe700 1 -- 172.21.15.27:0/3348374661 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11933 (secure 0 0 0) 0x7f216c0a93d0 con 0x7f2174111c30 2024-05-21T03:28:52.873 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:28:52.873 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/148854869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":148854869},{"type":"v1","addr":"172.21.15.27:6835","nonce":148854869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/4145544730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":4145544730},{"type":"v1","addr":"172.21.15.27:6837","nonce":4145544730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T03:20:34.055985+0000","modified":"2024-05-21T03:27:47.260575+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24472,"mds_2":24457,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/745486764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":745486764},{"type":"v1","addr":"172.21.15.27:6839","nonce":745486764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6841/3982752351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3982752351},{"type":"v1","addr":"172.21.15.27:6841","nonce":3982752351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/3497521608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3497521608},{"type":"v1","addr":"172.21.15.39:6835","nonce":3497521608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.165:6835/3576910218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3576910218},{"type":"v1","addr":"172.21.15.165:6835","nonce":3576910218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/3673846852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3673846852},{"type":"v1","addr":"172.21.15.39:6837","nonce":3673846852}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6837/691356139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":691356139},{"type":"v1","addr":"172.21.15.165:6837","nonce":691356139}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6839/4158170814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4158170814},{"type":"v1","addr":"172.21.15.165:6839","nonce":4158170814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6839/287607819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":287607819},{"type":"v1","addr":"172.21.15.39:6839","nonce":287607819}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24457,24452,24451,24472]},"id":1}]} 2024-05-21T03:28:52.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.875+0000 7f217b33e700 1 -- 172.21.15.27:0/3348374661 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f21600b2c00 msgr2=0x7f21600b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:52.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.875+0000 7f217b33e700 1 --2- 172.21.15.27:0/3348374661 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f21600b2c00 0x7f21600b50c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2164006010 tx=0x7f216401a040 comp rx=0 tx=0).stop 2024-05-21T03:28:52.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.875+0000 7f217b33e700 1 -- 172.21.15.27:0/3348374661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2174111c30 msgr2=0x7f217411a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:52.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.875+0000 7f217b33e700 1 --2- 172.21.15.27:0/3348374661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2174111c30 0x7f217411a240 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f216c00ba50 tx=0x7f216c00be10 comp rx=0 tx=0).stop 2024-05-21T03:28:52.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.875+0000 7f217b33e700 1 -- 172.21.15.27:0/3348374661 shutdown_connections 2024-05-21T03:28:52.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.876+0000 7f217b33e700 1 --2- 172.21.15.27:0/3348374661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f217411cd20 0x7f2174113520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:52.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.876+0000 7f217b33e700 1 --2- 172.21.15.27:0/3348374661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2174111c30 0x7f217411a240 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:52.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.876+0000 7f217b33e700 1 --2- 172.21.15.27:0/3348374661 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f21600b2c00 0x7f21600b50c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:52.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.876+0000 7f217b33e700 1 --2- 172.21.15.27:0/3348374661 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21741120a0 0x7f217411a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:52.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.876+0000 7f217b33e700 1 -- 172.21.15.27:0/3348374661 >> 172.21.15.27:0/3348374661 conn(0x7f217406e520 msgr2=0x7f217410f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:28:52.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.876+0000 7f217b33e700 1 -- 172.21.15.27:0/3348374661 shutdown_connections 2024-05-21T03:28:52.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:52.876+0000 7f217b33e700 1 -- 172.21.15.27:0/3348374661 wait complete. 2024-05-21T03:28:52.877 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 33 2024-05-21T03:28:53.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:52 smithi039 ceph-mon[36429]: pgmap v148: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 MiB/s wr, 5.26k op/s 2024-05-21T03:28:53.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:52 smithi165 ceph-mon[36269]: pgmap v148: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 MiB/s wr, 5.26k op/s 2024-05-21T03:28:53.691 DEBUG:tasks.quiescer.fs.[cephfs]:Dumping ops on rank 4 (f) to a remote file /var/run/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ops-72cf81f5-1-mds.f.json 2024-05-21T03:28:53.691 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell mds.1:4 ops --flags=locks --path=/var/run/ceph/ops-72cf81f5-1-mds.f.json 2024-05-21T03:28:53.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:53 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3348374661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:28:53.991 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:28:54.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:53 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/3348374661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:28:54.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:53 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/3348374661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:28:54.630 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:54 smithi027 ceph-mon[32179]: pgmap v149: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 52 MiB/s wr, 7.07k op/s 2024-05-21T03:28:55.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:54 smithi039 ceph-mon[36429]: pgmap v149: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 52 MiB/s wr, 7.07k op/s 2024-05-21T03:28:55.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:54 smithi165 ceph-mon[36269]: pgmap v149: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 52 MiB/s wr, 7.07k op/s 2024-05-21T03:28:55.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.957+0000 7fbbf3a58700 1 -- 172.21.15.27:0/3546917859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbec1035c0 msgr2=0x7fbbec103a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:55.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.957+0000 7fbbf3a58700 1 --2- 172.21.15.27:0/3546917859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbec1035c0 0x7fbbec103a40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc009990 tx=0x7fbbdc009ca0 comp rx=0 tx=0).stop 2024-05-21T03:28:55.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.958+0000 7fbbf3a58700 1 -- 172.21.15.27:0/3546917859 shutdown_connections 2024-05-21T03:28:55.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.958+0000 7fbbf3a58700 1 --2- 172.21.15.27:0/3546917859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbec103f80 0x7fbbec110410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:55.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.958+0000 7fbbf3a58700 1 --2- 172.21.15.27:0/3546917859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbec1095d0 0x7fbbec1099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:55.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.958+0000 7fbbf3a58700 1 --2- 172.21.15.27:0/3546917859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbec1035c0 0x7fbbec103a40 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:55.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.958+0000 7fbbf3a58700 1 -- 172.21.15.27:0/3546917859 >> 172.21.15.27:0/3546917859 conn(0x7fbbec07a6f0 msgr2=0x7fbbec07ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:28:55.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.958+0000 7fbbf3a58700 1 -- 172.21.15.27:0/3546917859 shutdown_connections 2024-05-21T03:28:55.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.958+0000 7fbbf3a58700 1 -- 172.21.15.27:0/3546917859 wait complete. 2024-05-21T03:28:55.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.958+0000 7fbbf3a58700 1 Processor -- start 2024-05-21T03:28:55.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.958+0000 7fbbf3a58700 1 -- start start 2024-05-21T03:28:55.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.959+0000 7fbbf3a58700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbec1035c0 0x7fbbec199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:55.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.959+0000 7fbbf3a58700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbec103f80 0x7fbbec19a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:55.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.959+0000 7fbbf3a58700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbec1095d0 0x7fbbec19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:55.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.959+0000 7fbbf3a58700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbbec113800 con 0x7fbbec1035c0 2024-05-21T03:28:55.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.959+0000 7fbbf3a58700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbbec113680 con 0x7fbbec103f80 2024-05-21T03:28:55.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.959+0000 7fbbf3a58700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbbec113980 con 0x7fbbec1095d0 2024-05-21T03:28:55.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.959+0000 7fbbf17f4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbec1035c0 0x7fbbec199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:55.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.959+0000 7fbbf17f4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbec1035c0 0x7fbbec199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41592/0 (socket says 172.21.15.27:41592) 2024-05-21T03:28:55.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.959+0000 7fbbf17f4700 1 -- 172.21.15.27:0/2871945625 learned_addr learned my addr 172.21.15.27:0/2871945625 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:28:55.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.959+0000 7fbbf0ff3700 1 --2- 172.21.15.27:0/2871945625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbec103f80 0x7fbbec19a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:55.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.959+0000 7fbbf1ff5700 1 --2- 172.21.15.27:0/2871945625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbec1095d0 0x7fbbec19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:55.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.959+0000 7fbbf17f4700 1 -- 172.21.15.27:0/2871945625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbec1095d0 msgr2=0x7fbbec19e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:55.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.959+0000 7fbbf17f4700 1 --2- 172.21.15.27:0/2871945625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbec1095d0 0x7fbbec19e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:55.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.959+0000 7fbbf17f4700 1 -- 172.21.15.27:0/2871945625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbec103f80 msgr2=0x7fbbec19a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:55.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.959+0000 7fbbf17f4700 1 --2- 172.21.15.27:0/2871945625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbec103f80 0x7fbbec19a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:55.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.959+0000 7fbbf17f4700 1 -- 172.21.15.27:0/2871945625 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbec19ed20 con 0x7fbbec1035c0 2024-05-21T03:28:55.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.959+0000 7fbbf17f4700 1 --2- 172.21.15.27:0/2871945625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbec1035c0 0x7fbbec199d10 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fbbe800b530 tx=0x7fbbe800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:28:55.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.960+0000 7fbbe27fc700 1 -- 172.21.15.27:0/2871945625 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8015020 con 0x7fbbec1035c0 2024-05-21T03:28:55.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.960+0000 7fbbe27fc700 1 -- 172.21.15.27:0/2871945625 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe8011ce0 con 0x7fbbec1035c0 2024-05-21T03:28:55.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.960+0000 7fbbf3a58700 1 -- 172.21.15.27:0/2871945625 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbec19f010 con 0x7fbbec1035c0 2024-05-21T03:28:55.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.960+0000 7fbbe27fc700 1 -- 172.21.15.27:0/2871945625 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8010e40 con 0x7fbbec1035c0 2024-05-21T03:28:55.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.960+0000 7fbbf3a58700 1 -- 172.21.15.27:0/2871945625 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbec1a6890 con 0x7fbbec1035c0 2024-05-21T03:28:55.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.962+0000 7fbbe27fc700 1 -- 172.21.15.27:0/2871945625 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7fbbe8018020 con 0x7fbbec1035c0 2024-05-21T03:28:55.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.963+0000 7fbbe27fc700 1 --2- 172.21.15.27:0/2871945625 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fbbd80b2a70 0x7fbbd80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:55.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.963+0000 7fbbf0ff3700 1 --2- 172.21.15.27:0/2871945625 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fbbd80b2a70 0x7fbbd80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:55.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.963+0000 7fbbe27fc700 1 -- 172.21.15.27:0/2871945625 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fbbe80e1d60 con 0x7fbbec1035c0 2024-05-21T03:28:55.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.964+0000 7fbbfe2eeb80 1 Processor -- start 2024-05-21T03:28:55.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.964+0000 7fbbf0ff3700 1 --2- 172.21.15.27:0/2871945625 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fbbd80b2a70 0x7fbbd80b4f30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00b5c0 tx=0x7fbbdc005960 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:28:55.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.964+0000 7fbbfe2eeb80 1 -- start start 2024-05-21T03:28:55.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.964+0000 7fbbfe2eeb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bc72a9700 0x560bc72a9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:55.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.964+0000 7fbbfe2eeb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc72a4bd0 0x560bc72a5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:55.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.964+0000 7fbbfe2eeb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560bc72a5590 0x560bc72b1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:55.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.964+0000 7fbbfe2eeb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560bc7202e50 con 0x560bc72a5590 2024-05-21T03:28:55.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.964+0000 7fbbfe2eeb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560bc71f29f0 con 0x560bc72a9700 2024-05-21T03:28:55.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.964+0000 7fbbfe2eeb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560bc71f6a30 con 0x560bc72a4bd0 2024-05-21T03:28:55.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.964+0000 7fbbf1ff5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560bc72a5590 0x560bc72b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:55.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.964+0000 7fbbf1ff5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560bc72a5590 0x560bc72b1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41594/0 (socket says 172.21.15.27:41594) 2024-05-21T03:28:55.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.964+0000 7fbbf1ff5700 1 -- 172.21.15.27:0/2122231798 learned_addr learned my addr 172.21.15.27:0/2122231798 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:28:55.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.964+0000 7fbbf17f4700 1 --2- 172.21.15.27:0/2122231798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bc72a9700 0x560bc72a9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:55.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.964+0000 7fbbf0ff3700 1 --2- 172.21.15.27:0/2122231798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc72a4bd0 0x560bc72a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:55.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.965+0000 7fbbf17f4700 1 -- 172.21.15.27:0/2122231798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc72a4bd0 msgr2=0x560bc72a5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:28:55.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.965+0000 7fbbf17f4700 1 --2- 172.21.15.27:0/2122231798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc72a4bd0 0x560bc72a5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:55.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.965+0000 7fbbf17f4700 1 -- 172.21.15.27:0/2122231798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560bc72a5590 msgr2=0x560bc72b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:28:55.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.965+0000 7fbbf17f4700 1 --2- 172.21.15.27:0/2122231798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560bc72a5590 0x560bc72b1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:55.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.965+0000 7fbbf17f4700 1 -- 172.21.15.27:0/2122231798 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bc719c090 con 0x560bc72a9700 2024-05-21T03:28:55.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.965+0000 7fbbf17f4700 1 --2- 172.21.15.27:0/2122231798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bc72a9700 0x560bc72a9ae0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80df5c0 tx=0x7fbbe80a88c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9587d44cbae92a2f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:28:55.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.965+0000 7fbbd77fe700 1 -- 172.21.15.27:0/2122231798 <== mon.2 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe80a9150 con 0x560bc72a9700 2024-05-21T03:28:55.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.965+0000 7fbbd77fe700 1 -- 172.21.15.27:0/2122231798 <== mon.2 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe80ad500 con 0x560bc72a9700 2024-05-21T03:28:55.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.965+0000 7fbbd77fe700 1 -- 172.21.15.27:0/2122231798 <== mon.2 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe80b1d80 con 0x560bc72a9700 2024-05-21T03:28:55.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.966+0000 7fbbfe2eeb80 1 -- 172.21.15.27:0/2122231798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bc72a9700 msgr2=0x560bc72a9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:55.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.966+0000 7fbbfe2eeb80 1 --2- 172.21.15.27:0/2122231798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bc72a9700 0x560bc72a9ae0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80df5c0 tx=0x7fbbe80a88c0 comp rx=0 tx=0).stop 2024-05-21T03:28:55.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.966+0000 7fbbfe2eeb80 1 -- 172.21.15.27:0/2122231798 shutdown_connections 2024-05-21T03:28:55.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.966+0000 7fbbfe2eeb80 1 --2- 172.21.15.27:0/2122231798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560bc72a5590 0x560bc72b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:55.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.966+0000 7fbbfe2eeb80 1 --2- 172.21.15.27:0/2122231798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bc72a9700 0x560bc72a9ae0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:55.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.966+0000 7fbbfe2eeb80 1 --2- 172.21.15.27:0/2122231798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc72a4bd0 0x560bc72a5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:55.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.966+0000 7fbbfe2eeb80 1 -- 172.21.15.27:0/2122231798 >> 172.21.15.27:0/2122231798 conn(0x560bc71d0320 msgr2=0x560bc729f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:28:55.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.966+0000 7fbbfe2eeb80 1 -- 172.21.15.27:0/2122231798 shutdown_connections 2024-05-21T03:28:55.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.966+0000 7fbbfe2eeb80 1 -- 172.21.15.27:0/2122231798 wait complete. 2024-05-21T03:28:55.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.966+0000 7fbbfe2eeb80 1 -- start start 2024-05-21T03:28:55.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.966+0000 7fbbfe2eeb80 1 Processor -- start 2024-05-21T03:28:55.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.967+0000 7fbbfe2eeb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc72a4bd0 0x560bc72b1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:55.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.967+0000 7fbbfe2eeb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bc72a5590 0x560bc72ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:55.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.967+0000 7fbbfe2eeb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560bc72a9700 0x560bc72aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:55.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.967+0000 7fbbfe2eeb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560bc70f9080 con 0x560bc72a9700 2024-05-21T03:28:55.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.967+0000 7fbbfe2eeb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560bc715ae80 con 0x560bc72a5590 2024-05-21T03:28:55.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.967+0000 7fbbfe2eeb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560bc72b1c60 con 0x560bc72a4bd0 2024-05-21T03:28:55.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.967+0000 7fbbfe2eeb80 10 client.0 _ll_register_callbacks cb 0x560bc718da30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:28:55.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.967+0000 7fbbf1ff5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560bc72a9700 0x560bc72aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:55.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.967+0000 7fbbf1ff5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560bc72a9700 0x560bc72aeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41602/0 (socket says 172.21.15.27:41602) 2024-05-21T03:28:55.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.967+0000 7fbbf1ff5700 1 -- 172.21.15.27:0/2075978326 learned_addr learned my addr 172.21.15.27:0/2075978326 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:28:55.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.967+0000 7fbbf0ff3700 1 --2- 172.21.15.27:0/2075978326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bc72a5590 0x560bc72ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:55.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.967+0000 7fbbf17f4700 1 --2- 172.21.15.27:0/2075978326 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc72a4bd0 0x560bc72b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:55.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.967+0000 7fbbf1ff5700 1 -- 172.21.15.27:0/2075978326 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc72a4bd0 msgr2=0x560bc72b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:28:55.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.967+0000 7fbbf1ff5700 1 --2- 172.21.15.27:0/2075978326 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc72a4bd0 0x560bc72b1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:55.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.967+0000 7fbbf1ff5700 1 -- 172.21.15.27:0/2075978326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bc72a5590 msgr2=0x560bc72ae5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:28:55.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.967+0000 7fbbf1ff5700 1 --2- 172.21.15.27:0/2075978326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bc72a5590 0x560bc72ae5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:55.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.967+0000 7fbbf1ff5700 1 -- 172.21.15.27:0/2075978326 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bc71429b0 con 0x560bc72a9700 2024-05-21T03:28:55.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.967+0000 7fbbf1ff5700 1 --2- 172.21.15.27:0/2075978326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560bc72a9700 0x560bc72aeb10 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fbbe400be80 tx=0x7fbbe400beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35d2573031a7802 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:28:55.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.967+0000 7fbbd4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:28:55.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.968+0000 7fbbd4ff9700 1 -- 172.21.15.27:0/2075978326 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4007920 con 0x560bc72a9700 2024-05-21T03:28:55.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.968+0000 7fbbfe2eeb80 1 -- 172.21.15.27:0/2075978326 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbe80ad500 con 0x560bc72a9700 2024-05-21T03:28:55.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.968+0000 7fbbd4ff9700 1 -- 172.21.15.27:0/2075978326 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fbbe4007ac0 con 0x560bc72a9700 2024-05-21T03:28:55.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.968+0000 7fbbd4ff9700 1 -- 172.21.15.27:0/2075978326 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbbe4007cc0 con 0x560bc72a9700 2024-05-21T03:28:55.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.968+0000 7fbbfe2eeb80 10 client.15627 fetch_fsmap learned FSMap version 33 2024-05-21T03:28:55.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.968+0000 7fbbfe2eeb80 1 -- 172.21.15.27:0/2075978326 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560bc71ac840 con 0x560bc72a9700 2024-05-21T03:28:55.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.968+0000 7fbbd4ff9700 1 -- 172.21.15.27:0/2075978326 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe80ad500 con 0x560bc72a9700 2024-05-21T03:28:55.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.968+0000 7fbbd4ff9700 1 -- 172.21.15.27:0/2075978326 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7fbbe401f3b0 con 0x560bc72a9700 2024-05-21T03:28:55.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.968+0000 7fbbfe2eeb80 10 client.15627 fetch_fsmap finished waiting for FSMap version 33 2024-05-21T03:28:55.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.968+0000 7fbbfe2eeb80 10 client.15627 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-21T03:28:55.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.968+0000 7fbbfe2eeb80 20 client.15627 populate_metadata read hostname 'smithi027' 2024-05-21T03:28:55.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.968+0000 7fbbfe2eeb80 1 --2- 172.21.15.27:0/2075978326 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x560bc72be870 0x560bc72c0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:55.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.968+0000 7fbbfe2eeb80 4 client.15627 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:28:55.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.968+0000 7fbbfe2eeb80 1 -- 172.21.15.27:0/2075978326 --> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bc7204da0 con 0x560bc72be870 2024-05-21T03:28:55.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.969+0000 7fbbf17f4700 1 --2- 172.21.15.27:0/2075978326 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x560bc72be870 0x560bc72c0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:55.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.969+0000 7fbbf17f4700 1 --2- 172.21.15.27:0/2075978326 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x560bc72be870 0x560bc72c0c60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a58b24438dc9bd02 server_cookie=f146c5d5f5b57b7c in_seq=0 out_seq=0 2024-05-21T03:28:55.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.969+0000 7fbbd4ff9700 10 client.15627 ms_handle_connect on v2:172.21.15.165:6834/3576910218 2024-05-21T03:28:55.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.970+0000 7fbbd4ff9700 1 -- 172.21.15.27:0/2075978326 <== mds.4 v2:172.21.15.165:6834/3576910218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560bc7204da0 con 0x560bc72be870 2024-05-21T03:28:55.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.970+0000 7fbbd4ff9700 10 client.15627 handle_command_reply: tid=0 2024-05-21T03:28:55.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.970+0000 7fbbfe2eeb80 1 client.15627 shutdown 2024-05-21T03:28:55.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.970+0000 7fbbfe2eeb80 1 -- 172.21.15.27:0/2075978326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560bc72a9700 msgr2=0x560bc72aeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:55.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.970+0000 7fbbfe2eeb80 1 --2- 172.21.15.27:0/2075978326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560bc72a9700 0x560bc72aeb10 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fbbe400be80 tx=0x7fbbe400beb0 comp rx=0 tx=0).stop 2024-05-21T03:28:55.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.970+0000 7fbbfe2eeb80 1 -- 172.21.15.27:0/2075978326 shutdown_connections 2024-05-21T03:28:55.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.970+0000 7fbbfe2eeb80 1 --2- 172.21.15.27:0/2075978326 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x560bc72be870 0x560bc72c0c60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:55.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.970+0000 7fbbfe2eeb80 1 --2- 172.21.15.27:0/2075978326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560bc72a9700 0x560bc72aeb10 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:55.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.970+0000 7fbbfe2eeb80 1 --2- 172.21.15.27:0/2075978326 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc72a4bd0 0x560bc72b1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:55.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.970+0000 7fbbfe2eeb80 1 --2- 172.21.15.27:0/2075978326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bc72a5590 0x560bc72ae5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:55.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.970+0000 7fbbfe2eeb80 1 -- 172.21.15.27:0/2075978326 >> 172.21.15.27:0/2075978326 conn(0x560bc71d0320 msgr2=0x560bc72a3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:28:55.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.970+0000 7fbbd4ff9700 0 client.15627 ms_handle_reset on v2:172.21.15.165:6834/3576910218 2024-05-21T03:28:55.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.970+0000 7fbbfe2eeb80 1 -- 172.21.15.27:0/2075978326 shutdown_connections 2024-05-21T03:28:55.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.970+0000 7fbbfe2eeb80 1 -- 172.21.15.27:0/2075978326 wait complete. 2024-05-21T03:28:55.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.970+0000 7fbbfe2eeb80 20 client.15627 trim_cache size 0 max 16384 2024-05-21T03:28:55.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.990+0000 7fbbfe2eeb80 1 Processor -- start 2024-05-21T03:28:55.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.990+0000 7fbbfe2eeb80 1 -- start start 2024-05-21T03:28:55.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.990+0000 7fbbfe2eeb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bc71d0320 0x560bc72ad530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:55.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.990+0000 7fbbfe2eeb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560bc72a4bd0 0x560bc7299e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:55.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.990+0000 7fbbfe2eeb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc72a5590 0x560bc72b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:55.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.990+0000 7fbbfe2eeb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbbec1adf10 con 0x560bc72a4bd0 2024-05-21T03:28:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.990+0000 7fbbfe2eeb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbbec1afb40 con 0x560bc71d0320 2024-05-21T03:28:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.990+0000 7fbbfe2eeb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbbec1ae5e0 con 0x560bc72a5590 2024-05-21T03:28:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.990+0000 7fbbf0ff3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560bc72a4bd0 0x560bc7299e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.990+0000 7fbbf0ff3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560bc72a4bd0 0x560bc7299e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41608/0 (socket says 172.21.15.27:41608) 2024-05-21T03:28:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.990+0000 7fbbf0ff3700 1 -- 172.21.15.27:0/2954776623 learned_addr learned my addr 172.21.15.27:0/2954776623 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:28:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.990+0000 7fbbf17f4700 1 --2- 172.21.15.27:0/2954776623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bc71d0320 0x560bc72ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.990+0000 7fbbf1ff5700 1 --2- 172.21.15.27:0/2954776623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc72a5590 0x560bc72b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.990+0000 7fbbf0ff3700 1 -- 172.21.15.27:0/2954776623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc72a5590 msgr2=0x560bc72b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:28:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.990+0000 7fbbf0ff3700 1 --2- 172.21.15.27:0/2954776623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc72a5590 0x560bc72b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.990+0000 7fbbf0ff3700 1 -- 172.21.15.27:0/2954776623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bc71d0320 msgr2=0x560bc72ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:28:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.990+0000 7fbbf0ff3700 1 --2- 172.21.15.27:0/2954776623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bc71d0320 0x560bc72ad530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.990+0000 7fbbf0ff3700 1 -- 172.21.15.27:0/2954776623 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bc72c1290 con 0x560bc72a4bd0 2024-05-21T03:28:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.990+0000 7fbbf17f4700 1 --2- 172.21.15.27:0/2954776623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bc71d0320 0x560bc72ad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:28:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.991+0000 7fbbf0ff3700 1 --2- 172.21.15.27:0/2954776623 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560bc72a4bd0 0x560bc7299e50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc000c00 tx=0x7fbbdc023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d3b8e07ad11c25 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:28:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.991+0000 7fbbd6ffd700 1 -- 172.21.15.27:0/2954776623 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc035070 con 0x560bc72a4bd0 2024-05-21T03:28:55.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.991+0000 7fbbd6ffd700 1 -- 172.21.15.27:0/2954776623 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbdc00f460 con 0x560bc72a4bd0 2024-05-21T03:28:55.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.991+0000 7fbbd6ffd700 1 -- 172.21.15.27:0/2954776623 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc030690 con 0x560bc72a4bd0 2024-05-21T03:28:55.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.991+0000 7fbbfe2eeb80 1 -- 172.21.15.27:0/2954776623 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560bc72a4bd0 msgr2=0x560bc7299e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:55.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.991+0000 7fbbfe2eeb80 1 --2- 172.21.15.27:0/2954776623 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560bc72a4bd0 0x560bc7299e50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc000c00 tx=0x7fbbdc023000 comp rx=0 tx=0).stop 2024-05-21T03:28:55.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.991+0000 7fbbfe2eeb80 1 -- 172.21.15.27:0/2954776623 shutdown_connections 2024-05-21T03:28:55.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.991+0000 7fbbfe2eeb80 1 --2- 172.21.15.27:0/2954776623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc72a5590 0x560bc72b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:55.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.991+0000 7fbbfe2eeb80 1 --2- 172.21.15.27:0/2954776623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bc71d0320 0x560bc72ad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:55.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.991+0000 7fbbfe2eeb80 1 --2- 172.21.15.27:0/2954776623 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560bc72a4bd0 0x560bc7299e50 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:55.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.991+0000 7fbbfe2eeb80 1 -- 172.21.15.27:0/2954776623 >> 172.21.15.27:0/2954776623 conn(0x560bc72a14a0 msgr2=0x560bc72afdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:28:55.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.991+0000 7fbbfe2eeb80 1 -- 172.21.15.27:0/2954776623 shutdown_connections 2024-05-21T03:28:55.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.991+0000 7fbbfe2eeb80 1 -- 172.21.15.27:0/2954776623 wait complete. 2024-05-21T03:28:55.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.992+0000 7fbbfe2eeb80 1 -- start start 2024-05-21T03:28:55.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.992+0000 7fbbfe2eeb80 1 Processor -- start 2024-05-21T03:28:55.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.992+0000 7fbbfe2eeb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc71d0320 0x560bc72b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:55.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.992+0000 7fbbfe2eeb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bc72a4bd0 0x560bc72b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:55.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.992+0000 7fbbfe2eeb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560bc72a5590 0x560bc7323380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:55.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.992+0000 7fbbfe2eeb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbbec1af3b0 con 0x560bc72a5590 2024-05-21T03:28:55.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.992+0000 7fbbfe2eeb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbbec1aece0 con 0x560bc72a4bd0 2024-05-21T03:28:55.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.992+0000 7fbbfe2eeb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbbec1abeb0 con 0x560bc71d0320 2024-05-21T03:28:55.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.992+0000 7fbbfe2eeb80 10 client.0 _ll_register_callbacks cb 0x560bc718da30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:28:55.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.992+0000 7fbbf1ff5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560bc72a5590 0x560bc7323380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:55.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.993+0000 7fbbf1ff5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560bc72a5590 0x560bc7323380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41610/0 (socket says 172.21.15.27:41610) 2024-05-21T03:28:55.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.993+0000 7fbbf1ff5700 1 -- 172.21.15.27:0/1412599115 learned_addr learned my addr 172.21.15.27:0/1412599115 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:28:55.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.993+0000 7fbbf17f4700 1 --2- 172.21.15.27:0/1412599115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc71d0320 0x560bc72b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:55.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.993+0000 7fbbf0ff3700 1 --2- 172.21.15.27:0/1412599115 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bc72a4bd0 0x560bc72b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:55.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.993+0000 7fbbf1ff5700 1 -- 172.21.15.27:0/1412599115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc71d0320 msgr2=0x560bc72b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:28:55.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.993+0000 7fbbf1ff5700 1 --2- 172.21.15.27:0/1412599115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc71d0320 0x560bc72b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:55.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.993+0000 7fbbf1ff5700 1 -- 172.21.15.27:0/1412599115 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bc72a4bd0 msgr2=0x560bc72b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:28:55.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.993+0000 7fbbf1ff5700 1 --2- 172.21.15.27:0/1412599115 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bc72a4bd0 0x560bc72b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:55.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.993+0000 7fbbf1ff5700 1 -- 172.21.15.27:0/1412599115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bc73239f0 con 0x560bc72a5590 2024-05-21T03:28:55.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.993+0000 7fbbf1ff5700 1 --2- 172.21.15.27:0/1412599115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560bc72a5590 0x560bc7323380 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4004c20 tx=0x7fbbe401f660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ca4ba5db3298ac server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:28:55.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.993+0000 7fbbd4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:28:55.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.993+0000 7fbbd4ff9700 1 -- 172.21.15.27:0/1412599115 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe401fe60 con 0x560bc72a5590 2024-05-21T03:28:55.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.993+0000 7fbbfe2eeb80 1 -- 172.21.15.27:0/1412599115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbdc00f460 con 0x560bc72a5590 2024-05-21T03:28:55.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.993+0000 7fbbd4ff9700 1 -- 172.21.15.27:0/1412599115 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fbbe4028420 con 0x560bc72a5590 2024-05-21T03:28:55.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.993+0000 7fbbd4ff9700 1 -- 172.21.15.27:0/1412599115 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbbe4007e60 con 0x560bc72a5590 2024-05-21T03:28:55.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.993+0000 7fbbfe2eeb80 10 client.15633 fetch_fsmap learned FSMap version 33 2024-05-21T03:28:55.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.993+0000 7fbbfe2eeb80 1 -- 172.21.15.27:0/1412599115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560bc732acc0 con 0x560bc72a5590 2024-05-21T03:28:55.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.994+0000 7fbbd4ff9700 1 -- 172.21.15.27:0/1412599115 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbdc00f460 con 0x560bc72a5590 2024-05-21T03:28:55.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.994+0000 7fbbd4ff9700 1 -- 172.21.15.27:0/1412599115 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7fbbe4030400 con 0x560bc72a5590 2024-05-21T03:28:55.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.994+0000 7fbbfe2eeb80 10 client.15633 fetch_fsmap finished waiting for FSMap version 33 2024-05-21T03:28:55.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.994+0000 7fbbfe2eeb80 10 client.15633 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-05-21T03:28:55.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.994+0000 7fbbfe2eeb80 20 client.15633 populate_metadata read hostname 'smithi027' 2024-05-21T03:28:55.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.994+0000 7fbbfe2eeb80 1 --2- 172.21.15.27:0/1412599115 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x560bc72a9700 0x560bc732d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:55.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.994+0000 7fbbfe2eeb80 4 client.15633 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-72cf81f5-1-mds.f.json"}] 2024-05-21T03:28:55.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.994+0000 7fbbfe2eeb80 1 -- 172.21.15.27:0/1412599115 --> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-72cf81f5-1-mds.f.json"}) -- 0x560bc71fe580 con 0x560bc72a9700 2024-05-21T03:28:55.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.994+0000 7fbbf17f4700 1 --2- 172.21.15.27:0/1412599115 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x560bc72a9700 0x560bc732d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:55.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.995+0000 7fbbf17f4700 1 --2- 172.21.15.27:0/1412599115 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x560bc72a9700 0x560bc732d480 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dabb6ac521d630a server_cookie=c824fe6e48e73395 in_seq=0 out_seq=0 2024-05-21T03:28:55.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:55.995+0000 7fbbd4ff9700 10 client.15633 ms_handle_connect on v2:172.21.15.165:6834/3576910218 2024-05-21T03:28:56.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:56.005+0000 7fbbd4ff9700 1 -- 172.21.15.27:0/1412599115 <== mds.4 v2:172.21.15.165:6834/3576910218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+58 (crc 0 0 0) 0x560bc71fe580 con 0x560bc72a9700 2024-05-21T03:28:56.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:56.005+0000 7fbbd4ff9700 10 client.15633 handle_command_reply: tid=0 2024-05-21T03:28:56.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:56.006+0000 7fbbfe2eeb80 1 client.15633 shutdown 2024-05-21T03:28:56.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:56.006+0000 7fbbfe2eeb80 1 -- 172.21.15.27:0/1412599115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560bc72a5590 msgr2=0x560bc7323380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:56.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:56.006+0000 7fbbfe2eeb80 1 --2- 172.21.15.27:0/1412599115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560bc72a5590 0x560bc7323380 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4004c20 tx=0x7fbbe401f660 comp rx=0 tx=0).stop 2024-05-21T03:28:56.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:56.006+0000 7fbbfe2eeb80 1 -- 172.21.15.27:0/1412599115 shutdown_connections 2024-05-21T03:28:56.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:56.006+0000 7fbbfe2eeb80 1 --2- 172.21.15.27:0/1412599115 >> [v2:172.21.15.165:6834/3576910218,v1:172.21.15.165:6835/3576910218] conn(0x560bc72a9700 0x560bc732d480 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:56.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:56.006+0000 7fbbfe2eeb80 1 --2- 172.21.15.27:0/1412599115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560bc72a5590 0x560bc7323380 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:56.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:56.006+0000 7fbbfe2eeb80 1 --2- 172.21.15.27:0/1412599115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc71d0320 0x560bc72b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:56.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:56.006+0000 7fbbfe2eeb80 1 --2- 172.21.15.27:0/1412599115 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bc72a4bd0 0x560bc72b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:56.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:56.006+0000 7fbbfe2eeb80 1 -- 172.21.15.27:0/1412599115 >> 172.21.15.27:0/1412599115 conn(0x560bc72a14a0 msgr2=0x560bc72b0640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:28:56.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:56.006+0000 7fbbd4ff9700 0 client.15633 ms_handle_reset on v2:172.21.15.165:6834/3576910218 2024-05-21T03:28:56.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:56.006+0000 7fbbfe2eeb80 1 -- 172.21.15.27:0/1412599115 shutdown_connections 2024-05-21T03:28:56.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:56.006+0000 7fbbfe2eeb80 1 -- 172.21.15.27:0/1412599115 wait complete. 2024-05-21T03:28:56.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:56.006+0000 7fbbfe2eeb80 20 client.15633 trim_cache size 0 max 16384 2024-05-21T03:28:56.008 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:28:56.008 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/var/run/ceph/ops-72cf81f5-1-mds.f.json" 2024-05-21T03:28:56.008 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:28:56.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:56.007+0000 7fbbf3a58700 1 -- 172.21.15.27:0/2871945625 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fbbd80b2a70 msgr2=0x7fbbd80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:56.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:56.007+0000 7fbbf3a58700 1 --2- 172.21.15.27:0/2871945625 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fbbd80b2a70 0x7fbbd80b4f30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00b5c0 tx=0x7fbbdc005960 comp rx=0 tx=0).stop 2024-05-21T03:28:56.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:56.007+0000 7fbbf3a58700 1 -- 172.21.15.27:0/2871945625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbec1035c0 msgr2=0x7fbbec199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:56.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:56.007+0000 7fbbf3a58700 1 --2- 172.21.15.27:0/2871945625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbec1035c0 0x7fbbec199d10 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fbbe800b530 tx=0x7fbbe800b840 comp rx=0 tx=0).stop 2024-05-21T03:28:56.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:56.007+0000 7fbbf3a58700 1 -- 172.21.15.27:0/2871945625 shutdown_connections 2024-05-21T03:28:56.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:56.007+0000 7fbbf3a58700 1 --2- 172.21.15.27:0/2871945625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbec1095d0 0x7fbbec19e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:56.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:56.007+0000 7fbbf3a58700 1 --2- 172.21.15.27:0/2871945625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbec1035c0 0x7fbbec199d10 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:56.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:56.007+0000 7fbbf3a58700 1 --2- 172.21.15.27:0/2871945625 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fbbd80b2a70 0x7fbbd80b4f30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:56.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:56.007+0000 7fbbf3a58700 1 --2- 172.21.15.27:0/2871945625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbec103f80 0x7fbbec19a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:56.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:56.007+0000 7fbbf3a58700 1 -- 172.21.15.27:0/2871945625 >> 172.21.15.27:0/2871945625 conn(0x7fbbec07a6f0 msgr2=0x7fbbec102490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:28:56.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:56.007+0000 7fbbf3a58700 1 -- 172.21.15.27:0/2871945625 shutdown_connections 2024-05-21T03:28:56.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:56.007+0000 7fbbf3a58700 1 -- 172.21.15.27:0/2871945625 wait complete. 2024-05-21T03:28:56.847 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:56 smithi027 ceph-mon[32179]: pgmap v150: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 32 MiB/s wr, 4.35k op/s 2024-05-21T03:28:56.848 DEBUG:tasks.quiescer.fs.[cephfs]:Dumping ops on rank 3 (h) to a remote file /var/run/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ops-72cf81f5-1-mds.h.json 2024-05-21T03:28:56.848 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell mds.1:3 ops --flags=locks --path=/var/run/ceph/ops-72cf81f5-1-mds.h.json 2024-05-21T03:28:57.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:56 smithi039 ceph-mon[36429]: pgmap v150: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 32 MiB/s wr, 4.35k op/s 2024-05-21T03:28:57.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:56 smithi165 ceph-mon[36269]: pgmap v150: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 32 MiB/s wr, 4.35k op/s 2024-05-21T03:28:57.100 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:28:58.716 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:28:58 smithi027 ceph-mon[32179]: pgmap v151: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 33 MiB/s wr, 4.46k op/s 2024-05-21T03:28:59.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:28:58 smithi039 ceph-mon[36429]: pgmap v151: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 33 MiB/s wr, 4.46k op/s 2024-05-21T03:28:59.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:28:58 smithi165 ceph-mon[36269]: pgmap v151: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 33 MiB/s wr, 4.46k op/s 2024-05-21T03:28:59.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.715+0000 7f3e61920700 1 -- 172.21.15.27:0/521529326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e5c1041a0 msgr2=0x7f3e5c1081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.715+0000 7f3e61920700 1 --2- 172.21.15.27:0/521529326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e5c1041a0 0x7f3e5c1081c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3e440099c0 tx=0x7f3e44009cd0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.715+0000 7f3e61920700 1 -- 172.21.15.27:0/521529326 shutdown_connections 2024-05-21T03:28:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.715+0000 7f3e61920700 1 --2- 172.21.15.27:0/521529326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e5c1088f0 0x7f3e5c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.715+0000 7f3e61920700 1 --2- 172.21.15.27:0/521529326 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e5c1037f0 0x7f3e5c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.715+0000 7f3e61920700 1 --2- 172.21.15.27:0/521529326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e5c1041a0 0x7f3e5c1081c0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.715+0000 7f3e61920700 1 -- 172.21.15.27:0/521529326 >> 172.21.15.27:0/521529326 conn(0x7f3e5c07a6a0 msgr2=0x7f3e5c07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:28:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.715+0000 7f3e61920700 1 -- 172.21.15.27:0/521529326 shutdown_connections 2024-05-21T03:28:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.715+0000 7f3e61920700 1 -- 172.21.15.27:0/521529326 wait complete. 2024-05-21T03:28:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.716+0000 7f3e61920700 1 Processor -- start 2024-05-21T03:28:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.716+0000 7f3e61920700 1 -- start start 2024-05-21T03:28:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.716+0000 7f3e61920700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e5c1037f0 0x7f3e5c075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.716+0000 7f3e61920700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e5c1041a0 0x7f3e5c075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.716+0000 7f3e61920700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e5c1088f0 0x7f3e5c0762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.716+0000 7f3e61920700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c1137b0 con 0x7f3e5c1041a0 2024-05-21T03:28:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.716+0000 7f3e61920700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c113630 con 0x7f3e5c1088f0 2024-05-21T03:28:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.716+0000 7f3e61920700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c113930 con 0x7f3e5c1037f0 2024-05-21T03:28:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.716+0000 7f3e5a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e5c1041a0 0x7f3e5c075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.716+0000 7f3e5a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e5c1041a0 0x7f3e5c075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41624/0 (socket says 172.21.15.27:41624) 2024-05-21T03:28:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.716+0000 7f3e5a7fc700 1 -- 172.21.15.27:0/1874230507 learned_addr learned my addr 172.21.15.27:0/1874230507 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:28:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.716+0000 7f3e5b7fe700 1 --2- 172.21.15.27:0/1874230507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e5c1088f0 0x7f3e5c0762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.716+0000 7f3e5affd700 1 --2- 172.21.15.27:0/1874230507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e5c1037f0 0x7f3e5c075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.717+0000 7f3e5a7fc700 1 -- 172.21.15.27:0/1874230507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e5c1037f0 msgr2=0x7f3e5c075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.717+0000 7f3e5a7fc700 1 --2- 172.21.15.27:0/1874230507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e5c1037f0 0x7f3e5c075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.717+0000 7f3e5a7fc700 1 -- 172.21.15.27:0/1874230507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e5c1088f0 msgr2=0x7f3e5c0762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.717+0000 7f3e5a7fc700 1 --2- 172.21.15.27:0/1874230507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e5c1088f0 0x7f3e5c0762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.717+0000 7f3e5a7fc700 1 -- 172.21.15.27:0/1874230507 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e5c1a8190 con 0x7f3e5c1041a0 2024-05-21T03:28:59.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.717+0000 7f3e5a7fc700 1 --2- 172.21.15.27:0/1874230507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e5c1041a0 0x7f3e5c075d80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400b580 tx=0x7f3e440047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:28:59.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.717+0000 7f3e6091e700 1 -- 172.21.15.27:0/1874230507 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e44027070 con 0x7f3e5c1041a0 2024-05-21T03:28:59.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.717+0000 7f3e6091e700 1 -- 172.21.15.27:0/1874230507 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e44004020 con 0x7f3e5c1041a0 2024-05-21T03:28:59.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.717+0000 7f3e61920700 1 -- 172.21.15.27:0/1874230507 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e5c1a8420 con 0x7f3e5c1041a0 2024-05-21T03:28:59.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.717+0000 7f3e6091e700 1 -- 172.21.15.27:0/1874230507 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e44022e10 con 0x7f3e5c1041a0 2024-05-21T03:28:59.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.717+0000 7f3e61920700 1 -- 172.21.15.27:0/1874230507 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e5c1a8870 con 0x7f3e5c1041a0 2024-05-21T03:28:59.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.720+0000 7f3e6091e700 1 -- 172.21.15.27:0/1874230507 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7f3e4402a070 con 0x7f3e5c1041a0 2024-05-21T03:28:59.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.721+0000 7f3e6091e700 1 --2- 172.21.15.27:0/1874230507 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f3e480b29f0 0x7f3e480b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:59.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.721+0000 7f3e5affd700 1 --2- 172.21.15.27:0/1874230507 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f3e480b29f0 0x7f3e480b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:59.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.721+0000 7f3e6091e700 1 -- 172.21.15.27:0/1874230507 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f3e440e3380 con 0x7f3e5c1041a0 2024-05-21T03:28:59.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.721+0000 7f3e6c1b6b80 1 Processor -- start 2024-05-21T03:28:59.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.721+0000 7f3e6c1b6b80 1 -- start start 2024-05-21T03:28:59.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.722+0000 7f3e6c1b6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5594b9405290 0x5594b9405670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:59.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.722+0000 7f3e6c1b6b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594b93ff8c0 0x5594b93ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:59.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.722+0000 7f3e6c1b6b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594b9400280 0x5594b940c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:59.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.722+0000 7f3e6c1b6b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5594b935ed70 con 0x5594b93ff8c0 2024-05-21T03:28:59.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.722+0000 7f3e5affd700 1 --2- 172.21.15.27:0/1874230507 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f3e480b29f0 0x7f3e480b4eb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c009a10 tx=0x7f3e4c009190 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:28:59.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.722+0000 7f3e6c1b6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5594b934e910 con 0x5594b9405290 2024-05-21T03:28:59.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.722+0000 7f3e6c1b6b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5594b9352950 con 0x5594b9400280 2024-05-21T03:28:59.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.722+0000 7f3e5a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594b93ff8c0 0x5594b93ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:59.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.722+0000 7f3e5a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594b93ff8c0 0x5594b93ffd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41626/0 (socket says 172.21.15.27:41626) 2024-05-21T03:28:59.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.722+0000 7f3e5a7fc700 1 -- 172.21.15.27:0/3926339840 learned_addr learned my addr 172.21.15.27:0/3926339840 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:28:59.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.722+0000 7f3e5affd700 1 --2- 172.21.15.27:0/3926339840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5594b9405290 0x5594b9405670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:59.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.722+0000 7f3e5a7fc700 1 -- 172.21.15.27:0/3926339840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594b9400280 msgr2=0x5594b940c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:28:59.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.722+0000 7f3e5a7fc700 1 --2- 172.21.15.27:0/3926339840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594b9400280 0x5594b940c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.722+0000 7f3e5a7fc700 1 -- 172.21.15.27:0/3926339840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5594b9405290 msgr2=0x5594b9405670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:28:59.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.722+0000 7f3e5a7fc700 1 --2- 172.21.15.27:0/3926339840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5594b9405290 0x5594b9405670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.722+0000 7f3e5a7fc700 1 -- 172.21.15.27:0/3926339840 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594b929e9b0 con 0x5594b93ff8c0 2024-05-21T03:28:59.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.722+0000 7f3e5affd700 1 --2- 172.21.15.27:0/3926339840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5594b9405290 0x5594b9405670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:28:59.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.722+0000 7f3e5a7fc700 1 --2- 172.21.15.27:0/3926339840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594b93ff8c0 0x5594b93ffd40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3e440a6770 tx=0x7f3e440aa110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c12d3f39fe83384 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:28:59.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.722+0000 7f3e51ffb700 1 -- 172.21.15.27:0/3926339840 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e440aa6f0 con 0x5594b93ff8c0 2024-05-21T03:28:59.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.722+0000 7f3e51ffb700 1 -- 172.21.15.27:0/3926339840 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e440aeaa0 con 0x5594b93ff8c0 2024-05-21T03:28:59.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.722+0000 7f3e51ffb700 1 -- 172.21.15.27:0/3926339840 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e440041d0 con 0x5594b93ff8c0 2024-05-21T03:28:59.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.723+0000 7f3e6c1b6b80 1 -- 172.21.15.27:0/3926339840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594b93ff8c0 msgr2=0x5594b93ffd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:59.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.723+0000 7f3e6c1b6b80 1 --2- 172.21.15.27:0/3926339840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594b93ff8c0 0x5594b93ffd40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3e440a6770 tx=0x7f3e440aa110 comp rx=0 tx=0).stop 2024-05-21T03:28:59.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.723+0000 7f3e6c1b6b80 1 -- 172.21.15.27:0/3926339840 shutdown_connections 2024-05-21T03:28:59.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.723+0000 7f3e6c1b6b80 1 --2- 172.21.15.27:0/3926339840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594b9400280 0x5594b940c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.723+0000 7f3e6c1b6b80 1 --2- 172.21.15.27:0/3926339840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5594b9405290 0x5594b9405670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.723+0000 7f3e6c1b6b80 1 --2- 172.21.15.27:0/3926339840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594b93ff8c0 0x5594b93ffd40 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.723+0000 7f3e6c1b6b80 1 -- 172.21.15.27:0/3926339840 >> 172.21.15.27:0/3926339840 conn(0x5594b932d160 msgr2=0x5594b93fa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:28:59.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.723+0000 7f3e6c1b6b80 1 -- 172.21.15.27:0/3926339840 shutdown_connections 2024-05-21T03:28:59.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.723+0000 7f3e6c1b6b80 1 -- 172.21.15.27:0/3926339840 wait complete. 2024-05-21T03:28:59.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.723+0000 7f3e6c1b6b80 1 -- start start 2024-05-21T03:28:59.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.724+0000 7f3e6c1b6b80 1 Processor -- start 2024-05-21T03:28:59.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.724+0000 7f3e6c1b6b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594b93ff8c0 0x5594b940be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:59.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.724+0000 7f3e6c1b6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5594b9400280 0x5594b9409180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:59.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.724+0000 7f3e6c1b6b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594b9405290 0x5594b94096c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:59.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.724+0000 7f3e6c1b6b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5594b9255080 con 0x5594b93ff8c0 2024-05-21T03:28:59.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.724+0000 7f3e6c1b6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5594b92f08a0 con 0x5594b9400280 2024-05-21T03:28:59.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.724+0000 7f3e6c1b6b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5594b9405e30 con 0x5594b9405290 2024-05-21T03:28:59.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.724+0000 7f3e6c1b6b80 10 client.0 _ll_register_callbacks cb 0x5594b92e9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:28:59.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.724+0000 7f3e5affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594b93ff8c0 0x5594b940be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:59.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.724+0000 7f3e5affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594b93ff8c0 0x5594b940be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41628/0 (socket says 172.21.15.27:41628) 2024-05-21T03:28:59.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.724+0000 7f3e5affd700 1 -- 172.21.15.27:0/1729911106 learned_addr learned my addr 172.21.15.27:0/1729911106 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:28:59.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.724+0000 7f3e5a7fc700 1 --2- 172.21.15.27:0/1729911106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5594b9400280 0x5594b9409180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:59.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.724+0000 7f3e5b7fe700 1 --2- 172.21.15.27:0/1729911106 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594b9405290 0x5594b94096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:59.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.724+0000 7f3e5affd700 1 -- 172.21.15.27:0/1729911106 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594b9405290 msgr2=0x5594b94096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:28:59.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.724+0000 7f3e5affd700 1 --2- 172.21.15.27:0/1729911106 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594b9405290 0x5594b94096c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.724+0000 7f3e5affd700 1 -- 172.21.15.27:0/1729911106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5594b9400280 msgr2=0x5594b9409180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:28:59.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.724+0000 7f3e5affd700 1 --2- 172.21.15.27:0/1729911106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5594b9400280 0x5594b9409180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.724+0000 7f3e5affd700 1 -- 172.21.15.27:0/1729911106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594b92ed770 con 0x5594b93ff8c0 2024-05-21T03:28:59.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.724+0000 7f3e5b7fe700 1 --2- 172.21.15.27:0/1729911106 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594b9405290 0x5594b94096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:28:59.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.724+0000 7f3e5a7fc700 1 --2- 172.21.15.27:0/1729911106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5594b9400280 0x5594b9409180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:28:59.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.725+0000 7f3e5affd700 1 --2- 172.21.15.27:0/1729911106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594b93ff8c0 0x5594b940be80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00ef40 tx=0x7f3e4c004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973cace1c4040601 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:28:59.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.725+0000 7f3e3affd700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:28:59.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.725+0000 7f3e3affd700 1 -- 172.21.15.27:0/1729911106 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c024020 con 0x5594b93ff8c0 2024-05-21T03:28:59.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.725+0000 7f3e3affd700 1 -- 172.21.15.27:0/1729911106 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f3e4c00f040 con 0x5594b93ff8c0 2024-05-21T03:28:59.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.725+0000 7f3e6c1b6b80 1 -- 172.21.15.27:0/1729911106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e440aeaa0 con 0x5594b93ff8c0 2024-05-21T03:28:59.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.725+0000 7f3e3affd700 1 -- 172.21.15.27:0/1729911106 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3e4c02f050 con 0x5594b93ff8c0 2024-05-21T03:28:59.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.725+0000 7f3e6c1b6b80 10 client.15645 fetch_fsmap learned FSMap version 33 2024-05-21T03:28:59.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.725+0000 7f3e6c1b6b80 1 -- 172.21.15.27:0/1729911106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5594b9416bd0 con 0x5594b93ff8c0 2024-05-21T03:28:59.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.725+0000 7f3e3affd700 1 -- 172.21.15.27:0/1729911106 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e440aeaa0 con 0x5594b93ff8c0 2024-05-21T03:28:59.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.725+0000 7f3e3affd700 1 -- 172.21.15.27:0/1729911106 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7f3e4c013450 con 0x5594b93ff8c0 2024-05-21T03:28:59.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.726+0000 7f3e6c1b6b80 10 client.15645 fetch_fsmap finished waiting for FSMap version 33 2024-05-21T03:28:59.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.726+0000 7f3e6c1b6b80 10 client.15645 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-05-21T03:28:59.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.726+0000 7f3e6c1b6b80 20 client.15645 populate_metadata read hostname 'smithi027' 2024-05-21T03:28:59.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.726+0000 7f3e6c1b6b80 1 --2- 172.21.15.27:0/1729911106 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x5594b9419210 0x5594b941b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:59.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.726+0000 7f3e6c1b6b80 4 client.15645 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:28:59.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.726+0000 7f3e6c1b6b80 1 -- 172.21.15.27:0/1729911106 --> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594b93cbac0 con 0x5594b9419210 2024-05-21T03:28:59.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.726+0000 7f3e5a7fc700 1 --2- 172.21.15.27:0/1729911106 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x5594b9419210 0x5594b941b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:59.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.726+0000 7f3e3affd700 10 client.15645 ms_handle_connect on v2:172.21.15.39:6836/3673846852 2024-05-21T03:28:59.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.726+0000 7f3e5a7fc700 1 --2- 172.21.15.27:0/1729911106 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x5594b9419210 0x5594b941b660 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=199dd811ad320722 server_cookie=18d46c63e0dedc07 in_seq=0 out_seq=0 2024-05-21T03:28:59.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.727+0000 7f3e3affd700 1 -- 172.21.15.27:0/1729911106 <== mds.3 v2:172.21.15.39:6836/3673846852 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5594b93cbac0 con 0x5594b9419210 2024-05-21T03:28:59.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.727+0000 7f3e3affd700 10 client.15645 handle_command_reply: tid=0 2024-05-21T03:28:59.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.727+0000 7f3e6c1b6b80 1 client.15645 shutdown 2024-05-21T03:28:59.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.727+0000 7f3e6c1b6b80 1 -- 172.21.15.27:0/1729911106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594b93ff8c0 msgr2=0x5594b940be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:59.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.727+0000 7f3e6c1b6b80 1 --2- 172.21.15.27:0/1729911106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594b93ff8c0 0x5594b940be80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00ef40 tx=0x7f3e4c004860 comp rx=0 tx=0).stop 2024-05-21T03:28:59.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.727+0000 7f3e6c1b6b80 1 -- 172.21.15.27:0/1729911106 shutdown_connections 2024-05-21T03:28:59.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.727+0000 7f3e6c1b6b80 1 --2- 172.21.15.27:0/1729911106 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594b9405290 0x5594b94096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.727+0000 7f3e6c1b6b80 1 --2- 172.21.15.27:0/1729911106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594b93ff8c0 0x5594b940be80 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.727+0000 7f3e6c1b6b80 1 --2- 172.21.15.27:0/1729911106 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x5594b9419210 0x5594b941b660 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.727+0000 7f3e6c1b6b80 1 --2- 172.21.15.27:0/1729911106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5594b9400280 0x5594b9409180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.727+0000 7f3e3affd700 0 client.15645 ms_handle_reset on v2:172.21.15.39:6836/3673846852 2024-05-21T03:28:59.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.727+0000 7f3e6c1b6b80 1 -- 172.21.15.27:0/1729911106 >> 172.21.15.27:0/1729911106 conn(0x5594b932d160 msgr2=0x5594b93f4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:28:59.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.727+0000 7f3e6c1b6b80 1 -- 172.21.15.27:0/1729911106 shutdown_connections 2024-05-21T03:28:59.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.727+0000 7f3e6c1b6b80 1 -- 172.21.15.27:0/1729911106 wait complete. 2024-05-21T03:28:59.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.727+0000 7f3e6c1b6b80 20 client.15645 trim_cache size 0 max 16384 2024-05-21T03:28:59.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.746+0000 7f3e6c1b6b80 1 Processor -- start 2024-05-21T03:28:59.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.746+0000 7f3e6c1b6b80 1 -- start start 2024-05-21T03:28:59.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.746+0000 7f3e6c1b6b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594b93ff8c0 0x5594b93fef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:59.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.746+0000 7f3e6c1b6b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594b9400280 0x5594b94154e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:59.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.746+0000 7f3e6c1b6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5594b9405290 0x5594b940b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:59.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.746+0000 7f3e6c1b6b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c1b0040 con 0x5594b9400280 2024-05-21T03:28:59.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.746+0000 7f3e6c1b6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c1b1c70 con 0x5594b9405290 2024-05-21T03:28:59.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.746+0000 7f3e6c1b6b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c1b0710 con 0x5594b93ff8c0 2024-05-21T03:28:59.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.746+0000 7f3e5a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594b9400280 0x5594b94154e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:59.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.746+0000 7f3e5a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594b9400280 0x5594b94154e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41642/0 (socket says 172.21.15.27:41642) 2024-05-21T03:28:59.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.746+0000 7f3e5a7fc700 1 -- 172.21.15.27:0/3999239389 learned_addr learned my addr 172.21.15.27:0/3999239389 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:28:59.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.747+0000 7f3e5b7fe700 1 --2- 172.21.15.27:0/3999239389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5594b9405290 0x5594b940b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:59.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.747+0000 7f3e5a7fc700 1 -- 172.21.15.27:0/3999239389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594b93ff8c0 msgr2=0x5594b93fef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:28:59.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.747+0000 7f3e5a7fc700 1 --2- 172.21.15.27:0/3999239389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594b93ff8c0 0x5594b93fef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.747+0000 7f3e5a7fc700 1 -- 172.21.15.27:0/3999239389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5594b9405290 msgr2=0x5594b940b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:28:59.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.747+0000 7f3e5a7fc700 1 --2- 172.21.15.27:0/3999239389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5594b9405290 0x5594b940b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.747+0000 7f3e5a7fc700 1 -- 172.21.15.27:0/3999239389 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594b9409cc0 con 0x5594b9400280 2024-05-21T03:28:59.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.747+0000 7f3e5b7fe700 1 --2- 172.21.15.27:0/3999239389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5594b9405290 0x5594b940b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:28:59.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.747+0000 7f3e5a7fc700 1 --2- 172.21.15.27:0/3999239389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594b9400280 0x5594b94154e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3e440e1050 tx=0x7f3e44009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95b75e081c5536d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:28:59.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.747+0000 7f3e517fa700 1 -- 172.21.15.27:0/3999239389 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e440aa6f0 con 0x5594b9400280 2024-05-21T03:28:59.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.747+0000 7f3e517fa700 1 -- 172.21.15.27:0/3999239389 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e44022570 con 0x5594b9400280 2024-05-21T03:28:59.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.747+0000 7f3e517fa700 1 -- 172.21.15.27:0/3999239389 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e440b3030 con 0x5594b9400280 2024-05-21T03:28:59.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.748+0000 7f3e6c1b6b80 1 -- 172.21.15.27:0/3999239389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594b9400280 msgr2=0x5594b94154e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:59.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.748+0000 7f3e6c1b6b80 1 --2- 172.21.15.27:0/3999239389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594b9400280 0x5594b94154e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3e440e1050 tx=0x7f3e44009cd0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.748+0000 7f3e6c1b6b80 1 -- 172.21.15.27:0/3999239389 shutdown_connections 2024-05-21T03:28:59.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.748+0000 7f3e6c1b6b80 1 --2- 172.21.15.27:0/3999239389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5594b9405290 0x5594b940b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.748+0000 7f3e6c1b6b80 1 --2- 172.21.15.27:0/3999239389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594b93ff8c0 0x5594b93fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.748+0000 7f3e6c1b6b80 1 --2- 172.21.15.27:0/3999239389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594b9400280 0x5594b94154e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.748+0000 7f3e6c1b6b80 1 -- 172.21.15.27:0/3999239389 >> 172.21.15.27:0/3999239389 conn(0x5594b932d160 msgr2=0x5594b93f4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:28:59.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.748+0000 7f3e6c1b6b80 1 -- 172.21.15.27:0/3999239389 shutdown_connections 2024-05-21T03:28:59.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.748+0000 7f3e6c1b6b80 1 -- 172.21.15.27:0/3999239389 wait complete. 2024-05-21T03:28:59.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.748+0000 7f3e6c1b6b80 1 -- start start 2024-05-21T03:28:59.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.748+0000 7f3e6c1b6b80 1 Processor -- start 2024-05-21T03:28:59.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.748+0000 7f3e6c1b6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5594b93ff8c0 0x5594b94118c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:59.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.748+0000 7f3e6c1b6b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594b9400280 0x5594b9411e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:59.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.748+0000 7f3e6c1b6b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594b9405290 0x5594b940c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:59.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.749+0000 7f3e6c1b6b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c1b14e0 con 0x5594b9405290 2024-05-21T03:28:59.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.749+0000 7f3e6c1b6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c1b0e10 con 0x5594b93ff8c0 2024-05-21T03:28:59.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.749+0000 7f3e6c1b6b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c1ae000 con 0x5594b9400280 2024-05-21T03:28:59.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.749+0000 7f3e6c1b6b80 10 client.0 _ll_register_callbacks cb 0x5594b92e9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:28:59.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.749+0000 7f3e5b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594b9405290 0x5594b940c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:59.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.749+0000 7f3e5b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594b9405290 0x5594b940c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41644/0 (socket says 172.21.15.27:41644) 2024-05-21T03:28:59.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.749+0000 7f3e5b7fe700 1 -- 172.21.15.27:0/2084621806 learned_addr learned my addr 172.21.15.27:0/2084621806 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:28:59.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.749+0000 7f3e5affd700 1 --2- 172.21.15.27:0/2084621806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5594b93ff8c0 0x5594b94118c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:59.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.749+0000 7f3e5b7fe700 1 -- 172.21.15.27:0/2084621806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594b9400280 msgr2=0x5594b9411e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:28:59.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.749+0000 7f3e5b7fe700 1 --2- 172.21.15.27:0/2084621806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594b9400280 0x5594b9411e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.749+0000 7f3e5b7fe700 1 -- 172.21.15.27:0/2084621806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5594b93ff8c0 msgr2=0x5594b94118c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:28:59.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.749+0000 7f3e5b7fe700 1 --2- 172.21.15.27:0/2084621806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5594b93ff8c0 0x5594b94118c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.749+0000 7f3e5b7fe700 1 -- 172.21.15.27:0/2084621806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594b940cd90 con 0x5594b9405290 2024-05-21T03:28:59.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.749+0000 7f3e5b7fe700 1 --2- 172.21.15.27:0/2084621806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594b9405290 0x5594b940c690 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3e5400acd0 tx=0x7f3e5400ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b2ad93ea7c5372 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:28:59.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.749+0000 7f3e3affd700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:28:59.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.749+0000 7f3e3affd700 1 -- 172.21.15.27:0/2084621806 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e54008040 con 0x5594b9405290 2024-05-21T03:28:59.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.749+0000 7f3e3affd700 1 -- 172.21.15.27:0/2084621806 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f3e54008a60 con 0x5594b9405290 2024-05-21T03:28:59.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.749+0000 7f3e6c1b6b80 1 -- 172.21.15.27:0/2084621806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e44022570 con 0x5594b9405290 2024-05-21T03:28:59.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.749+0000 7f3e3affd700 1 -- 172.21.15.27:0/2084621806 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3e54008c60 con 0x5594b9405290 2024-05-21T03:28:59.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.749+0000 7f3e6c1b6b80 10 client.15651 fetch_fsmap learned FSMap version 33 2024-05-21T03:28:59.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.749+0000 7f3e6c1b6b80 1 -- 172.21.15.27:0/2084621806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5594b94848b0 con 0x5594b9405290 2024-05-21T03:28:59.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.750+0000 7f3e3affd700 1 -- 172.21.15.27:0/2084621806 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e44022570 con 0x5594b9405290 2024-05-21T03:28:59.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.750+0000 7f3e3affd700 1 -- 172.21.15.27:0/2084621806 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7f3e54021450 con 0x5594b9405290 2024-05-21T03:28:59.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.750+0000 7f3e6c1b6b80 10 client.15651 fetch_fsmap finished waiting for FSMap version 33 2024-05-21T03:28:59.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.750+0000 7f3e6c1b6b80 10 client.15651 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-05-21T03:28:59.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.750+0000 7f3e6c1b6b80 20 client.15651 populate_metadata read hostname 'smithi027' 2024-05-21T03:28:59.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.750+0000 7f3e6c1b6b80 1 --2- 172.21.15.27:0/2084621806 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x5594b9485110 0x5594b9487530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:28:59.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.750+0000 7f3e6c1b6b80 4 client.15651 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-72cf81f5-1-mds.h.json"}] 2024-05-21T03:28:59.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.750+0000 7f3e6c1b6b80 1 -- 172.21.15.27:0/2084621806 --> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-72cf81f5-1-mds.h.json"}) -- 0x5594b9360cc0 con 0x5594b9485110 2024-05-21T03:28:59.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.750+0000 7f3e5affd700 1 --2- 172.21.15.27:0/2084621806 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x5594b9485110 0x5594b9487530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:28:59.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.750+0000 7f3e5affd700 1 --2- 172.21.15.27:0/2084621806 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x5594b9485110 0x5594b9487530 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1b20e1752ae0a9f2 server_cookie=c37dd9b068c955e3 in_seq=0 out_seq=0 2024-05-21T03:28:59.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.750+0000 7f3e3affd700 10 client.15651 ms_handle_connect on v2:172.21.15.39:6836/3673846852 2024-05-21T03:28:59.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.761+0000 7f3e3affd700 1 -- 172.21.15.27:0/2084621806 <== mds.3 v2:172.21.15.39:6836/3673846852 1 ==== command_reply(tid 0: 0 ) ==== 8+0+58 (crc 0 0 0) 0x5594b9360cc0 con 0x5594b9485110 2024-05-21T03:28:59.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.761+0000 7f3e3affd700 10 client.15651 handle_command_reply: tid=0 2024-05-21T03:28:59.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.761+0000 7f3e6c1b6b80 1 client.15651 shutdown 2024-05-21T03:28:59.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.761+0000 7f3e6c1b6b80 1 -- 172.21.15.27:0/2084621806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594b9405290 msgr2=0x5594b940c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:59.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.761+0000 7f3e6c1b6b80 1 --2- 172.21.15.27:0/2084621806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594b9405290 0x5594b940c690 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3e5400acd0 tx=0x7f3e5400ad00 comp rx=0 tx=0).stop 2024-05-21T03:28:59.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.762+0000 7f3e6c1b6b80 1 -- 172.21.15.27:0/2084621806 shutdown_connections 2024-05-21T03:28:59.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.762+0000 7f3e6c1b6b80 1 --2- 172.21.15.27:0/2084621806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5594b9405290 0x5594b940c690 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.762+0000 7f3e6c1b6b80 1 --2- 172.21.15.27:0/2084621806 >> [v2:172.21.15.39:6836/3673846852,v1:172.21.15.39:6837/3673846852] conn(0x5594b9485110 0x5594b9487530 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.762+0000 7f3e6c1b6b80 1 --2- 172.21.15.27:0/2084621806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5594b93ff8c0 0x5594b94118c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.762+0000 7f3e6c1b6b80 1 --2- 172.21.15.27:0/2084621806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5594b9400280 0x5594b9411e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.762+0000 7f3e3affd700 0 client.15651 ms_handle_reset on v2:172.21.15.39:6836/3673846852 2024-05-21T03:28:59.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.762+0000 7f3e6c1b6b80 1 -- 172.21.15.27:0/2084621806 >> 172.21.15.27:0/2084621806 conn(0x5594b932d160 msgr2=0x5594b9402d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:28:59.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.762+0000 7f3e6c1b6b80 1 -- 172.21.15.27:0/2084621806 shutdown_connections 2024-05-21T03:28:59.171 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:28:59.171 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/var/run/ceph/ops-72cf81f5-1-mds.h.json" 2024-05-21T03:28:59.171 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:28:59.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.762+0000 7f3e6c1b6b80 1 -- 172.21.15.27:0/2084621806 wait complete. 2024-05-21T03:28:59.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.762+0000 7f3e6c1b6b80 20 client.15651 trim_cache size 0 max 16384 2024-05-21T03:28:59.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.762+0000 7f3e61920700 1 -- 172.21.15.27:0/1874230507 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f3e480b29f0 msgr2=0x7f3e480b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:59.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.762+0000 7f3e61920700 1 --2- 172.21.15.27:0/1874230507 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f3e480b29f0 0x7f3e480b4eb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c009a10 tx=0x7f3e4c009190 comp rx=0 tx=0).stop 2024-05-21T03:28:59.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.762+0000 7f3e61920700 1 -- 172.21.15.27:0/1874230507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e5c1041a0 msgr2=0x7f3e5c075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:28:59.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.762+0000 7f3e61920700 1 --2- 172.21.15.27:0/1874230507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e5c1041a0 0x7f3e5c075d80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400b580 tx=0x7f3e440047b0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.762+0000 7f3e61920700 1 -- 172.21.15.27:0/1874230507 shutdown_connections 2024-05-21T03:28:59.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.762+0000 7f3e61920700 1 --2- 172.21.15.27:0/1874230507 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f3e480b29f0 0x7f3e480b4eb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.762+0000 7f3e61920700 1 --2- 172.21.15.27:0/1874230507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e5c1088f0 0x7f3e5c0762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.762+0000 7f3e61920700 1 --2- 172.21.15.27:0/1874230507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e5c1037f0 0x7f3e5c075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.762+0000 7f3e61920700 1 --2- 172.21.15.27:0/1874230507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e5c1041a0 0x7f3e5c075d80 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:28:59.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.762+0000 7f3e61920700 1 -- 172.21.15.27:0/1874230507 >> 172.21.15.27:0/1874230507 conn(0x7f3e5c07a6a0 msgr2=0x7f3e5c109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:28:59.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.762+0000 7f3e61920700 1 -- 172.21.15.27:0/1874230507 shutdown_connections 2024-05-21T03:28:59.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:28:58.762+0000 7f3e61920700 1 -- 172.21.15.27:0/1874230507 wait complete. 2024-05-21T03:28:59.821 DEBUG:tasks.quiescer.fs.[cephfs]:Dumping ops on rank 2 (c) to a remote file /var/run/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ops-72cf81f5-1-mds.c.json 2024-05-21T03:28:59.821 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 ops --flags=locks --path=/var/run/ceph/ops-72cf81f5-1-mds.c.json 2024-05-21T03:29:00.080 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:29:00.851 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:00 smithi027 ceph-mon[32179]: pgmap v152: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 36 MiB/s wr, 4.88k op/s 2024-05-21T03:29:00.851 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:00 smithi027 ceph-mon[32179]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:29:01.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:00 smithi039 ceph-mon[36429]: pgmap v152: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 36 MiB/s wr, 4.88k op/s 2024-05-21T03:29:01.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:00 smithi039 ceph-mon[36429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:29:01.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:00 smithi165 ceph-mon[36269]: pgmap v152: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 36 MiB/s wr, 4.88k op/s 2024-05-21T03:29:01.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:00 smithi165 ceph-mon[36269]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:29:01.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.680+0000 7ff818eb5700 1 -- 172.21.15.27:0/366274296 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8141084f0 msgr2=0x7ff8141088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:01.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.680+0000 7ff818eb5700 1 --2- 172.21.15.27:0/366274296 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8141084f0 0x7ff8141088d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff804009990 tx=0x7ff804009ca0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.680+0000 7ff818eb5700 1 -- 172.21.15.27:0/366274296 shutdown_connections 2024-05-21T03:29:01.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.680+0000 7ff818eb5700 1 --2- 172.21.15.27:0/366274296 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff814102e20 0x7ff81410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.680+0000 7ff818eb5700 1 --2- 172.21.15.27:0/366274296 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8141084f0 0x7ff8141088d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.680+0000 7ff818eb5700 1 --2- 172.21.15.27:0/366274296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff814102460 0x7ff8141028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.680+0000 7ff818eb5700 1 -- 172.21.15.27:0/366274296 >> 172.21.15.27:0/366274296 conn(0x7ff8140794e0 msgr2=0x7ff8140798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:01.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.680+0000 7ff818eb5700 1 -- 172.21.15.27:0/366274296 shutdown_connections 2024-05-21T03:29:01.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.680+0000 7ff818eb5700 1 -- 172.21.15.27:0/366274296 wait complete. 2024-05-21T03:29:01.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.681+0000 7ff818eb5700 1 Processor -- start 2024-05-21T03:29:01.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.681+0000 7ff818eb5700 1 -- start start 2024-05-21T03:29:01.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.681+0000 7ff818eb5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff814102460 0x7ff81419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:01.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.681+0000 7ff818eb5700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff814102e20 0x7ff81419af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:01.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.681+0000 7ff818eb5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8141084f0 0x7ff8141a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:01.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.681+0000 7ff818eb5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff814112620 con 0x7ff814102460 2024-05-21T03:29:01.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.681+0000 7ff818eb5700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff8141124a0 con 0x7ff8141084f0 2024-05-21T03:29:01.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.681+0000 7ff818eb5700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff8141127a0 con 0x7ff814102e20 2024-05-21T03:29:01.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.681+0000 7ff81259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff814102460 0x7ff81419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:01.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.681+0000 7ff81259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff814102460 0x7ff81419aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41670/0 (socket says 172.21.15.27:41670) 2024-05-21T03:29:01.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.681+0000 7ff81259c700 1 -- 172.21.15.27:0/3619011716 learned_addr learned my addr 172.21.15.27:0/3619011716 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:01.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.682+0000 7ff811d9b700 1 --2- 172.21.15.27:0/3619011716 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff814102e20 0x7ff81419af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:01.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.682+0000 7ff812d9d700 1 --2- 172.21.15.27:0/3619011716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8141084f0 0x7ff8141a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:01.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.682+0000 7ff81259c700 1 -- 172.21.15.27:0/3619011716 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff814102e20 msgr2=0x7ff81419af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:01.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.682+0000 7ff81259c700 1 --2- 172.21.15.27:0/3619011716 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff814102e20 0x7ff81419af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.682+0000 7ff81259c700 1 -- 172.21.15.27:0/3619011716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8141084f0 msgr2=0x7ff8141a5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:01.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.682+0000 7ff81259c700 1 --2- 172.21.15.27:0/3619011716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8141084f0 0x7ff8141a5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.682+0000 7ff81259c700 1 -- 172.21.15.27:0/3619011716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8141a5fe0 con 0x7ff814102460 2024-05-21T03:29:01.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.682+0000 7ff81259c700 1 --2- 172.21.15.27:0/3619011716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff814102460 0x7ff81419aa20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff804000c00 tx=0x7ff8040047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:01.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.682+0000 7ff8037fe700 1 -- 172.21.15.27:0/3619011716 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff804027070 con 0x7ff814102460 2024-05-21T03:29:01.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.682+0000 7ff8037fe700 1 -- 172.21.15.27:0/3619011716 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff804003f20 con 0x7ff814102460 2024-05-21T03:29:01.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.682+0000 7ff818eb5700 1 -- 172.21.15.27:0/3619011716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8141a61b0 con 0x7ff814102460 2024-05-21T03:29:01.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.682+0000 7ff8037fe700 1 -- 172.21.15.27:0/3619011716 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff804022cf0 con 0x7ff814102460 2024-05-21T03:29:01.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.682+0000 7ff818eb5700 1 -- 172.21.15.27:0/3619011716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8141a6600 con 0x7ff814102460 2024-05-21T03:29:01.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.685+0000 7ff8037fe700 1 -- 172.21.15.27:0/3619011716 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7ff80402a070 con 0x7ff814102460 2024-05-21T03:29:01.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.685+0000 7ff8037fe700 1 --2- 172.21.15.27:0/3619011716 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7ff7fc0b2a70 0x7ff7fc0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:01.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.686+0000 7ff811d9b700 1 --2- 172.21.15.27:0/3619011716 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7ff7fc0b2a70 0x7ff7fc0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:01.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.686+0000 7ff8037fe700 1 -- 172.21.15.27:0/3619011716 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7ff8040e3300 con 0x7ff814102460 2024-05-21T03:29:01.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.686+0000 7ff811d9b700 1 --2- 172.21.15.27:0/3619011716 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7ff7fc0b2a70 0x7ff7fc0b4f30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff814079ae0 tx=0x7ff80800a380 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:01.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.686+0000 7ff82374bb80 1 Processor -- start 2024-05-21T03:29:01.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.686+0000 7ff82374bb80 1 -- start start 2024-05-21T03:29:01.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.687+0000 7ff82374bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b50f263b10 0x55b50f263ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:01.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.687+0000 7ff82374bb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b50f25efe0 0x55b50f25f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:01.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.687+0000 7ff82374bb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b50f25f9a0 0x55b50f26ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:01.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.687+0000 7ff82374bb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b50f1b8e50 con 0x55b50f25f9a0 2024-05-21T03:29:01.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.687+0000 7ff82374bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b50f1a89f0 con 0x55b50f263b10 2024-05-21T03:29:01.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.687+0000 7ff82374bb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b50f1aca30 con 0x55b50f25efe0 2024-05-21T03:29:01.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.687+0000 7ff812d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b50f25f9a0 0x55b50f26ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:01.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.687+0000 7ff812d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b50f25f9a0 0x55b50f26ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41678/0 (socket says 172.21.15.27:41678) 2024-05-21T03:29:01.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.687+0000 7ff812d9d700 1 -- 172.21.15.27:0/2698344466 learned_addr learned my addr 172.21.15.27:0/2698344466 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:01.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.687+0000 7ff81259c700 1 --2- 172.21.15.27:0/2698344466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b50f263b10 0x55b50f263ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:01.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.687+0000 7ff811d9b700 1 --2- 172.21.15.27:0/2698344466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b50f25efe0 0x55b50f25f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:01.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.687+0000 7ff812d9d700 1 -- 172.21.15.27:0/2698344466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b50f25efe0 msgr2=0x55b50f25f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:01.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.687+0000 7ff812d9d700 1 --2- 172.21.15.27:0/2698344466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b50f25efe0 0x55b50f25f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.687+0000 7ff812d9d700 1 -- 172.21.15.27:0/2698344466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b50f263b10 msgr2=0x55b50f263ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:01.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.687+0000 7ff812d9d700 1 --2- 172.21.15.27:0/2698344466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b50f263b10 0x55b50f263ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.687+0000 7ff812d9d700 1 -- 172.21.15.27:0/2698344466 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b50f152090 con 0x55b50f25f9a0 2024-05-21T03:29:01.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.687+0000 7ff811d9b700 1 --2- 172.21.15.27:0/2698344466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b50f25efe0 0x55b50f25f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:29:01.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.687+0000 7ff812d9d700 1 --2- 172.21.15.27:0/2698344466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b50f25f9a0 0x55b50f26ba30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff80c009d20 tx=0x7ff80c0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75264086f0157c6f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:01.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.687+0000 7ff800ff9700 1 -- 172.21.15.27:0/2698344466 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80c004bb0 con 0x55b50f25f9a0 2024-05-21T03:29:01.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.688+0000 7ff800ff9700 1 -- 172.21.15.27:0/2698344466 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff80c004d50 con 0x55b50f25f9a0 2024-05-21T03:29:01.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.688+0000 7ff800ff9700 1 -- 172.21.15.27:0/2698344466 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80c014670 con 0x55b50f25f9a0 2024-05-21T03:29:01.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.688+0000 7ff82374bb80 1 -- 172.21.15.27:0/2698344466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b50f25f9a0 msgr2=0x55b50f26ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:01.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.688+0000 7ff82374bb80 1 --2- 172.21.15.27:0/2698344466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b50f25f9a0 0x55b50f26ba30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff80c009d20 tx=0x7ff80c0077e0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.688+0000 7ff82374bb80 1 -- 172.21.15.27:0/2698344466 shutdown_connections 2024-05-21T03:29:01.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.688+0000 7ff82374bb80 1 --2- 172.21.15.27:0/2698344466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b50f25f9a0 0x55b50f26ba30 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.688+0000 7ff82374bb80 1 --2- 172.21.15.27:0/2698344466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b50f263b10 0x55b50f263ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.688+0000 7ff82374bb80 1 --2- 172.21.15.27:0/2698344466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b50f25efe0 0x55b50f25f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.688+0000 7ff82374bb80 1 -- 172.21.15.27:0/2698344466 >> 172.21.15.27:0/2698344466 conn(0x55b50f186320 msgr2=0x55b50f2558c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:01.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.688+0000 7ff82374bb80 1 -- 172.21.15.27:0/2698344466 shutdown_connections 2024-05-21T03:29:01.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.688+0000 7ff82374bb80 1 -- 172.21.15.27:0/2698344466 wait complete. 2024-05-21T03:29:01.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.689+0000 7ff82374bb80 1 -- start start 2024-05-21T03:29:01.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.689+0000 7ff82374bb80 1 Processor -- start 2024-05-21T03:29:01.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.689+0000 7ff82374bb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b50f25efe0 0x55b50f26b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:01.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.689+0000 7ff82374bb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b50f25f9a0 0x55b50f2689a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:01.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.689+0000 7ff82374bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b50f263b10 0x55b50f268ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:01.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.689+0000 7ff82374bb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b50f0af080 con 0x55b50f25efe0 2024-05-21T03:29:01.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.689+0000 7ff82374bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b50f110e80 con 0x55b50f263b10 2024-05-21T03:29:01.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.689+0000 7ff82374bb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b50f26c090 con 0x55b50f25f9a0 2024-05-21T03:29:01.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.689+0000 7ff82374bb80 10 client.0 _ll_register_callbacks cb 0x55b50f143a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:29:01.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.689+0000 7ff81259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b50f25efe0 0x55b50f26b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:01.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.689+0000 7ff81259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b50f25efe0 0x55b50f26b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41680/0 (socket says 172.21.15.27:41680) 2024-05-21T03:29:01.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.689+0000 7ff81259c700 1 -- 172.21.15.27:0/624993013 learned_addr learned my addr 172.21.15.27:0/624993013 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:01.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.689+0000 7ff811d9b700 1 --2- 172.21.15.27:0/624993013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b50f25f9a0 0x55b50f2689a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:01.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.689+0000 7ff812d9d700 1 --2- 172.21.15.27:0/624993013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b50f263b10 0x55b50f268ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:01.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.690+0000 7ff81259c700 1 -- 172.21.15.27:0/624993013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b50f25f9a0 msgr2=0x55b50f2689a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:01.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.690+0000 7ff81259c700 1 --2- 172.21.15.27:0/624993013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b50f25f9a0 0x55b50f2689a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.690+0000 7ff81259c700 1 -- 172.21.15.27:0/624993013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b50f263b10 msgr2=0x55b50f268ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:01.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.690+0000 7ff81259c700 1 --2- 172.21.15.27:0/624993013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b50f263b10 0x55b50f268ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.690+0000 7ff81259c700 1 -- 172.21.15.27:0/624993013 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b50f0f89b0 con 0x55b50f25efe0 2024-05-21T03:29:01.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.690+0000 7ff81259c700 1 --2- 172.21.15.27:0/624993013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b50f25efe0 0x55b50f26b6a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff8040aa370 tx=0x7ff8040aabb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3775a3953aefaef server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:01.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.690+0000 7ff7f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:29:01.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.690+0000 7ff7f9ffb700 1 -- 172.21.15.27:0/624993013 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8040aad70 con 0x55b50f25efe0 2024-05-21T03:29:01.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.690+0000 7ff7f9ffb700 1 -- 172.21.15.27:0/624993013 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7ff8040afbb0 con 0x55b50f25efe0 2024-05-21T03:29:01.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.690+0000 7ff82374bb80 1 -- 172.21.15.27:0/624993013 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff80c004d50 con 0x55b50f25efe0 2024-05-21T03:29:01.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.690+0000 7ff7f9ffb700 1 -- 172.21.15.27:0/624993013 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff8040aa670 con 0x55b50f25efe0 2024-05-21T03:29:01.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.690+0000 7ff82374bb80 10 client.15663 fetch_fsmap learned FSMap version 34 2024-05-21T03:29:01.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.690+0000 7ff82374bb80 1 -- 172.21.15.27:0/624993013 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b50f162840 con 0x55b50f25efe0 2024-05-21T03:29:01.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.690+0000 7ff7f9ffb700 1 -- 172.21.15.27:0/624993013 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff80c004d50 con 0x55b50f25efe0 2024-05-21T03:29:01.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.691+0000 7ff7f9ffb700 1 -- 172.21.15.27:0/624993013 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7ff804090cb0 con 0x55b50f25efe0 2024-05-21T03:29:01.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.691+0000 7ff82374bb80 10 client.15663 fetch_fsmap finished waiting for FSMap version 34 2024-05-21T03:29:01.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.691+0000 7ff82374bb80 10 client.15663 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T03:29:01.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.691+0000 7ff82374bb80 20 client.15663 populate_metadata read hostname 'smithi027' 2024-05-21T03:29:01.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.691+0000 7ff82374bb80 1 --2- 172.21.15.27:0/624993013 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x55b50f278bb0 0x55b50f27afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:01.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.691+0000 7ff82374bb80 4 client.15663 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:29:01.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.691+0000 7ff82374bb80 1 -- 172.21.15.27:0/624993013 --> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b50f1bada0 con 0x55b50f278bb0 2024-05-21T03:29:01.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.691+0000 7ff811d9b700 1 --2- 172.21.15.27:0/624993013 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x55b50f278bb0 0x55b50f27afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:01.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.692+0000 7ff811d9b700 1 --2- 172.21.15.27:0/624993013 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x55b50f278bb0 0x55b50f27afa0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fbbdc5a461022369 server_cookie=c6b2f42882aea983 in_seq=0 out_seq=0 2024-05-21T03:29:01.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.692+0000 7ff7f9ffb700 10 client.15663 ms_handle_connect on v2:172.21.15.165:6836/691356139 2024-05-21T03:29:01.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.692+0000 7ff7f9ffb700 1 -- 172.21.15.27:0/624993013 <== mds.2 v2:172.21.15.165:6836/691356139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b50f1bada0 con 0x55b50f278bb0 2024-05-21T03:29:01.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.692+0000 7ff7f9ffb700 10 client.15663 handle_command_reply: tid=0 2024-05-21T03:29:01.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.692+0000 7ff82374bb80 1 client.15663 shutdown 2024-05-21T03:29:01.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.692+0000 7ff82374bb80 1 -- 172.21.15.27:0/624993013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b50f25efe0 msgr2=0x55b50f26b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:01.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.692+0000 7ff82374bb80 1 --2- 172.21.15.27:0/624993013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b50f25efe0 0x55b50f26b6a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff8040aa370 tx=0x7ff8040aabb0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.693+0000 7ff82374bb80 1 -- 172.21.15.27:0/624993013 shutdown_connections 2024-05-21T03:29:01.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.693+0000 7ff82374bb80 1 --2- 172.21.15.27:0/624993013 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x55b50f278bb0 0x55b50f27afa0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.693+0000 7ff82374bb80 1 --2- 172.21.15.27:0/624993013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b50f263b10 0x55b50f268ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.693+0000 7ff82374bb80 1 --2- 172.21.15.27:0/624993013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b50f25efe0 0x55b50f26b6a0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.693+0000 7ff82374bb80 1 --2- 172.21.15.27:0/624993013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b50f25f9a0 0x55b50f2689a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.693+0000 7ff82374bb80 1 -- 172.21.15.27:0/624993013 >> 172.21.15.27:0/624993013 conn(0x55b50f186320 msgr2=0x55b50f259bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:01.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.693+0000 7ff7f9ffb700 0 client.15663 ms_handle_reset on v2:172.21.15.165:6836/691356139 2024-05-21T03:29:01.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.693+0000 7ff82374bb80 1 -- 172.21.15.27:0/624993013 shutdown_connections 2024-05-21T03:29:01.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.693+0000 7ff82374bb80 1 -- 172.21.15.27:0/624993013 wait complete. 2024-05-21T03:29:01.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.693+0000 7ff82374bb80 20 client.15663 trim_cache size 0 max 16384 2024-05-21T03:29:01.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.712+0000 7ff82374bb80 1 Processor -- start 2024-05-21T03:29:01.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.712+0000 7ff82374bb80 1 -- start start 2024-05-21T03:29:01.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.712+0000 7ff82374bb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b50f25efe0 0x55b50f25a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:01.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.712+0000 7ff82374bb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b50f25f9a0 0x55b50f24fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:01.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.712+0000 7ff82374bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b50f263b10 0x55b50f268960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:01.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.712+0000 7ff82374bb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff8141aba80 con 0x55b50f25efe0 2024-05-21T03:29:01.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.712+0000 7ff82374bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff8141ad680 con 0x55b50f263b10 2024-05-21T03:29:01.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.712+0000 7ff82374bb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff8141ac150 con 0x55b50f25f9a0 2024-05-21T03:29:01.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.712+0000 7ff81259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b50f25efe0 0x55b50f25a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:01.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.712+0000 7ff81259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b50f25efe0 0x55b50f25a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41686/0 (socket says 172.21.15.27:41686) 2024-05-21T03:29:01.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.712+0000 7ff81259c700 1 -- 172.21.15.27:0/3057753998 learned_addr learned my addr 172.21.15.27:0/3057753998 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:01.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.712+0000 7ff812d9d700 1 --2- 172.21.15.27:0/3057753998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b50f263b10 0x55b50f268960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:01.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.712+0000 7ff81259c700 1 -- 172.21.15.27:0/3057753998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b50f25f9a0 msgr2=0x55b50f24fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:01.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.712+0000 7ff811d9b700 1 --2- 172.21.15.27:0/3057753998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b50f25f9a0 0x55b50f24fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:01.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.712+0000 7ff81259c700 1 --2- 172.21.15.27:0/3057753998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b50f25f9a0 0x55b50f24fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.712+0000 7ff81259c700 1 -- 172.21.15.27:0/3057753998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b50f263b10 msgr2=0x55b50f268960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:01.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.712+0000 7ff81259c700 1 --2- 172.21.15.27:0/3057753998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b50f263b10 0x55b50f268960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.712+0000 7ff81259c700 1 -- 172.21.15.27:0/3057753998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b50f260020 con 0x55b50f25efe0 2024-05-21T03:29:01.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.712+0000 7ff81259c700 1 --2- 172.21.15.27:0/3057753998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b50f25efe0 0x55b50f25a450 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff8040a66f0 tx=0x7ff804003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5d03ed55e9812d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:01.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.713+0000 7ff7fbfff700 1 -- 172.21.15.27:0/3057753998 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8040aad70 con 0x55b50f25efe0 2024-05-21T03:29:01.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.713+0000 7ff7fbfff700 1 -- 172.21.15.27:0/3057753998 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8040b3440 con 0x55b50f25efe0 2024-05-21T03:29:01.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.713+0000 7ff7fbfff700 1 -- 172.21.15.27:0/3057753998 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8040a9950 con 0x55b50f25efe0 2024-05-21T03:29:01.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.713+0000 7ff82374bb80 1 -- 172.21.15.27:0/3057753998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b50f25efe0 msgr2=0x55b50f25a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:01.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.713+0000 7ff82374bb80 1 --2- 172.21.15.27:0/3057753998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b50f25efe0 0x55b50f25a450 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff8040a66f0 tx=0x7ff804003780 comp rx=0 tx=0).stop 2024-05-21T03:29:01.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.713+0000 7ff82374bb80 1 -- 172.21.15.27:0/3057753998 shutdown_connections 2024-05-21T03:29:01.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.713+0000 7ff82374bb80 1 --2- 172.21.15.27:0/3057753998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b50f263b10 0x55b50f268960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.713+0000 7ff82374bb80 1 --2- 172.21.15.27:0/3057753998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b50f25efe0 0x55b50f25a450 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.713+0000 7ff82374bb80 1 --2- 172.21.15.27:0/3057753998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b50f25f9a0 0x55b50f24fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.713+0000 7ff82374bb80 1 -- 172.21.15.27:0/3057753998 >> 172.21.15.27:0/3057753998 conn(0x55b50f186320 msgr2=0x55b50f268250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:01.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.713+0000 7ff82374bb80 1 -- 172.21.15.27:0/3057753998 shutdown_connections 2024-05-21T03:29:01.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.713+0000 7ff82374bb80 1 -- 172.21.15.27:0/3057753998 wait complete. 2024-05-21T03:29:01.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.714+0000 7ff82374bb80 1 -- start start 2024-05-21T03:29:01.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.714+0000 7ff82374bb80 1 Processor -- start 2024-05-21T03:29:01.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.714+0000 7ff82374bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b50f25efe0 0x55b50f271260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:01.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.714+0000 7ff82374bb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b50f25f9a0 0x55b50f2717a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:01.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.714+0000 7ff82374bb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b50f263b10 0x55b50f26f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:01.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.714+0000 7ff82374bb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff8141acef0 con 0x55b50f25f9a0 2024-05-21T03:29:01.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.714+0000 7ff82374bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff8141ac820 con 0x55b50f25efe0 2024-05-21T03:29:01.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.714+0000 7ff82374bb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff8141a9a20 con 0x55b50f263b10 2024-05-21T03:29:01.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.714+0000 7ff82374bb80 10 client.0 _ll_register_callbacks cb 0x55b50f143a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:29:01.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.714+0000 7ff811d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b50f25f9a0 0x55b50f2717a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:01.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.714+0000 7ff811d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b50f25f9a0 0x55b50f2717a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41690/0 (socket says 172.21.15.27:41690) 2024-05-21T03:29:01.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.714+0000 7ff811d9b700 1 -- 172.21.15.27:0/1592862255 learned_addr learned my addr 172.21.15.27:0/1592862255 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:01.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.714+0000 7ff81259c700 1 --2- 172.21.15.27:0/1592862255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b50f25efe0 0x55b50f271260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:01.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.714+0000 7ff812d9d700 1 --2- 172.21.15.27:0/1592862255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b50f263b10 0x55b50f26f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:01.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.715+0000 7ff811d9b700 1 -- 172.21.15.27:0/1592862255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b50f263b10 msgr2=0x55b50f26f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:01.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.715+0000 7ff811d9b700 1 --2- 172.21.15.27:0/1592862255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b50f263b10 0x55b50f26f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.715+0000 7ff811d9b700 1 -- 172.21.15.27:0/1592862255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b50f25efe0 msgr2=0x55b50f271260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:01.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.715+0000 7ff811d9b700 1 --2- 172.21.15.27:0/1592862255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b50f25efe0 0x55b50f271260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.715+0000 7ff811d9b700 1 -- 172.21.15.27:0/1592862255 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b50f27b620 con 0x55b50f25f9a0 2024-05-21T03:29:01.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.715+0000 7ff811d9b700 1 --2- 172.21.15.27:0/1592862255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b50f25f9a0 0x55b50f2717a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x55b50f269f70 tx=0x7ff8080047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a350ea399d5387a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:01.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.715+0000 7ff7f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:29:01.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.715+0000 7ff7f9ffb700 1 -- 172.21.15.27:0/1592862255 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80802c060 con 0x55b50f25f9a0 2024-05-21T03:29:01.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.715+0000 7ff7f9ffb700 1 -- 172.21.15.27:0/1592862255 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7ff808003c30 con 0x55b50f25f9a0 2024-05-21T03:29:01.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.715+0000 7ff82374bb80 1 -- 172.21.15.27:0/1592862255 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8040b3440 con 0x55b50f25f9a0 2024-05-21T03:29:01.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.715+0000 7ff7f9ffb700 1 -- 172.21.15.27:0/1592862255 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff808016070 con 0x55b50f25f9a0 2024-05-21T03:29:01.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.715+0000 7ff82374bb80 10 client.15669 fetch_fsmap learned FSMap version 34 2024-05-21T03:29:01.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.715+0000 7ff82374bb80 1 -- 172.21.15.27:0/1592862255 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b50f2e40a0 con 0x55b50f25f9a0 2024-05-21T03:29:01.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.715+0000 7ff7f9ffb700 1 -- 172.21.15.27:0/1592862255 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8040b3440 con 0x55b50f25f9a0 2024-05-21T03:29:01.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.715+0000 7ff7f9ffb700 1 -- 172.21.15.27:0/1592862255 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7ff808019040 con 0x55b50f25f9a0 2024-05-21T03:29:01.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.716+0000 7ff82374bb80 10 client.15669 fetch_fsmap finished waiting for FSMap version 34 2024-05-21T03:29:01.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.716+0000 7ff82374bb80 10 client.15669 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T03:29:01.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.716+0000 7ff82374bb80 20 client.15669 populate_metadata read hostname 'smithi027' 2024-05-21T03:29:01.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.716+0000 7ff82374bb80 1 --2- 172.21.15.27:0/1592862255 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x55b50f2e4880 0x55b50f2e6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:01.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.716+0000 7ff82374bb80 4 client.15669 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-72cf81f5-1-mds.c.json"}] 2024-05-21T03:29:01.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.716+0000 7ff82374bb80 1 -- 172.21.15.27:0/1592862255 --> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-72cf81f5-1-mds.c.json"}) -- 0x55b50f1b4580 con 0x55b50f2e4880 2024-05-21T03:29:01.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.716+0000 7ff81259c700 1 --2- 172.21.15.27:0/1592862255 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x55b50f2e4880 0x55b50f2e6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:01.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.716+0000 7ff81259c700 1 --2- 172.21.15.27:0/1592862255 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x55b50f2e4880 0x55b50f2e6c70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d2871d6e7f2175e1 server_cookie=dc6963d8249218a9 in_seq=0 out_seq=0 2024-05-21T03:29:01.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.716+0000 7ff7f9ffb700 10 client.15669 ms_handle_connect on v2:172.21.15.165:6836/691356139 2024-05-21T03:29:01.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.729+0000 7ff7f9ffb700 1 -- 172.21.15.27:0/1592862255 <== mds.2 v2:172.21.15.165:6836/691356139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+58 (crc 0 0 0) 0x55b50f1b4580 con 0x55b50f2e4880 2024-05-21T03:29:01.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.729+0000 7ff7f9ffb700 10 client.15669 handle_command_reply: tid=0 2024-05-21T03:29:01.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.729+0000 7ff82374bb80 1 client.15669 shutdown 2024-05-21T03:29:01.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.729+0000 7ff82374bb80 1 -- 172.21.15.27:0/1592862255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b50f25f9a0 msgr2=0x55b50f2717a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:01.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.729+0000 7ff82374bb80 1 --2- 172.21.15.27:0/1592862255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b50f25f9a0 0x55b50f2717a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x55b50f269f70 tx=0x7ff8080047c0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.729+0000 7ff82374bb80 1 -- 172.21.15.27:0/1592862255 shutdown_connections 2024-05-21T03:29:01.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.729+0000 7ff82374bb80 1 --2- 172.21.15.27:0/1592862255 >> [v2:172.21.15.165:6836/691356139,v1:172.21.15.165:6837/691356139] conn(0x55b50f2e4880 0x55b50f2e6c70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.729+0000 7ff82374bb80 1 --2- 172.21.15.27:0/1592862255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b50f263b10 0x55b50f26f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.729+0000 7ff82374bb80 1 --2- 172.21.15.27:0/1592862255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b50f25efe0 0x55b50f271260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.729+0000 7ff82374bb80 1 --2- 172.21.15.27:0/1592862255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b50f25f9a0 0x55b50f2717a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.729+0000 7ff7f9ffb700 0 client.15669 ms_handle_reset on v2:172.21.15.165:6836/691356139 2024-05-21T03:29:01.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.729+0000 7ff82374bb80 1 -- 172.21.15.27:0/1592862255 >> 172.21.15.27:0/1592862255 conn(0x55b50f186320 msgr2=0x55b50f26aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:01.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.729+0000 7ff82374bb80 1 -- 172.21.15.27:0/1592862255 shutdown_connections 2024-05-21T03:29:01.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.729+0000 7ff82374bb80 1 -- 172.21.15.27:0/1592862255 wait complete. 2024-05-21T03:29:01.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.729+0000 7ff82374bb80 20 client.15669 trim_cache size 0 max 16384 2024-05-21T03:29:01.731 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:29:01.731 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/var/run/ceph/ops-72cf81f5-1-mds.c.json" 2024-05-21T03:29:01.731 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:29:01.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.730+0000 7ff818eb5700 1 -- 172.21.15.27:0/3619011716 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7ff7fc0b2a70 msgr2=0x7ff7fc0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:01.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.730+0000 7ff818eb5700 1 --2- 172.21.15.27:0/3619011716 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7ff7fc0b2a70 0x7ff7fc0b4f30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff814079ae0 tx=0x7ff80800a380 comp rx=0 tx=0).stop 2024-05-21T03:29:01.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.730+0000 7ff818eb5700 1 -- 172.21.15.27:0/3619011716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff814102460 msgr2=0x7ff81419aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:01.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.730+0000 7ff818eb5700 1 --2- 172.21.15.27:0/3619011716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff814102460 0x7ff81419aa20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff804000c00 tx=0x7ff8040047b0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.730+0000 7ff818eb5700 1 -- 172.21.15.27:0/3619011716 shutdown_connections 2024-05-21T03:29:01.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.730+0000 7ff818eb5700 1 --2- 172.21.15.27:0/3619011716 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7ff7fc0b2a70 0x7ff7fc0b4f30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.730+0000 7ff818eb5700 1 --2- 172.21.15.27:0/3619011716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8141084f0 0x7ff8141a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.730+0000 7ff818eb5700 1 --2- 172.21.15.27:0/3619011716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff814102460 0x7ff81419aa20 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.730+0000 7ff818eb5700 1 --2- 172.21.15.27:0/3619011716 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff814102e20 0x7ff81419af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:01.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.730+0000 7ff818eb5700 1 -- 172.21.15.27:0/3619011716 >> 172.21.15.27:0/3619011716 conn(0x7ff8140794e0 msgr2=0x7ff814105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:01.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.730+0000 7ff818eb5700 1 -- 172.21.15.27:0/3619011716 shutdown_connections 2024-05-21T03:29:01.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:01.730+0000 7ff818eb5700 1 -- 172.21.15.27:0/3619011716 wait complete. 2024-05-21T03:29:02.603 DEBUG:tasks.quiescer.fs.[cephfs]:Dumping ops on rank 1 (i) to a remote file /var/run/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ops-72cf81f5-1-mds.i.json 2024-05-21T03:29:02.603 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 ops --flags=locks --path=/var/run/ceph/ops-72cf81f5-1-mds.i.json 2024-05-21T03:29:02.862 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:29:02.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:02 smithi027 ceph-mon[32179]: pgmap v153: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 18 MiB/s wr, 2.38k op/s 2024-05-21T03:29:03.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:02 smithi039 ceph-mon[36429]: pgmap v153: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 18 MiB/s wr, 2.38k op/s 2024-05-21T03:29:03.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:02 smithi165 ceph-mon[36269]: pgmap v153: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 18 MiB/s wr, 2.38k op/s 2024-05-21T03:29:04.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.470+0000 7fbf40bd0700 1 -- 172.21.15.27:0/1891445858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf3c06c6d0 msgr2=0x7fbf3c110400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:04.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.470+0000 7fbf40bd0700 1 --2- 172.21.15.27:0/1891445858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf3c06c6d0 0x7fbf3c110400 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400b0d0 tx=0x7fbf3400b3e0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.470+0000 7fbf40bd0700 1 -- 172.21.15.27:0/1891445858 shutdown_connections 2024-05-21T03:29:04.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.470+0000 7fbf40bd0700 1 --2- 172.21.15.27:0/1891445858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf3c06c6d0 0x7fbf3c110400 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.470+0000 7fbf40bd0700 1 --2- 172.21.15.27:0/1891445858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf3c109630 0x7fbf3c109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.470+0000 7fbf40bd0700 1 --2- 172.21.15.27:0/1891445858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3c06bd10 0x7fbf3c06c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.470+0000 7fbf40bd0700 1 -- 172.21.15.27:0/1891445858 >> 172.21.15.27:0/1891445858 conn(0x7fbf3c0794e0 msgr2=0x7fbf3c0798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:04.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.470+0000 7fbf40bd0700 1 -- 172.21.15.27:0/1891445858 shutdown_connections 2024-05-21T03:29:04.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.470+0000 7fbf40bd0700 1 -- 172.21.15.27:0/1891445858 wait complete. 2024-05-21T03:29:04.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.471+0000 7fbf40bd0700 1 Processor -- start 2024-05-21T03:29:04.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.471+0000 7fbf40bd0700 1 -- start start 2024-05-21T03:29:04.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.471+0000 7fbf40bd0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3c06bd10 0x7fbf3c19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:04.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.471+0000 7fbf40bd0700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf3c06c6d0 0x7fbf3c19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:04.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.471+0000 7fbf40bd0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf3c109630 0x7fbf3c199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:04.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.471+0000 7fbf40bd0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c113740 con 0x7fbf3c109630 2024-05-21T03:29:04.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.471+0000 7fbf40bd0700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c1135c0 con 0x7fbf3c06bd10 2024-05-21T03:29:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.471+0000 7fbf40bd0700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c1138c0 con 0x7fbf3c06c6d0 2024-05-21T03:29:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.471+0000 7fbf3ad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf3c109630 0x7fbf3c199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.471+0000 7fbf3ad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf3c109630 0x7fbf3c199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44842/0 (socket says 172.21.15.27:44842) 2024-05-21T03:29:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.471+0000 7fbf3ad9d700 1 -- 172.21.15.27:0/3515199930 learned_addr learned my addr 172.21.15.27:0/3515199930 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.471+0000 7fbf39d9b700 1 --2- 172.21.15.27:0/3515199930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf3c06c6d0 0x7fbf3c19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.471+0000 7fbf3a59c700 1 --2- 172.21.15.27:0/3515199930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3c06bd10 0x7fbf3c19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.472+0000 7fbf3ad9d700 1 -- 172.21.15.27:0/3515199930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf3c06c6d0 msgr2=0x7fbf3c19fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.472+0000 7fbf3ad9d700 1 --2- 172.21.15.27:0/3515199930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf3c06c6d0 0x7fbf3c19fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.472+0000 7fbf3ad9d700 1 -- 172.21.15.27:0/3515199930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3c06bd10 msgr2=0x7fbf3c19f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.472+0000 7fbf3ad9d700 1 --2- 172.21.15.27:0/3515199930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3c06bd10 0x7fbf3c19f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.472+0000 7fbf3ad9d700 1 -- 172.21.15.27:0/3515199930 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf3c19a0e0 con 0x7fbf3c109630 2024-05-21T03:29:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.472+0000 7fbf3ad9d700 1 --2- 172.21.15.27:0/3515199930 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf3c109630 0x7fbf3c199980 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbf340095c0 tx=0x7fbf34007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.472+0000 7fbf2b7fe700 1 -- 172.21.15.27:0/3515199930 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3401f070 con 0x7fbf3c109630 2024-05-21T03:29:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.472+0000 7fbf2b7fe700 1 -- 172.21.15.27:0/3515199930 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf34007e20 con 0x7fbf3c109630 2024-05-21T03:29:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.472+0000 7fbf40bd0700 1 -- 172.21.15.27:0/3515199930 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf3c19a370 con 0x7fbf3c109630 2024-05-21T03:29:04.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.472+0000 7fbf2b7fe700 1 -- 172.21.15.27:0/3515199930 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3401b4a0 con 0x7fbf3c109630 2024-05-21T03:29:04.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.472+0000 7fbf40bd0700 1 -- 172.21.15.27:0/3515199930 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf3c1a6640 con 0x7fbf3c109630 2024-05-21T03:29:04.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.475+0000 7fbf2b7fe700 1 -- 172.21.15.27:0/3515199930 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7fbf34022070 con 0x7fbf3c109630 2024-05-21T03:29:04.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.475+0000 7fbf2b7fe700 1 --2- 172.21.15.27:0/3515199930 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fbf240b29f0 0x7fbf240b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:04.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.476+0000 7fbf3a59c700 1 --2- 172.21.15.27:0/3515199930 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fbf240b29f0 0x7fbf240b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:04.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.476+0000 7fbf2b7fe700 1 -- 172.21.15.27:0/3515199930 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fbf340e33d0 con 0x7fbf3c109630 2024-05-21T03:29:04.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.476+0000 7fbf4b466b80 1 Processor -- start 2024-05-21T03:29:04.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.476+0000 7fbf4b466b80 1 -- start start 2024-05-21T03:29:04.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.476+0000 7fbf3a59c700 1 --2- 172.21.15.27:0/3515199930 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fbf240b29f0 0x7fbf240b4eb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c100ba0 tx=0x7fbf2c00b410 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:04.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.476+0000 7fbf4b466b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560227501700 0x560227501ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:04.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.476+0000 7fbf4b466b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602274fcbd0 0x5602274fd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:04.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.476+0000 7fbf4b466b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602274fd590 0x560227509600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:04.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.476+0000 7fbf4b466b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56022745ae50 con 0x5602274fd590 2024-05-21T03:29:04.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.476+0000 7fbf4b466b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56022744a9f0 con 0x5602274fcbd0 2024-05-21T03:29:04.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.476+0000 7fbf4b466b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56022744ea30 con 0x560227501700 2024-05-21T03:29:04.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.476+0000 7fbf3ad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602274fd590 0x560227509600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:04.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.476+0000 7fbf3ad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602274fd590 0x560227509600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44846/0 (socket says 172.21.15.27:44846) 2024-05-21T03:29:04.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.476+0000 7fbf3ad9d700 1 -- 172.21.15.27:0/1454900976 learned_addr learned my addr 172.21.15.27:0/1454900976 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:04.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.476+0000 7fbf3a59c700 1 --2- 172.21.15.27:0/1454900976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560227501700 0x560227501ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:04.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.477+0000 7fbf39d9b700 1 --2- 172.21.15.27:0/1454900976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602274fcbd0 0x5602274fd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:04.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.477+0000 7fbf3ad9d700 1 -- 172.21.15.27:0/1454900976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560227501700 msgr2=0x560227501ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:04.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.477+0000 7fbf3ad9d700 1 --2- 172.21.15.27:0/1454900976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560227501700 0x560227501ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.477+0000 7fbf3ad9d700 1 -- 172.21.15.27:0/1454900976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602274fcbd0 msgr2=0x5602274fd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:04.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.477+0000 7fbf3ad9d700 1 --2- 172.21.15.27:0/1454900976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602274fcbd0 0x5602274fd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.477+0000 7fbf3ad9d700 1 -- 172.21.15.27:0/1454900976 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602273f4090 con 0x5602274fd590 2024-05-21T03:29:04.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.477+0000 7fbf3a59c700 1 --2- 172.21.15.27:0/1454900976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560227501700 0x560227501ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:29:04.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.477+0000 7fbf39d9b700 1 --2- 172.21.15.27:0/1454900976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602274fcbd0 0x5602274fd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:29:04.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.477+0000 7fbf3ad9d700 1 --2- 172.21.15.27:0/1454900976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602274fd590 0x560227509600 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbf340e1120 tx=0x7fbf340a9fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7efb68e2db5a451d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:04.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.477+0000 7fbf28ff9700 1 -- 172.21.15.27:0/1454900976 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf34029070 con 0x5602274fd590 2024-05-21T03:29:04.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.477+0000 7fbf28ff9700 1 -- 172.21.15.27:0/1454900976 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf340aa740 con 0x5602274fd590 2024-05-21T03:29:04.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.477+0000 7fbf28ff9700 1 -- 172.21.15.27:0/1454900976 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf340a4e60 con 0x5602274fd590 2024-05-21T03:29:04.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.478+0000 7fbf4b466b80 1 -- 172.21.15.27:0/1454900976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602274fd590 msgr2=0x560227509600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:04.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.478+0000 7fbf4b466b80 1 --2- 172.21.15.27:0/1454900976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602274fd590 0x560227509600 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbf340e1120 tx=0x7fbf340a9fb0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.478+0000 7fbf4b466b80 1 -- 172.21.15.27:0/1454900976 shutdown_connections 2024-05-21T03:29:04.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.478+0000 7fbf4b466b80 1 --2- 172.21.15.27:0/1454900976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602274fd590 0x560227509600 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.478+0000 7fbf4b466b80 1 --2- 172.21.15.27:0/1454900976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560227501700 0x560227501ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.478+0000 7fbf4b466b80 1 --2- 172.21.15.27:0/1454900976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602274fcbd0 0x5602274fd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.478+0000 7fbf4b466b80 1 -- 172.21.15.27:0/1454900976 >> 172.21.15.27:0/1454900976 conn(0x560227428320 msgr2=0x5602274f78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:04.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.478+0000 7fbf4b466b80 1 -- 172.21.15.27:0/1454900976 shutdown_connections 2024-05-21T03:29:04.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.478+0000 7fbf4b466b80 1 -- 172.21.15.27:0/1454900976 wait complete. 2024-05-21T03:29:04.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.478+0000 7fbf4b466b80 1 -- start start 2024-05-21T03:29:04.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.478+0000 7fbf4b466b80 1 Processor -- start 2024-05-21T03:29:04.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.479+0000 7fbf4b466b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602274fcbd0 0x560227509350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:04.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.479+0000 7fbf4b466b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602274fd590 0x560227506650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:04.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.479+0000 7fbf4b466b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560227501700 0x560227506b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:04.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.479+0000 7fbf4b466b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560227351080 con 0x560227501700 2024-05-21T03:29:04.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.479+0000 7fbf4b466b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5602273b2e80 con 0x5602274fd590 2024-05-21T03:29:04.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.479+0000 7fbf4b466b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56022745ae50 con 0x5602274fcbd0 2024-05-21T03:29:04.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.479+0000 7fbf4b466b80 10 client.0 _ll_register_callbacks cb 0x5602273e5a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:29:04.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.479+0000 7fbf3ad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560227501700 0x560227506b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:04.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.479+0000 7fbf3ad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560227501700 0x560227506b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44860/0 (socket says 172.21.15.27:44860) 2024-05-21T03:29:04.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.479+0000 7fbf3ad9d700 1 -- 172.21.15.27:0/1344870739 learned_addr learned my addr 172.21.15.27:0/1344870739 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:04.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.479+0000 7fbf39d9b700 1 --2- 172.21.15.27:0/1344870739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602274fd590 0x560227506650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:04.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.479+0000 7fbf3ad9d700 1 -- 172.21.15.27:0/1344870739 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602274fcbd0 msgr2=0x560227509350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:04.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.479+0000 7fbf3a59c700 1 --2- 172.21.15.27:0/1344870739 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602274fcbd0 0x560227509350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:04.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.479+0000 7fbf3ad9d700 1 --2- 172.21.15.27:0/1344870739 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602274fcbd0 0x560227509350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.479+0000 7fbf3ad9d700 1 -- 172.21.15.27:0/1344870739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602274fd590 msgr2=0x560227506650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:04.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.479+0000 7fbf3ad9d700 1 --2- 172.21.15.27:0/1344870739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602274fd590 0x560227506650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.479+0000 7fbf3ad9d700 1 -- 172.21.15.27:0/1344870739 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56022739a9b0 con 0x560227501700 2024-05-21T03:29:04.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.479+0000 7fbf39d9b700 1 --2- 172.21.15.27:0/1344870739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602274fd590 0x560227506650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:29:04.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.479+0000 7fbf3ad9d700 1 --2- 172.21.15.27:0/1344870739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560227501700 0x560227506b90 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbf340cd0b0 tx=0x7fbf340aafb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ec2e103bd87de8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:04.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.479+0000 7fbf21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:29:04.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.479+0000 7fbf21ffb700 1 -- 172.21.15.27:0/1344870739 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf340c4030 con 0x560227501700 2024-05-21T03:29:04.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.479+0000 7fbf21ffb700 1 -- 172.21.15.27:0/1344870739 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fbf340aa9a0 con 0x560227501700 2024-05-21T03:29:04.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.479+0000 7fbf4b466b80 1 -- 172.21.15.27:0/1344870739 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf340aa740 con 0x560227501700 2024-05-21T03:29:04.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.480+0000 7fbf21ffb700 1 -- 172.21.15.27:0/1344870739 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbf340c9050 con 0x560227501700 2024-05-21T03:29:04.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.480+0000 7fbf4b466b80 10 client.15681 fetch_fsmap learned FSMap version 34 2024-05-21T03:29:04.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.480+0000 7fbf4b466b80 1 -- 172.21.15.27:0/1344870739 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560227404840 con 0x560227501700 2024-05-21T03:29:04.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.480+0000 7fbf21ffb700 1 -- 172.21.15.27:0/1344870739 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf340aa740 con 0x560227501700 2024-05-21T03:29:04.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.480+0000 7fbf21ffb700 1 -- 172.21.15.27:0/1344870739 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7fbf34011070 con 0x560227501700 2024-05-21T03:29:04.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.480+0000 7fbf4b466b80 10 client.15681 fetch_fsmap finished waiting for FSMap version 34 2024-05-21T03:29:04.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.480+0000 7fbf4b466b80 10 client.15681 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T03:29:04.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.480+0000 7fbf4b466b80 20 client.15681 populate_metadata read hostname 'smithi027' 2024-05-21T03:29:04.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.480+0000 7fbf4b466b80 1 --2- 172.21.15.27:0/1344870739 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x5602275163c0 0x5602275187b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:04.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.480+0000 7fbf4b466b80 4 client.15681 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:29:04.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.480+0000 7fbf4b466b80 1 -- 172.21.15.27:0/1344870739 --> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56022745cda0 con 0x5602275163c0 2024-05-21T03:29:04.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.480+0000 7fbf3a59c700 1 --2- 172.21.15.27:0/1344870739 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x5602275163c0 0x5602275187b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:04.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.481+0000 7fbf21ffb700 10 client.15681 ms_handle_connect on v2:172.21.15.165:6838/4158170814 2024-05-21T03:29:04.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.481+0000 7fbf3a59c700 1 --2- 172.21.15.27:0/1344870739 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x5602275163c0 0x5602275187b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9c131cd2e548b9a1 server_cookie=354e08eed7219c10 in_seq=0 out_seq=0 2024-05-21T03:29:04.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.481+0000 7fbf21ffb700 1 -- 172.21.15.27:0/1344870739 <== mds.1 v2:172.21.15.165:6838/4158170814 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56022745cda0 con 0x5602275163c0 2024-05-21T03:29:04.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.481+0000 7fbf21ffb700 10 client.15681 handle_command_reply: tid=0 2024-05-21T03:29:04.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.482+0000 7fbf4b466b80 1 client.15681 shutdown 2024-05-21T03:29:04.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.482+0000 7fbf4b466b80 1 -- 172.21.15.27:0/1344870739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560227501700 msgr2=0x560227506b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:04.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.482+0000 7fbf4b466b80 1 --2- 172.21.15.27:0/1344870739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560227501700 0x560227506b90 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbf340cd0b0 tx=0x7fbf340aafb0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.482+0000 7fbf4b466b80 1 -- 172.21.15.27:0/1344870739 shutdown_connections 2024-05-21T03:29:04.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.482+0000 7fbf4b466b80 1 --2- 172.21.15.27:0/1344870739 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x5602275163c0 0x5602275187b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.482+0000 7fbf4b466b80 1 --2- 172.21.15.27:0/1344870739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560227501700 0x560227506b90 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.482+0000 7fbf4b466b80 1 --2- 172.21.15.27:0/1344870739 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602274fcbd0 0x560227509350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.482+0000 7fbf21ffb700 0 client.15681 ms_handle_reset on v2:172.21.15.165:6838/4158170814 2024-05-21T03:29:04.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.482+0000 7fbf4b466b80 1 --2- 172.21.15.27:0/1344870739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602274fd590 0x560227506650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.482+0000 7fbf4b466b80 1 -- 172.21.15.27:0/1344870739 >> 172.21.15.27:0/1344870739 conn(0x560227428320 msgr2=0x5602274fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:04.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.482+0000 7fbf4b466b80 1 -- 172.21.15.27:0/1344870739 shutdown_connections 2024-05-21T03:29:04.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.482+0000 7fbf4b466b80 1 -- 172.21.15.27:0/1344870739 wait complete. 2024-05-21T03:29:04.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.482+0000 7fbf4b466b80 20 client.15681 trim_cache size 0 max 16384 2024-05-21T03:29:04.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.501+0000 7fbf4b466b80 1 Processor -- start 2024-05-21T03:29:04.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.501+0000 7fbf4b466b80 1 -- start start 2024-05-21T03:29:04.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.501+0000 7fbf4b466b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602274fcbd0 0x5602274f1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:04.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.501+0000 7fbf4b466b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602274fd590 0x560227512a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:04.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.501+0000 7fbf4b466b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560227501700 0x560227502b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:04.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.501+0000 7fbf4b466b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c1abae0 con 0x560227501700 2024-05-21T03:29:04.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.501+0000 7fbf4b466b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c1ad6e0 con 0x5602274fd590 2024-05-21T03:29:04.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.501+0000 7fbf4b466b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c1ac1b0 con 0x5602274fcbd0 2024-05-21T03:29:04.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.501+0000 7fbf3ad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560227501700 0x560227502b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:04.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.501+0000 7fbf3ad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560227501700 0x560227502b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44868/0 (socket says 172.21.15.27:44868) 2024-05-21T03:29:04.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.501+0000 7fbf3ad9d700 1 -- 172.21.15.27:0/3835301592 learned_addr learned my addr 172.21.15.27:0/3835301592 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:04.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.501+0000 7fbf3a59c700 1 --2- 172.21.15.27:0/3835301592 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602274fcbd0 0x5602274f1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:04.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.501+0000 7fbf3ad9d700 1 -- 172.21.15.27:0/3835301592 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602274fcbd0 msgr2=0x5602274f1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:04.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.501+0000 7fbf3ad9d700 1 --2- 172.21.15.27:0/3835301592 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602274fcbd0 0x5602274f1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.501+0000 7fbf3ad9d700 1 -- 172.21.15.27:0/3835301592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602274fd590 msgr2=0x560227512a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:04.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.501+0000 7fbf39d9b700 1 --2- 172.21.15.27:0/3835301592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602274fd590 0x560227512a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:04.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.501+0000 7fbf3ad9d700 1 --2- 172.21.15.27:0/3835301592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602274fd590 0x560227512a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.501+0000 7fbf3ad9d700 1 -- 172.21.15.27:0/3835301592 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602274fdc10 con 0x560227501700 2024-05-21T03:29:04.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.502+0000 7fbf3ad9d700 1 --2- 172.21.15.27:0/3835301592 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560227501700 0x560227502b70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbf340a67c0 tx=0x7fbf340099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=522e7faf8cc91220 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:04.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.502+0000 7fbf23fff700 1 -- 172.21.15.27:0/3835301592 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf340c4070 con 0x560227501700 2024-05-21T03:29:04.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.502+0000 7fbf23fff700 1 -- 172.21.15.27:0/3835301592 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf340922e0 con 0x560227501700 2024-05-21T03:29:04.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.502+0000 7fbf23fff700 1 -- 172.21.15.27:0/3835301592 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf34029070 con 0x560227501700 2024-05-21T03:29:04.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.502+0000 7fbf4b466b80 1 -- 172.21.15.27:0/3835301592 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560227501700 msgr2=0x560227502b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:04.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.502+0000 7fbf4b466b80 1 --2- 172.21.15.27:0/3835301592 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560227501700 0x560227502b70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbf340a67c0 tx=0x7fbf340099e0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.502+0000 7fbf4b466b80 1 -- 172.21.15.27:0/3835301592 shutdown_connections 2024-05-21T03:29:04.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.502+0000 7fbf4b466b80 1 --2- 172.21.15.27:0/3835301592 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560227501700 0x560227502b70 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.502+0000 7fbf4b466b80 1 --2- 172.21.15.27:0/3835301592 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602274fcbd0 0x5602274f1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.502+0000 7fbf4b466b80 1 --2- 172.21.15.27:0/3835301592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602274fd590 0x560227512a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.502+0000 7fbf4b466b80 1 -- 172.21.15.27:0/3835301592 >> 172.21.15.27:0/3835301592 conn(0x560227428320 msgr2=0x560227502460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:04.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.503+0000 7fbf4b466b80 1 -- 172.21.15.27:0/3835301592 shutdown_connections 2024-05-21T03:29:04.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.503+0000 7fbf4b466b80 1 -- 172.21.15.27:0/3835301592 wait complete. 2024-05-21T03:29:04.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.503+0000 7fbf4b466b80 1 -- start start 2024-05-21T03:29:04.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.503+0000 7fbf4b466b80 1 Processor -- start 2024-05-21T03:29:04.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.503+0000 7fbf4b466b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602274fcbd0 0x56022751dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:04.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.503+0000 7fbf4b466b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602274fd590 0x56022751e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:04.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.503+0000 7fbf4b466b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560227501700 0x56022751e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:04.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.503+0000 7fbf4b466b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c1acf50 con 0x5602274fd590 2024-05-21T03:29:04.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.503+0000 7fbf4b466b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c1ac880 con 0x5602274fcbd0 2024-05-21T03:29:04.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.503+0000 7fbf4b466b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c1a9aa0 con 0x560227501700 2024-05-21T03:29:04.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.503+0000 7fbf4b466b80 10 client.0 _ll_register_callbacks cb 0x5602273e5a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:29:04.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.503+0000 7fbf39d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602274fd590 0x56022751e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:04.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.503+0000 7fbf39d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602274fd590 0x56022751e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44882/0 (socket says 172.21.15.27:44882) 2024-05-21T03:29:04.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.503+0000 7fbf39d9b700 1 -- 172.21.15.27:0/3047553581 learned_addr learned my addr 172.21.15.27:0/3047553581 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:04.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.503+0000 7fbf3ad9d700 1 --2- 172.21.15.27:0/3047553581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560227501700 0x56022751e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:04.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.504+0000 7fbf3a59c700 1 --2- 172.21.15.27:0/3047553581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602274fcbd0 0x56022751dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:04.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.504+0000 7fbf39d9b700 1 -- 172.21.15.27:0/3047553581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560227501700 msgr2=0x56022751e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:04.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.504+0000 7fbf39d9b700 1 --2- 172.21.15.27:0/3047553581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560227501700 0x56022751e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.504+0000 7fbf39d9b700 1 -- 172.21.15.27:0/3047553581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602274fcbd0 msgr2=0x56022751dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:04.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.504+0000 7fbf39d9b700 1 --2- 172.21.15.27:0/3047553581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602274fcbd0 0x56022751dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.504+0000 7fbf39d9b700 1 -- 172.21.15.27:0/3047553581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560227501350 con 0x5602274fd590 2024-05-21T03:29:04.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.504+0000 7fbf3a59c700 1 --2- 172.21.15.27:0/3047553581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602274fcbd0 0x56022751dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:29:04.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.504+0000 7fbf39d9b700 1 --2- 172.21.15.27:0/3047553581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602274fd590 0x56022751e1f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x560227507c20 tx=0x7fbf3000bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd9bc7a8064836a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:04.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.504+0000 7fbf21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:29:04.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.504+0000 7fbf21ffb700 1 -- 172.21.15.27:0/3047553581 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf30003cc0 con 0x5602274fd590 2024-05-21T03:29:04.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.504+0000 7fbf21ffb700 1 -- 172.21.15.27:0/3047553581 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fbf3001b440 con 0x5602274fd590 2024-05-21T03:29:04.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.504+0000 7fbf4b466b80 1 -- 172.21.15.27:0/3047553581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf340922e0 con 0x5602274fd590 2024-05-21T03:29:04.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.504+0000 7fbf21ffb700 1 -- 172.21.15.27:0/3047553581 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbf30003e60 con 0x5602274fd590 2024-05-21T03:29:04.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.504+0000 7fbf4b466b80 10 client.15687 fetch_fsmap learned FSMap version 34 2024-05-21T03:29:04.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.504+0000 7fbf4b466b80 1 -- 172.21.15.27:0/3047553581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5602275811d0 con 0x5602274fd590 2024-05-21T03:29:04.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.504+0000 7fbf21ffb700 1 -- 172.21.15.27:0/3047553581 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf340922e0 con 0x5602274fd590 2024-05-21T03:29:04.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.505+0000 7fbf21ffb700 1 -- 172.21.15.27:0/3047553581 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7fbf30019020 con 0x5602274fd590 2024-05-21T03:29:04.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.505+0000 7fbf4b466b80 10 client.15687 fetch_fsmap finished waiting for FSMap version 34 2024-05-21T03:29:04.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.505+0000 7fbf4b466b80 10 client.15687 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T03:29:04.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.505+0000 7fbf4b466b80 20 client.15687 populate_metadata read hostname 'smithi027' 2024-05-21T03:29:04.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.505+0000 7fbf4b466b80 1 --2- 172.21.15.27:0/3047553581 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x5602275818f0 0x560227583d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:04.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.505+0000 7fbf4b466b80 4 client.15687 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-72cf81f5-1-mds.i.json"}] 2024-05-21T03:29:04.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.505+0000 7fbf4b466b80 1 -- 172.21.15.27:0/3047553581 --> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-72cf81f5-1-mds.i.json"}) -- 0x560227456580 con 0x5602275818f0 2024-05-21T03:29:04.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.505+0000 7fbf3a59c700 1 --2- 172.21.15.27:0/3047553581 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x5602275818f0 0x560227583d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:04.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.505+0000 7fbf3a59c700 1 --2- 172.21.15.27:0/3047553581 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x5602275818f0 0x560227583d10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a6d75b846e17fa21 server_cookie=a8fdd7ab3ebf92f6 in_seq=0 out_seq=0 2024-05-21T03:29:04.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.505+0000 7fbf21ffb700 10 client.15687 ms_handle_connect on v2:172.21.15.165:6838/4158170814 2024-05-21T03:29:04.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.534+0000 7fbf21ffb700 1 -- 172.21.15.27:0/3047553581 <== mds.1 v2:172.21.15.165:6838/4158170814 1 ==== command_reply(tid 0: 0 ) ==== 8+0+58 (crc 0 0 0) 0x560227456580 con 0x5602275818f0 2024-05-21T03:29:04.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.534+0000 7fbf21ffb700 10 client.15687 handle_command_reply: tid=0 2024-05-21T03:29:04.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.534+0000 7fbf4b466b80 1 client.15687 shutdown 2024-05-21T03:29:04.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.534+0000 7fbf4b466b80 1 -- 172.21.15.27:0/3047553581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602274fd590 msgr2=0x56022751e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:04.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.534+0000 7fbf4b466b80 1 --2- 172.21.15.27:0/3047553581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602274fd590 0x56022751e1f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x560227507c20 tx=0x7fbf3000bc70 comp rx=0 tx=0).stop 2024-05-21T03:29:04.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.535+0000 7fbf4b466b80 1 -- 172.21.15.27:0/3047553581 shutdown_connections 2024-05-21T03:29:04.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.535+0000 7fbf4b466b80 1 --2- 172.21.15.27:0/3047553581 >> [v2:172.21.15.165:6838/4158170814,v1:172.21.15.165:6839/4158170814] conn(0x5602275818f0 0x560227583d10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.535+0000 7fbf4b466b80 1 --2- 172.21.15.27:0/3047553581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560227501700 0x56022751e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.535+0000 7fbf4b466b80 1 --2- 172.21.15.27:0/3047553581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602274fcbd0 0x56022751dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.535+0000 7fbf4b466b80 1 --2- 172.21.15.27:0/3047553581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602274fd590 0x56022751e1f0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.535+0000 7fbf4b466b80 1 -- 172.21.15.27:0/3047553581 >> 172.21.15.27:0/3047553581 conn(0x560227428320 msgr2=0x560227507e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:04.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.535+0000 7fbf21ffb700 0 client.15687 ms_handle_reset on v2:172.21.15.165:6838/4158170814 2024-05-21T03:29:04.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.535+0000 7fbf4b466b80 1 -- 172.21.15.27:0/3047553581 shutdown_connections 2024-05-21T03:29:04.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.535+0000 7fbf4b466b80 1 -- 172.21.15.27:0/3047553581 wait complete. 2024-05-21T03:29:04.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.535+0000 7fbf4b466b80 20 client.15687 trim_cache size 0 max 16384 2024-05-21T03:29:04.536 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:29:04.536 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/var/run/ceph/ops-72cf81f5-1-mds.i.json" 2024-05-21T03:29:04.536 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:29:04.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.535+0000 7fbf40bd0700 1 -- 172.21.15.27:0/3515199930 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fbf240b29f0 msgr2=0x7fbf240b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:04.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.535+0000 7fbf40bd0700 1 --2- 172.21.15.27:0/3515199930 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fbf240b29f0 0x7fbf240b4eb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c100ba0 tx=0x7fbf2c00b410 comp rx=0 tx=0).stop 2024-05-21T03:29:04.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.535+0000 7fbf40bd0700 1 -- 172.21.15.27:0/3515199930 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf3c109630 msgr2=0x7fbf3c199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:04.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.535+0000 7fbf40bd0700 1 --2- 172.21.15.27:0/3515199930 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf3c109630 0x7fbf3c199980 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbf340095c0 tx=0x7fbf34007a20 comp rx=0 tx=0).stop 2024-05-21T03:29:04.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.535+0000 7fbf40bd0700 1 -- 172.21.15.27:0/3515199930 shutdown_connections 2024-05-21T03:29:04.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.535+0000 7fbf40bd0700 1 --2- 172.21.15.27:0/3515199930 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf3c109630 0x7fbf3c199980 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.535+0000 7fbf40bd0700 1 --2- 172.21.15.27:0/3515199930 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fbf240b29f0 0x7fbf240b4eb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.535+0000 7fbf40bd0700 1 --2- 172.21.15.27:0/3515199930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3c06bd10 0x7fbf3c19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.535+0000 7fbf40bd0700 1 --2- 172.21.15.27:0/3515199930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf3c06c6d0 0x7fbf3c19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:04.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.535+0000 7fbf40bd0700 1 -- 172.21.15.27:0/3515199930 >> 172.21.15.27:0/3515199930 conn(0x7fbf3c0794e0 msgr2=0x7fbf3c100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:04.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.535+0000 7fbf40bd0700 1 -- 172.21.15.27:0/3515199930 shutdown_connections 2024-05-21T03:29:04.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:04.535+0000 7fbf40bd0700 1 -- 172.21.15.27:0/3515199930 wait complete. 2024-05-21T03:29:04.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:04 smithi027 ceph-mon[32179]: pgmap v154: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 18 MiB/s wr, 2.40k op/s 2024-05-21T03:29:05.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:04 smithi039 ceph-mon[36429]: pgmap v154: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 18 MiB/s wr, 2.40k op/s 2024-05-21T03:29:05.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:04 smithi165 ceph-mon[36269]: pgmap v154: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 18 MiB/s wr, 2.40k op/s 2024-05-21T03:29:05.269 DEBUG:tasks.quiescer.fs.[cephfs]:Dumping ops on rank 0 (b) to a remote file /var/run/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ops-72cf81f5-1-mds.b.json 2024-05-21T03:29:05.269 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 ops --flags=locks --path=/var/run/ceph/ops-72cf81f5-1-mds.b.json 2024-05-21T03:29:05.530 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:29:06.801 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:06 smithi027 ceph-mon[32179]: pgmap v155: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.3 MiB/s wr, 600 op/s 2024-05-21T03:29:07.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:06 smithi039 ceph-mon[36429]: pgmap v155: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.3 MiB/s wr, 600 op/s 2024-05-21T03:29:07.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:06 smithi165 ceph-mon[36269]: pgmap v155: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.3 MiB/s wr, 600 op/s 2024-05-21T03:29:07.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.338+0000 7f02f68fa700 1 -- 172.21.15.27:0/842057596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02f006b250 msgr2=0x7f02f006b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:07.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.338+0000 7f02f68fa700 1 --2- 172.21.15.27:0/842057596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02f006b250 0x7f02f006b630 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f02e0009990 tx=0x7f02e0009ca0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.338+0000 7f02f68fa700 1 -- 172.21.15.27:0/842057596 shutdown_connections 2024-05-21T03:29:07.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.338+0000 7f02f68fa700 1 --2- 172.21.15.27:0/842057596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02f010fff0 0x7f02f01123e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.338+0000 7f02f68fa700 1 --2- 172.21.15.27:0/842057596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02f006b250 0x7f02f006b630 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.338+0000 7f02f68fa700 1 --2- 172.21.15.27:0/842057596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02f006bb70 0x7f02f010fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.338+0000 7f02f68fa700 1 -- 172.21.15.27:0/842057596 >> 172.21.15.27:0/842057596 conn(0x7f02f00793f0 msgr2=0x7f02f0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:07.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.339+0000 7f02f68fa700 1 -- 172.21.15.27:0/842057596 shutdown_connections 2024-05-21T03:29:07.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.339+0000 7f02f68fa700 1 -- 172.21.15.27:0/842057596 wait complete. 2024-05-21T03:29:07.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.339+0000 7f02f68fa700 1 Processor -- start 2024-05-21T03:29:07.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.339+0000 7f02f68fa700 1 -- start start 2024-05-21T03:29:07.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.339+0000 7f02f68fa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02f006b250 0x7f02f019be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:07.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.339+0000 7f02f68fa700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02f006bb70 0x7f02f019c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:07.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.339+0000 7f02f68fa700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02f010fff0 0x7f02f01a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:07.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.339+0000 7f02f68fa700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f02f0115910 con 0x7f02f006b250 2024-05-21T03:29:07.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.339+0000 7f02f68fa700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f02f0115790 con 0x7f02f006bb70 2024-05-21T03:29:07.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.339+0000 7f02f68fa700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f02f0115a90 con 0x7f02f010fff0 2024-05-21T03:29:07.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.340+0000 7f02effff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02f006b250 0x7f02f019be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:07.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.340+0000 7f02effff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02f006b250 0x7f02f019be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44904/0 (socket says 172.21.15.27:44904) 2024-05-21T03:29:07.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.340+0000 7f02effff700 1 -- 172.21.15.27:0/1237598922 learned_addr learned my addr 172.21.15.27:0/1237598922 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:07.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.340+0000 7f02e7fff700 1 --2- 172.21.15.27:0/1237598922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02f006bb70 0x7f02f019c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:07.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.340+0000 7f02f4e97700 1 --2- 172.21.15.27:0/1237598922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02f010fff0 0x7f02f01a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:07.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.340+0000 7f02effff700 1 -- 172.21.15.27:0/1237598922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02f010fff0 msgr2=0x7f02f01a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:07.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.340+0000 7f02effff700 1 --2- 172.21.15.27:0/1237598922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02f010fff0 0x7f02f01a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.340+0000 7f02effff700 1 -- 172.21.15.27:0/1237598922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02f006bb70 msgr2=0x7f02f019c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:07.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.340+0000 7f02effff700 1 --2- 172.21.15.27:0/1237598922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02f006bb70 0x7f02f019c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.340+0000 7f02effff700 1 -- 172.21.15.27:0/1237598922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f01a0e20 con 0x7f02f006b250 2024-05-21T03:29:07.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.340+0000 7f02e7fff700 1 --2- 172.21.15.27:0/1237598922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02f006bb70 0x7f02f019c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:29:07.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.340+0000 7f02effff700 1 --2- 172.21.15.27:0/1237598922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02f006b250 0x7f02f019be60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f02e0000c00 tx=0x7f02e00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:07.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.340+0000 7f02edffb700 1 -- 172.21.15.27:0/1237598922 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e0027070 con 0x7f02f006b250 2024-05-21T03:29:07.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.340+0000 7f02edffb700 1 -- 172.21.15.27:0/1237598922 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02e0003fa0 con 0x7f02f006b250 2024-05-21T03:29:07.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.340+0000 7f02f68fa700 1 -- 172.21.15.27:0/1237598922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02f01a10b0 con 0x7f02f006b250 2024-05-21T03:29:07.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.340+0000 7f02edffb700 1 -- 172.21.15.27:0/1237598922 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e0022db0 con 0x7f02f006b250 2024-05-21T03:29:07.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.340+0000 7f02f68fa700 1 -- 172.21.15.27:0/1237598922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02f01a8990 con 0x7f02f006b250 2024-05-21T03:29:07.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.343+0000 7f02edffb700 1 -- 172.21.15.27:0/1237598922 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7f02e002a070 con 0x7f02f006b250 2024-05-21T03:29:07.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.344+0000 7f02edffb700 1 --2- 172.21.15.27:0/1237598922 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f02dc0b2a70 0x7f02dc0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:07.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.344+0000 7f02e7fff700 1 --2- 172.21.15.27:0/1237598922 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f02dc0b2a70 0x7f02dc0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:07.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.344+0000 7f02edffb700 1 -- 172.21.15.27:0/1237598922 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f02e00e32e0 con 0x7f02f006b250 2024-05-21T03:29:07.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.344+0000 7f02e7fff700 1 --2- 172.21.15.27:0/1237598922 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f02dc0b2a70 0x7f02dc0b4f30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f02d8005d90 tx=0x7f02d8005d00 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:07.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.345+0000 7f0301190b80 1 Processor -- start 2024-05-21T03:29:07.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.345+0000 7f0301190b80 1 -- start start 2024-05-21T03:29:07.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.345+0000 7f0301190b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570d9004700 0x5570d9004ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:07.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.345+0000 7f0301190b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570d8fffbd0 0x5570d9000050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:07.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.345+0000 7f0301190b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570d9000590 0x5570d900c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:07.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.345+0000 7f0301190b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5570d8f5de50 con 0x5570d9000590 2024-05-21T03:29:07.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.345+0000 7f0301190b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5570d8f4d9f0 con 0x5570d9004700 2024-05-21T03:29:07.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.345+0000 7f0301190b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5570d8f51a30 con 0x5570d8fffbd0 2024-05-21T03:29:07.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.345+0000 7f02f4e97700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570d9000590 0x5570d900c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:07.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.345+0000 7f02f4e97700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570d9000590 0x5570d900c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44912/0 (socket says 172.21.15.27:44912) 2024-05-21T03:29:07.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.345+0000 7f02f4e97700 1 -- 172.21.15.27:0/2463754498 learned_addr learned my addr 172.21.15.27:0/2463754498 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:07.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.345+0000 7f02effff700 1 --2- 172.21.15.27:0/2463754498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570d9004700 0x5570d9004ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:07.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.345+0000 7f02e7fff700 1 --2- 172.21.15.27:0/2463754498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570d8fffbd0 0x5570d9000050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:07.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.346+0000 7f02f4e97700 1 -- 172.21.15.27:0/2463754498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570d8fffbd0 msgr2=0x5570d9000050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:07.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.346+0000 7f02f4e97700 1 --2- 172.21.15.27:0/2463754498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570d8fffbd0 0x5570d9000050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.346+0000 7f02f4e97700 1 -- 172.21.15.27:0/2463754498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570d9004700 msgr2=0x5570d9004ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:07.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.346+0000 7f02f4e97700 1 --2- 172.21.15.27:0/2463754498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570d9004700 0x5570d9004ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.346+0000 7f02f4e97700 1 -- 172.21.15.27:0/2463754498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570d8ef7090 con 0x5570d9000590 2024-05-21T03:29:07.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.346+0000 7f02f4e97700 1 --2- 172.21.15.27:0/2463754498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570d9000590 0x5570d900c600 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f02e8009bf0 tx=0x7f02e8009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95817132749780a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:07.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.346+0000 7f02e67fc700 1 -- 172.21.15.27:0/2463754498 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e8019070 con 0x5570d9000590 2024-05-21T03:29:07.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.346+0000 7f02e67fc700 1 -- 172.21.15.27:0/2463754498 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02e80148b0 con 0x5570d9000590 2024-05-21T03:29:07.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.346+0000 7f02e67fc700 1 -- 172.21.15.27:0/2463754498 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e8013940 con 0x5570d9000590 2024-05-21T03:29:07.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.347+0000 7f0301190b80 1 -- 172.21.15.27:0/2463754498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570d9000590 msgr2=0x5570d900c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:07.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.347+0000 7f0301190b80 1 --2- 172.21.15.27:0/2463754498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570d9000590 0x5570d900c600 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f02e8009bf0 tx=0x7f02e8009f00 comp rx=0 tx=0).stop 2024-05-21T03:29:07.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.347+0000 7f0301190b80 1 -- 172.21.15.27:0/2463754498 shutdown_connections 2024-05-21T03:29:07.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.347+0000 7f0301190b80 1 --2- 172.21.15.27:0/2463754498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570d9000590 0x5570d900c600 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.347+0000 7f0301190b80 1 --2- 172.21.15.27:0/2463754498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570d9004700 0x5570d9004ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.347+0000 7f0301190b80 1 --2- 172.21.15.27:0/2463754498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570d8fffbd0 0x5570d9000050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.347+0000 7f0301190b80 1 -- 172.21.15.27:0/2463754498 >> 172.21.15.27:0/2463754498 conn(0x5570d8f2b320 msgr2=0x5570d8ffa8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:07.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.347+0000 7f0301190b80 1 -- 172.21.15.27:0/2463754498 shutdown_connections 2024-05-21T03:29:07.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.347+0000 7f0301190b80 1 -- 172.21.15.27:0/2463754498 wait complete. 2024-05-21T03:29:07.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.347+0000 7f0301190b80 1 -- start start 2024-05-21T03:29:07.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.348+0000 7f0301190b80 1 Processor -- start 2024-05-21T03:29:07.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.348+0000 7f0301190b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570d8fffbd0 0x5570d900c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:07.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.348+0000 7f0301190b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570d9000590 0x5570d9009540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:07.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.348+0000 7f0301190b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570d9004700 0x5570d9009a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:07.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.348+0000 7f0301190b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5570d8e54080 con 0x5570d8fffbd0 2024-05-21T03:29:07.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.348+0000 7f0301190b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5570d8eb5e80 con 0x5570d9004700 2024-05-21T03:29:07.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.348+0000 7f0301190b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5570d900cc60 con 0x5570d9000590 2024-05-21T03:29:07.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.348+0000 7f0301190b80 10 client.0 _ll_register_callbacks cb 0x5570d8ee8a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:29:07.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.348+0000 7f02effff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570d8fffbd0 0x5570d900c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:07.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.348+0000 7f02effff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570d8fffbd0 0x5570d900c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44918/0 (socket says 172.21.15.27:44918) 2024-05-21T03:29:07.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.348+0000 7f02effff700 1 -- 172.21.15.27:0/4215433284 learned_addr learned my addr 172.21.15.27:0/4215433284 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:07.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.348+0000 7f02e7fff700 1 --2- 172.21.15.27:0/4215433284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570d9000590 0x5570d9009540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:07.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.348+0000 7f02effff700 1 -- 172.21.15.27:0/4215433284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570d9000590 msgr2=0x5570d9009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:07.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.348+0000 7f02effff700 1 --2- 172.21.15.27:0/4215433284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570d9000590 0x5570d9009540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.348+0000 7f02effff700 1 -- 172.21.15.27:0/4215433284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570d9004700 msgr2=0x5570d9009a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:07.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.348+0000 7f02f4e97700 1 --2- 172.21.15.27:0/4215433284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570d9004700 0x5570d9009a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:07.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.348+0000 7f02effff700 1 --2- 172.21.15.27:0/4215433284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570d9004700 0x5570d9009a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.348+0000 7f02effff700 1 -- 172.21.15.27:0/4215433284 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570d8e9d9b0 con 0x5570d8fffbd0 2024-05-21T03:29:07.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.348+0000 7f02f4e97700 1 --2- 172.21.15.27:0/4215433284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570d9004700 0x5570d9009a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:29:07.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.349+0000 7f02effff700 1 --2- 172.21.15.27:0/4215433284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570d8fffbd0 0x5570d900c240 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f02e00aa5e0 tx=0x7f02e0022750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c342622b6b1657 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:07.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.349+0000 7f02cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:29:07.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.349+0000 7f02cf7fe700 1 -- 172.21.15.27:0/4215433284 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e00aa650 con 0x5570d8fffbd0 2024-05-21T03:29:07.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.349+0000 7f02cf7fe700 1 -- 172.21.15.27:0/4215433284 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f02e0004150 con 0x5570d8fffbd0 2024-05-21T03:29:07.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.349+0000 7f0301190b80 1 -- 172.21.15.27:0/4215433284 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02e80148b0 con 0x5570d8fffbd0 2024-05-21T03:29:07.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.349+0000 7f02cf7fe700 1 -- 172.21.15.27:0/4215433284 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f02e00aea00 con 0x5570d8fffbd0 2024-05-21T03:29:07.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.349+0000 7f0301190b80 10 client.15699 fetch_fsmap learned FSMap version 34 2024-05-21T03:29:07.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.349+0000 7f0301190b80 1 -- 172.21.15.27:0/4215433284 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5570d8f07840 con 0x5570d8fffbd0 2024-05-21T03:29:07.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.349+0000 7f02cf7fe700 1 -- 172.21.15.27:0/4215433284 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02e00cb030 con 0x5570d8fffbd0 2024-05-21T03:29:07.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.349+0000 7f02cf7fe700 1 -- 172.21.15.27:0/4215433284 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f02e00afb90 con 0x5570d8fffbd0 2024-05-21T03:29:07.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.349+0000 7f0301190b80 10 client.15699 fetch_fsmap finished waiting for FSMap version 34 2024-05-21T03:29:07.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.349+0000 7f0301190b80 10 client.15699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T03:29:07.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.349+0000 7f0301190b80 20 client.15699 populate_metadata read hostname 'smithi027' 2024-05-21T03:29:07.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.350+0000 7f0301190b80 1 --2- 172.21.15.27:0/4215433284 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x5570d9019870 0x5570d901bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:07.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.350+0000 7f0301190b80 4 client.15699 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:29:07.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.350+0000 7f0301190b80 1 -- 172.21.15.27:0/4215433284 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570d8f5fda0 con 0x5570d9019870 2024-05-21T03:29:07.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.350+0000 7f02e7fff700 1 --2- 172.21.15.27:0/4215433284 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x5570d9019870 0x5570d901bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:07.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.350+0000 7f02cf7fe700 10 client.15699 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:29:07.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.350+0000 7f02e7fff700 1 --2- 172.21.15.27:0/4215433284 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x5570d9019870 0x5570d901bcd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1298b0be0294a5a6 server_cookie=2698201200632617 in_seq=0 out_seq=0 2024-05-21T03:29:07.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.351+0000 7f02cf7fe700 1 -- 172.21.15.27:0/4215433284 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5570d8f5fda0 con 0x5570d9019870 2024-05-21T03:29:07.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.351+0000 7f02cf7fe700 10 client.15699 handle_command_reply: tid=0 2024-05-21T03:29:07.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.351+0000 7f0301190b80 1 client.15699 shutdown 2024-05-21T03:29:07.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.351+0000 7f0301190b80 1 -- 172.21.15.27:0/4215433284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570d8fffbd0 msgr2=0x5570d900c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:07.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.351+0000 7f0301190b80 1 --2- 172.21.15.27:0/4215433284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570d8fffbd0 0x5570d900c240 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f02e00aa5e0 tx=0x7f02e0022750 comp rx=0 tx=0).stop 2024-05-21T03:29:07.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.352+0000 7f0301190b80 1 -- 172.21.15.27:0/4215433284 shutdown_connections 2024-05-21T03:29:07.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.352+0000 7f0301190b80 1 --2- 172.21.15.27:0/4215433284 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x5570d9019870 0x5570d901bcd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.352+0000 7f0301190b80 1 --2- 172.21.15.27:0/4215433284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570d9004700 0x5570d9009a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.352+0000 7f0301190b80 1 --2- 172.21.15.27:0/4215433284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570d8fffbd0 0x5570d900c240 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.352+0000 7f0301190b80 1 --2- 172.21.15.27:0/4215433284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570d9000590 0x5570d9009540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.352+0000 7f02cf7fe700 0 client.15699 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:29:07.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.352+0000 7f0301190b80 1 -- 172.21.15.27:0/4215433284 >> 172.21.15.27:0/4215433284 conn(0x5570d8f2b320 msgr2=0x5570d8ffebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:07.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.352+0000 7f0301190b80 1 -- 172.21.15.27:0/4215433284 shutdown_connections 2024-05-21T03:29:07.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.352+0000 7f0301190b80 1 -- 172.21.15.27:0/4215433284 wait complete. 2024-05-21T03:29:07.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.352+0000 7f0301190b80 20 client.15699 trim_cache size 0 max 16384 2024-05-21T03:29:07.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.370+0000 7f0301190b80 1 Processor -- start 2024-05-21T03:29:07.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.370+0000 7f0301190b80 1 -- start start 2024-05-21T03:29:07.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.371+0000 7f0301190b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570d8fffbd0 0x5570d9003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:07.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.371+0000 7f0301190b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570d9000590 0x5570d8ff4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:07.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.371+0000 7f0301190b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570d9004700 0x5570d90215a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:07.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.371+0000 7f0301190b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f02f01afec0 con 0x5570d9000590 2024-05-21T03:29:07.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.371+0000 7f0301190b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f02f01b1ac0 con 0x5570d9004700 2024-05-21T03:29:07.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.371+0000 7f0301190b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f02f01b0590 con 0x5570d8fffbd0 2024-05-21T03:29:07.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.371+0000 7f02e7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570d9000590 0x5570d8ff4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:07.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.371+0000 7f02e7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570d9000590 0x5570d8ff4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44924/0 (socket says 172.21.15.27:44924) 2024-05-21T03:29:07.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.371+0000 7f02e7fff700 1 -- 172.21.15.27:0/3610399205 learned_addr learned my addr 172.21.15.27:0/3610399205 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:07.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.371+0000 7f02f4e97700 1 --2- 172.21.15.27:0/3610399205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570d9004700 0x5570d90215a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:07.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.371+0000 7f02effff700 1 --2- 172.21.15.27:0/3610399205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570d8fffbd0 0x5570d9003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:07.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.371+0000 7f02e7fff700 1 -- 172.21.15.27:0/3610399205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570d8fffbd0 msgr2=0x5570d9003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:07.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.371+0000 7f02e7fff700 1 --2- 172.21.15.27:0/3610399205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570d8fffbd0 0x5570d9003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.371+0000 7f02e7fff700 1 -- 172.21.15.27:0/3610399205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570d9004700 msgr2=0x5570d90215a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:07.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.371+0000 7f02e7fff700 1 --2- 172.21.15.27:0/3610399205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570d9004700 0x5570d90215a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.371+0000 7f02e7fff700 1 -- 172.21.15.27:0/3610399205 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570d9000c10 con 0x5570d9000590 2024-05-21T03:29:07.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.371+0000 7f02effff700 1 --2- 172.21.15.27:0/3610399205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570d8fffbd0 0x5570d9003f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:29:07.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.371+0000 7f02e7fff700 1 --2- 172.21.15.27:0/3610399205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570d9000590 0x5570d8ff4e50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x5570d900ab10 tx=0x7f02d8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c55b36c6b97b7c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:07.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.372+0000 7f02e5ffb700 1 -- 172.21.15.27:0/3610399205 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d8011b30 con 0x5570d9000590 2024-05-21T03:29:07.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.372+0000 7f02e5ffb700 1 -- 172.21.15.27:0/3610399205 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02d8017040 con 0x5570d9000590 2024-05-21T03:29:07.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.372+0000 7f02e5ffb700 1 -- 172.21.15.27:0/3610399205 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d8003c50 con 0x5570d9000590 2024-05-21T03:29:07.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.372+0000 7f0301190b80 1 -- 172.21.15.27:0/3610399205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570d9000590 msgr2=0x5570d8ff4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:07.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.372+0000 7f0301190b80 1 --2- 172.21.15.27:0/3610399205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570d9000590 0x5570d8ff4e50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x5570d900ab10 tx=0x7f02d8014000 comp rx=0 tx=0).stop 2024-05-21T03:29:07.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.372+0000 7f0301190b80 1 -- 172.21.15.27:0/3610399205 shutdown_connections 2024-05-21T03:29:07.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.372+0000 7f0301190b80 1 --2- 172.21.15.27:0/3610399205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570d9004700 0x5570d90215a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.372+0000 7f0301190b80 1 --2- 172.21.15.27:0/3610399205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570d8fffbd0 0x5570d9003f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.372+0000 7f0301190b80 1 --2- 172.21.15.27:0/3610399205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570d9000590 0x5570d8ff4e50 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.372+0000 7f0301190b80 1 -- 172.21.15.27:0/3610399205 >> 172.21.15.27:0/3610399205 conn(0x5570d8f2b320 msgr2=0x5570d9006e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:07.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.372+0000 7f0301190b80 1 -- 172.21.15.27:0/3610399205 shutdown_connections 2024-05-21T03:29:07.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.372+0000 7f0301190b80 1 -- 172.21.15.27:0/3610399205 wait complete. 2024-05-21T03:29:07.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.373+0000 7f0301190b80 1 -- start start 2024-05-21T03:29:07.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.373+0000 7f0301190b80 1 Processor -- start 2024-05-21T03:29:07.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.373+0000 7f0301190b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570d8fffbd0 0x5570d9012270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:07.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.373+0000 7f0301190b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570d9000590 0x5570d900ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:07.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.373+0000 7f0301190b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570d9004700 0x5570d900f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:07.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.373+0000 7f0301190b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f02f01b1330 con 0x5570d9000590 2024-05-21T03:29:07.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.373+0000 7f0301190b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f02f01b0c60 con 0x5570d8fffbd0 2024-05-21T03:29:07.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.373+0000 7f0301190b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f02f01ade60 con 0x5570d9004700 2024-05-21T03:29:07.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.373+0000 7f02e7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570d9000590 0x5570d900ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:07.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.373+0000 7f02e7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570d9000590 0x5570d900ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44932/0 (socket says 172.21.15.27:44932) 2024-05-21T03:29:07.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.373+0000 7f02e7fff700 1 -- 172.21.15.27:0/2700416805 learned_addr learned my addr 172.21.15.27:0/2700416805 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:07.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.373+0000 7f0301190b80 10 client.0 _ll_register_callbacks cb 0x5570d8ee8a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:29:07.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.373+0000 7f02effff700 1 --2- 172.21.15.27:0/2700416805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570d8fffbd0 0x5570d9012270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:07.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.374+0000 7f02f4e97700 1 --2- 172.21.15.27:0/2700416805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570d9004700 0x5570d900f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:07.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.374+0000 7f02e7fff700 1 -- 172.21.15.27:0/2700416805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570d9004700 msgr2=0x5570d900f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:07.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.374+0000 7f02e7fff700 1 --2- 172.21.15.27:0/2700416805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570d9004700 0x5570d900f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.374+0000 7f02e7fff700 1 -- 172.21.15.27:0/2700416805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570d8fffbd0 msgr2=0x5570d9012270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:07.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.374+0000 7f02e7fff700 1 --2- 172.21.15.27:0/2700416805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570d8fffbd0 0x5570d9012270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.374+0000 7f02e7fff700 1 -- 172.21.15.27:0/2700416805 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570d900f900 con 0x5570d9000590 2024-05-21T03:29:07.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.374+0000 7f02e7fff700 1 --2- 172.21.15.27:0/2700416805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570d9000590 0x5570d900ee80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f02d800ec20 tx=0x7f02d8009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550195f762345dd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:07.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.374+0000 7f02cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:29:07.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.374+0000 7f02cf7fe700 1 -- 172.21.15.27:0/2700416805 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d8011e00 con 0x5570d9000590 2024-05-21T03:29:07.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.374+0000 7f02cf7fe700 1 -- 172.21.15.27:0/2700416805 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f02d80272c0 con 0x5570d9000590 2024-05-21T03:29:07.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.374+0000 7f0301190b80 1 -- 172.21.15.27:0/2700416805 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02d8017040 con 0x5570d9000590 2024-05-21T03:29:07.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.375+0000 7f02cf7fe700 1 -- 172.21.15.27:0/2700416805 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f02d803f050 con 0x5570d9000590 2024-05-21T03:29:07.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.375+0000 7f0301190b80 10 client.15705 fetch_fsmap learned FSMap version 34 2024-05-21T03:29:07.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.375+0000 7f0301190b80 1 -- 172.21.15.27:0/2700416805 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5570d9085060 con 0x5570d9000590 2024-05-21T03:29:07.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.375+0000 7f02cf7fe700 1 -- 172.21.15.27:0/2700416805 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02d8017040 con 0x5570d9000590 2024-05-21T03:29:07.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.375+0000 7f02cf7fe700 1 -- 172.21.15.27:0/2700416805 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f02d800f040 con 0x5570d9000590 2024-05-21T03:29:07.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.375+0000 7f0301190b80 10 client.15705 fetch_fsmap finished waiting for FSMap version 34 2024-05-21T03:29:07.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.375+0000 7f0301190b80 10 client.15705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T03:29:07.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.375+0000 7f0301190b80 20 client.15705 populate_metadata read hostname 'smithi027' 2024-05-21T03:29:07.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.375+0000 7f0301190b80 1 --2- 172.21.15.27:0/2700416805 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x5570d90858c0 0x5570d9087ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:07.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.375+0000 7f0301190b80 4 client.15705 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-72cf81f5-1-mds.b.json"}] 2024-05-21T03:29:07.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.375+0000 7f0301190b80 1 -- 172.21.15.27:0/2700416805 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-72cf81f5-1-mds.b.json"}) -- 0x5570d8f59580 con 0x5570d90858c0 2024-05-21T03:29:07.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.375+0000 7f02effff700 1 --2- 172.21.15.27:0/2700416805 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x5570d90858c0 0x5570d9087ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:07.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.376+0000 7f02effff700 1 --2- 172.21.15.27:0/2700416805 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x5570d90858c0 0x5570d9087ce0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25016ac2ec0b4ea0 server_cookie=573e6dc22a37a1d0 in_seq=0 out_seq=0 2024-05-21T03:29:07.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.376+0000 7f02cf7fe700 10 client.15705 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:29:07.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.389+0000 7f02cf7fe700 1 -- 172.21.15.27:0/2700416805 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+58 (crc 0 0 0) 0x5570d8f59580 con 0x5570d90858c0 2024-05-21T03:29:07.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.389+0000 7f02cf7fe700 10 client.15705 handle_command_reply: tid=0 2024-05-21T03:29:07.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.389+0000 7f0301190b80 1 client.15705 shutdown 2024-05-21T03:29:07.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.389+0000 7f0301190b80 1 -- 172.21.15.27:0/2700416805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570d9000590 msgr2=0x5570d900ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:07.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.389+0000 7f0301190b80 1 --2- 172.21.15.27:0/2700416805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570d9000590 0x5570d900ee80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f02d800ec20 tx=0x7f02d8009810 comp rx=0 tx=0).stop 2024-05-21T03:29:07.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.389+0000 7f0301190b80 1 -- 172.21.15.27:0/2700416805 shutdown_connections 2024-05-21T03:29:07.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.389+0000 7f0301190b80 1 --2- 172.21.15.27:0/2700416805 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x5570d90858c0 0x5570d9087ce0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.389+0000 7f0301190b80 1 --2- 172.21.15.27:0/2700416805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570d9004700 0x5570d900f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.389+0000 7f0301190b80 1 --2- 172.21.15.27:0/2700416805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570d8fffbd0 0x5570d9012270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.389+0000 7f02cf7fe700 0 client.15705 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:29:07.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.389+0000 7f0301190b80 1 --2- 172.21.15.27:0/2700416805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570d9000590 0x5570d900ee80 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.389+0000 7f0301190b80 1 -- 172.21.15.27:0/2700416805 >> 172.21.15.27:0/2700416805 conn(0x5570d8f2b320 msgr2=0x5570d9006590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:07.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.389+0000 7f0301190b80 1 -- 172.21.15.27:0/2700416805 shutdown_connections 2024-05-21T03:29:07.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.389+0000 7f0301190b80 1 -- 172.21.15.27:0/2700416805 wait complete. 2024-05-21T03:29:07.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.389+0000 7f0301190b80 20 client.15705 trim_cache size 0 max 16384 2024-05-21T03:29:07.390 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:29:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "path": "/var/run/ceph/ops-72cf81f5-1-mds.b.json" 2024-05-21T03:29:07.390 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:29:07.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.390+0000 7f02f68fa700 1 -- 172.21.15.27:0/1237598922 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f02dc0b2a70 msgr2=0x7f02dc0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:07.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.390+0000 7f02f68fa700 1 --2- 172.21.15.27:0/1237598922 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f02dc0b2a70 0x7f02dc0b4f30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f02d8005d90 tx=0x7f02d8005d00 comp rx=0 tx=0).stop 2024-05-21T03:29:07.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.390+0000 7f02f68fa700 1 -- 172.21.15.27:0/1237598922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02f006b250 msgr2=0x7f02f019be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:07.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.390+0000 7f02f68fa700 1 --2- 172.21.15.27:0/1237598922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02f006b250 0x7f02f019be60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f02e0000c00 tx=0x7f02e00047b0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.390+0000 7f02f68fa700 1 -- 172.21.15.27:0/1237598922 shutdown_connections 2024-05-21T03:29:07.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.390+0000 7f02f68fa700 1 --2- 172.21.15.27:0/1237598922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02f010fff0 0x7f02f01a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.390+0000 7f02f68fa700 1 --2- 172.21.15.27:0/1237598922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02f006b250 0x7f02f019be60 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.390+0000 7f02f68fa700 1 --2- 172.21.15.27:0/1237598922 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f02dc0b2a70 0x7f02dc0b4f30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.390+0000 7f02f68fa700 1 --2- 172.21.15.27:0/1237598922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02f006bb70 0x7f02f019c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:07.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.390+0000 7f02f68fa700 1 -- 172.21.15.27:0/1237598922 >> 172.21.15.27:0/1237598922 conn(0x7f02f00793f0 msgr2=0x7f02f01114e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:07.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.390+0000 7f02f68fa700 1 -- 172.21.15.27:0/1237598922 shutdown_connections 2024-05-21T03:29:07.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:07.390+0000 7f02f68fa700 1 -- 172.21.15.27:0/1237598922 wait complete. 2024-05-21T03:29:08.222 DEBUG:teuthology.orchestra.run.smithi165:> mktemp 2024-05-21T03:29:08.241 INFO:teuthology.orchestra.run.smithi165.stdout:/tmp/tmp.Te3w3xbyB2 2024-05-21T03:29:08.242 DEBUG:teuthology.orchestra.run.smithi165:> sudo cp /var/run/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ops-72cf81f5-1-mds.f.json /tmp/tmp.Te3w3xbyB2 2024-05-21T03:29:08.314 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 0666 /tmp/tmp.Te3w3xbyB2 2024-05-21T03:29:08.444 DEBUG:teuthology.orchestra.remote:smithi165:/tmp/tmp.Te3w3xbyB2 is 316KB 2024-05-21T03:29:08.540 DEBUG:teuthology.orchestra.run.smithi165:> rm -fr /tmp/tmp.Te3w3xbyB2 2024-05-21T03:29:08.557 DEBUG:tasks.quiescer.fs.[cephfs]:read 323886B of ops from '/var/run/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ops-72cf81f5-1-mds.f.json' on mds.4 (f) 2024-05-21T03:29:08.616 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 4 (f) for 72cf81f5-1: 'peer_request:mds.0:118' 2024-05-21T03:29:08.617 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 4 (f) for 72cf81f5-1: 'peer_request:mds.0:119' 2024-05-21T03:29:08.618 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 4 (f) for 72cf81f5-1: 'peer_request:mds.0:120' 2024-05-21T03:29:08.619 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 4 (f) for 72cf81f5-1: 'peer_request:mds.0:121' 2024-05-21T03:29:08.619 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 4 (f) for 72cf81f5-1: 'peer_request:mds.0:122' 2024-05-21T03:29:08.620 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 4 (f) for 72cf81f5-1: 'peer_request:mds.0:123' 2024-05-21T03:29:08.621 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 4 (f) for 72cf81f5-1: 'peer_request:mds.0:124' 2024-05-21T03:29:08.622 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 4 (f) for 72cf81f5-1: 'peer_request:mds.0:125' 2024-05-21T03:29:08.623 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 4 (f) for 72cf81f5-1: 'peer_request:mds.0:126' 2024-05-21T03:29:08.624 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 4 (f) for 72cf81f5-1: 'peer_request:mds.0:127' 2024-05-21T03:29:08.625 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 4 (f) for 72cf81f5-1: 'peer_request:mds.0:128' 2024-05-21T03:29:08.626 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 4 (f) for 72cf81f5-1: 'peer_request:mds.0:129' 2024-05-21T03:29:08.626 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 4 (f) for 72cf81f5-1: 'peer_request:mds.0:213' 2024-05-21T03:29:08.627 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 4 (f) for 72cf81f5-1: 'peer_request:mds.0:214' 2024-05-21T03:29:08.628 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 4 (f) for 72cf81f5-1: 'peer_request:mds.0:215' 2024-05-21T03:29:08.629 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 4 (f) for 72cf81f5-1: 'peer_request:mds.0:216' 2024-05-21T03:29:08.630 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 4 (f) for 72cf81f5-1: 'peer_request:mds.0:217' 2024-05-21T03:29:08.631 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 4 (f) for 72cf81f5-1: 'peer_request:mds.0:218' 2024-05-21T03:29:08.632 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 4 (f) for 72cf81f5-1: 'peer_request:mds.0:219' 2024-05-21T03:29:08.633 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 4 (f) for 72cf81f5-1: 'peer_request:mds.0:220' 2024-05-21T03:29:08.634 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 4 (f) for 72cf81f5-1: 'peer_request:mds.0:221' 2024-05-21T03:29:08.635 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 4 (f) for 72cf81f5-1: 'peer_request:mds.0:222' 2024-05-21T03:29:08.636 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 4 (f) for 72cf81f5-1: 'peer_request:mds.0:223' 2024-05-21T03:29:08.637 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 4 (f) for 72cf81f5-1: 'peer_request:mds.0:224' 2024-05-21T03:29:08.640 INFO:tasks.quiescer.fs.[cephfs]:Pulled 138 ops from rank 4 (f) into home/teuthworker/archive/pdonnell-2024-05-21_01:33:07-fs:workload-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7717852/ops-72cf81f5-1-mds.f.json 2024-05-21T03:29:08.640 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -- /var/run/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ops-72cf81f5-1-mds.f.json 2024-05-21T03:29:08.666 DEBUG:teuthology.orchestra.run.smithi039:> mktemp 2024-05-21T03:29:08.685 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/tmp.baNV9YHfy9 2024-05-21T03:29:08.686 DEBUG:teuthology.orchestra.run.smithi039:> sudo cp /var/run/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ops-72cf81f5-1-mds.h.json /tmp/tmp.baNV9YHfy9 2024-05-21T03:29:08.760 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0666 /tmp/tmp.baNV9YHfy9 2024-05-21T03:29:08.888 DEBUG:teuthology.orchestra.remote:smithi039:/tmp/tmp.baNV9YHfy9 is 325KB 2024-05-21T03:29:08.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:08 smithi027 ceph-mon[32179]: pgmap v156: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.3 MiB/s wr, 600 op/s 2024-05-21T03:29:08.956 DEBUG:teuthology.orchestra.run.smithi039:> rm -fr /tmp/tmp.baNV9YHfy9 2024-05-21T03:29:08.972 DEBUG:tasks.quiescer.fs.[cephfs]:read 332823B of ops from '/var/run/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ops-72cf81f5-1-mds.h.json' on mds.3 (h) 2024-05-21T03:29:08.979 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:199' 2024-05-21T03:29:08.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:200' 2024-05-21T03:29:08.981 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:201' 2024-05-21T03:29:08.982 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:202' 2024-05-21T03:29:08.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:203' 2024-05-21T03:29:08.985 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:204' 2024-05-21T03:29:08.986 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:205' 2024-05-21T03:29:08.987 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:206' 2024-05-21T03:29:08.988 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:207' 2024-05-21T03:29:08.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:208' 2024-05-21T03:29:08.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:209' 2024-05-21T03:29:08.991 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:210' 2024-05-21T03:29:08.992 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:211' 2024-05-21T03:29:08.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:212' 2024-05-21T03:29:08.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:143' 2024-05-21T03:29:08.996 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:144' 2024-05-21T03:29:08.997 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:145' 2024-05-21T03:29:08.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:146' 2024-05-21T03:29:08.999 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:147' 2024-05-21T03:29:09.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:148' 2024-05-21T03:29:09.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:149' 2024-05-21T03:29:09.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:150' 2024-05-21T03:29:09.003 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:151' 2024-05-21T03:29:09.004 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:152' 2024-05-21T03:29:09.005 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:153' 2024-05-21T03:29:09.006 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:154' 2024-05-21T03:29:09.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:155' 2024-05-21T03:29:09.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:195' 2024-05-21T03:29:09.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:196' 2024-05-21T03:29:09.010 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:197' 2024-05-21T03:29:09.011 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 3 (h) for 72cf81f5-1: 'peer_request:mds.0:198' 2024-05-21T03:29:09.011 INFO:tasks.quiescer.fs.[cephfs]:Pulled 146 ops from rank 3 (h) into home/teuthworker/archive/pdonnell-2024-05-21_01:33:07-fs:workload-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7717852/ops-72cf81f5-1-mds.h.json 2024-05-21T03:29:09.012 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -- /var/run/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ops-72cf81f5-1-mds.h.json 2024-05-21T03:29:09.021 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:08 smithi039 ceph-mon[36429]: pgmap v156: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.3 MiB/s wr, 600 op/s 2024-05-21T03:29:09.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:08 smithi165 ceph-mon[36269]: pgmap v156: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.3 MiB/s wr, 600 op/s 2024-05-21T03:29:09.038 DEBUG:teuthology.orchestra.run.smithi165:> mktemp 2024-05-21T03:29:09.057 INFO:teuthology.orchestra.run.smithi165.stdout:/tmp/tmp.kl0p8Gl1L4 2024-05-21T03:29:09.057 DEBUG:teuthology.orchestra.run.smithi165:> sudo cp /var/run/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ops-72cf81f5-1-mds.c.json /tmp/tmp.kl0p8Gl1L4 2024-05-21T03:29:09.125 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 0666 /tmp/tmp.kl0p8Gl1L4 2024-05-21T03:29:09.253 DEBUG:teuthology.orchestra.remote:smithi165:/tmp/tmp.kl0p8Gl1L4 is 373KB 2024-05-21T03:29:09.320 DEBUG:teuthology.orchestra.run.smithi165:> rm -fr /tmp/tmp.kl0p8Gl1L4 2024-05-21T03:29:09.335 DEBUG:tasks.quiescer.fs.[cephfs]:read 381508B of ops from '/var/run/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ops-72cf81f5-1-mds.c.json' on mds.2 (c) 2024-05-21T03:29:09.342 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:169' 2024-05-21T03:29:09.343 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:170' 2024-05-21T03:29:09.344 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:171' 2024-05-21T03:29:09.345 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:172' 2024-05-21T03:29:09.347 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:173' 2024-05-21T03:29:09.348 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:174' 2024-05-21T03:29:09.349 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:175' 2024-05-21T03:29:09.350 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:176' 2024-05-21T03:29:09.351 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:177' 2024-05-21T03:29:09.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:178' 2024-05-21T03:29:09.353 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:179' 2024-05-21T03:29:09.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:180' 2024-05-21T03:29:09.356 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:181' 2024-05-21T03:29:09.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:182' 2024-05-21T03:29:09.358 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:183' 2024-05-21T03:29:09.359 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:184' 2024-05-21T03:29:09.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:185' 2024-05-21T03:29:09.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:186' 2024-05-21T03:29:09.362 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:187' 2024-05-21T03:29:09.363 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:188' 2024-05-21T03:29:09.365 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:189' 2024-05-21T03:29:09.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:190' 2024-05-21T03:29:09.367 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:191' 2024-05-21T03:29:09.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:192' 2024-05-21T03:29:09.369 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:193' 2024-05-21T03:29:09.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:162' 2024-05-21T03:29:09.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:194' 2024-05-21T03:29:09.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:163' 2024-05-21T03:29:09.372 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:164' 2024-05-21T03:29:09.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:165' 2024-05-21T03:29:09.375 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:166' 2024-05-21T03:29:09.376 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:167' 2024-05-21T03:29:09.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (c) for 72cf81f5-1: 'peer_request:mds.0:168' 2024-05-21T03:29:09.378 INFO:tasks.quiescer.fs.[cephfs]:Pulled 164 ops from rank 2 (c) into home/teuthworker/archive/pdonnell-2024-05-21_01:33:07-fs:workload-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7717852/ops-72cf81f5-1-mds.c.json 2024-05-21T03:29:09.378 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -- /var/run/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ops-72cf81f5-1-mds.c.json 2024-05-21T03:29:09.404 DEBUG:teuthology.orchestra.run.smithi165:> mktemp 2024-05-21T03:29:09.459 INFO:teuthology.orchestra.run.smithi165.stdout:/tmp/tmp.FKo7bc2tdR 2024-05-21T03:29:09.459 DEBUG:teuthology.orchestra.run.smithi165:> sudo cp /var/run/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ops-72cf81f5-1-mds.i.json /tmp/tmp.FKo7bc2tdR 2024-05-21T03:29:09.525 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 0666 /tmp/tmp.FKo7bc2tdR 2024-05-21T03:29:09.649 DEBUG:teuthology.orchestra.remote:smithi165:/tmp/tmp.FKo7bc2tdR is 910KB 2024-05-21T03:29:09.751 DEBUG:teuthology.orchestra.run.smithi165:> rm -fr /tmp/tmp.FKo7bc2tdR 2024-05-21T03:29:09.766 DEBUG:tasks.quiescer.fs.[cephfs]:read 932147B of ops from '/var/run/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ops-72cf81f5-1-mds.i.json' on mds.1 (i) 2024-05-21T03:29:09.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:51' 2024-05-21T03:29:09.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:73' 2024-05-21T03:29:09.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:80' 2024-05-21T03:29:09.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:94' 2024-05-21T03:29:09.782 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:108' 2024-05-21T03:29:09.782 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:114' 2024-05-21T03:29:09.782 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:126' 2024-05-21T03:29:09.783 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:142' 2024-05-21T03:29:09.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:60' 2024-05-21T03:29:09.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:65' 2024-05-21T03:29:09.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:89' 2024-05-21T03:29:09.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:91' 2024-05-21T03:29:09.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:114' 2024-05-21T03:29:09.786 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:120' 2024-05-21T03:29:09.787 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:78' 2024-05-21T03:29:09.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:74' 2024-05-21T03:29:09.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:84' 2024-05-21T03:29:09.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:100' 2024-05-21T03:29:09.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:127' 2024-05-21T03:29:09.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:143' 2024-05-21T03:29:09.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:52' 2024-05-21T03:29:09.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:66' 2024-05-21T03:29:09.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:81' 2024-05-21T03:29:09.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:95' 2024-05-21T03:29:09.792 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:115' 2024-05-21T03:29:09.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:61' 2024-05-21T03:29:09.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:75' 2024-05-21T03:29:09.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:90' 2024-05-21T03:29:09.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:92' 2024-05-21T03:29:09.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:121' 2024-05-21T03:29:09.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:128' 2024-05-21T03:29:09.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:144' 2024-05-21T03:29:09.798 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:79' 2024-05-21T03:29:09.798 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:67' 2024-05-21T03:29:09.798 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:85' 2024-05-21T03:29:09.798 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:101' 2024-05-21T03:29:09.800 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:27' 2024-05-21T03:29:09.801 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:53' 2024-05-21T03:29:09.801 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:76' 2024-05-21T03:29:09.802 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:82' 2024-05-21T03:29:09.802 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:96' 2024-05-21T03:29:09.803 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:116' 2024-05-21T03:29:09.803 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:129' 2024-05-21T03:29:09.803 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:145' 2024-05-21T03:29:09.803 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:28' 2024-05-21T03:29:09.804 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:62' 2024-05-21T03:29:09.804 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:68' 2024-05-21T03:29:09.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:91' 2024-05-21T03:29:09.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:93' 2024-05-21T03:29:09.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:122' 2024-05-21T03:29:09.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:29' 2024-05-21T03:29:09.808 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:80' 2024-05-21T03:29:09.808 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:77' 2024-05-21T03:29:09.808 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:86' 2024-05-21T03:29:09.808 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:102' 2024-05-21T03:29:09.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:130' 2024-05-21T03:29:09.810 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:30' 2024-05-21T03:29:09.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:54' 2024-05-21T03:29:09.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:69' 2024-05-21T03:29:09.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:83' 2024-05-21T03:29:09.812 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:97' 2024-05-21T03:29:09.812 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:117' 2024-05-21T03:29:09.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:31' 2024-05-21T03:29:09.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:27' 2024-05-21T03:29:09.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:63' 2024-05-21T03:29:09.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:78' 2024-05-21T03:29:09.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:92' 2024-05-21T03:29:09.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:94' 2024-05-21T03:29:09.816 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:123' 2024-05-21T03:29:09.816 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:131' 2024-05-21T03:29:09.817 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:48' 2024-05-21T03:29:09.817 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:81' 2024-05-21T03:29:09.817 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:70' 2024-05-21T03:29:09.818 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:87' 2024-05-21T03:29:09.818 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:103' 2024-05-21T03:29:09.820 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:32' 2024-05-21T03:29:09.820 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:27' 2024-05-21T03:29:09.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:55' 2024-05-21T03:29:09.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:79' 2024-05-21T03:29:09.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:84' 2024-05-21T03:29:09.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:98' 2024-05-21T03:29:09.822 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:118' 2024-05-21T03:29:09.822 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:132' 2024-05-21T03:29:09.823 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:33' 2024-05-21T03:29:09.824 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:64' 2024-05-21T03:29:09.824 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:71' 2024-05-21T03:29:09.824 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:93' 2024-05-21T03:29:09.824 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:104' 2024-05-21T03:29:09.825 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:124' 2024-05-21T03:29:09.826 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:34' 2024-05-21T03:29:09.827 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:28' 2024-05-21T03:29:09.827 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:82' 2024-05-21T03:29:09.827 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:80' 2024-05-21T03:29:09.827 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:88' 2024-05-21T03:29:09.828 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:99' 2024-05-21T03:29:09.828 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:110' 2024-05-21T03:29:09.829 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:35' 2024-05-21T03:29:09.830 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:49' 2024-05-21T03:29:09.830 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:56' 2024-05-21T03:29:09.831 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:72' 2024-05-21T03:29:09.831 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:85' 2024-05-21T03:29:09.831 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:105' 2024-05-21T03:29:09.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:127' 2024-05-21T03:29:09.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:119' 2024-05-21T03:29:09.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:133' 2024-05-21T03:29:09.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:36' 2024-05-21T03:29:09.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:28' 2024-05-21T03:29:09.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:65' 2024-05-21T03:29:09.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:81' 2024-05-21T03:29:09.834 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:94' 2024-05-21T03:29:09.834 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:100' 2024-05-21T03:29:09.834 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:116' 2024-05-21T03:29:09.835 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:125' 2024-05-21T03:29:09.835 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:37' 2024-05-21T03:29:09.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:29' 2024-05-21T03:29:09.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:57' 2024-05-21T03:29:09.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:73' 2024-05-21T03:29:09.837 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:89' 2024-05-21T03:29:09.837 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:106' 2024-05-21T03:29:09.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:134' 2024-05-21T03:29:09.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:38' 2024-05-21T03:29:09.839 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:50' 2024-05-21T03:29:09.839 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:66' 2024-05-21T03:29:09.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:82' 2024-05-21T03:29:09.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:86' 2024-05-21T03:29:09.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:101' 2024-05-21T03:29:09.841 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:120' 2024-05-21T03:29:09.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:39' 2024-05-21T03:29:09.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:59' 2024-05-21T03:29:09.843 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:58' 2024-05-21T03:29:09.843 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:74' 2024-05-21T03:29:09.843 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:95' 2024-05-21T03:29:09.843 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:107' 2024-05-21T03:29:09.844 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:111' 2024-05-21T03:29:09.844 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:126' 2024-05-21T03:29:09.844 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:135' 2024-05-21T03:29:09.845 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:40' 2024-05-21T03:29:09.846 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:67' 2024-05-21T03:29:09.846 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:83' 2024-05-21T03:29:09.846 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:90' 2024-05-21T03:29:09.846 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:102' 2024-05-21T03:29:09.847 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:128' 2024-05-21T03:29:09.847 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:136' 2024-05-21T03:29:09.848 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:41' 2024-05-21T03:29:09.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:59' 2024-05-21T03:29:09.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:75' 2024-05-21T03:29:09.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:87' 2024-05-21T03:29:09.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:108' 2024-05-21T03:29:09.850 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:117' 2024-05-21T03:29:09.850 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:121' 2024-05-21T03:29:09.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:42' 2024-05-21T03:29:09.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:47' 2024-05-21T03:29:09.852 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:68' 2024-05-21T03:29:09.852 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:84' 2024-05-21T03:29:09.852 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:96' 2024-05-21T03:29:09.852 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:103' 2024-05-21T03:29:09.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:127' 2024-05-21T03:29:09.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:137' 2024-05-21T03:29:09.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:43' 2024-05-21T03:29:09.857 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:26' 2024-05-21T03:29:09.858 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:60' 2024-05-21T03:29:09.858 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:76' 2024-05-21T03:29:09.858 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:91' 2024-05-21T03:29:09.858 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:109' 2024-05-21T03:29:09.859 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:112' 2024-05-21T03:29:09.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:44' 2024-05-21T03:29:09.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:69' 2024-05-21T03:29:09.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:85' 2024-05-21T03:29:09.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:88' 2024-05-21T03:29:09.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:104' 2024-05-21T03:29:09.863 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:118' 2024-05-21T03:29:09.863 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:122' 2024-05-21T03:29:09.863 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:138' 2024-05-21T03:29:09.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:45' 2024-05-21T03:29:09.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:61' 2024-05-21T03:29:09.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:77' 2024-05-21T03:29:09.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:97' 2024-05-21T03:29:09.866 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:110' 2024-05-21T03:29:09.867 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:46' 2024-05-21T03:29:09.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:70' 2024-05-21T03:29:09.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:86' 2024-05-21T03:29:09.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:92' 2024-05-21T03:29:09.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:105' 2024-05-21T03:29:09.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:123' 2024-05-21T03:29:09.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:139' 2024-05-21T03:29:09.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:62' 2024-05-21T03:29:09.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:78' 2024-05-21T03:29:09.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:89' 2024-05-21T03:29:09.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:111' 2024-05-21T03:29:09.876 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:71' 2024-05-21T03:29:09.876 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:87' 2024-05-21T03:29:09.876 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:98' 2024-05-21T03:29:09.876 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:106' 2024-05-21T03:29:09.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:113' 2024-05-21T03:29:09.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:124' 2024-05-21T03:29:09.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:140' 2024-05-21T03:29:09.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:63' 2024-05-21T03:29:09.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:79' 2024-05-21T03:29:09.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:93' 2024-05-21T03:29:09.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:112' 2024-05-21T03:29:09.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:109' 2024-05-21T03:29:09.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:129' 2024-05-21T03:29:09.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:72' 2024-05-21T03:29:09.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:88' 2024-05-21T03:29:09.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:90' 2024-05-21T03:29:09.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:107' 2024-05-21T03:29:09.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:126' 2024-05-21T03:29:09.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:119' 2024-05-21T03:29:09.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:125' 2024-05-21T03:29:09.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:141' 2024-05-21T03:29:09.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.3:64' 2024-05-21T03:29:09.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.2:83' 2024-05-21T03:29:09.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:99' 2024-05-21T03:29:09.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:113' 2024-05-21T03:29:09.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (i) for 72cf81f5-1: 'peer_request:mds.4:115' 2024-05-21T03:29:09.889 INFO:tasks.quiescer.fs.[cephfs]:Pulled 497 ops from rank 1 (i) into home/teuthworker/archive/pdonnell-2024-05-21_01:33:07-fs:workload-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7717852/ops-72cf81f5-1-mds.i.json 2024-05-21T03:29:09.889 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -- /var/run/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ops-72cf81f5-1-mds.i.json 2024-05-21T03:29:09.913 DEBUG:teuthology.orchestra.run.smithi039:> mktemp 2024-05-21T03:29:09.932 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/tmp.7XeRICnqBA 2024-05-21T03:29:09.932 DEBUG:teuthology.orchestra.run.smithi039:> sudo cp /var/run/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ops-72cf81f5-1-mds.b.json /tmp/tmp.7XeRICnqBA 2024-05-21T03:29:10.002 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0666 /tmp/tmp.7XeRICnqBA 2024-05-21T03:29:10.130 DEBUG:teuthology.orchestra.remote:smithi039:/tmp/tmp.7XeRICnqBA is 370KB 2024-05-21T03:29:10.227 DEBUG:teuthology.orchestra.run.smithi039:> rm -fr /tmp/tmp.7XeRICnqBA 2024-05-21T03:29:10.244 DEBUG:tasks.quiescer.fs.[cephfs]:read 379100B of ops from '/var/run/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ops-72cf81f5-1-mds.b.json' on mds.0 (b) 2024-05-21T03:29:10.252 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.2:16' 2024-05-21T03:29:10.252 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.1:21' 2024-05-21T03:29:10.253 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.4:15' 2024-05-21T03:29:10.253 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.2:22' 2024-05-21T03:29:10.254 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.3:17' 2024-05-21T03:29:10.254 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.4:20' 2024-05-21T03:29:10.255 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.1:16' 2024-05-21T03:29:10.256 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.3:23' 2024-05-21T03:29:10.257 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.2:17' 2024-05-21T03:29:10.258 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.1:22' 2024-05-21T03:29:10.259 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.4:16' 2024-05-21T03:29:10.259 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.2:23' 2024-05-21T03:29:10.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.1:17' 2024-05-21T03:29:10.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.3:24' 2024-05-21T03:29:10.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.2:18' 2024-05-21T03:29:10.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.4:21' 2024-05-21T03:29:10.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.3:18' 2024-05-21T03:29:10.264 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.3:25' 2024-05-21T03:29:10.267 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.3:19' 2024-05-21T03:29:10.267 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.1:23' 2024-05-21T03:29:10.268 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.4:17' 2024-05-21T03:29:10.268 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.2:24' 2024-05-21T03:29:10.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.1:18' 2024-05-21T03:29:10.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.4:22' 2024-05-21T03:29:10.270 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.2:19' 2024-05-21T03:29:10.271 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.3:20' 2024-05-21T03:29:10.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.1:19' 2024-05-21T03:29:10.273 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.1:24' 2024-05-21T03:29:10.273 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.2:20' 2024-05-21T03:29:10.274 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.2:25' 2024-05-21T03:29:10.275 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.3:26' 2024-05-21T03:29:10.276 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.4:23' 2024-05-21T03:29:10.277 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.3:21' 2024-05-21T03:29:10.277 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.1:25' 2024-05-21T03:29:10.278 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.4:18' 2024-05-21T03:29:10.278 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.2:26' 2024-05-21T03:29:10.279 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.1:20' 2024-05-21T03:29:10.279 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.4:24' 2024-05-21T03:29:10.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.2:21' 2024-05-21T03:29:10.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.4:25' 2024-05-21T03:29:10.281 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.4:19' 2024-05-21T03:29:10.284 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.3:22' 2024-05-21T03:29:10.285 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.3:16' 2024-05-21T03:29:10.287 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 72cf81f5-1: 'peer_request:mds.1:15' 2024-05-21T03:29:10.289 INFO:tasks.quiescer.fs.[cephfs]:Pulled 169 ops from rank 0 (b) into home/teuthworker/archive/pdonnell-2024-05-21_01:33:07-fs:workload-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7717852/ops-72cf81f5-1-mds.b.json 2024-05-21T03:29:10.289 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -- /var/run/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ops-72cf81f5-1-mds.b.json 2024-05-21T03:29:10.316 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:29:10.587 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:29:10.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:10 smithi027 ceph-mon[32179]: pgmap v157: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 3.5 MiB/s wr, 497 op/s 2024-05-21T03:29:10.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:10 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:29:11.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:10 smithi039 ceph-mon[36429]: pgmap v157: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 3.5 MiB/s wr, 497 op/s 2024-05-21T03:29:11.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:10 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:29:11.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:10 smithi165 ceph-mon[36269]: pgmap v157: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 3.5 MiB/s wr, 497 op/s 2024-05-21T03:29:11.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:10 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:29:11.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:11 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:29:11.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:11 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:29:11.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:11 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:29:12.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:11 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:29:12.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:11 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:29:12.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:11 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:29:12.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:11 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:29:12.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:11 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:29:12.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:11 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:29:12.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.322+0000 7f66ecae0700 1 -- 172.21.15.27:0/2037356430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66e8108600 msgr2=0x7f66e81089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:12.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.322+0000 7f66ecae0700 1 --2- 172.21.15.27:0/2037356430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66e8108600 0x7f66e81089e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009990 tx=0x7f66d8009ca0 comp rx=0 tx=0).stop 2024-05-21T03:29:12.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.322+0000 7f66ecae0700 1 -- 172.21.15.27:0/2037356430 shutdown_connections 2024-05-21T03:29:12.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.322+0000 7f66ecae0700 1 --2- 172.21.15.27:0/2037356430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66e81028a0 0x7f66e8110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:12.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.322+0000 7f66ecae0700 1 --2- 172.21.15.27:0/2037356430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66e8108600 0x7f66e81089e0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:12.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.322+0000 7f66ecae0700 1 --2- 172.21.15.27:0/2037356430 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66e8101ee0 0x7f66e8102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:12.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.322+0000 7f66ecae0700 1 -- 172.21.15.27:0/2037356430 >> 172.21.15.27:0/2037356430 conn(0x7f66e8077760 msgr2=0x7f66e8077b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:12.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.322+0000 7f66ecae0700 1 -- 172.21.15.27:0/2037356430 shutdown_connections 2024-05-21T03:29:12.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.322+0000 7f66ecae0700 1 -- 172.21.15.27:0/2037356430 wait complete. 2024-05-21T03:29:12.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.323+0000 7f66ecae0700 1 Processor -- start 2024-05-21T03:29:12.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.323+0000 7f66ecae0700 1 -- start start 2024-05-21T03:29:12.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.323+0000 7f66ecae0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66e8101ee0 0x7f66e819be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:12.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.323+0000 7f66ecae0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66e81028a0 0x7f66e819c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:12.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.323+0000 7f66ecae0700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66e8108600 0x7f66e81a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:12.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.323+0000 7f66ecae0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f66e8113790 con 0x7f66e8101ee0 2024-05-21T03:29:12.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.323+0000 7f66ecae0700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f66e8113610 con 0x7f66e81028a0 2024-05-21T03:29:12.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.323+0000 7f66ecae0700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f66e8113910 con 0x7f66e8108600 2024-05-21T03:29:12.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.323+0000 7f66e659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66e8101ee0 0x7f66e819be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:12.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.323+0000 7f66e659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66e8101ee0 0x7f66e819be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44960/0 (socket says 172.21.15.27:44960) 2024-05-21T03:29:12.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.323+0000 7f66e659c700 1 -- 172.21.15.27:0/2491552279 learned_addr learned my addr 172.21.15.27:0/2491552279 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:12.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.323+0000 7f66e6d9d700 1 --2- 172.21.15.27:0/2491552279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66e8108600 0x7f66e81a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:12.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.323+0000 7f66e5d9b700 1 --2- 172.21.15.27:0/2491552279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66e81028a0 0x7f66e819c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:12.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.324+0000 7f66e659c700 1 -- 172.21.15.27:0/2491552279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66e8108600 msgr2=0x7f66e81a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:12.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.324+0000 7f66e659c700 1 --2- 172.21.15.27:0/2491552279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66e8108600 0x7f66e81a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:12.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.324+0000 7f66e659c700 1 -- 172.21.15.27:0/2491552279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66e81028a0 msgr2=0x7f66e819c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:12.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.324+0000 7f66e659c700 1 --2- 172.21.15.27:0/2491552279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66e81028a0 0x7f66e819c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:12.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.324+0000 7f66e659c700 1 -- 172.21.15.27:0/2491552279 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e81a0e70 con 0x7f66e8101ee0 2024-05-21T03:29:12.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.324+0000 7f66e5d9b700 1 --2- 172.21.15.27:0/2491552279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66e81028a0 0x7f66e819c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:29:12.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.324+0000 7f66e659c700 1 --2- 172.21.15.27:0/2491552279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66e8101ee0 0x7f66e819be60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f66d8000c00 tx=0x7f66d8005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:12.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.324+0000 7f66d77fe700 1 -- 172.21.15.27:0/2491552279 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d8027070 con 0x7f66e8101ee0 2024-05-21T03:29:12.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.324+0000 7f66ecae0700 1 -- 172.21.15.27:0/2491552279 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e81a1100 con 0x7f66e8101ee0 2024-05-21T03:29:12.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.324+0000 7f66d77fe700 1 -- 172.21.15.27:0/2491552279 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66d8003f20 con 0x7f66e8101ee0 2024-05-21T03:29:12.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.324+0000 7f66d77fe700 1 -- 172.21.15.27:0/2491552279 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d8022c70 con 0x7f66e8101ee0 2024-05-21T03:29:12.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.324+0000 7f66ecae0700 1 -- 172.21.15.27:0/2491552279 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e81a89e0 con 0x7f66e8101ee0 2024-05-21T03:29:12.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.325+0000 7f66ecae0700 1 -- 172.21.15.27:0/2491552279 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66e806a230 con 0x7f66e8101ee0 2024-05-21T03:29:12.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.326+0000 7f66d77fe700 1 -- 172.21.15.27:0/2491552279 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7f66d802a070 con 0x7f66e8101ee0 2024-05-21T03:29:12.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.327+0000 7f66d77fe700 1 --2- 172.21.15.27:0/2491552279 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f66d00b29f0 0x7f66d00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:12.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.328+0000 7f66e5d9b700 1 --2- 172.21.15.27:0/2491552279 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f66d00b29f0 0x7f66d00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:12.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.328+0000 7f66d77fe700 1 -- 172.21.15.27:0/2491552279 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f66d80e3d10 con 0x7f66e8101ee0 2024-05-21T03:29:12.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.328+0000 7f66e5d9b700 1 --2- 172.21.15.27:0/2491552279 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f66d00b29f0 0x7f66d00b4eb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f66dc005fd0 tx=0x7f66dc005dc0 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:12.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.331+0000 7f66d77fe700 1 -- 172.21.15.27:0/2491552279 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f66d8031020 con 0x7f66e8101ee0 2024-05-21T03:29:12.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.593+0000 7f66ecae0700 1 -- 172.21.15.27:0/2491552279 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66e81a8bb0 con 0x7f66e8101ee0 2024-05-21T03:29:12.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.594+0000 7f66d77fe700 1 -- 172.21.15.27:0/2491552279 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11932 (secure 0 0 0) 0x7f66d8013070 con 0x7f66e8101ee0 2024-05-21T03:29:12.593 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:29:12.594 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/148854869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":148854869},{"type":"v1","addr":"172.21.15.27:6835","nonce":148854869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/4145544730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":4145544730},{"type":"v1","addr":"172.21.15.27:6837","nonce":4145544730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T03:20:34.055985+0000","modified":"2024-05-21T03:28:59.351119+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24472,"mds_2":24457,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/745486764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":745486764},{"type":"v1","addr":"172.21.15.27:6839","nonce":745486764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6841/3982752351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3982752351},{"type":"v1","addr":"172.21.15.27:6841","nonce":3982752351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/3497521608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3497521608},{"type":"v1","addr":"172.21.15.39:6835","nonce":3497521608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.165:6835/3576910218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3576910218},{"type":"v1","addr":"172.21.15.165:6835","nonce":3576910218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/3673846852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3673846852},{"type":"v1","addr":"172.21.15.39:6837","nonce":3673846852}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6837/691356139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":691356139},{"type":"v1","addr":"172.21.15.165:6837","nonce":691356139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6839/4158170814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4158170814},{"type":"v1","addr":"172.21.15.165:6839","nonce":4158170814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6839/287607819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":287607819},{"type":"v1","addr":"172.21.15.39:6839","nonce":287607819}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24472,24451,24452,24476,24457]},"id":1}]} 2024-05-21T03:29:12.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.596+0000 7f66ecae0700 1 -- 172.21.15.27:0/2491552279 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f66d00b29f0 msgr2=0x7f66d00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:12.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.596+0000 7f66ecae0700 1 --2- 172.21.15.27:0/2491552279 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f66d00b29f0 0x7f66d00b4eb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f66dc005fd0 tx=0x7f66dc005dc0 comp rx=0 tx=0).stop 2024-05-21T03:29:12.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.596+0000 7f66ecae0700 1 -- 172.21.15.27:0/2491552279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66e8101ee0 msgr2=0x7f66e819be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:12.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.596+0000 7f66ecae0700 1 --2- 172.21.15.27:0/2491552279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66e8101ee0 0x7f66e819be60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f66d8000c00 tx=0x7f66d8005ef0 comp rx=0 tx=0).stop 2024-05-21T03:29:12.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.596+0000 7f66ecae0700 1 -- 172.21.15.27:0/2491552279 shutdown_connections 2024-05-21T03:29:12.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.596+0000 7f66ecae0700 1 --2- 172.21.15.27:0/2491552279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66e8108600 0x7f66e81a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:12.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.596+0000 7f66ecae0700 1 --2- 172.21.15.27:0/2491552279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66e8101ee0 0x7f66e819be60 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:12.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.596+0000 7f66ecae0700 1 --2- 172.21.15.27:0/2491552279 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f66d00b29f0 0x7f66d00b4eb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:12.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.596+0000 7f66ecae0700 1 --2- 172.21.15.27:0/2491552279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66e81028a0 0x7f66e819c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:12.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.596+0000 7f66ecae0700 1 -- 172.21.15.27:0/2491552279 >> 172.21.15.27:0/2491552279 conn(0x7f66e8077760 msgr2=0x7f66e8100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:12.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.596+0000 7f66ecae0700 1 -- 172.21.15.27:0/2491552279 shutdown_connections 2024-05-21T03:29:12.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:12.596+0000 7f66ecae0700 1 -- 172.21.15.27:0/2491552279 wait complete. 2024-05-21T03:29:12.597 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 34 2024-05-21T03:29:12.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:12 smithi027 ceph-mon[32179]: pgmap v158: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-21T03:29:12.911 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:12 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2491552279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:29:13.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:12 smithi039 ceph-mon[36429]: pgmap v158: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-21T03:29:13.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:12 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/2491552279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:29:13.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:12 smithi165 ceph-mon[36269]: pgmap v158: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-21T03:29:13.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:12 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/2491552279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:29:13.278 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 72cf81f5 --await-for 45.0' 2024-05-21T03:29:13.278 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 72cf81f5 --await-for 45.0 2024-05-21T03:29:13.534 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:29:14.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:14 smithi027 ceph-mon[32179]: pgmap v159: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-05-21T03:29:15.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:14 smithi039 ceph-mon[36429]: pgmap v159: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-05-21T03:29:15.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:14 smithi165 ceph-mon[36269]: pgmap v159: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-05-21T03:29:15.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.230+0000 7f8c2e82e700 1 -- 172.21.15.27:0/4175168639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c2806bbe0 msgr2=0x7f8c281103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:15.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.230+0000 7f8c2e82e700 1 --2- 172.21.15.27:0/4175168639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c2806bbe0 0x7f8c281103d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000b0a0 tx=0x7f8c2000b3b0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.230+0000 7f8c2e82e700 1 -- 172.21.15.27:0/4175168639 shutdown_connections 2024-05-21T03:29:15.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.230+0000 7f8c2e82e700 1 --2- 172.21.15.27:0/4175168639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c2806bbe0 0x7f8c281103d0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.230+0000 7f8c2e82e700 1 --2- 172.21.15.27:0/4175168639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c281095b0 0x7f8c28109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.230+0000 7f8c2e82e700 1 --2- 172.21.15.27:0/4175168639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c2806b220 0x7f8c2806b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.230+0000 7f8c2e82e700 1 -- 172.21.15.27:0/4175168639 >> 172.21.15.27:0/4175168639 conn(0x7f8c280793f0 msgr2=0x7f8c28079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:15.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.230+0000 7f8c2e82e700 1 -- 172.21.15.27:0/4175168639 shutdown_connections 2024-05-21T03:29:15.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.230+0000 7f8c2e82e700 1 -- 172.21.15.27:0/4175168639 wait complete. 2024-05-21T03:29:15.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.231+0000 7f8c2e82e700 1 Processor -- start 2024-05-21T03:29:15.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.231+0000 7f8c2e82e700 1 -- start start 2024-05-21T03:29:15.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.231+0000 7f8c2e82e700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c2806b220 0x7f8c28199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:15.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.231+0000 7f8c2e82e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c2806bbe0 0x7f8c2819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:15.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.231+0000 7f8c2e82e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c281095b0 0x7f8c2819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:15.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.231+0000 7f8c2e82e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8c281137c0 con 0x7f8c2806bbe0 2024-05-21T03:29:15.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.231+0000 7f8c2e82e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8c28113640 con 0x7f8c281095b0 2024-05-21T03:29:15.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.231+0000 7f8c2e82e700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8c28113940 con 0x7f8c2806b220 2024-05-21T03:29:15.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.231+0000 7f8c1ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c2806bbe0 0x7f8c2819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:15.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.231+0000 7f8c1ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c2806bbe0 0x7f8c2819a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37876/0 (socket says 172.21.15.27:37876) 2024-05-21T03:29:15.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.231+0000 7f8c1ffff700 1 -- 172.21.15.27:0/3008530357 learned_addr learned my addr 172.21.15.27:0/3008530357 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:15.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.231+0000 7f8c27fff700 1 --2- 172.21.15.27:0/3008530357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c2806b220 0x7f8c28199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:15.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.231+0000 7f8c2cdcb700 1 --2- 172.21.15.27:0/3008530357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c281095b0 0x7f8c2819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:15.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.232+0000 7f8c1ffff700 1 -- 172.21.15.27:0/3008530357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c2806b220 msgr2=0x7f8c28199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:15.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.232+0000 7f8c1ffff700 1 --2- 172.21.15.27:0/3008530357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c2806b220 0x7f8c28199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.232+0000 7f8c1ffff700 1 -- 172.21.15.27:0/3008530357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c281095b0 msgr2=0x7f8c2819e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:15.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.232+0000 7f8c1ffff700 1 --2- 172.21.15.27:0/3008530357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c281095b0 0x7f8c2819e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.232+0000 7f8c1ffff700 1 -- 172.21.15.27:0/3008530357 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c2819ed20 con 0x7f8c2806bbe0 2024-05-21T03:29:15.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.232+0000 7f8c1ffff700 1 --2- 172.21.15.27:0/3008530357 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c2806bbe0 0x7f8c2819a250 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8c1000b530 tx=0x7f8c1000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:15.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.232+0000 7f8c25ffb700 1 -- 172.21.15.27:0/3008530357 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c10015020 con 0x7f8c2806bbe0 2024-05-21T03:29:15.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.232+0000 7f8c25ffb700 1 -- 172.21.15.27:0/3008530357 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c10011ce0 con 0x7f8c2806bbe0 2024-05-21T03:29:15.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.232+0000 7f8c25ffb700 1 -- 172.21.15.27:0/3008530357 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c10010d20 con 0x7f8c2806bbe0 2024-05-21T03:29:15.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.232+0000 7f8c2e82e700 1 -- 172.21.15.27:0/3008530357 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c2819f010 con 0x7f8c2806bbe0 2024-05-21T03:29:15.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.232+0000 7f8c2e82e700 1 -- 172.21.15.27:0/3008530357 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c28102440 con 0x7f8c2806bbe0 2024-05-21T03:29:15.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.234+0000 7f8c25ffb700 1 -- 172.21.15.27:0/3008530357 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7f8c10018020 con 0x7f8c2806bbe0 2024-05-21T03:29:15.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.235+0000 7f8c25ffb700 1 --2- 172.21.15.27:0/3008530357 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f8c140b29f0 0x7f8c140b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:15.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.235+0000 7f8c27fff700 1 --2- 172.21.15.27:0/3008530357 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f8c140b29f0 0x7f8c140b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:15.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.235+0000 7f8c25ffb700 1 -- 172.21.15.27:0/3008530357 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f8c100e1d50 con 0x7f8c2806bbe0 2024-05-21T03:29:15.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.236+0000 7f8c390c4b80 1 Processor -- start 2024-05-21T03:29:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.236+0000 7f8c27fff700 1 --2- 172.21.15.27:0/3008530357 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f8c140b29f0 0x7f8c140b4eb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8c18006fd0 tx=0x7f8c18008040 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.236+0000 7f8c390c4b80 1 -- start start 2024-05-21T03:29:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.236+0000 7f8c390c4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5e7c7c720 0x55a5e7c7cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.236+0000 7f8c390c4b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e7c77bf0 0x55a5e7c78070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.236+0000 7f8c390c4b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e7c785b0 0x55a5e7c84620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.236+0000 7f8c390c4b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a5e7bd5e70 con 0x55a5e7c77bf0 2024-05-21T03:29:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.236+0000 7f8c390c4b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a5e7bc5a10 con 0x55a5e7c7c720 2024-05-21T03:29:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.236+0000 7f8c390c4b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a5e7bc9a50 con 0x55a5e7c785b0 2024-05-21T03:29:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.236+0000 7f8c1ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e7c77bf0 0x55a5e7c78070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.236+0000 7f8c1ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e7c77bf0 0x55a5e7c78070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37880/0 (socket says 172.21.15.27:37880) 2024-05-21T03:29:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.236+0000 7f8c1ffff700 1 -- 172.21.15.27:0/879026798 learned_addr learned my addr 172.21.15.27:0/879026798 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.236+0000 7f8c27fff700 1 --2- 172.21.15.27:0/879026798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5e7c7c720 0x55a5e7c7cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:15.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.236+0000 7f8c2cdcb700 1 --2- 172.21.15.27:0/879026798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e7c785b0 0x55a5e7c84620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:15.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.236+0000 7f8c1ffff700 1 -- 172.21.15.27:0/879026798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e7c785b0 msgr2=0x55a5e7c84620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:15.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.237+0000 7f8c1ffff700 1 --2- 172.21.15.27:0/879026798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e7c785b0 0x55a5e7c84620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.237+0000 7f8c1ffff700 1 -- 172.21.15.27:0/879026798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5e7c7c720 msgr2=0x55a5e7c7cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:15.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.237+0000 7f8c1ffff700 1 --2- 172.21.15.27:0/879026798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5e7c7c720 0x55a5e7c7cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.237+0000 7f8c1ffff700 1 -- 172.21.15.27:0/879026798 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5e7b6f0b0 con 0x55a5e7c77bf0 2024-05-21T03:29:15.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.237+0000 7f8c1ffff700 1 --2- 172.21.15.27:0/879026798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e7c77bf0 0x55a5e7c78070 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8c100a90d0 tx=0x7f8c100a5130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=839c431e01861337 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:15.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.237+0000 7f8c1e7fc700 1 -- 172.21.15.27:0/879026798 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c100c3070 con 0x55a5e7c77bf0 2024-05-21T03:29:15.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.237+0000 7f8c1e7fc700 1 -- 172.21.15.27:0/879026798 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c100a9140 con 0x55a5e7c77bf0 2024-05-21T03:29:15.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.237+0000 7f8c1e7fc700 1 -- 172.21.15.27:0/879026798 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c100a95a0 con 0x55a5e7c77bf0 2024-05-21T03:29:15.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.238+0000 7f8c390c4b80 1 -- 172.21.15.27:0/879026798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e7c77bf0 msgr2=0x55a5e7c78070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:15.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.238+0000 7f8c390c4b80 1 --2- 172.21.15.27:0/879026798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e7c77bf0 0x55a5e7c78070 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8c100a90d0 tx=0x7f8c100a5130 comp rx=0 tx=0).stop 2024-05-21T03:29:15.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.238+0000 7f8c390c4b80 1 -- 172.21.15.27:0/879026798 shutdown_connections 2024-05-21T03:29:15.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.238+0000 7f8c390c4b80 1 --2- 172.21.15.27:0/879026798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e7c785b0 0x55a5e7c84620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.238+0000 7f8c390c4b80 1 --2- 172.21.15.27:0/879026798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5e7c7c720 0x55a5e7c7cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.238+0000 7f8c390c4b80 1 --2- 172.21.15.27:0/879026798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e7c77bf0 0x55a5e7c78070 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.238+0000 7f8c390c4b80 1 -- 172.21.15.27:0/879026798 >> 172.21.15.27:0/879026798 conn(0x55a5e7ba3340 msgr2=0x55a5e7c728e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:15.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.238+0000 7f8c390c4b80 1 -- 172.21.15.27:0/879026798 shutdown_connections 2024-05-21T03:29:15.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.238+0000 7f8c390c4b80 1 -- 172.21.15.27:0/879026798 wait complete. 2024-05-21T03:29:15.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.238+0000 7f8c390c4b80 1 -- start start 2024-05-21T03:29:15.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.238+0000 7f8c390c4b80 1 Processor -- start 2024-05-21T03:29:15.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.239+0000 7f8c390c4b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e7c77bf0 0x55a5e7c843d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:15.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.239+0000 7f8c390c4b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e7c785b0 0x55a5e7c816d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:15.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.239+0000 7f8c390c4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5e7c7c720 0x55a5e7c81c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:15.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.239+0000 7f8c390c4b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a5e7acc0a0 con 0x55a5e7c785b0 2024-05-21T03:29:15.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.239+0000 7f8c390c4b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a5e7b2dea0 con 0x55a5e7c7c720 2024-05-21T03:29:15.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.239+0000 7f8c390c4b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a5e7c84c80 con 0x55a5e7c77bf0 2024-05-21T03:29:15.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.239+0000 7f8c390c4b80 10 client.0 _ll_register_callbacks cb 0x55a5e7b60a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:29:15.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.239+0000 7f8c1ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e7c785b0 0x55a5e7c816d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:15.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.239+0000 7f8c1ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e7c785b0 0x55a5e7c816d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37890/0 (socket says 172.21.15.27:37890) 2024-05-21T03:29:15.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.239+0000 7f8c1ffff700 1 -- 172.21.15.27:0/2682116206 learned_addr learned my addr 172.21.15.27:0/2682116206 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:15.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.239+0000 7f8c27fff700 1 --2- 172.21.15.27:0/2682116206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e7c77bf0 0x55a5e7c843d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:15.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.239+0000 7f8c2cdcb700 1 --2- 172.21.15.27:0/2682116206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5e7c7c720 0x55a5e7c81c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:15.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.239+0000 7f8c1ffff700 1 -- 172.21.15.27:0/2682116206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e7c77bf0 msgr2=0x55a5e7c843d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:15.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.239+0000 7f8c1ffff700 1 --2- 172.21.15.27:0/2682116206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e7c77bf0 0x55a5e7c843d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.239+0000 7f8c1ffff700 1 -- 172.21.15.27:0/2682116206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5e7c7c720 msgr2=0x55a5e7c81c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:15.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.239+0000 7f8c1ffff700 1 --2- 172.21.15.27:0/2682116206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5e7c7c720 0x55a5e7c81c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.239+0000 7f8c1ffff700 1 -- 172.21.15.27:0/2682116206 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5e7b159d0 con 0x55a5e7c785b0 2024-05-21T03:29:15.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.239+0000 7f8c1ffff700 1 --2- 172.21.15.27:0/2682116206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e7c785b0 0x55a5e7c816d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f8c100df660 tx=0x7f8c100ad3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd9341ebc76b645 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:15.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.239+0000 7f8c077fe700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:29:15.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.239+0000 7f8c077fe700 1 -- 172.21.15.27:0/2682116206 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c100c3050 con 0x55a5e7c785b0 2024-05-21T03:29:15.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.239+0000 7f8c077fe700 1 -- 172.21.15.27:0/2682116206 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f8c100a5730 con 0x55a5e7c785b0 2024-05-21T03:29:15.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.239+0000 7f8c390c4b80 1 -- 172.21.15.27:0/2682116206 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c100a9140 con 0x55a5e7c785b0 2024-05-21T03:29:15.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.240+0000 7f8c077fe700 1 -- 172.21.15.27:0/2682116206 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8c100ad4f0 con 0x55a5e7c785b0 2024-05-21T03:29:15.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.240+0000 7f8c390c4b80 10 client.15723 fetch_fsmap learned FSMap version 34 2024-05-21T03:29:15.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.240+0000 7f8c390c4b80 1 -- 172.21.15.27:0/2682116206 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a5e7b7f860 con 0x55a5e7c785b0 2024-05-21T03:29:15.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.240+0000 7f8c077fe700 1 -- 172.21.15.27:0/2682116206 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c100a9140 con 0x55a5e7c785b0 2024-05-21T03:29:15.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.240+0000 7f8c077fe700 1 -- 172.21.15.27:0/2682116206 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f8c100a83a0 con 0x55a5e7c785b0 2024-05-21T03:29:15.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.240+0000 7f8c390c4b80 10 client.15723 fetch_fsmap finished waiting for FSMap version 34 2024-05-21T03:29:15.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.240+0000 7f8c390c4b80 10 client.15723 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-21T03:29:15.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.240+0000 7f8c390c4b80 20 client.15723 populate_metadata read hostname 'smithi027' 2024-05-21T03:29:15.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.240+0000 7f8c390c4b80 1 --2- 172.21.15.27:0/2682116206 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55a5e7c917b0 0x55a5e7c93ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:15.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.240+0000 7f8c390c4b80 4 client.15723 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:29:15.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.240+0000 7f8c390c4b80 1 -- 172.21.15.27:0/2682116206 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5e7bd7dc0 con 0x55a5e7c917b0 2024-05-21T03:29:15.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.241+0000 7f8c27fff700 1 --2- 172.21.15.27:0/2682116206 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55a5e7c917b0 0x55a5e7c93ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:15.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.241+0000 7f8c27fff700 1 --2- 172.21.15.27:0/2682116206 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55a5e7c917b0 0x55a5e7c93ba0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e19b42baaa0a72f6 server_cookie=8f0e94f050707d4e in_seq=0 out_seq=0 2024-05-21T03:29:15.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.241+0000 7f8c077fe700 10 client.15723 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:29:15.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.242+0000 7f8c077fe700 1 -- 172.21.15.27:0/2682116206 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a5e7bd7dc0 con 0x55a5e7c917b0 2024-05-21T03:29:15.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.242+0000 7f8c077fe700 10 client.15723 handle_command_reply: tid=0 2024-05-21T03:29:15.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.242+0000 7f8c390c4b80 1 client.15723 shutdown 2024-05-21T03:29:15.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.242+0000 7f8c390c4b80 1 -- 172.21.15.27:0/2682116206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e7c785b0 msgr2=0x55a5e7c816d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:15.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.242+0000 7f8c390c4b80 1 --2- 172.21.15.27:0/2682116206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e7c785b0 0x55a5e7c816d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f8c100df660 tx=0x7f8c100ad3a0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.242+0000 7f8c390c4b80 1 -- 172.21.15.27:0/2682116206 shutdown_connections 2024-05-21T03:29:15.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.242+0000 7f8c390c4b80 1 --2- 172.21.15.27:0/2682116206 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55a5e7c917b0 0x55a5e7c93ba0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.242+0000 7f8c077fe700 0 client.15723 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:29:15.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.242+0000 7f8c390c4b80 1 --2- 172.21.15.27:0/2682116206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5e7c7c720 0x55a5e7c81c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.242+0000 7f8c390c4b80 1 --2- 172.21.15.27:0/2682116206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e7c77bf0 0x55a5e7c843d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.242+0000 7f8c390c4b80 1 --2- 172.21.15.27:0/2682116206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e7c785b0 0x55a5e7c816d0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.242+0000 7f8c390c4b80 1 -- 172.21.15.27:0/2682116206 >> 172.21.15.27:0/2682116206 conn(0x55a5e7ba3340 msgr2=0x55a5e7c76be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:15.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.242+0000 7f8c390c4b80 1 -- 172.21.15.27:0/2682116206 shutdown_connections 2024-05-21T03:29:15.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.242+0000 7f8c390c4b80 1 -- 172.21.15.27:0/2682116206 wait complete. 2024-05-21T03:29:15.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.242+0000 7f8c390c4b80 20 client.15723 trim_cache size 0 max 16384 2024-05-21T03:29:15.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.267+0000 7f8c390c4b80 1 Processor -- start 2024-05-21T03:29:15.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.267+0000 7f8c390c4b80 1 -- start start 2024-05-21T03:29:15.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.267+0000 7f8c390c4b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e7c77bf0 0x55a5e7c6ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:15.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.267+0000 7f8c390c4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5e7c785b0 0x55a5e7c7dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:15.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.267+0000 7f8c390c4b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e7c7c720 0x55a5e7c96bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:15.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.267+0000 7f8c390c4b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8c281b01f0 con 0x55a5e7c7c720 2024-05-21T03:29:15.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.267+0000 7f8c390c4b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8c281b1df0 con 0x55a5e7c785b0 2024-05-21T03:29:15.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.267+0000 7f8c390c4b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8c281b08c0 con 0x55a5e7c77bf0 2024-05-21T03:29:15.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.268+0000 7f8c2cdcb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e7c7c720 0x55a5e7c96bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:15.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.268+0000 7f8c2cdcb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e7c7c720 0x55a5e7c96bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37898/0 (socket says 172.21.15.27:37898) 2024-05-21T03:29:15.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.268+0000 7f8c2cdcb700 1 -- 172.21.15.27:0/3215899740 learned_addr learned my addr 172.21.15.27:0/3215899740 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:15.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.268+0000 7f8c1ffff700 1 --2- 172.21.15.27:0/3215899740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5e7c785b0 0x55a5e7c7dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:15.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.268+0000 7f8c27fff700 1 --2- 172.21.15.27:0/3215899740 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e7c77bf0 0x55a5e7c6ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:15.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.268+0000 7f8c2cdcb700 1 -- 172.21.15.27:0/3215899740 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e7c77bf0 msgr2=0x55a5e7c6ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:15.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.268+0000 7f8c2cdcb700 1 --2- 172.21.15.27:0/3215899740 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e7c77bf0 0x55a5e7c6ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.268+0000 7f8c2cdcb700 1 -- 172.21.15.27:0/3215899740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5e7c785b0 msgr2=0x55a5e7c7dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:15.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.268+0000 7f8c2cdcb700 1 --2- 172.21.15.27:0/3215899740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5e7c785b0 0x55a5e7c7dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.268+0000 7f8c2cdcb700 1 -- 172.21.15.27:0/3215899740 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5e7c7bee0 con 0x55a5e7c7c720 2024-05-21T03:29:15.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.268+0000 7f8c1ffff700 1 --2- 172.21.15.27:0/3215899740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5e7c785b0 0x55a5e7c7dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:29:15.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.268+0000 7f8c2cdcb700 1 --2- 172.21.15.27:0/3215899740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e7c7c720 0x55a5e7c96bc0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009310 tx=0x7f8c20009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5a258d3f46a016 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:15.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.268+0000 7f8c1dffb700 1 -- 172.21.15.27:0/3215899740 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c20009c70 con 0x55a5e7c7c720 2024-05-21T03:29:15.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.268+0000 7f8c1dffb700 1 -- 172.21.15.27:0/3215899740 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c20004c50 con 0x55a5e7c7c720 2024-05-21T03:29:15.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.268+0000 7f8c1dffb700 1 -- 172.21.15.27:0/3215899740 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c20005340 con 0x55a5e7c7c720 2024-05-21T03:29:15.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.269+0000 7f8c390c4b80 1 -- 172.21.15.27:0/3215899740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e7c7c720 msgr2=0x55a5e7c96bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:15.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.269+0000 7f8c390c4b80 1 --2- 172.21.15.27:0/3215899740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e7c7c720 0x55a5e7c96bc0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009310 tx=0x7f8c20009340 comp rx=0 tx=0).stop 2024-05-21T03:29:15.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.269+0000 7f8c390c4b80 1 -- 172.21.15.27:0/3215899740 shutdown_connections 2024-05-21T03:29:15.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.269+0000 7f8c390c4b80 1 --2- 172.21.15.27:0/3215899740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e7c7c720 0x55a5e7c96bc0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.269+0000 7f8c390c4b80 1 --2- 172.21.15.27:0/3215899740 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e7c77bf0 0x55a5e7c6ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.269+0000 7f8c390c4b80 1 --2- 172.21.15.27:0/3215899740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5e7c785b0 0x55a5e7c7dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.269+0000 7f8c390c4b80 1 -- 172.21.15.27:0/3215899740 >> 172.21.15.27:0/3215899740 conn(0x55a5e7ba3340 msgr2=0x55a5e7c7d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:15.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.269+0000 7f8c390c4b80 1 -- 172.21.15.27:0/3215899740 shutdown_connections 2024-05-21T03:29:15.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.269+0000 7f8c390c4b80 1 -- 172.21.15.27:0/3215899740 wait complete. 2024-05-21T03:29:15.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.269+0000 7f8c390c4b80 1 -- start start 2024-05-21T03:29:15.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.270+0000 7f8c390c4b80 1 Processor -- start 2024-05-21T03:29:15.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.270+0000 7f8c390c4b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e7c77bf0 0x55a5e7c970a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:15.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.270+0000 7f8c390c4b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e7c785b0 0x55a5e7c95450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:15.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.270+0000 7f8c390c4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5e7c7c720 0x55a5e7c95990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:15.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.270+0000 7f8c390c4b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8c281b1660 con 0x55a5e7c785b0 2024-05-21T03:29:15.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.270+0000 7f8c390c4b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8c281b0f90 con 0x55a5e7c7c720 2024-05-21T03:29:15.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.270+0000 7f8c390c4b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8c281ae190 con 0x55a5e7c77bf0 2024-05-21T03:29:15.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.270+0000 7f8c390c4b80 10 client.0 _ll_register_callbacks cb 0x55a5e7b60a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:29:15.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.270+0000 7f8c1ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e7c785b0 0x55a5e7c95450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:15.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.270+0000 7f8c1ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e7c785b0 0x55a5e7c95450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37912/0 (socket says 172.21.15.27:37912) 2024-05-21T03:29:15.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.270+0000 7f8c1ffff700 1 -- 172.21.15.27:0/1312432643 learned_addr learned my addr 172.21.15.27:0/1312432643 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:15.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.270+0000 7f8c27fff700 1 --2- 172.21.15.27:0/1312432643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e7c77bf0 0x55a5e7c970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:15.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.270+0000 7f8c2cdcb700 1 --2- 172.21.15.27:0/1312432643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5e7c7c720 0x55a5e7c95990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:15.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.270+0000 7f8c1ffff700 1 -- 172.21.15.27:0/1312432643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e7c77bf0 msgr2=0x55a5e7c970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:15.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.270+0000 7f8c1ffff700 1 --2- 172.21.15.27:0/1312432643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e7c77bf0 0x55a5e7c970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.270+0000 7f8c1ffff700 1 -- 172.21.15.27:0/1312432643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5e7c7c720 msgr2=0x55a5e7c95990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:15.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.270+0000 7f8c1ffff700 1 --2- 172.21.15.27:0/1312432643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5e7c7c720 0x55a5e7c95990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.270+0000 7f8c1ffff700 1 -- 172.21.15.27:0/1312432643 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5e7c95f80 con 0x55a5e7c785b0 2024-05-21T03:29:15.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.271+0000 7f8c1ffff700 1 --2- 172.21.15.27:0/1312432643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e7c785b0 0x55a5e7c95450 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8c100b1ec0 tx=0x7f8c100a5130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0fbc14d5cde91d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:15.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.271+0000 7f8c077fe700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:29:15.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.271+0000 7f8c077fe700 1 -- 172.21.15.27:0/1312432643 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c100c3070 con 0x55a5e7c785b0 2024-05-21T03:29:15.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.271+0000 7f8c077fe700 1 -- 172.21.15.27:0/1312432643 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f8c100a5730 con 0x55a5e7c785b0 2024-05-21T03:29:15.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.271+0000 7f8c390c4b80 1 -- 172.21.15.27:0/1312432643 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c20004c50 con 0x55a5e7c785b0 2024-05-21T03:29:15.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.271+0000 7f8c077fe700 1 -- 172.21.15.27:0/1312432643 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8c100a9140 con 0x55a5e7c785b0 2024-05-21T03:29:15.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.271+0000 7f8c390c4b80 10 client.15729 fetch_fsmap learned FSMap version 34 2024-05-21T03:29:15.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.271+0000 7f8c390c4b80 1 -- 172.21.15.27:0/1312432643 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a5e7cfce50 con 0x55a5e7c785b0 2024-05-21T03:29:15.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.271+0000 7f8c077fe700 1 -- 172.21.15.27:0/1312432643 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c20004c50 con 0x55a5e7c785b0 2024-05-21T03:29:15.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.271+0000 7f8c077fe700 1 -- 172.21.15.27:0/1312432643 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f8c1008c080 con 0x55a5e7c785b0 2024-05-21T03:29:15.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.271+0000 7f8c390c4b80 10 client.15729 fetch_fsmap finished waiting for FSMap version 34 2024-05-21T03:29:15.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.271+0000 7f8c390c4b80 10 client.15729 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-21T03:29:15.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.271+0000 7f8c390c4b80 20 client.15729 populate_metadata read hostname 'smithi027' 2024-05-21T03:29:15.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.271+0000 7f8c390c4b80 1 --2- 172.21.15.27:0/1312432643 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55a5e7cfd2a0 0x55a5e7cff6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:15.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.271+0000 7f8c390c4b80 4 client.15729 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "72cf81f5", "await_for": 45.0}] 2024-05-21T03:29:15.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.271+0000 7f8c390c4b80 1 -- 172.21.15.27:0/1312432643 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "quiesce db", "set_id": "72cf81f5", "await_for": 45.0}) -- 0x55a5e7bd15a0 con 0x55a5e7cfd2a0 2024-05-21T03:29:15.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.272+0000 7f8c27fff700 1 --2- 172.21.15.27:0/1312432643 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55a5e7cfd2a0 0x55a5e7cff6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:15.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.272+0000 7f8c077fe700 10 client.15729 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:29:15.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.272+0000 7f8c27fff700 1 --2- 172.21.15.27:0/1312432643 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55a5e7cfd2a0 0x55a5e7cff6c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c4cfcb913955f66 server_cookie=75f46a8f2b2ffd38 in_seq=0 out_seq=0 2024-05-21T03:29:15.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.273+0000 7f8c077fe700 1 -- 172.21.15.27:0/1312432643 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+592 (crc 0 0 0) 0x55a5e7bd15a0 con 0x55a5e7cfd2a0 2024-05-21T03:29:15.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.273+0000 7f8c077fe700 10 client.15729 handle_command_reply: tid=0 2024-05-21T03:29:15.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.273+0000 7f8c390c4b80 1 client.15729 shutdown 2024-05-21T03:29:15.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.273+0000 7f8c390c4b80 1 -- 172.21.15.27:0/1312432643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e7c785b0 msgr2=0x55a5e7c95450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:15.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.273+0000 7f8c390c4b80 1 --2- 172.21.15.27:0/1312432643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e7c785b0 0x55a5e7c95450 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8c100b1ec0 tx=0x7f8c100a5130 comp rx=0 tx=0).stop 2024-05-21T03:29:15.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.273+0000 7f8c390c4b80 1 -- 172.21.15.27:0/1312432643 shutdown_connections 2024-05-21T03:29:15.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.273+0000 7f8c390c4b80 1 --2- 172.21.15.27:0/1312432643 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55a5e7cfd2a0 0x55a5e7cff6c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.273+0000 7f8c390c4b80 1 --2- 172.21.15.27:0/1312432643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5e7c7c720 0x55a5e7c95990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.273+0000 7f8c390c4b80 1 --2- 172.21.15.27:0/1312432643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e7c77bf0 0x55a5e7c970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.273+0000 7f8c390c4b80 1 --2- 172.21.15.27:0/1312432643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e7c785b0 0x55a5e7c95450 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.273+0000 7f8c390c4b80 1 -- 172.21.15.27:0/1312432643 >> 172.21.15.27:0/1312432643 conn(0x55a5e7ba3340 msgr2=0x55a5e7c81770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:15.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.273+0000 7f8c077fe700 0 client.15729 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:29:15.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.273+0000 7f8c390c4b80 1 -- 172.21.15.27:0/1312432643 shutdown_connections 2024-05-21T03:29:15.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.273+0000 7f8c390c4b80 1 -- 172.21.15.27:0/1312432643 wait complete. 2024-05-21T03:29:15.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.273+0000 7f8c390c4b80 20 client.15729 trim_cache size 0 max 16384 2024-05-21T03:29:15.278 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:29:15.278 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 34, 2024-05-21T03:29:15.278 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24476, 2024-05-21T03:29:15.278 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 11, 2024-05-21T03:29:15.278 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-05-21T03:29:15.278 INFO:teuthology.orchestra.run.smithi027.stdout: "72cf81f5": { 2024-05-21T03:29:15.278 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 11, 2024-05-21T03:29:15.278 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-05-21T03:29:15.278 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-05-21T03:29:15.279 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-05-21T03:29:15.279 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-05-21T03:29:15.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:29:15.279 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-05-21T03:29:15.279 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 141.4, 2024-05-21T03:29:15.279 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-05-21T03:29:15.279 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-05-21T03:29:15.279 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-05-21T03:29:15.279 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-05-21T03:29:15.279 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-05-21T03:29:15.279 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 23.2 2024-05-21T03:29:15.279 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:29:15.279 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:29:15.279 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:29:15.280 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:29:15.280 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:29:15.280 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:29:15.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.274+0000 7f8c2e82e700 1 -- 172.21.15.27:0/3008530357 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f8c140b29f0 msgr2=0x7f8c140b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:15.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.274+0000 7f8c2e82e700 1 --2- 172.21.15.27:0/3008530357 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f8c140b29f0 0x7f8c140b4eb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8c18006fd0 tx=0x7f8c18008040 comp rx=0 tx=0).stop 2024-05-21T03:29:15.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.274+0000 7f8c2e82e700 1 -- 172.21.15.27:0/3008530357 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c2806bbe0 msgr2=0x7f8c2819a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:15.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.274+0000 7f8c2e82e700 1 --2- 172.21.15.27:0/3008530357 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c2806bbe0 0x7f8c2819a250 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8c1000b530 tx=0x7f8c1000b8f0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.274+0000 7f8c2e82e700 1 -- 172.21.15.27:0/3008530357 shutdown_connections 2024-05-21T03:29:15.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.274+0000 7f8c2e82e700 1 --2- 172.21.15.27:0/3008530357 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f8c140b29f0 0x7f8c140b4eb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.274+0000 7f8c2e82e700 1 --2- 172.21.15.27:0/3008530357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c281095b0 0x7f8c2819e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.274+0000 7f8c2e82e700 1 --2- 172.21.15.27:0/3008530357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c2806b220 0x7f8c28199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.274+0000 7f8c2e82e700 1 --2- 172.21.15.27:0/3008530357 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c2806bbe0 0x7f8c2819a250 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:15.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.274+0000 7f8c2e82e700 1 -- 172.21.15.27:0/3008530357 >> 172.21.15.27:0/3008530357 conn(0x7f8c280793f0 msgr2=0x7f8c28101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:15.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.274+0000 7f8c2e82e700 1 -- 172.21.15.27:0/3008530357 shutdown_connections 2024-05-21T03:29:15.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:15.274+0000 7f8c2e82e700 1 -- 172.21.15.27:0/3008530357 wait complete. 2024-05-21T03:29:15.983 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '72cf81f5', quiesce took 77.8 seconds. Will release after: -56.349999999999994 2024-05-21T03:29:15.984 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:29:16.237 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:29:16.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:16 smithi027 ceph-mon[32179]: pgmap v160: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T03:29:17.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:16 smithi039 ceph-mon[36429]: pgmap v160: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T03:29:17.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:16 smithi165 ceph-mon[36269]: pgmap v160: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T03:29:17.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.922+0000 7f96a7fbd700 1 -- 172.21.15.27:0/440879135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96a0107000 msgr2=0x7f96a010eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:17.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.922+0000 7f96a7fbd700 1 --2- 172.21.15.27:0/440879135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96a0107000 0x7f96a010eb40 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9698009990 tx=0x7f9698009ca0 comp rx=0 tx=0).stop 2024-05-21T03:29:17.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.922+0000 7f96a7fbd700 1 -- 172.21.15.27:0/440879135 shutdown_connections 2024-05-21T03:29:17.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.922+0000 7f96a7fbd700 1 --2- 172.21.15.27:0/440879135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96a0107000 0x7f96a010eb40 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:17.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.922+0000 7f96a7fbd700 1 --2- 172.21.15.27:0/440879135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96a0102160 0x7f96a0102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:17.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.922+0000 7f96a7fbd700 1 --2- 172.21.15.27:0/440879135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96a0102b10 0x7f96a01068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:17.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.922+0000 7f96a7fbd700 1 -- 172.21.15.27:0/440879135 >> 172.21.15.27:0/440879135 conn(0x7f96a0077780 msgr2=0x7f96a0077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:17.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.922+0000 7f96a7fbd700 1 -- 172.21.15.27:0/440879135 shutdown_connections 2024-05-21T03:29:17.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.922+0000 7f96a7fbd700 1 -- 172.21.15.27:0/440879135 wait complete. 2024-05-21T03:29:17.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.923+0000 7f96a7fbd700 1 Processor -- start 2024-05-21T03:29:17.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.923+0000 7f96a7fbd700 1 -- start start 2024-05-21T03:29:17.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.923+0000 7f96a7fbd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96a0102160 0x7f96a019e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:17.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.923+0000 7f96a7fbd700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96a0102b10 0x7f96a019ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:17.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.923+0000 7f96a7fbd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96a0107000 0x7f96a0198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:17.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.923+0000 7f96a7fbd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f96a0111e40 con 0x7f96a0102160 2024-05-21T03:29:17.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.923+0000 7f96a7fbd700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f96a0111cc0 con 0x7f96a0107000 2024-05-21T03:29:17.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.923+0000 7f96a7fbd700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f96a0111fc0 con 0x7f96a0102b10 2024-05-21T03:29:17.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.923+0000 7f96a5d59700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96a0102160 0x7f96a019e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:17.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.923+0000 7f96a5d59700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96a0102160 0x7f96a019e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37918/0 (socket says 172.21.15.27:37918) 2024-05-21T03:29:17.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.923+0000 7f96a5d59700 1 -- 172.21.15.27:0/2732174826 learned_addr learned my addr 172.21.15.27:0/2732174826 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:17.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.924+0000 7f96a655a700 1 --2- 172.21.15.27:0/2732174826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96a0107000 0x7f96a0198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:17.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.924+0000 7f96a5558700 1 --2- 172.21.15.27:0/2732174826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96a0102b10 0x7f96a019ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:17.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.924+0000 7f96a5d59700 1 -- 172.21.15.27:0/2732174826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96a0102b10 msgr2=0x7f96a019ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:17.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.924+0000 7f96a5d59700 1 --2- 172.21.15.27:0/2732174826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96a0102b10 0x7f96a019ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:17.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.924+0000 7f96a5d59700 1 -- 172.21.15.27:0/2732174826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96a0107000 msgr2=0x7f96a0198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:17.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.924+0000 7f96a5d59700 1 --2- 172.21.15.27:0/2732174826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96a0107000 0x7f96a0198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:17.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.924+0000 7f96a5d59700 1 -- 172.21.15.27:0/2732174826 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a0199060 con 0x7f96a0102160 2024-05-21T03:29:17.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.924+0000 7f96a5558700 1 --2- 172.21.15.27:0/2732174826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96a0102b10 0x7f96a019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:29:17.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.924+0000 7f96a5d59700 1 --2- 172.21.15.27:0/2732174826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96a0102160 0x7f96a019e740 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f969c00ada0 tx=0x7f969c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:17.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.924+0000 7f9696ffd700 1 -- 172.21.15.27:0/2732174826 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c017020 con 0x7f96a0102160 2024-05-21T03:29:17.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.924+0000 7f96a7fbd700 1 -- 172.21.15.27:0/2732174826 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a0199350 con 0x7f96a0102160 2024-05-21T03:29:17.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.924+0000 7f9696ffd700 1 -- 172.21.15.27:0/2732174826 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f969c00ce30 con 0x7f96a0102160 2024-05-21T03:29:17.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.924+0000 7f9696ffd700 1 -- 172.21.15.27:0/2732174826 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c005560 con 0x7f96a0102160 2024-05-21T03:29:17.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.924+0000 7f96a7fbd700 1 -- 172.21.15.27:0/2732174826 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a01a5690 con 0x7f96a0102160 2024-05-21T03:29:17.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.925+0000 7f96a7fbd700 1 -- 172.21.15.27:0/2732174826 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96a006a230 con 0x7f96a0102160 2024-05-21T03:29:17.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.926+0000 7f9696ffd700 1 -- 172.21.15.27:0/2732174826 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7f969c004ad0 con 0x7f96a0102160 2024-05-21T03:29:17.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.927+0000 7f9696ffd700 1 --2- 172.21.15.27:0/2732174826 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f968c0b29f0 0x7f968c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:17.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.927+0000 7f96a5558700 1 --2- 172.21.15.27:0/2732174826 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f968c0b29f0 0x7f968c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:17.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.927+0000 7f9696ffd700 1 -- 172.21.15.27:0/2732174826 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f969c0e2470 con 0x7f96a0102160 2024-05-21T03:29:17.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.928+0000 7f96a5558700 1 --2- 172.21.15.27:0/2732174826 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f968c0b29f0 0x7f968c0b4eb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f96a0105d70 tx=0x7f9690008040 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:17.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:17.931+0000 7f9696ffd700 1 -- 172.21.15.27:0/2732174826 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f969c021020 con 0x7f96a0102160 2024-05-21T03:29:18.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:18.192+0000 7f96a7fbd700 1 -- 172.21.15.27:0/2732174826 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96a0199e30 con 0x7f96a0102160 2024-05-21T03:29:18.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:18.193+0000 7f9696ffd700 1 -- 172.21.15.27:0/2732174826 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11932 (secure 0 0 0) 0x7f96a0199e30 con 0x7f96a0102160 2024-05-21T03:29:18.193 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:29:18.194 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/148854869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":148854869},{"type":"v1","addr":"172.21.15.27:6835","nonce":148854869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/4145544730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":4145544730},{"type":"v1","addr":"172.21.15.27:6837","nonce":4145544730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T03:20:34.055985+0000","modified":"2024-05-21T03:28:59.351119+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24472,"mds_2":24457,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/745486764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":745486764},{"type":"v1","addr":"172.21.15.27:6839","nonce":745486764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6841/3982752351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3982752351},{"type":"v1","addr":"172.21.15.27:6841","nonce":3982752351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/3497521608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3497521608},{"type":"v1","addr":"172.21.15.39:6835","nonce":3497521608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.165:6835/3576910218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3576910218},{"type":"v1","addr":"172.21.15.165:6835","nonce":3576910218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/3673846852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3673846852},{"type":"v1","addr":"172.21.15.39:6837","nonce":3673846852}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6837/691356139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":691356139},{"type":"v1","addr":"172.21.15.165:6837","nonce":691356139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6839/4158170814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4158170814},{"type":"v1","addr":"172.21.15.165:6839","nonce":4158170814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6839/287607819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":287607819},{"type":"v1","addr":"172.21.15.39:6839","nonce":287607819}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24472,24451,24452,24476,24457]},"id":1}]} 2024-05-21T03:29:18.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:18.195+0000 7f96a7fbd700 1 -- 172.21.15.27:0/2732174826 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f968c0b29f0 msgr2=0x7f968c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:18.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:18.195+0000 7f96a7fbd700 1 --2- 172.21.15.27:0/2732174826 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f968c0b29f0 0x7f968c0b4eb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f96a0105d70 tx=0x7f9690008040 comp rx=0 tx=0).stop 2024-05-21T03:29:18.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:18.195+0000 7f96a7fbd700 1 -- 172.21.15.27:0/2732174826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96a0102160 msgr2=0x7f96a019e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:18.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:18.195+0000 7f96a7fbd700 1 --2- 172.21.15.27:0/2732174826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96a0102160 0x7f96a019e740 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f969c00ada0 tx=0x7f969c00c330 comp rx=0 tx=0).stop 2024-05-21T03:29:18.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:18.195+0000 7f96a7fbd700 1 -- 172.21.15.27:0/2732174826 shutdown_connections 2024-05-21T03:29:18.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:18.195+0000 7f96a7fbd700 1 --2- 172.21.15.27:0/2732174826 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f968c0b29f0 0x7f968c0b4eb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:18.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:18.195+0000 7f96a7fbd700 1 --2- 172.21.15.27:0/2732174826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96a0107000 0x7f96a0198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:18.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:18.195+0000 7f96a7fbd700 1 --2- 172.21.15.27:0/2732174826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96a0102160 0x7f96a019e740 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:18.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:18.195+0000 7f96a7fbd700 1 --2- 172.21.15.27:0/2732174826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96a0102b10 0x7f96a019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:18.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:18.195+0000 7f96a7fbd700 1 -- 172.21.15.27:0/2732174826 >> 172.21.15.27:0/2732174826 conn(0x7f96a0077780 msgr2=0x7f96a0104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:18.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:18.196+0000 7f96a7fbd700 1 -- 172.21.15.27:0/2732174826 shutdown_connections 2024-05-21T03:29:18.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:18.196+0000 7f96a7fbd700 1 -- 172.21.15.27:0/2732174826 wait complete. 2024-05-21T03:29:18.197 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 34 2024-05-21T03:29:18.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:18 smithi027 ceph-mon[32179]: pgmap v161: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T03:29:18.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:18 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2732174826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:29:18.976 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 72cf81f5 --release --await' 2024-05-21T03:29:18.977 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 72cf81f5 --release --await 2024-05-21T03:29:19.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:18 smithi039 ceph-mon[36429]: pgmap v161: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T03:29:19.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:18 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/2732174826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:29:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:18 smithi165 ceph-mon[36269]: pgmap v161: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T03:29:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:18 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/2732174826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:29:19.231 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:29:20.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:20 smithi027 ceph-mon[32179]: pgmap v162: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-05-21T03:29:21.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:20 smithi039 ceph-mon[36429]: pgmap v162: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-05-21T03:29:21.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:20 smithi165 ceph-mon[36269]: pgmap v162: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-05-21T03:29:21.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.062+0000 7ff18187a700 1 -- 172.21.15.27:0/3328385998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff17c1084f0 msgr2=0x7ff17c1088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:21.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.062+0000 7ff18187a700 1 --2- 172.21.15.27:0/3328385998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff17c1084f0 0x7ff17c1088d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00b0a0 tx=0x7ff16c00b3b0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.062+0000 7ff18187a700 1 -- 172.21.15.27:0/3328385998 shutdown_connections 2024-05-21T03:29:21.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.062+0000 7ff18187a700 1 --2- 172.21.15.27:0/3328385998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff17c102e30 0x7ff17c10f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.062+0000 7ff18187a700 1 --2- 172.21.15.27:0/3328385998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff17c1084f0 0x7ff17c1088d0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.062+0000 7ff18187a700 1 --2- 172.21.15.27:0/3328385998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff17c102470 0x7ff17c1028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.062+0000 7ff18187a700 1 -- 172.21.15.27:0/3328385998 >> 172.21.15.27:0/3328385998 conn(0x7ff17c0793e0 msgr2=0x7ff17c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:21.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.062+0000 7ff18187a700 1 -- 172.21.15.27:0/3328385998 shutdown_connections 2024-05-21T03:29:21.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.062+0000 7ff18187a700 1 -- 172.21.15.27:0/3328385998 wait complete. 2024-05-21T03:29:21.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.063+0000 7ff18187a700 1 Processor -- start 2024-05-21T03:29:21.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.063+0000 7ff18187a700 1 -- start start 2024-05-21T03:29:21.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.064+0000 7ff18187a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff17c102470 0x7ff17c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:21.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.064+0000 7ff18187a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff17c102e30 0x7ff17c19b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:21.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.064+0000 7ff18187a700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff17c1084f0 0x7ff17c1a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:21.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.064+0000 7ff18187a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c112670 con 0x7ff17c102e30 2024-05-21T03:29:21.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.064+0000 7ff18187a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c1124f0 con 0x7ff17c102470 2024-05-21T03:29:21.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.064+0000 7ff18187a700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c1127f0 con 0x7ff17c1084f0 2024-05-21T03:29:21.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.064+0000 7ff17a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff17c102e30 0x7ff17c19b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:21.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.064+0000 7ff17a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff17c102e30 0x7ff17c19b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37932/0 (socket says 172.21.15.27:37932) 2024-05-21T03:29:21.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.064+0000 7ff17a7fc700 1 -- 172.21.15.27:0/2451162012 learned_addr learned my addr 172.21.15.27:0/2451162012 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:21.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.064+0000 7ff17affd700 1 --2- 172.21.15.27:0/2451162012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff17c102470 0x7ff17c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:21.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.064+0000 7ff17b7fe700 1 --2- 172.21.15.27:0/2451162012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff17c1084f0 0x7ff17c1a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:21.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.064+0000 7ff17a7fc700 1 -- 172.21.15.27:0/2451162012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff17c1084f0 msgr2=0x7ff17c1a5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:21.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.064+0000 7ff17a7fc700 1 --2- 172.21.15.27:0/2451162012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff17c1084f0 0x7ff17c1a5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.064+0000 7ff17a7fc700 1 -- 172.21.15.27:0/2451162012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff17c102470 msgr2=0x7ff17c19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:21.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.064+0000 7ff17a7fc700 1 --2- 172.21.15.27:0/2451162012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff17c102470 0x7ff17c19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.064+0000 7ff17a7fc700 1 -- 172.21.15.27:0/2451162012 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff17c1a6090 con 0x7ff17c102e30 2024-05-21T03:29:21.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.064+0000 7ff17a7fc700 1 --2- 172.21.15.27:0/2451162012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff17c102e30 0x7ff17c19b010 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff17000b530 tx=0x7ff17000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:21.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.064+0000 7ff180878700 1 -- 172.21.15.27:0/2451162012 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff170015020 con 0x7ff17c102e30 2024-05-21T03:29:21.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.065+0000 7ff18187a700 1 -- 172.21.15.27:0/2451162012 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff17c1a62c0 con 0x7ff17c102e30 2024-05-21T03:29:21.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.065+0000 7ff180878700 1 -- 172.21.15.27:0/2451162012 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff170011ce0 con 0x7ff17c102e30 2024-05-21T03:29:21.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.065+0000 7ff180878700 1 -- 172.21.15.27:0/2451162012 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff170010dd0 con 0x7ff17c102e30 2024-05-21T03:29:21.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.065+0000 7ff18187a700 1 -- 172.21.15.27:0/2451162012 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff17c1a6800 con 0x7ff17c102e30 2024-05-21T03:29:21.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.067+0000 7ff180878700 1 -- 172.21.15.27:0/2451162012 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7ff170018020 con 0x7ff17c102e30 2024-05-21T03:29:21.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.068+0000 7ff180878700 1 --2- 172.21.15.27:0/2451162012 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7ff1640b29f0 0x7ff1640b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:21.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.068+0000 7ff17affd700 1 --2- 172.21.15.27:0/2451162012 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7ff1640b29f0 0x7ff1640b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:21.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.068+0000 7ff180878700 1 -- 172.21.15.27:0/2451162012 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7ff1700e1c40 con 0x7ff17c102e30 2024-05-21T03:29:21.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.068+0000 7ff17affd700 1 --2- 172.21.15.27:0/2451162012 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7ff1640b29f0 0x7ff1640b4eb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff16c009e60 tx=0x7ff16c00b7b0 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:21.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.068+0000 7ff18c110b80 1 Processor -- start 2024-05-21T03:29:21.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.068+0000 7ff18c110b80 1 -- start start 2024-05-21T03:29:21.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.069+0000 7ff18c110b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bedba2cb30 0x55bedba2cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:21.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.069+0000 7ff18c110b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bedba28000 0x55bedba28480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:21.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.069+0000 7ff18c110b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bedba289c0 0x55bedba34a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:21.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.069+0000 7ff18c110b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bedb981e70 con 0x55bedba28000 2024-05-21T03:29:21.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.069+0000 7ff18c110b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bedb971a10 con 0x55bedba289c0 2024-05-21T03:29:21.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.069+0000 7ff18c110b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bedb975a50 con 0x55bedba2cb30 2024-05-21T03:29:21.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.069+0000 7ff17a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bedba28000 0x55bedba28480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:21.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.069+0000 7ff17a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bedba28000 0x55bedba28480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37944/0 (socket says 172.21.15.27:37944) 2024-05-21T03:29:21.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.069+0000 7ff17a7fc700 1 -- 172.21.15.27:0/1260487285 learned_addr learned my addr 172.21.15.27:0/1260487285 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:21.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.069+0000 7ff17affd700 1 --2- 172.21.15.27:0/1260487285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bedba2cb30 0x55bedba2cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:21.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.069+0000 7ff17b7fe700 1 --2- 172.21.15.27:0/1260487285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bedba289c0 0x55bedba34a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:21.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.069+0000 7ff17a7fc700 1 -- 172.21.15.27:0/1260487285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bedba2cb30 msgr2=0x55bedba2cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:21.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.069+0000 7ff17a7fc700 1 --2- 172.21.15.27:0/1260487285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bedba2cb30 0x55bedba2cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.069+0000 7ff17a7fc700 1 -- 172.21.15.27:0/1260487285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bedba289c0 msgr2=0x55bedba34a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:21.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.069+0000 7ff17a7fc700 1 --2- 172.21.15.27:0/1260487285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bedba289c0 0x55bedba34a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.069+0000 7ff17a7fc700 1 -- 172.21.15.27:0/1260487285 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bedb91b0b0 con 0x55bedba28000 2024-05-21T03:29:21.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.069+0000 7ff17affd700 1 --2- 172.21.15.27:0/1260487285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bedba2cb30 0x55bedba2cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:29:21.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.069+0000 7ff17b7fe700 1 --2- 172.21.15.27:0/1260487285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bedba289c0 0x55bedba34a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:29:21.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.069+0000 7ff17a7fc700 1 --2- 172.21.15.27:0/1260487285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bedba28000 0x55bedba28480 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff1700a8cd0 tx=0x7ff1700a87c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3a0f729eb11152 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:21.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.069+0000 7ff161ffb700 1 -- 172.21.15.27:0/1260487285 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1700a8fd0 con 0x55bedba28000 2024-05-21T03:29:21.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.070+0000 7ff161ffb700 1 -- 172.21.15.27:0/1260487285 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1700ad380 con 0x55bedba28000 2024-05-21T03:29:21.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.070+0000 7ff161ffb700 1 -- 172.21.15.27:0/1260487285 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1700a8230 con 0x55bedba28000 2024-05-21T03:29:21.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.070+0000 7ff18c110b80 1 -- 172.21.15.27:0/1260487285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bedba28000 msgr2=0x55bedba28480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:21.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.070+0000 7ff18c110b80 1 --2- 172.21.15.27:0/1260487285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bedba28000 0x55bedba28480 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff1700a8cd0 tx=0x7ff1700a87c0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.070+0000 7ff18c110b80 1 -- 172.21.15.27:0/1260487285 shutdown_connections 2024-05-21T03:29:21.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.070+0000 7ff18c110b80 1 --2- 172.21.15.27:0/1260487285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bedba289c0 0x55bedba34a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.070+0000 7ff18c110b80 1 --2- 172.21.15.27:0/1260487285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bedba2cb30 0x55bedba2cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.070+0000 7ff18c110b80 1 --2- 172.21.15.27:0/1260487285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bedba28000 0x55bedba28480 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.070+0000 7ff18c110b80 1 -- 172.21.15.27:0/1260487285 >> 172.21.15.27:0/1260487285 conn(0x55bedb94f340 msgr2=0x55bedba1e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:21.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.070+0000 7ff18c110b80 1 -- 172.21.15.27:0/1260487285 shutdown_connections 2024-05-21T03:29:21.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.070+0000 7ff18c110b80 1 -- 172.21.15.27:0/1260487285 wait complete. 2024-05-21T03:29:21.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.071+0000 7ff18c110b80 1 -- start start 2024-05-21T03:29:21.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.071+0000 7ff18c110b80 1 Processor -- start 2024-05-21T03:29:21.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.071+0000 7ff18c110b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bedba28000 0x55bedba346f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:21.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.071+0000 7ff18c110b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bedba289c0 0x55bedba319f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:21.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.071+0000 7ff18c110b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bedba2cb30 0x55bedba31f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:21.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.071+0000 7ff18c110b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bedb8780a0 con 0x55bedba289c0 2024-05-21T03:29:21.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.071+0000 7ff18c110b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bedb8d9ea0 con 0x55bedba2cb30 2024-05-21T03:29:21.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.071+0000 7ff18c110b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bedba350b0 con 0x55bedba28000 2024-05-21T03:29:21.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.071+0000 7ff18c110b80 10 client.0 _ll_register_callbacks cb 0x55bedb90ca50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:29:21.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.071+0000 7ff17a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bedba289c0 0x55bedba319f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:21.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.071+0000 7ff17a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bedba289c0 0x55bedba319f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37958/0 (socket says 172.21.15.27:37958) 2024-05-21T03:29:21.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.071+0000 7ff17a7fc700 1 -- 172.21.15.27:0/4065712566 learned_addr learned my addr 172.21.15.27:0/4065712566 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:21.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.072+0000 7ff17affd700 1 --2- 172.21.15.27:0/4065712566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bedba28000 0x55bedba346f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:21.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.072+0000 7ff17b7fe700 1 --2- 172.21.15.27:0/4065712566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bedba2cb30 0x55bedba31f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:21.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.072+0000 7ff17a7fc700 1 -- 172.21.15.27:0/4065712566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bedba28000 msgr2=0x55bedba346f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:21.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.072+0000 7ff17a7fc700 1 --2- 172.21.15.27:0/4065712566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bedba28000 0x55bedba346f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.072+0000 7ff17a7fc700 1 -- 172.21.15.27:0/4065712566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bedba2cb30 msgr2=0x55bedba31f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:21.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.072+0000 7ff17a7fc700 1 --2- 172.21.15.27:0/4065712566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bedba2cb30 0x55bedba31f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.072+0000 7ff17a7fc700 1 -- 172.21.15.27:0/4065712566 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bedb8c19d0 con 0x55bedba289c0 2024-05-21T03:29:21.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.072+0000 7ff17b7fe700 1 --2- 172.21.15.27:0/4065712566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bedba2cb30 0x55bedba31f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:29:21.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.072+0000 7ff17a7fc700 1 --2- 172.21.15.27:0/4065712566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bedba289c0 0x55bedba319f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff17000b500 tx=0x7ff1700107e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fcd3009f0e6cc3a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:21.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.072+0000 7ff15affd700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:29:21.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.072+0000 7ff15affd700 1 -- 172.21.15.27:0/4065712566 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1700c9020 con 0x55bedba289c0 2024-05-21T03:29:21.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.072+0000 7ff15affd700 1 -- 172.21.15.27:0/4065712566 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7ff1700a55c0 con 0x55bedba289c0 2024-05-21T03:29:21.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.072+0000 7ff18c110b80 1 -- 172.21.15.27:0/4065712566 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1700ad380 con 0x55bedba289c0 2024-05-21T03:29:21.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.072+0000 7ff15affd700 1 -- 172.21.15.27:0/4065712566 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff1700a8fd0 con 0x55bedba289c0 2024-05-21T03:29:21.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.072+0000 7ff18c110b80 10 client.15747 fetch_fsmap learned FSMap version 35 2024-05-21T03:29:21.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.072+0000 7ff18c110b80 1 -- 172.21.15.27:0/4065712566 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55bedb92b860 con 0x55bedba289c0 2024-05-21T03:29:21.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.073+0000 7ff15affd700 1 -- 172.21.15.27:0/4065712566 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1700ad380 con 0x55bedba289c0 2024-05-21T03:29:21.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.073+0000 7ff15affd700 1 -- 172.21.15.27:0/4065712566 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7ff1700a4940 con 0x55bedba289c0 2024-05-21T03:29:21.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.073+0000 7ff18c110b80 10 client.15747 fetch_fsmap finished waiting for FSMap version 35 2024-05-21T03:29:21.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.073+0000 7ff18c110b80 10 client.15747 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-21T03:29:21.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.073+0000 7ff18c110b80 20 client.15747 populate_metadata read hostname 'smithi027' 2024-05-21T03:29:21.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.073+0000 7ff18c110b80 1 --2- 172.21.15.27:0/4065712566 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55bedba41a30 0x55bedba43e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:21.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.073+0000 7ff18c110b80 4 client.15747 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:29:21.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.073+0000 7ff18c110b80 1 -- 172.21.15.27:0/4065712566 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bedb983dc0 con 0x55bedba41a30 2024-05-21T03:29:21.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.073+0000 7ff17affd700 1 --2- 172.21.15.27:0/4065712566 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55bedba41a30 0x55bedba43e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:21.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.074+0000 7ff17affd700 1 --2- 172.21.15.27:0/4065712566 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55bedba41a30 0x55bedba43e20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f641f774f0dae15 server_cookie=fb47006177bbd491 in_seq=0 out_seq=0 2024-05-21T03:29:21.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.074+0000 7ff15affd700 10 client.15747 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:29:21.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.074+0000 7ff15affd700 1 -- 172.21.15.27:0/4065712566 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55bedb983dc0 con 0x55bedba41a30 2024-05-21T03:29:21.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.074+0000 7ff15affd700 10 client.15747 handle_command_reply: tid=0 2024-05-21T03:29:21.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.074+0000 7ff18c110b80 1 client.15747 shutdown 2024-05-21T03:29:21.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.075+0000 7ff18c110b80 1 -- 172.21.15.27:0/4065712566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bedba289c0 msgr2=0x55bedba319f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:21.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.075+0000 7ff18c110b80 1 --2- 172.21.15.27:0/4065712566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bedba289c0 0x55bedba319f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff17000b500 tx=0x7ff1700107e0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.075+0000 7ff18c110b80 1 -- 172.21.15.27:0/4065712566 shutdown_connections 2024-05-21T03:29:21.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.075+0000 7ff18c110b80 1 --2- 172.21.15.27:0/4065712566 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55bedba41a30 0x55bedba43e20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.075+0000 7ff18c110b80 1 --2- 172.21.15.27:0/4065712566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bedba2cb30 0x55bedba31f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.075+0000 7ff18c110b80 1 --2- 172.21.15.27:0/4065712566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bedba28000 0x55bedba346f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.075+0000 7ff15affd700 0 client.15747 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:29:21.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.075+0000 7ff18c110b80 1 --2- 172.21.15.27:0/4065712566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bedba289c0 0x55bedba319f0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.075+0000 7ff18c110b80 1 -- 172.21.15.27:0/4065712566 >> 172.21.15.27:0/4065712566 conn(0x55bedb94f340 msgr2=0x55bedba22be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:21.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.075+0000 7ff18c110b80 1 -- 172.21.15.27:0/4065712566 shutdown_connections 2024-05-21T03:29:21.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.075+0000 7ff18c110b80 1 -- 172.21.15.27:0/4065712566 wait complete. 2024-05-21T03:29:21.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.075+0000 7ff18c110b80 20 client.15747 trim_cache size 0 max 16384 2024-05-21T03:29:21.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.095+0000 7ff18c110b80 1 Processor -- start 2024-05-21T03:29:21.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.095+0000 7ff18c110b80 1 -- start start 2024-05-21T03:29:21.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.095+0000 7ff18c110b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bedba28000 0x55bedba18e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:21.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.095+0000 7ff18c110b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bedba289c0 0x55bedba2dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:21.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.095+0000 7ff18c110b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bedba2cb30 0x55bedba46e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:21.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.095+0000 7ff18c110b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c1abcc0 con 0x55bedba28000 2024-05-21T03:29:21.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.095+0000 7ff18c110b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c1ad830 con 0x55bedba289c0 2024-05-21T03:29:21.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.095+0000 7ff18c110b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c1ac390 con 0x55bedba2cb30 2024-05-21T03:29:21.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.095+0000 7ff17affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bedba28000 0x55bedba18e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:21.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.095+0000 7ff17affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bedba28000 0x55bedba18e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37970/0 (socket says 172.21.15.27:37970) 2024-05-21T03:29:21.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.095+0000 7ff17affd700 1 -- 172.21.15.27:0/365912024 learned_addr learned my addr 172.21.15.27:0/365912024 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:21.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.095+0000 7ff17a7fc700 1 --2- 172.21.15.27:0/365912024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bedba289c0 0x55bedba2dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:21.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.095+0000 7ff17affd700 1 -- 172.21.15.27:0/365912024 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bedba2cb30 msgr2=0x55bedba46e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:21.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.095+0000 7ff17affd700 1 --2- 172.21.15.27:0/365912024 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bedba2cb30 0x55bedba46e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.096+0000 7ff17affd700 1 -- 172.21.15.27:0/365912024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bedba289c0 msgr2=0x55bedba2dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:21.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.096+0000 7ff17affd700 1 --2- 172.21.15.27:0/365912024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bedba289c0 0x55bedba2dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.096+0000 7ff17affd700 1 -- 172.21.15.27:0/365912024 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bedba2f960 con 0x55bedba28000 2024-05-21T03:29:21.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.096+0000 7ff17a7fc700 1 --2- 172.21.15.27:0/365912024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bedba289c0 0x55bedba2dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:29:21.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.096+0000 7ff17affd700 1 --2- 172.21.15.27:0/365912024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bedba28000 0x55bedba18e70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff16c000f80 tx=0x7ff16c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ccfc61b266fb5a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:21.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.096+0000 7ff1617fa700 1 -- 172.21.15.27:0/365912024 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c038070 con 0x55bedba28000 2024-05-21T03:29:21.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.096+0000 7ff1617fa700 1 -- 172.21.15.27:0/365912024 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff16c032440 con 0x55bedba28000 2024-05-21T03:29:21.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.096+0000 7ff1617fa700 1 -- 172.21.15.27:0/365912024 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c0052d0 con 0x55bedba28000 2024-05-21T03:29:21.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.097+0000 7ff18c110b80 1 -- 172.21.15.27:0/365912024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bedba28000 msgr2=0x55bedba18e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:21.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.097+0000 7ff18c110b80 1 --2- 172.21.15.27:0/365912024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bedba28000 0x55bedba18e70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff16c000f80 tx=0x7ff16c020000 comp rx=0 tx=0).stop 2024-05-21T03:29:21.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.097+0000 7ff18c110b80 1 -- 172.21.15.27:0/365912024 shutdown_connections 2024-05-21T03:29:21.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.097+0000 7ff18c110b80 1 --2- 172.21.15.27:0/365912024 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bedba2cb30 0x55bedba46e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.097+0000 7ff18c110b80 1 --2- 172.21.15.27:0/365912024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bedba28000 0x55bedba18e70 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.097+0000 7ff18c110b80 1 --2- 172.21.15.27:0/365912024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bedba289c0 0x55bedba2dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.097+0000 7ff18c110b80 1 -- 172.21.15.27:0/365912024 >> 172.21.15.27:0/365912024 conn(0x55bedb94f340 msgr2=0x55bedba2d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:21.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.097+0000 7ff18c110b80 1 -- 172.21.15.27:0/365912024 shutdown_connections 2024-05-21T03:29:21.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.097+0000 7ff18c110b80 1 -- 172.21.15.27:0/365912024 wait complete. 2024-05-21T03:29:21.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.097+0000 7ff18c110b80 1 -- start start 2024-05-21T03:29:21.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.097+0000 7ff18c110b80 1 Processor -- start 2024-05-21T03:29:21.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.098+0000 7ff18c110b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bedba28000 0x55bedba47120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:21.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.098+0000 7ff18c110b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bedba289c0 0x55bedba47660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:21.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.098+0000 7ff18c110b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bedba2cb30 0x55bedba45630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:21.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.098+0000 7ff18c110b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c1ad130 con 0x55bedba2cb30 2024-05-21T03:29:21.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.098+0000 7ff18c110b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c1aca60 con 0x55bedba28000 2024-05-21T03:29:21.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.098+0000 7ff18c110b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c1a9c80 con 0x55bedba289c0 2024-05-21T03:29:21.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.098+0000 7ff18c110b80 10 client.0 _ll_register_callbacks cb 0x55bedb90ca50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:29:21.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.098+0000 7ff17b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bedba2cb30 0x55bedba45630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:21.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.098+0000 7ff17b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bedba2cb30 0x55bedba45630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37982/0 (socket says 172.21.15.27:37982) 2024-05-21T03:29:21.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.098+0000 7ff17b7fe700 1 -- 172.21.15.27:0/2648928625 learned_addr learned my addr 172.21.15.27:0/2648928625 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:21.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.098+0000 7ff17affd700 1 --2- 172.21.15.27:0/2648928625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bedba28000 0x55bedba47120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:21.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.098+0000 7ff17a7fc700 1 --2- 172.21.15.27:0/2648928625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bedba289c0 0x55bedba47660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:21.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.098+0000 7ff17b7fe700 1 -- 172.21.15.27:0/2648928625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bedba289c0 msgr2=0x55bedba47660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:21.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.098+0000 7ff17b7fe700 1 --2- 172.21.15.27:0/2648928625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bedba289c0 0x55bedba47660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.098+0000 7ff17b7fe700 1 -- 172.21.15.27:0/2648928625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bedba28000 msgr2=0x55bedba47120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:21.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.098+0000 7ff17b7fe700 1 --2- 172.21.15.27:0/2648928625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bedba28000 0x55bedba47120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.098+0000 7ff17b7fe700 1 -- 172.21.15.27:0/2648928625 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bedba45e70 con 0x55bedba2cb30 2024-05-21T03:29:21.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.098+0000 7ff17a7fc700 1 --2- 172.21.15.27:0/2648928625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bedba289c0 0x55bedba47660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:29:21.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.098+0000 7ff17b7fe700 1 --2- 172.21.15.27:0/2648928625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bedba2cb30 0x55bedba45630 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff174003900 tx=0x7ff174003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d14407bc4dc8483 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:21.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.098+0000 7ff15affd700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:29:21.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.098+0000 7ff15affd700 1 -- 172.21.15.27:0/2648928625 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17400bce0 con 0x55bedba2cb30 2024-05-21T03:29:21.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.098+0000 7ff15affd700 1 -- 172.21.15.27:0/2648928625 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7ff17401b470 con 0x55bedba2cb30 2024-05-21T03:29:21.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.098+0000 7ff18c110b80 1 -- 172.21.15.27:0/2648928625 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff16c032440 con 0x55bedba2cb30 2024-05-21T03:29:21.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.099+0000 7ff15affd700 1 -- 172.21.15.27:0/2648928625 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff17401b670 con 0x55bedba2cb30 2024-05-21T03:29:21.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.099+0000 7ff18c110b80 10 client.15753 fetch_fsmap learned FSMap version 35 2024-05-21T03:29:21.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.099+0000 7ff18c110b80 1 -- 172.21.15.27:0/2648928625 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55bedbaacf20 con 0x55bedba2cb30 2024-05-21T03:29:21.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.099+0000 7ff15affd700 1 -- 172.21.15.27:0/2648928625 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff16c032440 con 0x55bedba2cb30 2024-05-21T03:29:21.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.099+0000 7ff15affd700 1 -- 172.21.15.27:0/2648928625 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7ff174012860 con 0x55bedba2cb30 2024-05-21T03:29:21.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.099+0000 7ff18c110b80 10 client.15753 fetch_fsmap finished waiting for FSMap version 35 2024-05-21T03:29:21.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.099+0000 7ff18c110b80 10 client.15753 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-21T03:29:21.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.099+0000 7ff18c110b80 20 client.15753 populate_metadata read hostname 'smithi027' 2024-05-21T03:29:21.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.099+0000 7ff18c110b80 1 --2- 172.21.15.27:0/2648928625 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55bedbaad690 0x55bedbaafab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:21.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.099+0000 7ff18c110b80 4 client.15753 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "72cf81f5", "release": true, "await": true}] 2024-05-21T03:29:21.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.099+0000 7ff18c110b80 1 -- 172.21.15.27:0/2648928625 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "quiesce db", "set_id": "72cf81f5", "release": true, "await": true}) -- 0x55bedb97d5a0 con 0x55bedbaad690 2024-05-21T03:29:21.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.099+0000 7ff17affd700 1 --2- 172.21.15.27:0/2648928625 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55bedbaad690 0x55bedbaafab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:21.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.100+0000 7ff17affd700 1 --2- 172.21.15.27:0/2648928625 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55bedbaad690 0x55bedbaafab0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bfd9c2efe3ce99a server_cookie=1af35e70d3ad7cf6 in_seq=0 out_seq=0 2024-05-21T03:29:21.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.100+0000 7ff15affd700 10 client.15753 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:29:21.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.111+0000 7ff15affd700 1 -- 172.21.15.27:0/2648928625 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55bedb97d5a0 con 0x55bedbaad690 2024-05-21T03:29:21.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.111+0000 7ff15affd700 10 client.15753 handle_command_reply: tid=0 2024-05-21T03:29:21.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.112+0000 7ff18c110b80 1 client.15753 shutdown 2024-05-21T03:29:21.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.113+0000 7ff18c110b80 1 -- 172.21.15.27:0/2648928625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bedba2cb30 msgr2=0x55bedba45630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:21.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.113+0000 7ff18c110b80 1 --2- 172.21.15.27:0/2648928625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bedba2cb30 0x55bedba45630 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff174003900 tx=0x7ff174003c10 comp rx=0 tx=0).stop 2024-05-21T03:29:21.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.113+0000 7ff18c110b80 1 -- 172.21.15.27:0/2648928625 shutdown_connections 2024-05-21T03:29:21.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.113+0000 7ff18c110b80 1 --2- 172.21.15.27:0/2648928625 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55bedbaad690 0x55bedbaafab0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.113+0000 7ff18c110b80 1 --2- 172.21.15.27:0/2648928625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bedba2cb30 0x55bedba45630 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.113+0000 7ff18c110b80 1 --2- 172.21.15.27:0/2648928625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bedba28000 0x55bedba47120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.113+0000 7ff18c110b80 1 --2- 172.21.15.27:0/2648928625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bedba289c0 0x55bedba47660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.114+0000 7ff18c110b80 1 -- 172.21.15.27:0/2648928625 >> 172.21.15.27:0/2648928625 conn(0x55bedb94f340 msgr2=0x55bedba33ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:21.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.114+0000 7ff15affd700 0 client.15753 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:29:21.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.114+0000 7ff18c110b80 1 -- 172.21.15.27:0/2648928625 shutdown_connections 2024-05-21T03:29:21.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.114+0000 7ff18c110b80 1 -- 172.21.15.27:0/2648928625 wait complete. 2024-05-21T03:29:21.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.114+0000 7ff18c110b80 20 client.15753 trim_cache size 0 max 16384 2024-05-21T03:29:21.120 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:29:21.120 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 35, 2024-05-21T03:29:21.120 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24476, 2024-05-21T03:29:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 13, 2024-05-21T03:29:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-05-21T03:29:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: "72cf81f5": { 2024-05-21T03:29:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 13, 2024-05-21T03:29:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-05-21T03:29:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-05-21T03:29:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-05-21T03:29:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-05-21T03:29:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:29:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-05-21T03:29:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 141.4, 2024-05-21T03:29:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-05-21T03:29:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-05-21T03:29:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-05-21T03:29:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-05-21T03:29:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-05-21T03:29:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-05-21T03:29:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:29:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:29:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:29:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:29:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:29:21.122 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:29:21.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.120+0000 7ff18187a700 1 -- 172.21.15.27:0/2451162012 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7ff1640b29f0 msgr2=0x7ff1640b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:21.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.120+0000 7ff18187a700 1 --2- 172.21.15.27:0/2451162012 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7ff1640b29f0 0x7ff1640b4eb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff16c009e60 tx=0x7ff16c00b7b0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.120+0000 7ff18187a700 1 -- 172.21.15.27:0/2451162012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff17c102e30 msgr2=0x7ff17c19b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:21.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.120+0000 7ff18187a700 1 --2- 172.21.15.27:0/2451162012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff17c102e30 0x7ff17c19b010 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff17000b530 tx=0x7ff17000b840 comp rx=0 tx=0).stop 2024-05-21T03:29:21.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.121+0000 7ff18187a700 1 -- 172.21.15.27:0/2451162012 shutdown_connections 2024-05-21T03:29:21.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.121+0000 7ff18187a700 1 --2- 172.21.15.27:0/2451162012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff17c1084f0 0x7ff17c1a5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.121+0000 7ff18187a700 1 --2- 172.21.15.27:0/2451162012 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7ff1640b29f0 0x7ff1640b4eb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.121+0000 7ff18187a700 1 --2- 172.21.15.27:0/2451162012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff17c102470 0x7ff17c19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.121+0000 7ff18187a700 1 --2- 172.21.15.27:0/2451162012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff17c102e30 0x7ff17c19b010 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:21.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.121+0000 7ff18187a700 1 -- 172.21.15.27:0/2451162012 >> 172.21.15.27:0/2451162012 conn(0x7ff17c0793e0 msgr2=0x7ff17c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:21.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.121+0000 7ff18187a700 1 -- 172.21.15.27:0/2451162012 shutdown_connections 2024-05-21T03:29:21.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:21.121+0000 7ff18187a700 1 -- 172.21.15.27:0/2451162012 wait complete. 2024-05-21T03:29:21.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:21 smithi027 ceph-mon[32179]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:29:22.006 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '72cf81f5', total seconds elapsed: 83.8 2024-05-21T03:29:22.006 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.45 2024-05-21T03:29:22.007 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:29:22.018 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:21 smithi039 ceph-mon[36429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:29:22.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:21 smithi165 ceph-mon[36269]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:29:22.422 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:29:22.844 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:22 smithi027 ceph-mon[32179]: pgmap v163: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T03:29:23.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:22 smithi039 ceph-mon[36429]: pgmap v163: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T03:29:23.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:22 smithi165 ceph-mon[36269]: pgmap v163: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T03:29:24.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.530+0000 7fb282619700 1 -- 172.21.15.27:0/2746376468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb27c103f50 msgr2=0x7fb27c110430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:24.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.530+0000 7fb282619700 1 --2- 172.21.15.27:0/2746376468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb27c103f50 0x7fb27c110430 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb26c009990 tx=0x7fb26c009ca0 comp rx=0 tx=0).stop 2024-05-21T03:29:24.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.530+0000 7fb282619700 1 -- 172.21.15.27:0/2746376468 shutdown_connections 2024-05-21T03:29:24.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.530+0000 7fb282619700 1 --2- 172.21.15.27:0/2746376468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb27c103f50 0x7fb27c110430 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:24.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.530+0000 7fb282619700 1 --2- 172.21.15.27:0/2746376468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb27c1096a0 0x7fb27c109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:24.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.531+0000 7fb282619700 1 --2- 172.21.15.27:0/2746376468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb27c103590 0x7fb27c103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:24.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.531+0000 7fb282619700 1 -- 172.21.15.27:0/2746376468 >> 172.21.15.27:0/2746376468 conn(0x7fb27c07a6f0 msgr2=0x7fb27c07ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:24.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.531+0000 7fb282619700 1 -- 172.21.15.27:0/2746376468 shutdown_connections 2024-05-21T03:29:24.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.532+0000 7fb282619700 1 -- 172.21.15.27:0/2746376468 wait complete. 2024-05-21T03:29:24.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.532+0000 7fb282619700 1 Processor -- start 2024-05-21T03:29:24.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.532+0000 7fb282619700 1 -- start start 2024-05-21T03:29:24.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.533+0000 7fb282619700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb27c103590 0x7fb27c075060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:24.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.533+0000 7fb282619700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb27c103f50 0x7fb27c06d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:24.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.534+0000 7fb27bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb27c103590 0x7fb27c075060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:24.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.534+0000 7fb27bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb27c103590 0x7fb27c075060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38598/0 (socket says 172.21.15.27:38598) 2024-05-21T03:29:24.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.534+0000 7fb282619700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb27c1096a0 0x7fb27c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:24.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.534+0000 7fb282619700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c113810 con 0x7fb27c103590 2024-05-21T03:29:24.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.534+0000 7fb282619700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c113690 con 0x7fb27c103f50 2024-05-21T03:29:24.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.534+0000 7fb282619700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c113990 con 0x7fb27c1096a0 2024-05-21T03:29:24.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.534+0000 7fb27bfff700 1 -- 172.21.15.27:0/1942900401 learned_addr learned my addr 172.21.15.27:0/1942900401 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:24.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.534+0000 7fb27b7fe700 1 --2- 172.21.15.27:0/1942900401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb27c103f50 0x7fb27c06d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:24.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.535+0000 7fb280bb6700 1 --2- 172.21.15.27:0/1942900401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb27c1096a0 0x7fb27c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:24.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.535+0000 7fb27b7fe700 1 -- 172.21.15.27:0/1942900401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb27c1096a0 msgr2=0x7fb27c06d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:24.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.535+0000 7fb27b7fe700 1 --2- 172.21.15.27:0/1942900401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb27c1096a0 0x7fb27c06d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:24.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.535+0000 7fb27b7fe700 1 -- 172.21.15.27:0/1942900401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb27c103590 msgr2=0x7fb27c075060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:24.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.535+0000 7fb27b7fe700 1 --2- 172.21.15.27:0/1942900401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb27c103590 0x7fb27c075060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:24.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.535+0000 7fb27b7fe700 1 -- 172.21.15.27:0/1942900401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27c06dde0 con 0x7fb27c103f50 2024-05-21T03:29:24.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.535+0000 7fb27bfff700 1 --2- 172.21.15.27:0/1942900401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb27c103590 0x7fb27c075060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:29:24.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.535+0000 7fb27b7fe700 1 --2- 172.21.15.27:0/1942900401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb27c103f50 0x7fb27c06d030 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb27000b450 tx=0x7fb27000b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:24.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.535+0000 7fb2797fa700 1 -- 172.21.15.27:0/1942900401 <== mon.2 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb270015020 con 0x7fb27c103f50 2024-05-21T03:29:24.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.536+0000 7fb282619700 1 -- 172.21.15.27:0/1942900401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb27c1aeb10 con 0x7fb27c103f50 2024-05-21T03:29:24.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.536+0000 7fb2797fa700 1 -- 172.21.15.27:0/1942900401 <== mon.2 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb270011ce0 con 0x7fb27c103f50 2024-05-21T03:29:24.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.536+0000 7fb282619700 1 -- 172.21.15.27:0/1942900401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb27c1af070 con 0x7fb27c103f50 2024-05-21T03:29:24.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.537+0000 7fb2797fa700 1 -- 172.21.15.27:0/1942900401 <== mon.2 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb270015020 con 0x7fb27c103f50 2024-05-21T03:29:24.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.537+0000 7fb282619700 1 -- 172.21.15.27:0/1942900401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb268005330 con 0x7fb27c103f50 2024-05-21T03:29:24.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.541+0000 7fb2797fa700 1 -- 172.21.15.27:0/1942900401 <== mon.2 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7fb270018020 con 0x7fb27c103f50 2024-05-21T03:29:24.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.543+0000 7fb2797fa700 1 --2- 172.21.15.27:0/1942900401 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fb2640b2e70 0x7fb2640b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:24.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.543+0000 7fb2797fa700 1 -- 172.21.15.27:0/1942900401 <== mon.2 v2:172.21.15.39:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fb270011630 con 0x7fb27c103f50 2024-05-21T03:29:24.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.544+0000 7fb27bfff700 1 --2- 172.21.15.27:0/1942900401 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fb2640b2e70 0x7fb2640b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:24.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.544+0000 7fb2797fa700 1 -- 172.21.15.27:0/1942900401 <== mon.2 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb2700a96e0 con 0x7fb27c103f50 2024-05-21T03:29:24.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:24.548+0000 7fb27bfff700 1 --2- 172.21.15.27:0/1942900401 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fb2640b2e70 0x7fb2640b5330 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb274007350 tx=0x7fb274009610 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:25.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:24 smithi039 ceph-mon[36429]: pgmap v164: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 989 KiB/s wr, 263 op/s 2024-05-21T03:29:25.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:24 smithi165 ceph-mon[36269]: pgmap v164: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 989 KiB/s wr, 263 op/s 2024-05-21T03:29:25.049 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:24 smithi027 ceph-mon[32179]: pgmap v164: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 989 KiB/s wr, 263 op/s 2024-05-21T03:29:25.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:25.048+0000 7fb282619700 1 -- 172.21.15.27:0/1942900401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb268005620 con 0x7fb27c103f50 2024-05-21T03:29:25.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:25.050+0000 7fb2797fa700 1 -- 172.21.15.27:0/1942900401 <== mon.2 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11926 (secure 0 0 0) 0x7fb2700a96e0 con 0x7fb27c103f50 2024-05-21T03:29:25.050 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:29:25.051 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/148854869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":148854869},{"type":"v1","addr":"172.21.15.27:6835","nonce":148854869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/4145544730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":4145544730},{"type":"v1","addr":"172.21.15.27:6837","nonce":4145544730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T03:20:34.055985+0000","modified":"2024-05-21T03:29:20.250695+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24472,"mds_2":24457,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/745486764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":745486764},{"type":"v1","addr":"172.21.15.27:6839","nonce":745486764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6841/3982752351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3982752351},{"type":"v1","addr":"172.21.15.27:6841","nonce":3982752351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/3497521608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3497521608},{"type":"v1","addr":"172.21.15.39:6835","nonce":3497521608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.165:6835/3576910218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3576910218},{"type":"v1","addr":"172.21.15.165:6835","nonce":3576910218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/3673846852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3673846852},{"type":"v1","addr":"172.21.15.39:6837","nonce":3673846852}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6837/691356139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":691356139},{"type":"v1","addr":"172.21.15.165:6837","nonce":691356139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6839/4158170814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4158170814},{"type":"v1","addr":"172.21.15.165:6839","nonce":4158170814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6839/287607819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":287607819},{"type":"v1","addr":"172.21.15.39:6839","nonce":287607819}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24457,24476,24452,24472,24451]},"id":1}]} 2024-05-21T03:29:25.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:25.054+0000 7fb282619700 1 -- 172.21.15.27:0/1942900401 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fb2640b2e70 msgr2=0x7fb2640b5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:25.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:25.054+0000 7fb282619700 1 --2- 172.21.15.27:0/1942900401 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fb2640b2e70 0x7fb2640b5330 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb274007350 tx=0x7fb274009610 comp rx=0 tx=0).stop 2024-05-21T03:29:25.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:25.054+0000 7fb282619700 1 -- 172.21.15.27:0/1942900401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb27c103f50 msgr2=0x7fb27c06d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:25.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:25.054+0000 7fb282619700 1 --2- 172.21.15.27:0/1942900401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb27c103f50 0x7fb27c06d030 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb27000b450 tx=0x7fb27000b810 comp rx=0 tx=0).stop 2024-05-21T03:29:25.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:25.055+0000 7fb282619700 1 -- 172.21.15.27:0/1942900401 shutdown_connections 2024-05-21T03:29:25.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:25.055+0000 7fb282619700 1 --2- 172.21.15.27:0/1942900401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb27c1096a0 0x7fb27c06d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:25.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:25.055+0000 7fb282619700 1 --2- 172.21.15.27:0/1942900401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb27c103590 0x7fb27c075060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:25.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:25.055+0000 7fb282619700 1 --2- 172.21.15.27:0/1942900401 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fb2640b2e70 0x7fb2640b5330 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:25.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:25.055+0000 7fb282619700 1 --2- 172.21.15.27:0/1942900401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb27c103f50 0x7fb27c06d030 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:25.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:25.055+0000 7fb282619700 1 -- 172.21.15.27:0/1942900401 >> 172.21.15.27:0/1942900401 conn(0x7fb27c07a6f0 msgr2=0x7fb27c102100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:25.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:25.055+0000 7fb282619700 1 -- 172.21.15.27:0/1942900401 shutdown_connections 2024-05-21T03:29:25.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:25.055+0000 7fb282619700 1 -- 172.21.15.27:0/1942900401 wait complete. 2024-05-21T03:29:25.057 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 35 2024-05-21T03:29:25.929 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 141.45 --await-for 45.0' 2024-05-21T03:29:25.929 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 141.45 --await-for 45.0 2024-05-21T03:29:25.970 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:25 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/1942900401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:29:26.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:25 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/1942900401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:29:26.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:25 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/1942900401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:29:26.353 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:29:26.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:26 smithi027 ceph-mon[32179]: pgmap v165: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 989 KiB/s wr, 258 op/s 2024-05-21T03:29:26.959 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:26 smithi027 ceph-mon[32179]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:29:27.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:26 smithi039 ceph-mon[36429]: pgmap v165: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 989 KiB/s wr, 258 op/s 2024-05-21T03:29:27.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:26 smithi039 ceph-mon[36429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:29:27.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:26 smithi165 ceph-mon[36269]: pgmap v165: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 989 KiB/s wr, 258 op/s 2024-05-21T03:29:27.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:26 smithi165 ceph-mon[36269]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:29:28.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.457+0000 7f34e913e700 1 -- 172.21.15.27:0/2100473454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34e41084f0 msgr2=0x7f34e4105fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:28.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.457+0000 7f34e913e700 1 --2- 172.21.15.27:0/2100473454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34e41084f0 0x7f34e4105fa0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f34d40098f0 tx=0x7f34d4009c00 comp rx=0 tx=0).stop 2024-05-21T03:29:28.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.458+0000 7f34e913e700 1 -- 172.21.15.27:0/2100473454 shutdown_connections 2024-05-21T03:29:28.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.458+0000 7f34e913e700 1 --2- 172.21.15.27:0/2100473454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34e406b240 0x7f34e406b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:28.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.458+0000 7f34e913e700 1 --2- 172.21.15.27:0/2100473454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34e41084f0 0x7f34e4105fa0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:28.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.458+0000 7f34e913e700 1 --2- 172.21.15.27:0/2100473454 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34e41064e0 0x7f34e4106960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:28.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.458+0000 7f34e913e700 1 -- 172.21.15.27:0/2100473454 >> 172.21.15.27:0/2100473454 conn(0x7f34e40793e0 msgr2=0x7f34e40797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:28.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.458+0000 7f34e913e700 1 -- 172.21.15.27:0/2100473454 shutdown_connections 2024-05-21T03:29:28.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.458+0000 7f34e913e700 1 -- 172.21.15.27:0/2100473454 wait complete. 2024-05-21T03:29:28.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.459+0000 7f34e913e700 1 Processor -- start 2024-05-21T03:29:28.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.459+0000 7f34e913e700 1 -- start start 2024-05-21T03:29:28.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.459+0000 7f34e913e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34e406b240 0x7f34e41a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:28.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.459+0000 7f34e913e700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34e41064e0 0x7f34e41a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:28.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.459+0000 7f34e913e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34e41084f0 0x7f34e41a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:28.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.460+0000 7f34e913e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f34e4119ce0 con 0x7f34e41084f0 2024-05-21T03:29:28.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.460+0000 7f34e913e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f34e4119b60 con 0x7f34e406b240 2024-05-21T03:29:28.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.460+0000 7f34e913e700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f34e4119e60 con 0x7f34e41064e0 2024-05-21T03:29:28.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.460+0000 7f34e259c700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34e41064e0 0x7f34e41a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:28.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.460+0000 7f34e259c700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34e41064e0 0x7f34e41a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.27:35616/0 (socket says 172.21.15.27:35616) 2024-05-21T03:29:28.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.460+0000 7f34e259c700 1 -- 172.21.15.27:0/3879379464 learned_addr learned my addr 172.21.15.27:0/3879379464 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:28.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.460+0000 7f34e2d9d700 1 --2- 172.21.15.27:0/3879379464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34e406b240 0x7f34e41a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:28.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.460+0000 7f34e359e700 1 --2- 172.21.15.27:0/3879379464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34e41084f0 0x7f34e41a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:28.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.460+0000 7f34e259c700 1 -- 172.21.15.27:0/3879379464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34e406b240 msgr2=0x7f34e41a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:28.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.460+0000 7f34e259c700 1 --2- 172.21.15.27:0/3879379464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34e406b240 0x7f34e41a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:28.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.460+0000 7f34e259c700 1 -- 172.21.15.27:0/3879379464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34e41084f0 msgr2=0x7f34e41a4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:28.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.460+0000 7f34e259c700 1 --2- 172.21.15.27:0/3879379464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34e41084f0 0x7f34e41a4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:28.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.460+0000 7f34e259c700 1 -- 172.21.15.27:0/3879379464 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e41a51f0 con 0x7f34e41064e0 2024-05-21T03:29:28.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.460+0000 7f34e359e700 1 --2- 172.21.15.27:0/3879379464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34e41084f0 0x7f34e41a4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:29:28.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.460+0000 7f34e2d9d700 1 --2- 172.21.15.27:0/3879379464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34e406b240 0x7f34e41a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:29:28.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.461+0000 7f34e259c700 1 --2- 172.21.15.27:0/3879379464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34e41064e0 0x7f34e41a0770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f34d800e9a0 tx=0x7f34d800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:28.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.461+0000 7f34cbfff700 1 -- 172.21.15.27:0/3879379464 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d800ca10 con 0x7f34e41064e0 2024-05-21T03:29:28.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.461+0000 7f34cbfff700 1 -- 172.21.15.27:0/3879379464 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34d800cbb0 con 0x7f34e41064e0 2024-05-21T03:29:28.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.461+0000 7f34cbfff700 1 -- 172.21.15.27:0/3879379464 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8018680 con 0x7f34e41064e0 2024-05-21T03:29:28.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.461+0000 7f34e913e700 1 -- 172.21.15.27:0/3879379464 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e41a54e0 con 0x7f34e41064e0 2024-05-21T03:29:28.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.461+0000 7f34e913e700 1 -- 172.21.15.27:0/3879379464 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e41acd60 con 0x7f34e41064e0 2024-05-21T03:29:28.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.467+0000 7f34cbfff700 1 -- 172.21.15.27:0/3879379464 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7f34d8010430 con 0x7f34e41064e0 2024-05-21T03:29:28.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.468+0000 7f34cbfff700 1 --2- 172.21.15.27:0/3879379464 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f34cc0b2ba0 0x7f34cc0b5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:28.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.469+0000 7f34cbfff700 1 -- 172.21.15.27:0/3879379464 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f34d80e2700 con 0x7f34e41064e0 2024-05-21T03:29:28.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.469+0000 7f34f39d4b80 1 Processor -- start 2024-05-21T03:29:28.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.469+0000 7f34e2d9d700 1 --2- 172.21.15.27:0/3879379464 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f34cc0b2ba0 0x7f34cc0b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:28.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.469+0000 7f34f39d4b80 1 -- start start 2024-05-21T03:29:28.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.470+0000 7f34f39d4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b584e7c0 0x55e3b584eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:28.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.470+0000 7f34f39d4b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b5849c90 0x55e3b584a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:28.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.470+0000 7f34f39d4b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e3b584a650 0x55e3b58566c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:28.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.470+0000 7f34f39d4b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e3b57a7f10 con 0x55e3b584a650 2024-05-21T03:29:28.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.470+0000 7f34f39d4b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e3b5797ab0 con 0x55e3b584e7c0 2024-05-21T03:29:28.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.470+0000 7f34f39d4b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e3b579baf0 con 0x55e3b5849c90 2024-05-21T03:29:28.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.470+0000 7f34e359e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e3b584a650 0x55e3b58566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:28.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.470+0000 7f34e359e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e3b584a650 0x55e3b58566c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38620/0 (socket says 172.21.15.27:38620) 2024-05-21T03:29:28.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.470+0000 7f34e359e700 1 -- 172.21.15.27:0/3152995293 learned_addr learned my addr 172.21.15.27:0/3152995293 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:28.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.471+0000 7f34e2d9d700 1 --2- 172.21.15.27:0/3152995293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b584e7c0 0x55e3b584eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:28.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.471+0000 7f34e359e700 1 -- 172.21.15.27:0/3152995293 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b5849c90 msgr2=0x55e3b584a110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T03:29:28.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.471+0000 7f34e359e700 1 --2- 172.21.15.27:0/3152995293 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b5849c90 0x55e3b584a110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:28.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.471+0000 7f34e359e700 1 -- 172.21.15.27:0/3152995293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b584e7c0 msgr2=0x55e3b584eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:28.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.471+0000 7f34e359e700 1 --2- 172.21.15.27:0/3152995293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b584e7c0 0x55e3b584eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:28.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.471+0000 7f34e359e700 1 -- 172.21.15.27:0/3152995293 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3b5741150 con 0x55e3b584a650 2024-05-21T03:29:28.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.471+0000 7f34e2d9d700 1 --2- 172.21.15.27:0/3152995293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b584e7c0 0x55e3b584eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:29:28.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.472+0000 7f34e359e700 1 --2- 172.21.15.27:0/3152995293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e3b584a650 0x55e3b58566c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00e980 tx=0x7f34dc0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=647cfef1029b02e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:28.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.472+0000 7f34e2d9d700 1 --2- 172.21.15.27:0/3879379464 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f34cc0b2ba0 0x7f34cc0b5060 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f34d4009fd0 tx=0x7f34d4005dc0 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:28.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.472+0000 7f34c97fa700 1 -- 172.21.15.27:0/3152995293 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34dc019070 con 0x55e3b584a650 2024-05-21T03:29:28.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.472+0000 7f34c97fa700 1 -- 172.21.15.27:0/3152995293 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34dc011b50 con 0x55e3b584a650 2024-05-21T03:29:28.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.473+0000 7f34c97fa700 1 -- 172.21.15.27:0/3152995293 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34dc007ac0 con 0x55e3b584a650 2024-05-21T03:29:28.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.473+0000 7f34f39d4b80 1 -- 172.21.15.27:0/3152995293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e3b584a650 msgr2=0x55e3b58566c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:28.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.473+0000 7f34f39d4b80 1 --2- 172.21.15.27:0/3152995293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e3b584a650 0x55e3b58566c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00e980 tx=0x7f34dc0099e0 comp rx=0 tx=0).stop 2024-05-21T03:29:28.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.473+0000 7f34f39d4b80 1 -- 172.21.15.27:0/3152995293 shutdown_connections 2024-05-21T03:29:28.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.474+0000 7f34f39d4b80 1 --2- 172.21.15.27:0/3152995293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e3b584a650 0x55e3b58566c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:28.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.474+0000 7f34f39d4b80 1 --2- 172.21.15.27:0/3152995293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b584e7c0 0x55e3b584eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:28.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.474+0000 7f34f39d4b80 1 --2- 172.21.15.27:0/3152995293 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b5849c90 0x55e3b584a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:28.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.474+0000 7f34f39d4b80 1 -- 172.21.15.27:0/3152995293 >> 172.21.15.27:0/3152995293 conn(0x55e3b57753e0 msgr2=0x55e3b5844980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:28.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.474+0000 7f34f39d4b80 1 -- 172.21.15.27:0/3152995293 shutdown_connections 2024-05-21T03:29:28.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.475+0000 7f34f39d4b80 1 -- 172.21.15.27:0/3152995293 wait complete. 2024-05-21T03:29:28.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.475+0000 7f34f39d4b80 1 -- start start 2024-05-21T03:29:28.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.476+0000 7f34f39d4b80 1 Processor -- start 2024-05-21T03:29:28.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.476+0000 7f34f39d4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b5849c90 0x55e3b58562e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:28.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.477+0000 7f34f39d4b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e3b584a650 0x55e3b58535e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:28.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.477+0000 7f34e2d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b5849c90 0x55e3b58562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:28.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.477+0000 7f34e2d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b5849c90 0x55e3b58562e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.27:36510/0 (socket says 172.21.15.27:36510) 2024-05-21T03:29:28.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.477+0000 7f34e2d9d700 1 -- 172.21.15.27:0/3937785997 learned_addr learned my addr 172.21.15.27:0/3937785997 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:28.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.477+0000 7f34f39d4b80 1 --2- 172.21.15.27:0/3937785997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b584e7c0 0x55e3b5853b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:28.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.477+0000 7f34f39d4b80 1 -- 172.21.15.27:0/3937785997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e3b569e140 con 0x55e3b584a650 2024-05-21T03:29:28.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.477+0000 7f34e359e700 1 --2- 172.21.15.27:0/3937785997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b584e7c0 0x55e3b5853b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:28.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.477+0000 7f34f39d4b80 1 -- 172.21.15.27:0/3937785997 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e3b56fff40 con 0x55e3b5849c90 2024-05-21T03:29:28.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.477+0000 7f34f39d4b80 1 -- 172.21.15.27:0/3937785997 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e3b5856d20 con 0x55e3b584e7c0 2024-05-21T03:29:28.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.477+0000 7f34f39d4b80 10 client.0 _ll_register_callbacks cb 0x55e3b5732af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:29:28.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.477+0000 7f34e259c700 1 --2- 172.21.15.27:0/3937785997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e3b584a650 0x55e3b58535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:28.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.478+0000 7f34e259c700 1 -- 172.21.15.27:0/3937785997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b584e7c0 msgr2=0x55e3b5853b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:28.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.478+0000 7f34e259c700 1 --2- 172.21.15.27:0/3937785997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b584e7c0 0x55e3b5853b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:28.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.478+0000 7f34e259c700 1 -- 172.21.15.27:0/3937785997 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b5849c90 msgr2=0x55e3b58562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:28.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.478+0000 7f34e259c700 1 --2- 172.21.15.27:0/3937785997 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b5849c90 0x55e3b58562e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:28.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.478+0000 7f34e259c700 1 -- 172.21.15.27:0/3937785997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3b56e7a70 con 0x55e3b584a650 2024-05-21T03:29:28.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.479+0000 7f34e259c700 1 --2- 172.21.15.27:0/3937785997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e3b584a650 0x55e3b58535e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f34d80dfdf0 tx=0x7f34d80e0350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245aab54e7e26c83 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:28.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.479+0000 7f34c6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:29:28.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.479+0000 7f34c6ff5700 1 -- 172.21.15.27:0/3937785997 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d80cc2c0 con 0x55e3b584a650 2024-05-21T03:29:28.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.479+0000 7f34c6ff5700 1 -- 172.21.15.27:0/3937785997 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f34d80b2350 con 0x55e3b584a650 2024-05-21T03:29:28.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.479+0000 7f34f39d4b80 1 -- 172.21.15.27:0/3937785997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34dc011b50 con 0x55e3b584a650 2024-05-21T03:29:28.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.480+0000 7f34c6ff5700 1 -- 172.21.15.27:0/3937785997 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f34d80aa080 con 0x55e3b584a650 2024-05-21T03:29:28.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.480+0000 7f34f39d4b80 10 client.15768 fetch_fsmap learned FSMap version 36 2024-05-21T03:29:28.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.480+0000 7f34f39d4b80 1 -- 172.21.15.27:0/3937785997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e3b5751900 con 0x55e3b584a650 2024-05-21T03:29:28.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.481+0000 7f34c6ff5700 1 -- 172.21.15.27:0/3937785997 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34dc011b50 con 0x55e3b584a650 2024-05-21T03:29:28.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.482+0000 7f34c6ff5700 1 -- 172.21.15.27:0/3937785997 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f34d80107b0 con 0x55e3b584a650 2024-05-21T03:29:28.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.482+0000 7f34f39d4b80 10 client.15768 fetch_fsmap finished waiting for FSMap version 36 2024-05-21T03:29:28.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.482+0000 7f34f39d4b80 10 client.15768 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-21T03:29:28.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.482+0000 7f34f39d4b80 20 client.15768 populate_metadata read hostname 'smithi027' 2024-05-21T03:29:28.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.482+0000 7f34f39d4b80 1 --2- 172.21.15.27:0/3937785997 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55e3b58637a0 0x55e3b5865b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:28.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.482+0000 7f34f39d4b80 4 client.15768 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:29:28.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.482+0000 7f34f39d4b80 1 -- 172.21.15.27:0/3937785997 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3b57a9e60 con 0x55e3b58637a0 2024-05-21T03:29:28.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.483+0000 7f34e2d9d700 1 --2- 172.21.15.27:0/3937785997 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55e3b58637a0 0x55e3b5865b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:28.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.484+0000 7f34e2d9d700 1 --2- 172.21.15.27:0/3937785997 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55e3b58637a0 0x55e3b5865b90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edd53004d736368a server_cookie=433f035e2ef15ebf in_seq=0 out_seq=0 2024-05-21T03:29:28.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.484+0000 7f34c6ff5700 10 client.15768 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:29:28.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.485+0000 7f34c6ff5700 1 -- 172.21.15.27:0/3937785997 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e3b57a9e60 con 0x55e3b58637a0 2024-05-21T03:29:28.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.485+0000 7f34c6ff5700 10 client.15768 handle_command_reply: tid=0 2024-05-21T03:29:28.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.485+0000 7f34f39d4b80 1 client.15768 shutdown 2024-05-21T03:29:28.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.485+0000 7f34f39d4b80 1 -- 172.21.15.27:0/3937785997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e3b584a650 msgr2=0x55e3b58535e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:28.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.485+0000 7f34f39d4b80 1 --2- 172.21.15.27:0/3937785997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e3b584a650 0x55e3b58535e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f34d80dfdf0 tx=0x7f34d80e0350 comp rx=0 tx=0).stop 2024-05-21T03:29:28.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.486+0000 7f34f39d4b80 1 -- 172.21.15.27:0/3937785997 shutdown_connections 2024-05-21T03:29:28.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.486+0000 7f34f39d4b80 1 --2- 172.21.15.27:0/3937785997 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55e3b58637a0 0x55e3b5865b90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:28.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.486+0000 7f34c6ff5700 0 client.15768 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:29:28.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.486+0000 7f34f39d4b80 1 --2- 172.21.15.27:0/3937785997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b584e7c0 0x55e3b5853b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:28.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.486+0000 7f34f39d4b80 1 --2- 172.21.15.27:0/3937785997 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b5849c90 0x55e3b58562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:28.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.486+0000 7f34f39d4b80 1 --2- 172.21.15.27:0/3937785997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e3b584a650 0x55e3b58535e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:28.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.486+0000 7f34f39d4b80 1 -- 172.21.15.27:0/3937785997 >> 172.21.15.27:0/3937785997 conn(0x55e3b57753e0 msgr2=0x55e3b5848c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:28.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.486+0000 7f34f39d4b80 1 -- 172.21.15.27:0/3937785997 shutdown_connections 2024-05-21T03:29:28.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.486+0000 7f34f39d4b80 1 -- 172.21.15.27:0/3937785997 wait complete. 2024-05-21T03:29:28.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.486+0000 7f34f39d4b80 20 client.15768 trim_cache size 0 max 16384 2024-05-21T03:29:28.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.525+0000 7f34f39d4b80 1 Processor -- start 2024-05-21T03:29:28.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.525+0000 7f34f39d4b80 1 -- start start 2024-05-21T03:29:28.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.526+0000 7f34f39d4b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e3b5849c90 0x55e3b583ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:28.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.526+0000 7f34f39d4b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b584a650 0x55e3b5849390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:28.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.526+0000 7f34e2d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e3b5849c90 0x55e3b583ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:28.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.526+0000 7f34e2d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e3b5849c90 0x55e3b583ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38640/0 (socket says 172.21.15.27:38640) 2024-05-21T03:29:28.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.526+0000 7f34e2d9d700 1 -- 172.21.15.27:0/958103784 learned_addr learned my addr 172.21.15.27:0/958103784 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:28.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.527+0000 7f34f39d4b80 1 --2- 172.21.15.27:0/958103784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b584e7c0 0x55e3b586b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:28.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.527+0000 7f34e259c700 1 --2- 172.21.15.27:0/958103784 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b584a650 0x55e3b5849390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:28.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.527+0000 7f34e359e700 1 --2- 172.21.15.27:0/958103784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b584e7c0 0x55e3b586b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:28.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.527+0000 7f34f39d4b80 1 -- 172.21.15.27:0/958103784 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f34e41b4420 con 0x55e3b5849c90 2024-05-21T03:29:28.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.527+0000 7f34f39d4b80 1 -- 172.21.15.27:0/958103784 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f34e41b5f60 con 0x55e3b584e7c0 2024-05-21T03:29:28.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.527+0000 7f34f39d4b80 1 -- 172.21.15.27:0/958103784 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f34e41b4af0 con 0x55e3b584a650 2024-05-21T03:29:28.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.528+0000 7f34e359e700 1 -- 172.21.15.27:0/958103784 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b584a650 msgr2=0x55e3b5849390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:28.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.528+0000 7f34e359e700 1 --2- 172.21.15.27:0/958103784 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b584a650 0x55e3b5849390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:28.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.528+0000 7f34e359e700 1 -- 172.21.15.27:0/958103784 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e3b5849c90 msgr2=0x55e3b583ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:28.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.528+0000 7f34e359e700 1 --2- 172.21.15.27:0/958103784 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e3b5849c90 0x55e3b583ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:28.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.528+0000 7f34e359e700 1 -- 172.21.15.27:0/958103784 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3b5853c10 con 0x55e3b584e7c0 2024-05-21T03:29:28.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.528+0000 7f34e359e700 1 --2- 172.21.15.27:0/958103784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b584e7c0 0x55e3b586b3f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f34dc000f80 tx=0x7f34dc007700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5697439f6f558a06 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:28.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.528+0000 7f34c8ff9700 1 -- 172.21.15.27:0/958103784 <== mon.2 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34dc019040 con 0x55e3b584e7c0 2024-05-21T03:29:28.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.528+0000 7f34c8ff9700 1 -- 172.21.15.27:0/958103784 <== mon.2 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34dc007770 con 0x55e3b584e7c0 2024-05-21T03:29:28.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.528+0000 7f34c8ff9700 1 -- 172.21.15.27:0/958103784 <== mon.2 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34dc0118e0 con 0x55e3b584e7c0 2024-05-21T03:29:28.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.530+0000 7f34f39d4b80 1 -- 172.21.15.27:0/958103784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b584e7c0 msgr2=0x55e3b586b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:28.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.530+0000 7f34f39d4b80 1 --2- 172.21.15.27:0/958103784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b584e7c0 0x55e3b586b3f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f34dc000f80 tx=0x7f34dc007700 comp rx=0 tx=0).stop 2024-05-21T03:29:28.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.530+0000 7f34f39d4b80 1 -- 172.21.15.27:0/958103784 shutdown_connections 2024-05-21T03:29:28.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.530+0000 7f34f39d4b80 1 --2- 172.21.15.27:0/958103784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b584e7c0 0x55e3b586b3f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:28.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.530+0000 7f34f39d4b80 1 --2- 172.21.15.27:0/958103784 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e3b5849c90 0x55e3b583ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:28.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.530+0000 7f34f39d4b80 1 --2- 172.21.15.27:0/958103784 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b584a650 0x55e3b5849390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:28.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.530+0000 7f34f39d4b80 1 -- 172.21.15.27:0/958103784 >> 172.21.15.27:0/958103784 conn(0x55e3b57753e0 msgr2=0x55e3b5848c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:28.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.531+0000 7f34f39d4b80 1 -- 172.21.15.27:0/958103784 shutdown_connections 2024-05-21T03:29:28.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.531+0000 7f34f39d4b80 1 -- 172.21.15.27:0/958103784 wait complete. 2024-05-21T03:29:28.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.532+0000 7f34f39d4b80 1 -- start start 2024-05-21T03:29:28.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.532+0000 7f34f39d4b80 1 Processor -- start 2024-05-21T03:29:28.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.533+0000 7f34f39d4b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e3b5849c90 0x55e3b585c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:28.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.533+0000 7f34f39d4b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b584a650 0x55e3b5858f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:28.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.533+0000 7f34f39d4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b584e7c0 0x55e3b58594c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:28.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.533+0000 7f34f39d4b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f34e41b5890 con 0x55e3b5849c90 2024-05-21T03:29:28.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.533+0000 7f34e259c700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b584a650 0x55e3b5858f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:28.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.533+0000 7f34e259c700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b584a650 0x55e3b5858f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.27:35644/0 (socket says 172.21.15.27:35644) 2024-05-21T03:29:28.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.533+0000 7f34e259c700 1 -- 172.21.15.27:0/3971872595 learned_addr learned my addr 172.21.15.27:0/3971872595 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:28.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.533+0000 7f34f39d4b80 1 -- 172.21.15.27:0/3971872595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f34e41b51c0 con 0x55e3b584e7c0 2024-05-21T03:29:28.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.533+0000 7f34f39d4b80 1 -- 172.21.15.27:0/3971872595 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f34e41b23c0 con 0x55e3b584a650 2024-05-21T03:29:28.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.533+0000 7f34e359e700 1 --2- 172.21.15.27:0/3971872595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b584e7c0 0x55e3b58594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:28.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.533+0000 7f34f39d4b80 10 client.0 _ll_register_callbacks cb 0x55e3b5732af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:29:28.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.533+0000 7f34e2d9d700 1 --2- 172.21.15.27:0/3971872595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e3b5849c90 0x55e3b585c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:28.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.534+0000 7f34e359e700 1 -- 172.21.15.27:0/3971872595 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b584a650 msgr2=0x55e3b5858f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:28.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.534+0000 7f34e359e700 1 --2- 172.21.15.27:0/3971872595 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b584a650 0x55e3b5858f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:28.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.534+0000 7f34e359e700 1 -- 172.21.15.27:0/3971872595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e3b5849c90 msgr2=0x55e3b585c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:28.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.534+0000 7f34e359e700 1 --2- 172.21.15.27:0/3971872595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e3b5849c90 0x55e3b585c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:28.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.534+0000 7f34e359e700 1 -- 172.21.15.27:0/3971872595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3b5859cc0 con 0x55e3b584e7c0 2024-05-21T03:29:28.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.534+0000 7f34e359e700 1 --2- 172.21.15.27:0/3971872595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b584e7c0 0x55e3b58594c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f34dc009a20 tx=0x7f34dc007b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76b5da8bdf68927b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:28.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.534+0000 7f34c6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-21T03:29:28.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.534+0000 7f34c6ff5700 1 -- 172.21.15.27:0/3971872595 <== mon.2 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34dc02b070 con 0x55e3b584e7c0 2024-05-21T03:29:28.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.534+0000 7f34f39d4b80 1 -- 172.21.15.27:0/3971872595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34dc007770 con 0x55e3b584e7c0 2024-05-21T03:29:28.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.534+0000 7f34c6ff5700 1 -- 172.21.15.27:0/3971872595 <== mon.2 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f34dc0116a0 con 0x55e3b584e7c0 2024-05-21T03:29:28.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.534+0000 7f34c6ff5700 1 -- 172.21.15.27:0/3971872595 <== mon.2 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f34dc019040 con 0x55e3b584e7c0 2024-05-21T03:29:28.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.534+0000 7f34f39d4b80 10 client.25322 fetch_fsmap learned FSMap version 36 2024-05-21T03:29:28.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.534+0000 7f34f39d4b80 1 -- 172.21.15.27:0/3971872595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e3b58cf200 con 0x55e3b584e7c0 2024-05-21T03:29:28.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.535+0000 7f34c6ff5700 1 -- 172.21.15.27:0/3971872595 <== mon.2 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34dc007770 con 0x55e3b584e7c0 2024-05-21T03:29:28.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.535+0000 7f34c6ff5700 1 -- 172.21.15.27:0/3971872595 <== mon.2 v2:172.21.15.39:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f34dc01e610 con 0x55e3b584e7c0 2024-05-21T03:29:28.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.535+0000 7f34f39d4b80 10 client.25322 fetch_fsmap finished waiting for FSMap version 36 2024-05-21T03:29:28.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.535+0000 7f34f39d4b80 10 client.25322 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-21T03:29:28.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.535+0000 7f34f39d4b80 20 client.25322 populate_metadata read hostname 'smithi027' 2024-05-21T03:29:28.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.535+0000 7f34f39d4b80 1 --2- 172.21.15.27:0/3971872595 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55e3b58637a0 0x55e3b58d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:28.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.535+0000 7f34f39d4b80 4 client.25322 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.45, "await_for": 45.0}] 2024-05-21T03:29:28.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.535+0000 7f34f39d4b80 1 -- 172.21.15.27:0/3971872595 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.45, "await_for": 45.0}) -- 0x55e3b57a3640 con 0x55e3b58637a0 2024-05-21T03:29:28.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.535+0000 7f34e2d9d700 1 --2- 172.21.15.27:0/3971872595 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55e3b58637a0 0x55e3b58d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:28.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.536+0000 7f34c6ff5700 10 client.25322 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:29:28.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:28.537+0000 7f34e2d9d700 1 --2- 172.21.15.27:0/3971872595 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55e3b58637a0 0x55e3b58d1730 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a524170cb43cc5b2 server_cookie=f31e1dc6aff4fa64 in_seq=0 out_seq=0 2024-05-21T03:29:28.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:28 smithi027 ceph-mon[32179]: pgmap v166: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 2.1 MiB/s wr, 486 op/s 2024-05-21T03:29:29.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:28 smithi039 ceph-mon[36429]: pgmap v166: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 2.1 MiB/s wr, 486 op/s 2024-05-21T03:29:29.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:28 smithi165 ceph-mon[36269]: pgmap v166: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 2.1 MiB/s wr, 486 op/s 2024-05-21T03:29:31.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:30 smithi039 ceph-mon[36429]: pgmap v167: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 18 MiB/s wr, 3.65k op/s 2024-05-21T03:29:31.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:30 smithi165 ceph-mon[36269]: pgmap v167: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 18 MiB/s wr, 3.65k op/s 2024-05-21T03:29:31.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:30 smithi027 ceph-mon[32179]: pgmap v167: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 18 MiB/s wr, 3.65k op/s 2024-05-21T03:29:33.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:32 smithi039 ceph-mon[36429]: pgmap v168: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 18 MiB/s wr, 3.65k op/s 2024-05-21T03:29:33.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:32 smithi165 ceph-mon[36269]: pgmap v168: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 18 MiB/s wr, 3.65k op/s 2024-05-21T03:29:33.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:32 smithi027 ceph-mon[32179]: pgmap v168: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 18 MiB/s wr, 3.65k op/s 2024-05-21T03:29:35.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:34 smithi039 ceph-mon[36429]: pgmap v169: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 61 MiB/s wr, 12.84k op/s 2024-05-21T03:29:35.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:34 smithi165 ceph-mon[36269]: pgmap v169: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 61 MiB/s wr, 12.84k op/s 2024-05-21T03:29:35.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:34 smithi027 ceph-mon[32179]: pgmap v169: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 61 MiB/s wr, 12.84k op/s 2024-05-21T03:29:36.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:36.820+0000 7f34c6ff5700 1 -- 172.21.15.27:0/3971872595 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e3b57a3640 con 0x55e3b58637a0 2024-05-21T03:29:36.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:36.820+0000 7f34c6ff5700 10 client.25322 handle_command_reply: tid=0 2024-05-21T03:29:36.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:36.820+0000 7f34f39d4b80 1 client.25322 shutdown 2024-05-21T03:29:36.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:36.821+0000 7f34f39d4b80 1 -- 172.21.15.27:0/3971872595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b584e7c0 msgr2=0x55e3b58594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:36.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:36.821+0000 7f34f39d4b80 1 --2- 172.21.15.27:0/3971872595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b584e7c0 0x55e3b58594c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f34dc009a20 tx=0x7f34dc007b30 comp rx=0 tx=0).stop 2024-05-21T03:29:36.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:36.821+0000 7f34f39d4b80 1 -- 172.21.15.27:0/3971872595 shutdown_connections 2024-05-21T03:29:36.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:36.821+0000 7f34f39d4b80 1 --2- 172.21.15.27:0/3971872595 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55e3b58637a0 0x55e3b58d1730 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:36.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:36.821+0000 7f34f39d4b80 1 --2- 172.21.15.27:0/3971872595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b584e7c0 0x55e3b58594c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:36.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:36.821+0000 7f34f39d4b80 1 --2- 172.21.15.27:0/3971872595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e3b5849c90 0x55e3b585c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:36.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:36.821+0000 7f34f39d4b80 1 --2- 172.21.15.27:0/3971872595 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b584a650 0x55e3b5858f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:36.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:36.821+0000 7f34c6ff5700 0 client.25322 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:29:36.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:36.821+0000 7f34f39d4b80 1 -- 172.21.15.27:0/3971872595 >> 172.21.15.27:0/3971872595 conn(0x55e3b57753e0 msgr2=0x55e3b584d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:36.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:36.821+0000 7f34f39d4b80 1 -- 172.21.15.27:0/3971872595 shutdown_connections 2024-05-21T03:29:36.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:36.821+0000 7f34f39d4b80 1 -- 172.21.15.27:0/3971872595 wait complete. 2024-05-21T03:29:36.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:36.821+0000 7f34f39d4b80 20 client.25322 trim_cache size 0 max 16384 2024-05-21T03:29:36.823 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:29:36.823 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 36, 2024-05-21T03:29:36.823 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24476, 2024-05-21T03:29:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 15, 2024-05-21T03:29:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-05-21T03:29:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: "f36cc8d4": { 2024-05-21T03:29:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 15, 2024-05-21T03:29:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-05-21T03:29:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-05-21T03:29:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-05-21T03:29:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-05-21T03:29:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:29:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-05-21T03:29:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 141.4, 2024-05-21T03:29:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-05-21T03:29:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-05-21T03:29:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-05-21T03:29:36.825 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-05-21T03:29:36.825 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-05-21T03:29:36.825 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-05-21T03:29:36.825 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:29:36.825 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:29:36.825 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:29:36.825 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:29:36.825 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:29:36.825 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:29:36.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:36.821+0000 7f34e913e700 1 -- 172.21.15.27:0/3879379464 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f34cc0b2ba0 msgr2=0x7f34cc0b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:36.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:36.821+0000 7f34e913e700 1 --2- 172.21.15.27:0/3879379464 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f34cc0b2ba0 0x7f34cc0b5060 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f34d4009fd0 tx=0x7f34d4005dc0 comp rx=0 tx=0).stop 2024-05-21T03:29:36.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:36.821+0000 7f34e913e700 1 -- 172.21.15.27:0/3879379464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34e41064e0 msgr2=0x7f34e41a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:36.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:36.821+0000 7f34e913e700 1 --2- 172.21.15.27:0/3879379464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34e41064e0 0x7f34e41a0770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f34d800e9a0 tx=0x7f34d800ecb0 comp rx=0 tx=0).stop 2024-05-21T03:29:36.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:36.822+0000 7f34e913e700 1 -- 172.21.15.27:0/3879379464 shutdown_connections 2024-05-21T03:29:36.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:36.822+0000 7f34e913e700 1 --2- 172.21.15.27:0/3879379464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34e41084f0 0x7f34e41a4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:36.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:36.822+0000 7f34e913e700 1 --2- 172.21.15.27:0/3879379464 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f34cc0b2ba0 0x7f34cc0b5060 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:36.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:36.822+0000 7f34e913e700 1 --2- 172.21.15.27:0/3879379464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34e406b240 0x7f34e41a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:36.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:36.822+0000 7f34e913e700 1 --2- 172.21.15.27:0/3879379464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34e41064e0 0x7f34e41a0770 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:36.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:36.822+0000 7f34e913e700 1 -- 172.21.15.27:0/3879379464 >> 172.21.15.27:0/3879379464 conn(0x7f34e40793e0 msgr2=0x7f34e41002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:36.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:36.822+0000 7f34e913e700 1 -- 172.21.15.27:0/3879379464 shutdown_connections 2024-05-21T03:29:36.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:36.822+0000 7f34e913e700 1 -- 172.21.15.27:0/3879379464 wait complete. 2024-05-21T03:29:37.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:36 smithi039 ceph-mon[36429]: pgmap v170: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 60 MiB/s wr, 12.58k op/s 2024-05-21T03:29:37.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:36 smithi165 ceph-mon[36269]: pgmap v170: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 60 MiB/s wr, 12.58k op/s 2024-05-21T03:29:37.148 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:36 smithi027 ceph-mon[32179]: pgmap v170: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 60 MiB/s wr, 12.58k op/s 2024-05-21T03:29:37.512 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f36cc8d4', quiesce took 15.5 seconds. Will release after: 5.949999999999999 2024-05-21T03:29:39.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:38 smithi039 ceph-mon[36429]: pgmap v171: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 60 MiB/s wr, 12.58k op/s 2024-05-21T03:29:39.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:38 smithi039 ceph-mon[36429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:29:39.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:38 smithi165 ceph-mon[36269]: pgmap v171: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 60 MiB/s wr, 12.58k op/s 2024-05-21T03:29:39.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:38 smithi165 ceph-mon[36269]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:29:39.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:38 smithi027 ceph-mon[32179]: pgmap v171: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 60 MiB/s wr, 12.58k op/s 2024-05-21T03:29:39.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:38 smithi027 ceph-mon[32179]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:29:41.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:40 smithi039 ceph-mon[36429]: pgmap v172: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 90 MiB/s wr, 18.88k op/s 2024-05-21T03:29:41.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:40 smithi165 ceph-mon[36269]: pgmap v172: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 90 MiB/s wr, 18.88k op/s 2024-05-21T03:29:41.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:40 smithi027 ceph-mon[32179]: pgmap v172: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 90 MiB/s wr, 18.88k op/s 2024-05-21T03:29:43.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:42 smithi039 ceph-mon[36429]: pgmap v173: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 75 MiB/s wr, 15.71k op/s 2024-05-21T03:29:43.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:42 smithi165 ceph-mon[36269]: pgmap v173: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 75 MiB/s wr, 15.71k op/s 2024-05-21T03:29:43.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:42 smithi027 ceph-mon[32179]: pgmap v173: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 75 MiB/s wr, 15.71k op/s 2024-05-21T03:29:43.463 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:29:43.718 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:29:45.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:44 smithi039 ceph-mon[36429]: pgmap v174: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 84 MiB/s wr, 17.64k op/s 2024-05-21T03:29:45.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:44 smithi165 ceph-mon[36269]: pgmap v174: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 84 MiB/s wr, 17.64k op/s 2024-05-21T03:29:45.158 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:44 smithi027 ceph-mon[32179]: pgmap v174: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 84 MiB/s wr, 17.64k op/s 2024-05-21T03:29:45.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.709+0000 7f9e9111c700 1 -- 172.21.15.27:0/810554525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e8c102e00 msgr2=0x7f9e8c10f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:45.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.709+0000 7f9e9111c700 1 --2- 172.21.15.27:0/810554525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e8c102e00 0x7f9e8c10f2f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9e8400b0a0 tx=0x7f9e8400b3b0 comp rx=0 tx=0).stop 2024-05-21T03:29:45.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.709+0000 7f9e9111c700 1 -- 172.21.15.27:0/810554525 shutdown_connections 2024-05-21T03:29:45.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.709+0000 7f9e9111c700 1 --2- 172.21.15.27:0/810554525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e8c102e00 0x7f9e8c10f2f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:45.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.709+0000 7f9e9111c700 1 --2- 172.21.15.27:0/810554525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e8c108520 0x7f9e8c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:45.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.709+0000 7f9e9111c700 1 --2- 172.21.15.27:0/810554525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e8c102440 0x7f9e8c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:45.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.709+0000 7f9e9111c700 1 -- 172.21.15.27:0/810554525 >> 172.21.15.27:0/810554525 conn(0x7f9e8c0793f0 msgr2=0x7f9e8c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:45.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.709+0000 7f9e9111c700 1 -- 172.21.15.27:0/810554525 shutdown_connections 2024-05-21T03:29:45.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.709+0000 7f9e9111c700 1 -- 172.21.15.27:0/810554525 wait complete. 2024-05-21T03:29:45.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.710+0000 7f9e9111c700 1 Processor -- start 2024-05-21T03:29:45.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.710+0000 7f9e9111c700 1 -- start start 2024-05-21T03:29:45.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.710+0000 7f9e9111c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e8c102440 0x7f9e8c104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:45.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.710+0000 7f9e9111c700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e8c102e00 0x7f9e8c1045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:45.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.710+0000 7f9e9111c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e8c108520 0x7f9e8c104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:45.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.710+0000 7f9e9111c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8c1125f0 con 0x7f9e8c102440 2024-05-21T03:29:45.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.710+0000 7f9e9111c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8c112470 con 0x7f9e8c108520 2024-05-21T03:29:45.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.711+0000 7f9e9111c700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8c112770 con 0x7f9e8c102e00 2024-05-21T03:29:45.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.711+0000 7f9e8ad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e8c102440 0x7f9e8c104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:45.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.711+0000 7f9e8ad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e8c102440 0x7f9e8c104080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59920/0 (socket says 172.21.15.27:59920) 2024-05-21T03:29:45.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.711+0000 7f9e8ad9d700 1 -- 172.21.15.27:0/3763360186 learned_addr learned my addr 172.21.15.27:0/3763360186 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:45.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.711+0000 7f9e8a59c700 1 --2- 172.21.15.27:0/3763360186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e8c102e00 0x7f9e8c1045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:45.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.711+0000 7f9e8b59e700 1 --2- 172.21.15.27:0/3763360186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e8c108520 0x7f9e8c104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:45.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.711+0000 7f9e8ad9d700 1 -- 172.21.15.27:0/3763360186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e8c102e00 msgr2=0x7f9e8c1045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:45.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.711+0000 7f9e8ad9d700 1 --2- 172.21.15.27:0/3763360186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e8c102e00 0x7f9e8c1045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:45.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.711+0000 7f9e8ad9d700 1 -- 172.21.15.27:0/3763360186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e8c108520 msgr2=0x7f9e8c104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:45.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.711+0000 7f9e8ad9d700 1 --2- 172.21.15.27:0/3763360186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e8c108520 0x7f9e8c104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:45.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.711+0000 7f9e8ad9d700 1 -- 172.21.15.27:0/3763360186 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e8c1a8100 con 0x7f9e8c102440 2024-05-21T03:29:45.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.711+0000 7f9e8ad9d700 1 --2- 172.21.15.27:0/3763360186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e8c102440 0x7f9e8c104080 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00c920 tx=0x7f9e7c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:45.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.711+0000 7f9e73fff700 1 -- 172.21.15.27:0/3763360186 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c004a30 con 0x7f9e8c102440 2024-05-21T03:29:45.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.711+0000 7f9e9111c700 1 -- 172.21.15.27:0/3763360186 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e8c1a83f0 con 0x7f9e8c102440 2024-05-21T03:29:45.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.711+0000 7f9e73fff700 1 -- 172.21.15.27:0/3763360186 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e7c0054d0 con 0x7f9e8c102440 2024-05-21T03:29:45.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.711+0000 7f9e73fff700 1 -- 172.21.15.27:0/3763360186 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c007bd0 con 0x7f9e8c102440 2024-05-21T03:29:45.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.711+0000 7f9e9111c700 1 -- 172.21.15.27:0/3763360186 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e8c1a8980 con 0x7f9e8c102440 2024-05-21T03:29:45.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.712+0000 7f9e9111c700 1 -- 172.21.15.27:0/3763360186 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e8c06a230 con 0x7f9e8c102440 2024-05-21T03:29:45.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.714+0000 7f9e73fff700 1 -- 172.21.15.27:0/3763360186 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7f9e7c004bd0 con 0x7f9e8c102440 2024-05-21T03:29:45.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.715+0000 7f9e73fff700 1 --2- 172.21.15.27:0/3763360186 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f9e740b29f0 0x7f9e740b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:45.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.715+0000 7f9e8a59c700 1 --2- 172.21.15.27:0/3763360186 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f9e740b29f0 0x7f9e740b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:45.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.715+0000 7f9e73fff700 1 -- 172.21.15.27:0/3763360186 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f9e7c0e24d0 con 0x7f9e8c102440 2024-05-21T03:29:45.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.715+0000 7f9e8a59c700 1 --2- 172.21.15.27:0/3763360186 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f9e740b29f0 0x7f9e740b4eb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c079b40 tx=0x7f9e8000b3c0 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:45.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.718+0000 7f9e73fff700 1 -- 172.21.15.27:0/3763360186 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9e7c014030 con 0x7f9e8c102440 2024-05-21T03:29:45.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.979+0000 7f9e9111c700 1 -- 172.21.15.27:0/3763360186 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e8c105560 con 0x7f9e8c102440 2024-05-21T03:29:45.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.979+0000 7f9e73fff700 1 -- 172.21.15.27:0/3763360186 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11924 (secure 0 0 0) 0x7f9e7c0a9eb0 con 0x7f9e8c102440 2024-05-21T03:29:45.979 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:29:45.979 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/148854869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":148854869},{"type":"v1","addr":"172.21.15.27:6835","nonce":148854869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/4145544730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":4145544730},{"type":"v1","addr":"172.21.15.27:6837","nonce":4145544730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T03:20:34.055985+0000","modified":"2024-05-21T03:29:37.263175+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24472,"mds_2":24457,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/745486764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":745486764},{"type":"v1","addr":"172.21.15.27:6839","nonce":745486764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6841/3982752351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3982752351},{"type":"v1","addr":"172.21.15.27:6841","nonce":3982752351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/3497521608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3497521608},{"type":"v1","addr":"172.21.15.39:6835","nonce":3497521608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.165:6835/3576910218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3576910218},{"type":"v1","addr":"172.21.15.165:6835","nonce":3576910218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/3673846852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3673846852},{"type":"v1","addr":"172.21.15.39:6837","nonce":3673846852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6837/691356139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":691356139},{"type":"v1","addr":"172.21.15.165:6837","nonce":691356139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6839/4158170814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4158170814},{"type":"v1","addr":"172.21.15.165:6839","nonce":4158170814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6839/287607819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":287607819},{"type":"v1","addr":"172.21.15.39:6839","nonce":287607819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24457,24452,24451,24472]},"id":1}]} 2024-05-21T03:29:45.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.982+0000 7f9e9111c700 1 -- 172.21.15.27:0/3763360186 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f9e740b29f0 msgr2=0x7f9e740b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:45.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.982+0000 7f9e9111c700 1 --2- 172.21.15.27:0/3763360186 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f9e740b29f0 0x7f9e740b4eb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c079b40 tx=0x7f9e8000b3c0 comp rx=0 tx=0).stop 2024-05-21T03:29:45.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.982+0000 7f9e9111c700 1 -- 172.21.15.27:0/3763360186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e8c102440 msgr2=0x7f9e8c104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:45.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.982+0000 7f9e9111c700 1 --2- 172.21.15.27:0/3763360186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e8c102440 0x7f9e8c104080 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00c920 tx=0x7f9e7c00cc30 comp rx=0 tx=0).stop 2024-05-21T03:29:45.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.982+0000 7f9e9111c700 1 -- 172.21.15.27:0/3763360186 shutdown_connections 2024-05-21T03:29:45.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.982+0000 7f9e9111c700 1 --2- 172.21.15.27:0/3763360186 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f9e740b29f0 0x7f9e740b4eb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:45.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.982+0000 7f9e9111c700 1 --2- 172.21.15.27:0/3763360186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e8c108520 0x7f9e8c104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:45.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.982+0000 7f9e9111c700 1 --2- 172.21.15.27:0/3763360186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e8c102440 0x7f9e8c104080 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:45.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.982+0000 7f9e9111c700 1 --2- 172.21.15.27:0/3763360186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e8c102e00 0x7f9e8c1045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:45.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.982+0000 7f9e9111c700 1 -- 172.21.15.27:0/3763360186 >> 172.21.15.27:0/3763360186 conn(0x7f9e8c0793f0 msgr2=0x7f9e8c06b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:45.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.982+0000 7f9e9111c700 1 -- 172.21.15.27:0/3763360186 shutdown_connections 2024-05-21T03:29:45.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:45.982+0000 7f9e9111c700 1 -- 172.21.15.27:0/3763360186 wait complete. 2024-05-21T03:29:45.983 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 37 2024-05-21T03:29:46.786 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id f36cc8d4 --release --await' 2024-05-21T03:29:46.786 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id f36cc8d4 --release --await 2024-05-21T03:29:47.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:46 smithi165 ceph-mon[36269]: pgmap v175: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 MiB/s wr, 8.44k op/s 2024-05-21T03:29:47.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:46 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/3763360186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:29:47.045 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:29:47.168 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:46 smithi027 ceph-mon[32179]: pgmap v175: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 MiB/s wr, 8.44k op/s 2024-05-21T03:29:47.168 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:46 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3763360186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:29:47.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:46 smithi039 ceph-mon[36429]: pgmap v175: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 MiB/s wr, 8.44k op/s 2024-05-21T03:29:47.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:46 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/3763360186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:29:48.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.683+0000 7fa2d762a700 1 -- 172.21.15.27:0/2710125018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2d006b220 msgr2=0x7fa2d006b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:48.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.683+0000 7fa2d762a700 1 --2- 172.21.15.27:0/2710125018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2d006b220 0x7fa2d006b6a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0009990 tx=0x7fa2c0009ca0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.683+0000 7fa2d762a700 1 -- 172.21.15.27:0/2710125018 shutdown_connections 2024-05-21T03:29:48.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.683+0000 7fa2d762a700 1 --2- 172.21.15.27:0/2710125018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2d006bbe0 0x7fa2d01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.683+0000 7fa2d762a700 1 --2- 172.21.15.27:0/2710125018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2d01095b0 0x7fa2d0109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.683+0000 7fa2d762a700 1 --2- 172.21.15.27:0/2710125018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2d006b220 0x7fa2d006b6a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.683+0000 7fa2d762a700 1 -- 172.21.15.27:0/2710125018 >> 172.21.15.27:0/2710125018 conn(0x7fa2d00793f0 msgr2=0x7fa2d0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:48.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.684+0000 7fa2d762a700 1 -- 172.21.15.27:0/2710125018 shutdown_connections 2024-05-21T03:29:48.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.684+0000 7fa2d762a700 1 -- 172.21.15.27:0/2710125018 wait complete. 2024-05-21T03:29:48.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.684+0000 7fa2d762a700 1 Processor -- start 2024-05-21T03:29:48.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.684+0000 7fa2d762a700 1 -- start start 2024-05-21T03:29:48.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.684+0000 7fa2d762a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2d006b220 0x7fa2d0102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:48.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.684+0000 7fa2d762a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2d006bbe0 0x7fa2d0102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:48.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.685+0000 7fa2d762a700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2d01095b0 0x7fa2d0102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:48.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.685+0000 7fa2d762a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d01136c0 con 0x7fa2d006bbe0 2024-05-21T03:29:48.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.685+0000 7fa2d762a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d0113540 con 0x7fa2d006b220 2024-05-21T03:29:48.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.685+0000 7fa2d762a700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d0113840 con 0x7fa2d01095b0 2024-05-21T03:29:48.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.685+0000 7fa2d4bc5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2d006bbe0 0x7fa2d0102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:48.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.685+0000 7fa2d4bc5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2d006bbe0 0x7fa2d0102a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59940/0 (socket says 172.21.15.27:59940) 2024-05-21T03:29:48.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.685+0000 7fa2d4bc5700 1 -- 172.21.15.27:0/794284733 learned_addr learned my addr 172.21.15.27:0/794284733 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:48.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.685+0000 7fa2d53c6700 1 --2- 172.21.15.27:0/794284733 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2d006b220 0x7fa2d0102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:48.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.685+0000 7fa2d5bc7700 1 --2- 172.21.15.27:0/794284733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2d01095b0 0x7fa2d0102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.685+0000 7fa2d4bc5700 1 -- 172.21.15.27:0/794284733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2d01095b0 msgr2=0x7fa2d0102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.685+0000 7fa2d4bc5700 1 --2- 172.21.15.27:0/794284733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2d01095b0 0x7fa2d0102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.685+0000 7fa2d4bc5700 1 -- 172.21.15.27:0/794284733 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2d006b220 msgr2=0x7fa2d0102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.685+0000 7fa2d4bc5700 1 --2- 172.21.15.27:0/794284733 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2d006b220 0x7fa2d0102510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.685+0000 7fa2d4bc5700 1 -- 172.21.15.27:0/794284733 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d01ac5a0 con 0x7fa2d006bbe0 2024-05-21T03:29:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.685+0000 7fa2d53c6700 1 --2- 172.21.15.27:0/794284733 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2d006b220 0x7fa2d0102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:29:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.685+0000 7fa2d5bc7700 1 --2- 172.21.15.27:0/794284733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2d01095b0 0x7fa2d0102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:29:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.685+0000 7fa2d4bc5700 1 --2- 172.21.15.27:0/794284733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2d006bbe0 0x7fa2d0102a50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00057f0 tx=0x7fa2c0003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.685+0000 7fa2c67fc700 1 -- 172.21.15.27:0/794284733 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0027070 con 0x7fa2d006bbe0 2024-05-21T03:29:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.685+0000 7fa2c67fc700 1 -- 172.21.15.27:0/794284733 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2c00041c0 con 0x7fa2d006bbe0 2024-05-21T03:29:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.685+0000 7fa2d762a700 1 -- 172.21.15.27:0/794284733 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d01ac7d0 con 0x7fa2d006bbe0 2024-05-21T03:29:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.685+0000 7fa2c67fc700 1 -- 172.21.15.27:0/794284733 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0022e60 con 0x7fa2d006bbe0 2024-05-21T03:29:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.686+0000 7fa2d762a700 1 -- 172.21.15.27:0/794284733 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d01acd60 con 0x7fa2d006bbe0 2024-05-21T03:29:48.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.688+0000 7fa2c67fc700 1 -- 172.21.15.27:0/794284733 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7fa2c002a070 con 0x7fa2d006bbe0 2024-05-21T03:29:48.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.689+0000 7fa2c67fc700 1 --2- 172.21.15.27:0/794284733 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fa2bc0b2a70 0x7fa2bc0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:48.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.689+0000 7fa2d53c6700 1 --2- 172.21.15.27:0/794284733 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fa2bc0b2a70 0x7fa2bc0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:48.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.689+0000 7fa2c67fc700 1 -- 172.21.15.27:0/794284733 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fa2c00e3330 con 0x7fa2d006bbe0 2024-05-21T03:29:48.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.689+0000 7fa2e1ec0b80 1 Processor -- start 2024-05-21T03:29:48.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.690+0000 7fa2e1ec0b80 1 -- start start 2024-05-21T03:29:48.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.690+0000 7fa2d53c6700 1 --2- 172.21.15.27:0/794284733 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fa2bc0b2a70 0x7fa2bc0b4f30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00e350 tx=0x7fa2cc009040 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:48.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.690+0000 7fa2e1ec0b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d9c5782b0 0x560d9c578690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:48.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.690+0000 7fa2e1ec0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d9c5728e0 0x560d9c572d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:48.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.690+0000 7fa2e1ec0b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d9c5732a0 0x560d9c57f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:48.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.690+0000 7fa2e1ec0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560d9c4d1d90 con 0x560d9c5728e0 2024-05-21T03:29:48.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.690+0000 7fa2e1ec0b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560d9c4c1930 con 0x560d9c5732a0 2024-05-21T03:29:48.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.690+0000 7fa2e1ec0b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560d9c4c5970 con 0x560d9c5782b0 2024-05-21T03:29:48.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.690+0000 7fa2d4bc5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d9c5728e0 0x560d9c572d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:48.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.690+0000 7fa2d4bc5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d9c5728e0 0x560d9c572d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59946/0 (socket says 172.21.15.27:59946) 2024-05-21T03:29:48.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.690+0000 7fa2d4bc5700 1 -- 172.21.15.27:0/647196035 learned_addr learned my addr 172.21.15.27:0/647196035 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:48.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.690+0000 7fa2d53c6700 1 --2- 172.21.15.27:0/647196035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d9c5782b0 0x560d9c578690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:48.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.690+0000 7fa2d5bc7700 1 --2- 172.21.15.27:0/647196035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d9c5732a0 0x560d9c57f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:48.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.690+0000 7fa2d4bc5700 1 -- 172.21.15.27:0/647196035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d9c5782b0 msgr2=0x560d9c578690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:48.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.690+0000 7fa2d4bc5700 1 --2- 172.21.15.27:0/647196035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d9c5782b0 0x560d9c578690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.690+0000 7fa2d4bc5700 1 -- 172.21.15.27:0/647196035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d9c5732a0 msgr2=0x560d9c57f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:48.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.690+0000 7fa2d4bc5700 1 --2- 172.21.15.27:0/647196035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d9c5732a0 0x560d9c57f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.690+0000 7fa2d4bc5700 1 -- 172.21.15.27:0/647196035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d9c4119d0 con 0x560d9c5728e0 2024-05-21T03:29:48.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.690+0000 7fa2d53c6700 1 --2- 172.21.15.27:0/647196035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d9c5782b0 0x560d9c578690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:29:48.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.690+0000 7fa2d5bc7700 1 --2- 172.21.15.27:0/647196035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d9c5732a0 0x560d9c57f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:29:48.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.690+0000 7fa2d4bc5700 1 --2- 172.21.15.27:0/647196035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d9c5728e0 0x560d9c572d60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00aa630 tx=0x7fa2c0022930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d25a705d172616 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:48.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.691+0000 7fa2bb7fe700 1 -- 172.21.15.27:0/647196035 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c00aada0 con 0x560d9c5728e0 2024-05-21T03:29:48.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.691+0000 7fa2bb7fe700 1 -- 172.21.15.27:0/647196035 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2c00aa6a0 con 0x560d9c5728e0 2024-05-21T03:29:48.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.691+0000 7fa2bb7fe700 1 -- 172.21.15.27:0/647196035 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c00afbe0 con 0x560d9c5728e0 2024-05-21T03:29:48.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.691+0000 7fa2e1ec0b80 1 -- 172.21.15.27:0/647196035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d9c5728e0 msgr2=0x560d9c572d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:48.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.691+0000 7fa2e1ec0b80 1 --2- 172.21.15.27:0/647196035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d9c5728e0 0x560d9c572d60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00aa630 tx=0x7fa2c0022930 comp rx=0 tx=0).stop 2024-05-21T03:29:48.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.691+0000 7fa2e1ec0b80 1 -- 172.21.15.27:0/647196035 shutdown_connections 2024-05-21T03:29:48.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.691+0000 7fa2e1ec0b80 1 --2- 172.21.15.27:0/647196035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d9c5732a0 0x560d9c57f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.691+0000 7fa2e1ec0b80 1 --2- 172.21.15.27:0/647196035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d9c5782b0 0x560d9c578690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.691+0000 7fa2e1ec0b80 1 --2- 172.21.15.27:0/647196035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d9c5728e0 0x560d9c572d60 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.691+0000 7fa2e1ec0b80 1 -- 172.21.15.27:0/647196035 >> 172.21.15.27:0/647196035 conn(0x560d9c4a0180 msgr2=0x560d9c56d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:48.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.692+0000 7fa2e1ec0b80 1 -- 172.21.15.27:0/647196035 shutdown_connections 2024-05-21T03:29:48.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.692+0000 7fa2e1ec0b80 1 -- 172.21.15.27:0/647196035 wait complete. 2024-05-21T03:29:48.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.692+0000 7fa2e1ec0b80 1 -- start start 2024-05-21T03:29:48.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.692+0000 7fa2e1ec0b80 1 Processor -- start 2024-05-21T03:29:48.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.692+0000 7fa2e1ec0b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d9c5728e0 0x560d9c57eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:48.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.692+0000 7fa2e1ec0b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d9c5732a0 0x560d9c57c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:48.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.692+0000 7fa2e1ec0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d9c5782b0 0x560d9c57c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:48.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.692+0000 7fa2e1ec0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560d9c3c80a0 con 0x560d9c5782b0 2024-05-21T03:29:48.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.692+0000 7fa2e1ec0b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560d9c4638c0 con 0x560d9c5728e0 2024-05-21T03:29:48.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.692+0000 7fa2e1ec0b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560d9c578e50 con 0x560d9c5732a0 2024-05-21T03:29:48.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.693+0000 7fa2e1ec0b80 10 client.0 _ll_register_callbacks cb 0x560d9c45ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:29:48.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.693+0000 7fa2d5bc7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d9c5782b0 0x560d9c57c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:48.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.693+0000 7fa2d53c6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d9c5728e0 0x560d9c57eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:48.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.693+0000 7fa2d5bc7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d9c5782b0 0x560d9c57c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59956/0 (socket says 172.21.15.27:59956) 2024-05-21T03:29:48.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.693+0000 7fa2d5bc7700 1 -- 172.21.15.27:0/2077096431 learned_addr learned my addr 172.21.15.27:0/2077096431 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:48.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.693+0000 7fa2d53c6700 1 --2- 172.21.15.27:0/2077096431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d9c5728e0 0x560d9c57eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.27:56768/0 (socket says 172.21.15.27:56768) 2024-05-21T03:29:48.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.693+0000 7fa2d4bc5700 1 --2- 172.21.15.27:0/2077096431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d9c5732a0 0x560d9c57c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:48.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.693+0000 7fa2d5bc7700 1 -- 172.21.15.27:0/2077096431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d9c5732a0 msgr2=0x560d9c57c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:48.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.693+0000 7fa2d5bc7700 1 --2- 172.21.15.27:0/2077096431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d9c5732a0 0x560d9c57c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.693+0000 7fa2d5bc7700 1 -- 172.21.15.27:0/2077096431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d9c5728e0 msgr2=0x560d9c57eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:48.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.693+0000 7fa2d5bc7700 1 --2- 172.21.15.27:0/2077096431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d9c5728e0 0x560d9c57eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.693+0000 7fa2d5bc7700 1 -- 172.21.15.27:0/2077096431 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d9c460790 con 0x560d9c5782b0 2024-05-21T03:29:48.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.693+0000 7fa2d5bc7700 1 --2- 172.21.15.27:0/2077096431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d9c5782b0 0x560d9c57c6e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800f7c0 tx=0x7fa2c800fad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f184600e8d6ef990 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:48.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.693+0000 7fa2b8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:29:48.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.693+0000 7fa2b8ff9700 1 -- 172.21.15.27:0/2077096431 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8010960 con 0x560d9c5782b0 2024-05-21T03:29:48.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.693+0000 7fa2e1ec0b80 1 -- 172.21.15.27:0/2077096431 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2c00aa6a0 con 0x560d9c5782b0 2024-05-21T03:29:48.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.693+0000 7fa2b8ff9700 1 -- 172.21.15.27:0/2077096431 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fa2c8010b00 con 0x560d9c5782b0 2024-05-21T03:29:48.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.694+0000 7fa2b8ff9700 1 -- 172.21.15.27:0/2077096431 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa2c8010d00 con 0x560d9c5782b0 2024-05-21T03:29:48.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.694+0000 7fa2e1ec0b80 10 client.15792 fetch_fsmap learned FSMap version 37 2024-05-21T03:29:48.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.694+0000 7fa2e1ec0b80 1 -- 172.21.15.27:0/2077096431 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560d9c589cd0 con 0x560d9c5782b0 2024-05-21T03:29:48.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.694+0000 7fa2b8ff9700 1 -- 172.21.15.27:0/2077096431 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2c00aa6a0 con 0x560d9c5782b0 2024-05-21T03:29:48.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.694+0000 7fa2b8ff9700 1 -- 172.21.15.27:0/2077096431 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fa2c801b610 con 0x560d9c5782b0 2024-05-21T03:29:48.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.694+0000 7fa2e1ec0b80 10 client.15792 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T03:29:48.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.694+0000 7fa2e1ec0b80 10 client.15792 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-21T03:29:48.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.694+0000 7fa2e1ec0b80 20 client.15792 populate_metadata read hostname 'smithi027' 2024-05-21T03:29:48.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.694+0000 7fa2e1ec0b80 1 --2- 172.21.15.27:0/2077096431 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x560d9c58c260 0x560d9c58e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:48.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.694+0000 7fa2e1ec0b80 4 client.15792 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:29:48.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.694+0000 7fa2e1ec0b80 1 -- 172.21.15.27:0/2077096431 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d9c53eae0 con 0x560d9c58c260 2024-05-21T03:29:48.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.694+0000 7fa2d53c6700 1 --2- 172.21.15.27:0/2077096431 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x560d9c58c260 0x560d9c58e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:48.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.695+0000 7fa2d53c6700 1 --2- 172.21.15.27:0/2077096431 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x560d9c58c260 0x560d9c58e650 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6fd6c18b395252f server_cookie=222a637f912ba7b3 in_seq=0 out_seq=0 2024-05-21T03:29:48.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.695+0000 7fa2b8ff9700 10 client.15792 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:29:48.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.696+0000 7fa2b8ff9700 1 -- 172.21.15.27:0/2077096431 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560d9c53eae0 con 0x560d9c58c260 2024-05-21T03:29:48.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.696+0000 7fa2b8ff9700 10 client.15792 handle_command_reply: tid=0 2024-05-21T03:29:48.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.696+0000 7fa2e1ec0b80 1 client.15792 shutdown 2024-05-21T03:29:48.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.696+0000 7fa2e1ec0b80 1 -- 172.21.15.27:0/2077096431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d9c5782b0 msgr2=0x560d9c57c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:48.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.696+0000 7fa2e1ec0b80 1 --2- 172.21.15.27:0/2077096431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d9c5782b0 0x560d9c57c6e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800f7c0 tx=0x7fa2c800fad0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.696+0000 7fa2e1ec0b80 1 -- 172.21.15.27:0/2077096431 shutdown_connections 2024-05-21T03:29:48.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.696+0000 7fa2e1ec0b80 1 --2- 172.21.15.27:0/2077096431 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x560d9c58c260 0x560d9c58e650 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.696+0000 7fa2e1ec0b80 1 --2- 172.21.15.27:0/2077096431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d9c5782b0 0x560d9c57c6e0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.696+0000 7fa2e1ec0b80 1 --2- 172.21.15.27:0/2077096431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d9c5728e0 0x560d9c57eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.696+0000 7fa2b8ff9700 0 client.15792 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:29:48.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.696+0000 7fa2e1ec0b80 1 --2- 172.21.15.27:0/2077096431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d9c5732a0 0x560d9c57c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.696+0000 7fa2e1ec0b80 1 -- 172.21.15.27:0/2077096431 >> 172.21.15.27:0/2077096431 conn(0x560d9c4a0180 msgr2=0x560d9c567ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:48.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.696+0000 7fa2e1ec0b80 1 -- 172.21.15.27:0/2077096431 shutdown_connections 2024-05-21T03:29:48.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.696+0000 7fa2e1ec0b80 1 -- 172.21.15.27:0/2077096431 wait complete. 2024-05-21T03:29:48.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.696+0000 7fa2e1ec0b80 20 client.15792 trim_cache size 0 max 16384 2024-05-21T03:29:48.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.716+0000 7fa2e1ec0b80 1 Processor -- start 2024-05-21T03:29:48.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.716+0000 7fa2e1ec0b80 1 -- start start 2024-05-21T03:29:48.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.717+0000 7fa2e1ec0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d9c5728e0 0x560d9c57baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:48.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.717+0000 7fa2e1ec0b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d9c5732a0 0x560d9c57e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:48.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.717+0000 7fa2e1ec0b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d9c5782b0 0x560d9c5761f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:48.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.717+0000 7fa2e1ec0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d01b43b0 con 0x560d9c5728e0 2024-05-21T03:29:48.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.717+0000 7fa2e1ec0b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d01b5ef0 con 0x560d9c5732a0 2024-05-21T03:29:48.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.717+0000 7fa2e1ec0b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d01b4a80 con 0x560d9c5782b0 2024-05-21T03:29:48.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.717+0000 7fa2d53c6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d9c5728e0 0x560d9c57baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:48.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.717+0000 7fa2d53c6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d9c5728e0 0x560d9c57baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59964/0 (socket says 172.21.15.27:59964) 2024-05-21T03:29:48.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.717+0000 7fa2d53c6700 1 -- 172.21.15.27:0/1426466151 learned_addr learned my addr 172.21.15.27:0/1426466151 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:48.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.717+0000 7fa2d4bc5700 1 --2- 172.21.15.27:0/1426466151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d9c5732a0 0x560d9c57e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:48.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.717+0000 7fa2d5bc7700 1 --2- 172.21.15.27:0/1426466151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d9c5782b0 0x560d9c5761f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:48.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.717+0000 7fa2d53c6700 1 -- 172.21.15.27:0/1426466151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d9c5782b0 msgr2=0x560d9c5761f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:48.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.717+0000 7fa2d53c6700 1 --2- 172.21.15.27:0/1426466151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d9c5782b0 0x560d9c5761f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.717+0000 7fa2d53c6700 1 -- 172.21.15.27:0/1426466151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d9c5732a0 msgr2=0x560d9c57e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:48.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.717+0000 7fa2d53c6700 1 --2- 172.21.15.27:0/1426466151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d9c5732a0 0x560d9c57e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.717+0000 7fa2d53c6700 1 -- 172.21.15.27:0/1426466151 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d9c56e2e0 con 0x560d9c5728e0 2024-05-21T03:29:48.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.717+0000 7fa2d53c6700 1 --2- 172.21.15.27:0/1426466151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d9c5728e0 0x560d9c57baa0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc008fd0 tx=0x7fa2cc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f250adaad4347f1f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:48.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.718+0000 7fa2baffd700 1 -- 172.21.15.27:0/1426466151 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc015df0 con 0x560d9c5728e0 2024-05-21T03:29:48.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.718+0000 7fa2baffd700 1 -- 172.21.15.27:0/1426466151 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2cc004020 con 0x560d9c5728e0 2024-05-21T03:29:48.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.718+0000 7fa2baffd700 1 -- 172.21.15.27:0/1426466151 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc02e480 con 0x560d9c5728e0 2024-05-21T03:29:48.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.718+0000 7fa2e1ec0b80 1 -- 172.21.15.27:0/1426466151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d9c5728e0 msgr2=0x560d9c57baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:48.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.718+0000 7fa2e1ec0b80 1 --2- 172.21.15.27:0/1426466151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d9c5728e0 0x560d9c57baa0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc008fd0 tx=0x7fa2cc014000 comp rx=0 tx=0).stop 2024-05-21T03:29:48.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.718+0000 7fa2e1ec0b80 1 -- 172.21.15.27:0/1426466151 shutdown_connections 2024-05-21T03:29:48.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.718+0000 7fa2e1ec0b80 1 --2- 172.21.15.27:0/1426466151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d9c5782b0 0x560d9c5761f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.718+0000 7fa2e1ec0b80 1 --2- 172.21.15.27:0/1426466151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d9c5728e0 0x560d9c57baa0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.718+0000 7fa2e1ec0b80 1 --2- 172.21.15.27:0/1426466151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d9c5732a0 0x560d9c57e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.718+0000 7fa2e1ec0b80 1 -- 172.21.15.27:0/1426466151 >> 172.21.15.27:0/1426466151 conn(0x560d9c4a0180 msgr2=0x560d9c575ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:48.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.719+0000 7fa2e1ec0b80 1 -- 172.21.15.27:0/1426466151 shutdown_connections 2024-05-21T03:29:48.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.719+0000 7fa2e1ec0b80 1 -- 172.21.15.27:0/1426466151 wait complete. 2024-05-21T03:29:48.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.719+0000 7fa2e1ec0b80 1 -- start start 2024-05-21T03:29:48.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.719+0000 7fa2e1ec0b80 1 Processor -- start 2024-05-21T03:29:48.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.719+0000 7fa2e1ec0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d9c5728e0 0x560d9c584c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:48.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.719+0000 7fa2e1ec0b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d9c5732a0 0x560d9c57f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:48.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.719+0000 7fa2e1ec0b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d9c5782b0 0x560d9c57fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:48.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.719+0000 7fa2e1ec0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d01b5820 con 0x560d9c5728e0 2024-05-21T03:29:48.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.719+0000 7fa2e1ec0b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d01b5150 con 0x560d9c5782b0 2024-05-21T03:29:48.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.719+0000 7fa2e1ec0b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d01b2350 con 0x560d9c5732a0 2024-05-21T03:29:48.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.719+0000 7fa2e1ec0b80 10 client.0 _ll_register_callbacks cb 0x560d9c45ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:29:48.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.720+0000 7fa2d53c6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d9c5728e0 0x560d9c584c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:48.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.720+0000 7fa2d53c6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d9c5728e0 0x560d9c584c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59968/0 (socket says 172.21.15.27:59968) 2024-05-21T03:29:48.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.720+0000 7fa2d53c6700 1 -- 172.21.15.27:0/153542742 learned_addr learned my addr 172.21.15.27:0/153542742 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:29:48.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.720+0000 7fa2d5bc7700 1 --2- 172.21.15.27:0/153542742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d9c5782b0 0x560d9c57fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:48.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.720+0000 7fa2d4bc5700 1 --2- 172.21.15.27:0/153542742 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d9c5732a0 0x560d9c57f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:48.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.720+0000 7fa2d53c6700 1 -- 172.21.15.27:0/153542742 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d9c5732a0 msgr2=0x560d9c57f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:48.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.720+0000 7fa2d53c6700 1 --2- 172.21.15.27:0/153542742 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d9c5732a0 0x560d9c57f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.720+0000 7fa2d53c6700 1 -- 172.21.15.27:0/153542742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d9c5782b0 msgr2=0x560d9c57fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:29:48.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.720+0000 7fa2d53c6700 1 --2- 172.21.15.27:0/153542742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d9c5782b0 0x560d9c57fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.720+0000 7fa2d53c6700 1 -- 172.21.15.27:0/153542742 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d9c5802a0 con 0x560d9c5728e0 2024-05-21T03:29:48.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.720+0000 7fa2d4bc5700 1 --2- 172.21.15.27:0/153542742 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d9c5732a0 0x560d9c57f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:29:48.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.720+0000 7fa2d53c6700 1 --2- 172.21.15.27:0/153542742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d9c5728e0 0x560d9c584c60 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc030040 tx=0x7fa2cc00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1089d593502289c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:29:48.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.720+0000 7fa2b8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:29:48.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.720+0000 7fa2b8ff9700 1 -- 172.21.15.27:0/153542742 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc03d060 con 0x560d9c5728e0 2024-05-21T03:29:48.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.720+0000 7fa2b8ff9700 1 -- 172.21.15.27:0/153542742 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fa2cc010040 con 0x560d9c5728e0 2024-05-21T03:29:48.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.720+0000 7fa2e1ec0b80 1 -- 172.21.15.27:0/153542742 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2cc004020 con 0x560d9c5728e0 2024-05-21T03:29:48.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.721+0000 7fa2b8ff9700 1 -- 172.21.15.27:0/153542742 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa2cc0043e0 con 0x560d9c5728e0 2024-05-21T03:29:48.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.721+0000 7fa2e1ec0b80 10 client.15798 fetch_fsmap learned FSMap version 37 2024-05-21T03:29:48.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.721+0000 7fa2e1ec0b80 1 -- 172.21.15.27:0/153542742 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560d9c5f7cb0 con 0x560d9c5728e0 2024-05-21T03:29:48.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.721+0000 7fa2b8ff9700 1 -- 172.21.15.27:0/153542742 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2cc004020 con 0x560d9c5728e0 2024-05-21T03:29:48.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.721+0000 7fa2b8ff9700 1 -- 172.21.15.27:0/153542742 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fa2cc014070 con 0x560d9c5728e0 2024-05-21T03:29:48.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.721+0000 7fa2e1ec0b80 10 client.15798 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T03:29:48.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.721+0000 7fa2e1ec0b80 10 client.15798 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-21T03:29:48.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.721+0000 7fa2e1ec0b80 20 client.15798 populate_metadata read hostname 'smithi027' 2024-05-21T03:29:48.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.721+0000 7fa2e1ec0b80 1 --2- 172.21.15.27:0/153542742 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x560d9c58c260 0x560d9c5fa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:29:48.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.721+0000 7fa2e1ec0b80 4 client.15798 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f36cc8d4", "release": true, "await": true}] 2024-05-21T03:29:48.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.721+0000 7fa2e1ec0b80 1 -- 172.21.15.27:0/153542742 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f36cc8d4", "release": true, "await": true}) -- 0x560d9c4d3ce0 con 0x560d9c58c260 2024-05-21T03:29:48.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.721+0000 7fa2d4bc5700 1 --2- 172.21.15.27:0/153542742 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x560d9c58c260 0x560d9c5fa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:29:48.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.722+0000 7fa2b8ff9700 10 client.15798 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:29:48.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.722+0000 7fa2d4bc5700 1 --2- 172.21.15.27:0/153542742 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x560d9c58c260 0x560d9c5fa3d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=308eafd70d3a8e41 server_cookie=a673accdcedfd3d4 in_seq=0 out_seq=0 2024-05-21T03:29:48.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.736+0000 7fa2b8ff9700 1 -- 172.21.15.27:0/153542742 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560d9c4d3ce0 con 0x560d9c58c260 2024-05-21T03:29:48.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.736+0000 7fa2b8ff9700 10 client.15798 handle_command_reply: tid=0 2024-05-21T03:29:48.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.736+0000 7fa2e1ec0b80 1 client.15798 shutdown 2024-05-21T03:29:48.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.736+0000 7fa2e1ec0b80 1 -- 172.21.15.27:0/153542742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d9c5728e0 msgr2=0x560d9c584c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.737+0000 7fa2e1ec0b80 1 --2- 172.21.15.27:0/153542742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d9c5728e0 0x560d9c584c60 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc030040 tx=0x7fa2cc00ef90 comp rx=0 tx=0).stop 2024-05-21T03:29:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.737+0000 7fa2e1ec0b80 1 -- 172.21.15.27:0/153542742 shutdown_connections 2024-05-21T03:29:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.737+0000 7fa2e1ec0b80 1 --2- 172.21.15.27:0/153542742 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x560d9c58c260 0x560d9c5fa3d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.737+0000 7fa2e1ec0b80 1 --2- 172.21.15.27:0/153542742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d9c5782b0 0x560d9c57fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.737+0000 7fa2b8ff9700 0 client.15798 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:29:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.737+0000 7fa2e1ec0b80 1 --2- 172.21.15.27:0/153542742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560d9c5728e0 0x560d9c584c60 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.737+0000 7fa2e1ec0b80 1 --2- 172.21.15.27:0/153542742 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560d9c5732a0 0x560d9c57f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.737+0000 7fa2e1ec0b80 1 -- 172.21.15.27:0/153542742 >> 172.21.15.27:0/153542742 conn(0x560d9c4a0180 msgr2=0x560d9c5797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.737+0000 7fa2e1ec0b80 1 -- 172.21.15.27:0/153542742 shutdown_connections 2024-05-21T03:29:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.737+0000 7fa2e1ec0b80 1 -- 172.21.15.27:0/153542742 wait complete. 2024-05-21T03:29:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.737+0000 7fa2e1ec0b80 20 client.15798 trim_cache size 0 max 16384 2024-05-21T03:29:48.738 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:29:48.738 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 37, 2024-05-21T03:29:48.738 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24476, 2024-05-21T03:29:48.738 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 17, 2024-05-21T03:29:48.738 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-05-21T03:29:48.738 INFO:teuthology.orchestra.run.smithi027.stdout: "f36cc8d4": { 2024-05-21T03:29:48.738 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 17, 2024-05-21T03:29:48.738 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-05-21T03:29:48.738 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-05-21T03:29:48.738 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-05-21T03:29:48.738 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-05-21T03:29:48.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:29:48.739 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-05-21T03:29:48.739 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 141.4, 2024-05-21T03:29:48.739 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-05-21T03:29:48.739 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-05-21T03:29:48.739 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-05-21T03:29:48.739 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-05-21T03:29:48.739 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-05-21T03:29:48.739 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-05-21T03:29:48.739 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:29:48.739 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:29:48.739 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:29:48.739 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:29:48.739 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:29:48.740 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:29:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.738+0000 7fa2d762a700 1 -- 172.21.15.27:0/794284733 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fa2bc0b2a70 msgr2=0x7fa2bc0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.738+0000 7fa2d762a700 1 --2- 172.21.15.27:0/794284733 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fa2bc0b2a70 0x7fa2bc0b4f30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00e350 tx=0x7fa2cc009040 comp rx=0 tx=0).stop 2024-05-21T03:29:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.738+0000 7fa2d762a700 1 -- 172.21.15.27:0/794284733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2d006bbe0 msgr2=0x7fa2d0102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:29:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.738+0000 7fa2d762a700 1 --2- 172.21.15.27:0/794284733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2d006bbe0 0x7fa2d0102a50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00057f0 tx=0x7fa2c0003730 comp rx=0 tx=0).stop 2024-05-21T03:29:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.738+0000 7fa2d762a700 1 -- 172.21.15.27:0/794284733 shutdown_connections 2024-05-21T03:29:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.738+0000 7fa2d762a700 1 --2- 172.21.15.27:0/794284733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2d01095b0 0x7fa2d0102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.738+0000 7fa2d762a700 1 --2- 172.21.15.27:0/794284733 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fa2bc0b2a70 0x7fa2bc0b4f30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.738+0000 7fa2d762a700 1 --2- 172.21.15.27:0/794284733 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2d006b220 0x7fa2d0102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.738+0000 7fa2d762a700 1 --2- 172.21.15.27:0/794284733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2d006bbe0 0x7fa2d0102a50 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:29:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.738+0000 7fa2d762a700 1 -- 172.21.15.27:0/794284733 >> 172.21.15.27:0/794284733 conn(0x7fa2d00793f0 msgr2=0x7fa2d0101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:29:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.738+0000 7fa2d762a700 1 -- 172.21.15.27:0/794284733 shutdown_connections 2024-05-21T03:29:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:29:48.738+0000 7fa2d762a700 1 -- 172.21.15.27:0/794284733 wait complete. 2024-05-21T03:29:49.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:48 smithi027 ceph-mon[32179]: pgmap v176: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 MiB/s wr, 8.44k op/s 2024-05-21T03:29:49.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:48 smithi039 ceph-mon[36429]: pgmap v176: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 MiB/s wr, 8.44k op/s 2024-05-21T03:29:49.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:48 smithi165 ceph-mon[36269]: pgmap v176: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 MiB/s wr, 8.44k op/s 2024-05-21T03:29:49.480 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f36cc8d4', total seconds elapsed: 27.5 2024-05-21T03:29:51.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:50 smithi027 ceph-mon[32179]: pgmap v177: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 42 MiB/s wr, 8.77k op/s 2024-05-21T03:29:51.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:50 smithi039 ceph-mon[36429]: pgmap v177: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 42 MiB/s wr, 8.77k op/s 2024-05-21T03:29:51.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:50 smithi165 ceph-mon[36269]: pgmap v177: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 42 MiB/s wr, 8.77k op/s 2024-05-21T03:29:53.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:52 smithi027 ceph-mon[32179]: pgmap v178: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 2.25k op/s 2024-05-21T03:29:53.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:52 smithi039 ceph-mon[36429]: pgmap v178: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 2.25k op/s 2024-05-21T03:29:53.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:52 smithi165 ceph-mon[36269]: pgmap v178: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 2.25k op/s 2024-05-21T03:29:55.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:54 smithi027 ceph-mon[32179]: pgmap v179: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 MiB/s wr, 5.05k op/s 2024-05-21T03:29:55.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:54 smithi039 ceph-mon[36429]: pgmap v179: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 MiB/s wr, 5.05k op/s 2024-05-21T03:29:55.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:54 smithi165 ceph-mon[36269]: pgmap v179: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 MiB/s wr, 5.05k op/s 2024-05-21T03:29:56.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:55 smithi027 ceph-mon[32179]: pgmap v180: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 3.13k op/s 2024-05-21T03:29:56.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:55 smithi039 ceph-mon[36429]: pgmap v180: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 3.13k op/s 2024-05-21T03:29:56.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:55 smithi165 ceph-mon[36269]: pgmap v180: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 3.13k op/s 2024-05-21T03:29:58.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:29:58 smithi027 ceph-mon[32179]: pgmap v181: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 3.13k op/s 2024-05-21T03:29:58.970 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:29:58 smithi039 ceph-mon[36429]: pgmap v181: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 3.13k op/s 2024-05-21T03:29:58.972 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:29:58 smithi165 ceph-mon[36269]: pgmap v181: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 3.13k op/s 2024-05-21T03:30:00.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:00 smithi027 ceph-mon[32179]: pgmap v182: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 26 MiB/s wr, 5.49k op/s 2024-05-21T03:30:00.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:00 smithi027 ceph-mon[32179]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:30:00.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:00 smithi027 ceph-mon[32179]: overall HEALTH_OK 2024-05-21T03:30:01.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:00 smithi039 ceph-mon[36429]: pgmap v182: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 26 MiB/s wr, 5.49k op/s 2024-05-21T03:30:01.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:00 smithi039 ceph-mon[36429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:30:01.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:00 smithi039 ceph-mon[36429]: overall HEALTH_OK 2024-05-21T03:30:01.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:00 smithi165 ceph-mon[36269]: pgmap v182: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 26 MiB/s wr, 5.49k op/s 2024-05-21T03:30:01.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:00 smithi165 ceph-mon[36269]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:30:01.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:00 smithi165 ceph-mon[36269]: overall HEALTH_OK 2024-05-21T03:30:02.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:02 smithi027 ceph-mon[32179]: pgmap v183: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 MiB/s wr, 5.17k op/s 2024-05-21T03:30:03.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:02 smithi039 ceph-mon[36429]: pgmap v183: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 MiB/s wr, 5.17k op/s 2024-05-21T03:30:03.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:02 smithi165 ceph-mon[36269]: pgmap v183: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 MiB/s wr, 5.17k op/s 2024-05-21T03:30:04.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:04 smithi027 ceph-mon[32179]: pgmap v184: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 32 MiB/s wr, 6.89k op/s 2024-05-21T03:30:05.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:04 smithi039 ceph-mon[36429]: pgmap v184: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 32 MiB/s wr, 6.89k op/s 2024-05-21T03:30:05.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:04 smithi165 ceph-mon[36269]: pgmap v184: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 32 MiB/s wr, 6.89k op/s 2024-05-21T03:30:05.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:05 smithi027 ceph-mon[32179]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:30:06.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:05 smithi039 ceph-mon[36429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:30:06.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:05 smithi165 ceph-mon[36269]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:30:06.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:06 smithi027 ceph-mon[32179]: pgmap v185: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 23 MiB/s wr, 5.13k op/s 2024-05-21T03:30:07.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:06 smithi039 ceph-mon[36429]: pgmap v185: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 23 MiB/s wr, 5.13k op/s 2024-05-21T03:30:07.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:06 smithi165 ceph-mon[36269]: pgmap v185: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 23 MiB/s wr, 5.13k op/s 2024-05-21T03:30:08.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:08 smithi027 ceph-mon[32179]: pgmap v186: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 23 MiB/s wr, 5.13k op/s 2024-05-21T03:30:09.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:08 smithi039 ceph-mon[36429]: pgmap v186: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 23 MiB/s wr, 5.13k op/s 2024-05-21T03:30:09.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:08 smithi165 ceph-mon[36269]: pgmap v186: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 23 MiB/s wr, 5.13k op/s 2024-05-21T03:30:10.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:10 smithi027 ceph-mon[32179]: pgmap v187: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 34 MiB/s wr, 7.54k op/s 2024-05-21T03:30:11.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:10 smithi039 ceph-mon[36429]: pgmap v187: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 34 MiB/s wr, 7.54k op/s 2024-05-21T03:30:11.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:10 smithi165 ceph-mon[36269]: pgmap v187: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 34 MiB/s wr, 7.54k op/s 2024-05-21T03:30:12.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:12 smithi027 ceph-mon[32179]: pgmap v188: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 24 MiB/s wr, 5.18k op/s 2024-05-21T03:30:12.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:12 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:30:13.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:12 smithi039 ceph-mon[36429]: pgmap v188: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 24 MiB/s wr, 5.18k op/s 2024-05-21T03:30:13.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:12 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:30:13.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:12 smithi165 ceph-mon[36269]: pgmap v188: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 24 MiB/s wr, 5.18k op/s 2024-05-21T03:30:13.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:12 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:30:13.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:13 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:30:13.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:13 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:30:13.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:13 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:30:14.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:13 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:30:14.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:13 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:30:14.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:13 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:30:14.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:13 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:30:14.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:13 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:30:14.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:13 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:30:14.930 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:14 smithi027 ceph-mon[32179]: pgmap v189: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 26 MiB/s wr, 5.71k op/s 2024-05-21T03:30:15.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:14 smithi039 ceph-mon[36429]: pgmap v189: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 26 MiB/s wr, 5.71k op/s 2024-05-21T03:30:15.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:14 smithi165 ceph-mon[36269]: pgmap v189: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 26 MiB/s wr, 5.71k op/s 2024-05-21T03:30:16.859 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:16 smithi027 ceph-mon[32179]: pgmap v190: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 22 MiB/s wr, 4.72k op/s 2024-05-21T03:30:17.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:16 smithi039 ceph-mon[36429]: pgmap v190: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 22 MiB/s wr, 4.72k op/s 2024-05-21T03:30:17.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:16 smithi165 ceph-mon[36269]: pgmap v190: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 22 MiB/s wr, 4.72k op/s 2024-05-21T03:30:18.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:18 smithi027 ceph-mon[32179]: pgmap v191: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 17 MiB/s wr, 3.68k op/s 2024-05-21T03:30:19.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:18 smithi039 ceph-mon[36429]: pgmap v191: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 17 MiB/s wr, 3.68k op/s 2024-05-21T03:30:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:18 smithi165 ceph-mon[36269]: pgmap v191: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 17 MiB/s wr, 3.68k op/s 2024-05-21T03:30:20.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:20 smithi027 ceph-mon[32179]: pgmap v192: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 41 MiB/s wr, 8.85k op/s 2024-05-21T03:30:21.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:20 smithi039 ceph-mon[36429]: pgmap v192: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 41 MiB/s wr, 8.85k op/s 2024-05-21T03:30:21.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:20 smithi165 ceph-mon[36269]: pgmap v192: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 41 MiB/s wr, 8.85k op/s 2024-05-21T03:30:22.941 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:22 smithi027 ceph-mon[32179]: pgmap v193: 97 pgs: 97 active+clean; 15 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 38 MiB/s wr, 8.32k op/s 2024-05-21T03:30:23.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:22 smithi039 ceph-mon[36429]: pgmap v193: 97 pgs: 97 active+clean; 15 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 38 MiB/s wr, 8.32k op/s 2024-05-21T03:30:23.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:22 smithi165 ceph-mon[36269]: pgmap v193: 97 pgs: 97 active+clean; 15 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 38 MiB/s wr, 8.32k op/s 2024-05-21T03:30:24.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:24 smithi027 ceph-mon[32179]: pgmap v194: 97 pgs: 97 active+clean; 15 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 40 KiB/s rd, 51 MiB/s wr, 11.24k op/s 2024-05-21T03:30:25.018 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:24 smithi039 ceph-mon[36429]: pgmap v194: 97 pgs: 97 active+clean; 15 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 40 KiB/s rd, 51 MiB/s wr, 11.24k op/s 2024-05-21T03:30:25.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:24 smithi165 ceph-mon[36269]: pgmap v194: 97 pgs: 97 active+clean; 15 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 40 KiB/s rd, 51 MiB/s wr, 11.24k op/s 2024-05-21T03:30:26.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:26 smithi027 ceph-mon[32179]: pgmap v195: 97 pgs: 97 active+clean; 15 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 27 KiB/s rd, 75 MiB/s wr, 16.60k op/s 2024-05-21T03:30:27.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:26 smithi039 ceph-mon[36429]: pgmap v195: 97 pgs: 97 active+clean; 15 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 27 KiB/s rd, 75 MiB/s wr, 16.60k op/s 2024-05-21T03:30:27.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:26 smithi165 ceph-mon[36269]: pgmap v195: 97 pgs: 97 active+clean; 15 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 27 KiB/s rd, 75 MiB/s wr, 16.60k op/s 2024-05-21T03:30:28.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:28 smithi027 ceph-mon[32179]: pgmap v196: 97 pgs: 97 active+clean; 15 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 26 KiB/s rd, 72 MiB/s wr, 15.86k op/s 2024-05-21T03:30:29.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:28 smithi039 ceph-mon[36429]: pgmap v196: 97 pgs: 97 active+clean; 15 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 26 KiB/s rd, 72 MiB/s wr, 15.86k op/s 2024-05-21T03:30:29.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:28 smithi165 ceph-mon[36269]: pgmap v196: 97 pgs: 97 active+clean; 15 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 26 KiB/s rd, 72 MiB/s wr, 15.86k op/s 2024-05-21T03:30:30.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:30 smithi027 ceph-mon[32179]: pgmap v197: 97 pgs: 97 active+clean; 15 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 58 KiB/s rd, 81 MiB/s wr, 18.06k op/s 2024-05-21T03:30:31.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:30 smithi039 ceph-mon[36429]: pgmap v197: 97 pgs: 97 active+clean; 15 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 58 KiB/s rd, 81 MiB/s wr, 18.06k op/s 2024-05-21T03:30:31.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:30 smithi165 ceph-mon[36269]: pgmap v197: 97 pgs: 97 active+clean; 15 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 58 KiB/s rd, 81 MiB/s wr, 18.06k op/s 2024-05-21T03:30:32.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:32 smithi027 ceph-mon[32179]: pgmap v198: 97 pgs: 97 active+clean; 15 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 54 KiB/s rd, 60 MiB/s wr, 13.58k op/s 2024-05-21T03:30:33.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:32 smithi039 ceph-mon[36429]: pgmap v198: 97 pgs: 97 active+clean; 15 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 54 KiB/s rd, 60 MiB/s wr, 13.58k op/s 2024-05-21T03:30:33.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:32 smithi165 ceph-mon[36269]: pgmap v198: 97 pgs: 97 active+clean; 15 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 54 KiB/s rd, 60 MiB/s wr, 13.58k op/s 2024-05-21T03:30:34.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:34 smithi027 ceph-mon[32179]: pgmap v199: 97 pgs: 97 active+clean; 16 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 62 KiB/s rd, 58 MiB/s wr, 13.08k op/s 2024-05-21T03:30:35.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:34 smithi039 ceph-mon[36429]: pgmap v199: 97 pgs: 97 active+clean; 16 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 62 KiB/s rd, 58 MiB/s wr, 13.08k op/s 2024-05-21T03:30:35.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:34 smithi165 ceph-mon[36269]: pgmap v199: 97 pgs: 97 active+clean; 16 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 62 KiB/s rd, 58 MiB/s wr, 13.08k op/s 2024-05-21T03:30:36.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:36 smithi027 ceph-mon[32179]: pgmap v200: 97 pgs: 97 active+clean; 16 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 56 KiB/s rd, 54 MiB/s wr, 12.01k op/s 2024-05-21T03:30:37.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:36 smithi039 ceph-mon[36429]: pgmap v200: 97 pgs: 97 active+clean; 16 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 56 KiB/s rd, 54 MiB/s wr, 12.01k op/s 2024-05-21T03:30:37.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:36 smithi165 ceph-mon[36269]: pgmap v200: 97 pgs: 97 active+clean; 16 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 56 KiB/s rd, 54 MiB/s wr, 12.01k op/s 2024-05-21T03:30:38.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:38 smithi027 ceph-mon[32179]: pgmap v201: 97 pgs: 97 active+clean; 16 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 51 KiB/s rd, 31 MiB/s wr, 7.00k op/s 2024-05-21T03:30:39.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:38 smithi039 ceph-mon[36429]: pgmap v201: 97 pgs: 97 active+clean; 16 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 51 KiB/s rd, 31 MiB/s wr, 7.00k op/s 2024-05-21T03:30:39.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:38 smithi165 ceph-mon[36269]: pgmap v201: 97 pgs: 97 active+clean; 16 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 51 KiB/s rd, 31 MiB/s wr, 7.00k op/s 2024-05-21T03:30:40.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:40 smithi027 ceph-mon[32179]: pgmap v202: 97 pgs: 97 active+clean; 16 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 72 KiB/s rd, 44 MiB/s wr, 9.66k op/s 2024-05-21T03:30:41.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:40 smithi039 ceph-mon[36429]: pgmap v202: 97 pgs: 97 active+clean; 16 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 72 KiB/s rd, 44 MiB/s wr, 9.66k op/s 2024-05-21T03:30:41.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:40 smithi165 ceph-mon[36269]: pgmap v202: 97 pgs: 97 active+clean; 16 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 72 KiB/s rd, 44 MiB/s wr, 9.66k op/s 2024-05-21T03:30:42.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:42 smithi027 ceph-mon[32179]: pgmap v203: 97 pgs: 97 active+clean; 16 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 40 KiB/s rd, 34 MiB/s wr, 7.46k op/s 2024-05-21T03:30:43.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:42 smithi039 ceph-mon[36429]: pgmap v203: 97 pgs: 97 active+clean; 16 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 40 KiB/s rd, 34 MiB/s wr, 7.46k op/s 2024-05-21T03:30:43.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:42 smithi165 ceph-mon[36269]: pgmap v203: 97 pgs: 97 active+clean; 16 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 40 KiB/s rd, 34 MiB/s wr, 7.46k op/s 2024-05-21T03:30:44.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:44 smithi027 ceph-mon[32179]: pgmap v204: 97 pgs: 97 active+clean; 16 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 47 KiB/s rd, 42 MiB/s wr, 8.85k op/s 2024-05-21T03:30:45.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:44 smithi039 ceph-mon[36429]: pgmap v204: 97 pgs: 97 active+clean; 16 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 47 KiB/s rd, 42 MiB/s wr, 8.85k op/s 2024-05-21T03:30:45.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:44 smithi165 ceph-mon[36269]: pgmap v204: 97 pgs: 97 active+clean; 16 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 47 KiB/s rd, 42 MiB/s wr, 8.85k op/s 2024-05-21T03:30:46.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:46 smithi027 ceph-mon[32179]: pgmap v205: 97 pgs: 97 active+clean; 16 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 43 KiB/s rd, 42 MiB/s wr, 8.80k op/s 2024-05-21T03:30:47.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:46 smithi039 ceph-mon[36429]: pgmap v205: 97 pgs: 97 active+clean; 16 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 43 KiB/s rd, 42 MiB/s wr, 8.80k op/s 2024-05-21T03:30:47.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:46 smithi165 ceph-mon[36269]: pgmap v205: 97 pgs: 97 active+clean; 16 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 43 KiB/s rd, 42 MiB/s wr, 8.80k op/s 2024-05-21T03:30:48.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:48 smithi027 ceph-mon[32179]: pgmap v206: 97 pgs: 97 active+clean; 16 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 42 KiB/s rd, 39 MiB/s wr, 8.08k op/s 2024-05-21T03:30:49.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:48 smithi039 ceph-mon[36429]: pgmap v206: 97 pgs: 97 active+clean; 16 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 42 KiB/s rd, 39 MiB/s wr, 8.08k op/s 2024-05-21T03:30:49.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:48 smithi165 ceph-mon[36269]: pgmap v206: 97 pgs: 97 active+clean; 16 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 42 KiB/s rd, 39 MiB/s wr, 8.08k op/s 2024-05-21T03:30:50.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:50 smithi027 ceph-mon[32179]: pgmap v207: 97 pgs: 97 active+clean; 16 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 53 KiB/s rd, 52 MiB/s wr, 10.50k op/s 2024-05-21T03:30:51.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:50 smithi039 ceph-mon[36429]: pgmap v207: 97 pgs: 97 active+clean; 16 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 53 KiB/s rd, 52 MiB/s wr, 10.50k op/s 2024-05-21T03:30:51.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:50 smithi165 ceph-mon[36269]: pgmap v207: 97 pgs: 97 active+clean; 16 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 53 KiB/s rd, 52 MiB/s wr, 10.50k op/s 2024-05-21T03:30:53.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:52 smithi039 ceph-mon[36429]: pgmap v208: 97 pgs: 97 active+clean; 16 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 32 KiB/s rd, 40 MiB/s wr, 7.83k op/s 2024-05-21T03:30:53.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:52 smithi165 ceph-mon[36269]: pgmap v208: 97 pgs: 97 active+clean; 16 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 32 KiB/s rd, 40 MiB/s wr, 7.83k op/s 2024-05-21T03:30:53.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:52 smithi027 ceph-mon[32179]: pgmap v208: 97 pgs: 97 active+clean; 16 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 32 KiB/s rd, 40 MiB/s wr, 7.83k op/s 2024-05-21T03:30:55.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:54 smithi039 ceph-mon[36429]: pgmap v209: 97 pgs: 97 active+clean; 16 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 46 KiB/s rd, 50 MiB/s wr, 9.94k op/s 2024-05-21T03:30:55.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:54 smithi165 ceph-mon[36269]: pgmap v209: 97 pgs: 97 active+clean; 16 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 46 KiB/s rd, 50 MiB/s wr, 9.94k op/s 2024-05-21T03:30:55.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:54 smithi027 ceph-mon[32179]: pgmap v209: 97 pgs: 97 active+clean; 16 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 46 KiB/s rd, 50 MiB/s wr, 9.94k op/s 2024-05-21T03:30:57.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:56 smithi039 ceph-mon[36429]: pgmap v210: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 36 KiB/s rd, 47 MiB/s wr, 9.43k op/s 2024-05-21T03:30:57.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:56 smithi165 ceph-mon[36269]: pgmap v210: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 36 KiB/s rd, 47 MiB/s wr, 9.43k op/s 2024-05-21T03:30:57.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:56 smithi027 ceph-mon[32179]: pgmap v210: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 36 KiB/s rd, 47 MiB/s wr, 9.43k op/s 2024-05-21T03:30:59.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:30:58 smithi039 ceph-mon[36429]: pgmap v211: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 31 KiB/s rd, 40 MiB/s wr, 8.10k op/s 2024-05-21T03:30:59.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:30:58 smithi165 ceph-mon[36269]: pgmap v211: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 31 KiB/s rd, 40 MiB/s wr, 8.10k op/s 2024-05-21T03:30:59.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:30:58 smithi027 ceph-mon[32179]: pgmap v211: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 31 KiB/s rd, 40 MiB/s wr, 8.10k op/s 2024-05-21T03:31:01.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:00 smithi039 ceph-mon[36429]: pgmap v212: 97 pgs: 97 active+clean; 17 GiB data, 56 GiB used, 1016 GiB / 1.0 TiB avail; 51 KiB/s rd, 50 MiB/s wr, 10.19k op/s 2024-05-21T03:31:01.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:00 smithi165 ceph-mon[36269]: pgmap v212: 97 pgs: 97 active+clean; 17 GiB data, 56 GiB used, 1016 GiB / 1.0 TiB avail; 51 KiB/s rd, 50 MiB/s wr, 10.19k op/s 2024-05-21T03:31:01.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:00 smithi027 ceph-mon[32179]: pgmap v212: 97 pgs: 97 active+clean; 17 GiB data, 56 GiB used, 1016 GiB / 1.0 TiB avail; 51 KiB/s rd, 50 MiB/s wr, 10.19k op/s 2024-05-21T03:31:03.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:02 smithi039 ceph-mon[36429]: pgmap v213: 97 pgs: 97 active+clean; 17 GiB data, 56 GiB used, 1016 GiB / 1.0 TiB avail; 39 KiB/s rd, 33 MiB/s wr, 6.89k op/s 2024-05-21T03:31:03.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:02 smithi165 ceph-mon[36269]: pgmap v213: 97 pgs: 97 active+clean; 17 GiB data, 56 GiB used, 1016 GiB / 1.0 TiB avail; 39 KiB/s rd, 33 MiB/s wr, 6.89k op/s 2024-05-21T03:31:03.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:02 smithi027 ceph-mon[32179]: pgmap v213: 97 pgs: 97 active+clean; 17 GiB data, 56 GiB used, 1016 GiB / 1.0 TiB avail; 39 KiB/s rd, 33 MiB/s wr, 6.89k op/s 2024-05-21T03:31:04.978 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:04 smithi027 ceph-mon[32179]: pgmap v214: 97 pgs: 97 active+clean; 17 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 41 KiB/s rd, 36 MiB/s wr, 7.40k op/s 2024-05-21T03:31:05.018 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:04 smithi039 ceph-mon[36429]: pgmap v214: 97 pgs: 97 active+clean; 17 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 41 KiB/s rd, 36 MiB/s wr, 7.40k op/s 2024-05-21T03:31:05.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:04 smithi165 ceph-mon[36269]: pgmap v214: 97 pgs: 97 active+clean; 17 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 41 KiB/s rd, 36 MiB/s wr, 7.40k op/s 2024-05-21T03:31:07.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:06 smithi039 ceph-mon[36429]: pgmap v215: 97 pgs: 97 active+clean; 17 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 48 KiB/s rd, 42 MiB/s wr, 8.53k op/s 2024-05-21T03:31:07.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:06 smithi039 ceph-mon[36429]: osdmap e84: 12 total, 12 up, 12 in 2024-05-21T03:31:07.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:06 smithi165 ceph-mon[36269]: pgmap v215: 97 pgs: 97 active+clean; 17 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 48 KiB/s rd, 42 MiB/s wr, 8.53k op/s 2024-05-21T03:31:07.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:06 smithi165 ceph-mon[36269]: osdmap e84: 12 total, 12 up, 12 in 2024-05-21T03:31:07.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:06 smithi027 ceph-mon[32179]: pgmap v215: 97 pgs: 97 active+clean; 17 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 48 KiB/s rd, 42 MiB/s wr, 8.53k op/s 2024-05-21T03:31:07.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:06 smithi027 ceph-mon[32179]: osdmap e84: 12 total, 12 up, 12 in 2024-05-21T03:31:09.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:08 smithi039 ceph-mon[36429]: pgmap v217: 97 pgs: 97 active+clean; 17 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 52 KiB/s rd, 41 MiB/s wr, 8.36k op/s 2024-05-21T03:31:09.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:08 smithi165 ceph-mon[36269]: pgmap v217: 97 pgs: 97 active+clean; 17 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 52 KiB/s rd, 41 MiB/s wr, 8.36k op/s 2024-05-21T03:31:09.153 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.3 2024-05-21T03:31:09.154 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:31:09.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:08 smithi027 ceph-mon[32179]: pgmap v217: 97 pgs: 97 active+clean; 17 GiB data, 57 GiB used, 1015 GiB / 1.0 TiB avail; 52 KiB/s rd, 41 MiB/s wr, 8.36k op/s 2024-05-21T03:31:09.645 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:31:11.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:10 smithi039 ceph-mon[36429]: pgmap v218: 97 pgs: 97 active+clean; 17 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 53 KiB/s rd, 38 MiB/s wr, 7.66k op/s 2024-05-21T03:31:11.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:10 smithi165 ceph-mon[36269]: pgmap v218: 97 pgs: 97 active+clean; 17 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 53 KiB/s rd, 38 MiB/s wr, 7.66k op/s 2024-05-21T03:31:11.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:10 smithi027 ceph-mon[32179]: pgmap v218: 97 pgs: 97 active+clean; 17 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 53 KiB/s rd, 38 MiB/s wr, 7.66k op/s 2024-05-21T03:31:12.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.005+0000 7f17d7382700 1 -- 172.21.15.27:0/1577706319 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17d006c470 msgr2=0x7f17d01103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:31:12.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.005+0000 7f17d7382700 1 --2- 172.21.15.27:0/1577706319 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17d006c470 0x7f17d01103c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f17c800b0a0 tx=0x7f17c800b3b0 comp rx=0 tx=0).stop 2024-05-21T03:31:12.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.005+0000 7f17d7382700 1 -- 172.21.15.27:0/1577706319 shutdown_connections 2024-05-21T03:31:12.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.005+0000 7f17d7382700 1 --2- 172.21.15.27:0/1577706319 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17d006c470 0x7f17d01103c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:12.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.005+0000 7f17d7382700 1 --2- 172.21.15.27:0/1577706319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17d01095a0 0x7f17d0109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:12.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.005+0000 7f17d7382700 1 --2- 172.21.15.27:0/1577706319 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17d006bab0 0x7f17d006bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:12.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.005+0000 7f17d7382700 1 -- 172.21.15.27:0/1577706319 >> 172.21.15.27:0/1577706319 conn(0x7f17d00793b0 msgr2=0x7f17d00797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:31:12.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.005+0000 7f17d7382700 1 -- 172.21.15.27:0/1577706319 shutdown_connections 2024-05-21T03:31:12.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.005+0000 7f17d7382700 1 -- 172.21.15.27:0/1577706319 wait complete. 2024-05-21T03:31:12.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.006+0000 7f17d7382700 1 Processor -- start 2024-05-21T03:31:12.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.006+0000 7f17d7382700 1 -- start start 2024-05-21T03:31:12.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.006+0000 7f17d7382700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17d006bab0 0x7f17d0117970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:12.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.006+0000 7f17d7382700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17d006c470 0x7f17d0117eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:12.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.006+0000 7f17d7382700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17d01095a0 0x7f17d0112ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:12.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.007+0000 7f17d7382700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f17d0103f70 con 0x7f17d01095a0 2024-05-21T03:31:12.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.007+0000 7f17d7382700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f17d0103df0 con 0x7f17d006bab0 2024-05-21T03:31:12.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.007+0000 7f17d7382700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f17d01040f0 con 0x7f17d006c470 2024-05-21T03:31:12.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.007+0000 7f17d591f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17d01095a0 0x7f17d0112ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:12.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.007+0000 7f17d591f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17d01095a0 0x7f17d0112ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52092/0 (socket says 172.21.15.27:52092) 2024-05-21T03:31:12.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.007+0000 7f17d591f700 1 -- 172.21.15.27:0/494606086 learned_addr learned my addr 172.21.15.27:0/494606086 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:31:12.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.007+0000 7f17d511e700 1 --2- 172.21.15.27:0/494606086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17d006bab0 0x7f17d0117970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:12.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.007+0000 7f17d591f700 1 -- 172.21.15.27:0/494606086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17d006c470 msgr2=0x7f17d0117eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:31:12.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.007+0000 7f17d591f700 1 --2- 172.21.15.27:0/494606086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17d006c470 0x7f17d0117eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:12.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.007+0000 7f17d591f700 1 -- 172.21.15.27:0/494606086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17d006bab0 msgr2=0x7f17d0117970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:31:12.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.007+0000 7f17d591f700 1 --2- 172.21.15.27:0/494606086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17d006bab0 0x7f17d0117970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:12.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.007+0000 7f17d591f700 1 -- 172.21.15.27:0/494606086 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d01133e0 con 0x7f17d01095a0 2024-05-21T03:31:12.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.007+0000 7f17d591f700 1 --2- 172.21.15.27:0/494606086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17d01095a0 0x7f17d0112ae0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f17c80062a0 tx=0x7f17c8004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:31:12.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.008+0000 7f17c67fc700 1 -- 172.21.15.27:0/494606086 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c8021070 con 0x7f17d01095a0 2024-05-21T03:31:12.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.008+0000 7f17c67fc700 1 -- 172.21.15.27:0/494606086 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17c8007e30 con 0x7f17d01095a0 2024-05-21T03:31:12.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.008+0000 7f17c67fc700 1 -- 172.21.15.27:0/494606086 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c8004780 con 0x7f17d01095a0 2024-05-21T03:31:12.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.008+0000 7f17d7382700 1 -- 172.21.15.27:0/494606086 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d0113670 con 0x7f17d01095a0 2024-05-21T03:31:12.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.008+0000 7f17d7382700 1 -- 172.21.15.27:0/494606086 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d011d860 con 0x7f17d01095a0 2024-05-21T03:31:12.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.009+0000 7f17d7382700 1 -- 172.21.15.27:0/494606086 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17d0104c60 con 0x7f17d01095a0 2024-05-21T03:31:12.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.012+0000 7f17c67fc700 1 -- 172.21.15.27:0/494606086 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7f17c8007980 con 0x7f17d01095a0 2024-05-21T03:31:12.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.013+0000 7f17c67fc700 1 --2- 172.21.15.27:0/494606086 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f17bc0b2b40 0x7f17bc0b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:12.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.013+0000 7f17c67fc700 1 -- 172.21.15.27:0/494606086 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8615+0+0 (secure 0 0 0) 0x7f17c80e3530 con 0x7f17d01095a0 2024-05-21T03:31:12.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.013+0000 7f17d511e700 1 --2- 172.21.15.27:0/494606086 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f17bc0b2b40 0x7f17bc0b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:12.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.014+0000 7f17d511e700 1 --2- 172.21.15.27:0/494606086 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f17bc0b2b40 0x7f17bc0b5000 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009be0 tx=0x7f17cc009260 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:31:12.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.019+0000 7f17c67fc700 1 -- 172.21.15.27:0/494606086 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f17c8004000 con 0x7f17d01095a0 2024-05-21T03:31:12.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.520+0000 7f17d7382700 1 -- 172.21.15.27:0/494606086 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17d006a230 con 0x7f17d01095a0 2024-05-21T03:31:12.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.531+0000 7f17c67fc700 1 -- 172.21.15.27:0/494606086 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11931 (secure 0 0 0) 0x7f17d006a230 con 0x7f17d01095a0 2024-05-21T03:31:12.530 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:31:12.531 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/148854869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":148854869},{"type":"v1","addr":"172.21.15.27:6835","nonce":148854869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/4145544730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":4145544730},{"type":"v1","addr":"172.21.15.27:6837","nonce":4145544730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T03:20:34.055985+0000","modified":"2024-05-21T03:30:04.352448+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24472,"mds_2":24457,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/745486764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":745486764},{"type":"v1","addr":"172.21.15.27:6839","nonce":745486764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6841/3982752351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3982752351},{"type":"v1","addr":"172.21.15.27:6841","nonce":3982752351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/3497521608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3497521608},{"type":"v1","addr":"172.21.15.39:6835","nonce":3497521608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.165:6835/3576910218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3576910218},{"type":"v1","addr":"172.21.15.165:6835","nonce":3576910218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/3673846852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3673846852},{"type":"v1","addr":"172.21.15.39:6837","nonce":3673846852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6837/691356139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":691356139},{"type":"v1","addr":"172.21.15.165:6837","nonce":691356139}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6839/4158170814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4158170814},{"type":"v1","addr":"172.21.15.165:6839","nonce":4158170814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6839/287607819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":287607819},{"type":"v1","addr":"172.21.15.39:6839","nonce":287607819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24457,24476,24452,24472,24451]},"id":1}]} 2024-05-21T03:31:12.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.535+0000 7f17d7382700 1 -- 172.21.15.27:0/494606086 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f17bc0b2b40 msgr2=0x7f17bc0b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:31:12.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.535+0000 7f17d7382700 1 --2- 172.21.15.27:0/494606086 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f17bc0b2b40 0x7f17bc0b5000 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009be0 tx=0x7f17cc009260 comp rx=0 tx=0).stop 2024-05-21T03:31:12.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.535+0000 7f17d7382700 1 -- 172.21.15.27:0/494606086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17d01095a0 msgr2=0x7f17d0112ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:31:12.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.535+0000 7f17d7382700 1 --2- 172.21.15.27:0/494606086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17d01095a0 0x7f17d0112ae0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f17c80062a0 tx=0x7f17c8004c80 comp rx=0 tx=0).stop 2024-05-21T03:31:12.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.535+0000 7f17d7382700 1 -- 172.21.15.27:0/494606086 shutdown_connections 2024-05-21T03:31:12.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.535+0000 7f17d7382700 1 --2- 172.21.15.27:0/494606086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17d01095a0 0x7f17d0112ae0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:12.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.535+0000 7f17d7382700 1 --2- 172.21.15.27:0/494606086 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f17bc0b2b40 0x7f17bc0b5000 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:12.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.535+0000 7f17d7382700 1 --2- 172.21.15.27:0/494606086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17d006bab0 0x7f17d0117970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:12.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.535+0000 7f17d7382700 1 --2- 172.21.15.27:0/494606086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17d006c470 0x7f17d0117eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:12.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.535+0000 7f17d7382700 1 -- 172.21.15.27:0/494606086 >> 172.21.15.27:0/494606086 conn(0x7f17d00793b0 msgr2=0x7f17d01010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:31:12.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.537+0000 7f17d7382700 1 -- 172.21.15.27:0/494606086 shutdown_connections 2024-05-21T03:31:12.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:12.537+0000 7f17d7382700 1 -- 172.21.15.27:0/494606086 wait complete. 2024-05-21T03:31:12.539 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 39 2024-05-21T03:31:12.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:12 smithi027 ceph-mon[32179]: pgmap v219: 97 pgs: 97 active+clean; 17 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 53 KiB/s rd, 38 MiB/s wr, 7.66k op/s 2024-05-21T03:31:12.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:12 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/494606086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:31:13.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:12 smithi039 ceph-mon[36429]: pgmap v219: 97 pgs: 97 active+clean; 17 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 53 KiB/s rd, 38 MiB/s wr, 7.66k op/s 2024-05-21T03:31:13.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:12 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/494606086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:31:13.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:12 smithi165 ceph-mon[36269]: pgmap v219: 97 pgs: 97 active+clean; 17 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 53 KiB/s rd, 38 MiB/s wr, 7.66k op/s 2024-05-21T03:31:13.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:12 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/494606086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:31:13.616 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 147.3 --await-for 45.0' 2024-05-21T03:31:13.616 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 147.3 --await-for 45.0 2024-05-21T03:31:13.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:13 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:31:13.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:13 smithi027 ceph-mon[32179]: osdmap e85: 12 total, 12 up, 12 in 2024-05-21T03:31:14.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:13 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:31:14.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:13 smithi039 ceph-mon[36429]: osdmap e85: 12 total, 12 up, 12 in 2024-05-21T03:31:14.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:13 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:31:14.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:13 smithi165 ceph-mon[36269]: osdmap e85: 12 total, 12 up, 12 in 2024-05-21T03:31:14.252 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:31:15.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:14 smithi039 ceph-mon[36429]: pgmap v221: 97 pgs: 97 active+clean; 17 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 33 KiB/s rd, 19 MiB/s wr, 3.94k op/s 2024-05-21T03:31:15.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:14 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:31:15.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:14 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:31:15.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:14 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:31:15.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:14 smithi165 ceph-mon[36269]: pgmap v221: 97 pgs: 97 active+clean; 17 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 33 KiB/s rd, 19 MiB/s wr, 3.94k op/s 2024-05-21T03:31:15.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:14 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:31:15.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:14 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:31:15.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:14 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:31:15.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:14 smithi027 ceph-mon[32179]: pgmap v221: 97 pgs: 97 active+clean; 17 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 33 KiB/s rd, 19 MiB/s wr, 3.94k op/s 2024-05-21T03:31:15.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:14 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:31:15.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:14 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:31:15.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:14 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:31:17.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:16 smithi039 ceph-mon[36429]: pgmap v222: 97 pgs: 97 active+clean; 17 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 45 KiB/s rd, 40 MiB/s wr, 8.11k op/s 2024-05-21T03:31:17.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:16 smithi165 ceph-mon[36269]: pgmap v222: 97 pgs: 97 active+clean; 17 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 45 KiB/s rd, 40 MiB/s wr, 8.11k op/s 2024-05-21T03:31:17.041 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:16 smithi027 ceph-mon[32179]: pgmap v222: 97 pgs: 97 active+clean; 17 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 45 KiB/s rd, 40 MiB/s wr, 8.11k op/s 2024-05-21T03:31:17.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:17.989+0000 7f184005a700 1 -- 172.21.15.27:0/2617530609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1838114230 msgr2=0x7f1838116620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:31:17.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:17.989+0000 7f184005a700 1 --2- 172.21.15.27:0/2617530609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1838114230 0x7f1838116620 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1828009990 tx=0x7f1828009ca0 comp rx=0 tx=0).stop 2024-05-21T03:31:17.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:17.989+0000 7f184005a700 1 -- 172.21.15.27:0/2617530609 shutdown_connections 2024-05-21T03:31:17.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:17.989+0000 7f184005a700 1 --2- 172.21.15.27:0/2617530609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1838114230 0x7f1838116620 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:17.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:17.989+0000 7f184005a700 1 --2- 172.21.15.27:0/2617530609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1838077560 0x7f1838075770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:17.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:17.989+0000 7f184005a700 1 --2- 172.21.15.27:0/2617530609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1838075cb0 0x7f1838076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:17.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:17.990+0000 7f184005a700 1 -- 172.21.15.27:0/2617530609 >> 172.21.15.27:0/2617530609 conn(0x7f183807a610 msgr2=0x7f183807aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:31:17.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:17.995+0000 7f184005a700 1 -- 172.21.15.27:0/2617530609 shutdown_connections 2024-05-21T03:31:17.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:17.996+0000 7f184005a700 1 -- 172.21.15.27:0/2617530609 wait complete. 2024-05-21T03:31:17.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:17.997+0000 7f184005a700 1 Processor -- start 2024-05-21T03:31:17.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:17.997+0000 7f184005a700 1 -- start start 2024-05-21T03:31:17.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:17.998+0000 7f184005a700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1838075cb0 0x7f1838074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:17.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:17.998+0000 7f184005a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1838114230 0x7f183806ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:17.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:17.998+0000 7f184005a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f183806e2f0 0x7f183806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:17.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:17.998+0000 7f184005a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1838119dc0 con 0x7f183806e2f0 2024-05-21T03:31:17.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:17.998+0000 7f184005a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1838119c40 con 0x7f1838114230 2024-05-21T03:31:17.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:17.998+0000 7f184005a700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1838119f40 con 0x7f1838075cb0 2024-05-21T03:31:18.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:17.998+0000 7f183e5f7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f183806e2f0 0x7f183806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:18.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:17.998+0000 7f183e5f7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f183806e2f0 0x7f183806e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34288/0 (socket says 172.21.15.27:34288) 2024-05-21T03:31:18.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:17.998+0000 7f183e5f7700 1 -- 172.21.15.27:0/1364027092 learned_addr learned my addr 172.21.15.27:0/1364027092 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:31:18.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:17.998+0000 7f183d5f5700 1 --2- 172.21.15.27:0/1364027092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1838114230 0x7f183806ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:18.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:17.999+0000 7f183e5f7700 1 -- 172.21.15.27:0/1364027092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1838075cb0 msgr2=0x7f1838074d20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T03:31:18.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:17.999+0000 7f183e5f7700 1 --2- 172.21.15.27:0/1364027092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1838075cb0 0x7f1838074d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:18.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:17.999+0000 7f183e5f7700 1 -- 172.21.15.27:0/1364027092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1838114230 msgr2=0x7f183806ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:31:18.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:17.999+0000 7f183e5f7700 1 --2- 172.21.15.27:0/1364027092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1838114230 0x7f183806ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:18.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:17.999+0000 7f183e5f7700 1 -- 172.21.15.27:0/1364027092 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1838072850 con 0x7f183806e2f0 2024-05-21T03:31:18.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:17.999+0000 7f183e5f7700 1 --2- 172.21.15.27:0/1364027092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f183806e2f0 0x7f183806e770 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f182800b5c0 tx=0x7f1828005eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:31:18.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:17.999+0000 7f1836ffd700 1 -- 172.21.15.27:0/1364027092 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1828027070 con 0x7f183806e2f0 2024-05-21T03:31:18.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:17.999+0000 7f184005a700 1 -- 172.21.15.27:0/1364027092 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1838072ae0 con 0x7f183806e2f0 2024-05-21T03:31:18.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:17.999+0000 7f184005a700 1 -- 172.21.15.27:0/1364027092 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1838073010 con 0x7f183806e2f0 2024-05-21T03:31:18.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.005+0000 7f184a8f0b80 1 Processor -- start 2024-05-21T03:31:18.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.005+0000 7f184a8f0b80 1 -- start start 2024-05-21T03:31:18.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.006+0000 7f184a8f0b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56508610b340 0x56508610b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:18.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.006+0000 7f184a8f0b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x565086105970 0x565086105df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:18.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.006+0000 7f184a8f0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565086106330 0x565086112170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:18.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.006+0000 7f184a8f0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x565086064e20 con 0x565086106330 2024-05-21T03:31:18.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.006+0000 7f184a8f0b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5650860549c0 con 0x565086105970 2024-05-21T03:31:18.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.006+0000 7f184a8f0b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x565086058a00 con 0x56508610b340 2024-05-21T03:31:18.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.006+0000 7f1836ffd700 1 -- 172.21.15.27:0/1364027092 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1828022920 con 0x7f183806e2f0 2024-05-21T03:31:18.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.006+0000 7f1836ffd700 1 -- 172.21.15.27:0/1364027092 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182800f790 con 0x7f183806e2f0 2024-05-21T03:31:18.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.006+0000 7f183d5f5700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56508610b340 0x56508610b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:18.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.006+0000 7f183d5f5700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56508610b340 0x56508610b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.27:37426/0 (socket says 172.21.15.27:37426) 2024-05-21T03:31:18.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.006+0000 7f183d5f5700 1 -- 172.21.15.27:0/767027821 learned_addr learned my addr 172.21.15.27:0/767027821 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:31:18.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.011+0000 7f183d5f5700 1 --2- 172.21.15.27:0/767027821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565086106330 0x565086112170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:18.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.011+0000 7f183d5f5700 1 -- 172.21.15.27:0/767027821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x565086105970 msgr2=0x565086105df0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T03:31:18.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.011+0000 7f183d5f5700 1 --2- 172.21.15.27:0/767027821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x565086105970 0x565086105df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:18.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.011+0000 7f183d5f5700 1 -- 172.21.15.27:0/767027821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565086106330 msgr2=0x565086112170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:31:18.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.012+0000 7f183d5f5700 1 --2- 172.21.15.27:0/767027821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565086106330 0x565086112170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:18.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.012+0000 7f183d5f5700 1 -- 172.21.15.27:0/767027821 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565085fa4a60 con 0x56508610b340 2024-05-21T03:31:18.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.012+0000 7f1836ffd700 1 -- 172.21.15.27:0/1364027092 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7f1828022470 con 0x7f183806e2f0 2024-05-21T03:31:18.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.013+0000 7f1836ffd700 1 --2- 172.21.15.27:0/1364027092 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f18240b2ba0 0x7f18240b5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:18.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.014+0000 7f1836ffd700 1 -- 172.21.15.27:0/1364027092 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8571+0+0 (secure 0 0 0) 0x7f18280e3230 con 0x7f183806e2f0 2024-05-21T03:31:18.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.015+0000 7f183d5f5700 1 --2- 172.21.15.27:0/767027821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56508610b340 0x56508610b720 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1830011590 tx=0x7f183000a7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ef2211a9922b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:31:18.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.017+0000 7f182ffff700 1 -- 172.21.15.27:0/767027821 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1830004220 con 0x56508610b340 2024-05-21T03:31:18.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.018+0000 7f182ffff700 1 -- 172.21.15.27:0/767027821 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18300043c0 con 0x56508610b340 2024-05-21T03:31:18.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.019+0000 7f184a8f0b80 1 -- 172.21.15.27:0/767027821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56508610b340 msgr2=0x56508610b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:31:18.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.019+0000 7f184a8f0b80 1 --2- 172.21.15.27:0/767027821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56508610b340 0x56508610b720 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1830011590 tx=0x7f183000a7c0 comp rx=0 tx=0).stop 2024-05-21T03:31:18.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.019+0000 7f184a8f0b80 1 -- 172.21.15.27:0/767027821 shutdown_connections 2024-05-21T03:31:18.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.019+0000 7f184a8f0b80 1 --2- 172.21.15.27:0/767027821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565086106330 0x565086112170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:18.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.019+0000 7f184a8f0b80 1 --2- 172.21.15.27:0/767027821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56508610b340 0x56508610b720 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:18.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.019+0000 7f184a8f0b80 1 --2- 172.21.15.27:0/767027821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x565086105970 0x565086105df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:18.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.019+0000 7f184a8f0b80 1 -- 172.21.15.27:0/767027821 >> 172.21.15.27:0/767027821 conn(0x565086033210 msgr2=0x565086100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:31:18.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.019+0000 7f183ddf6700 1 --2- 172.21.15.27:0/1364027092 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f18240b2ba0 0x7f18240b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:18.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.020+0000 7f184a8f0b80 1 -- 172.21.15.27:0/767027821 shutdown_connections 2024-05-21T03:31:18.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.020+0000 7f184a8f0b80 1 -- 172.21.15.27:0/767027821 wait complete. 2024-05-21T03:31:18.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.021+0000 7f184a8f0b80 1 -- start start 2024-05-21T03:31:18.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.021+0000 7f184a8f0b80 1 Processor -- start 2024-05-21T03:31:18.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.022+0000 7f184a8f0b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x565086105970 0x565086111dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:18.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.022+0000 7f184a8f0b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565086106330 0x56508610f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:18.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.022+0000 7f184a8f0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56508610b340 0x56508610f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:18.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.022+0000 7f183e5f7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56508610b340 0x56508610f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:18.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.022+0000 7f183e5f7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56508610b340 0x56508610f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34306/0 (socket says 172.21.15.27:34306) 2024-05-21T03:31:18.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.023+0000 7f183e5f7700 1 -- 172.21.15.27:0/136695820 learned_addr learned my addr 172.21.15.27:0/136695820 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:31:18.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.022+0000 7f184a8f0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x565085f5b130 con 0x56508610b340 2024-05-21T03:31:18.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.023+0000 7f184a8f0b80 1 -- 172.21.15.27:0/136695820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x565085ff6950 con 0x565086105970 2024-05-21T03:31:18.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.023+0000 7f184a8f0b80 1 -- 172.21.15.27:0/136695820 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5650861127d0 con 0x565086106330 2024-05-21T03:31:18.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.024+0000 7f184a8f0b80 10 client.0 _ll_register_callbacks cb 0x565085fefab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:31:18.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.024+0000 7f183e5f7700 1 -- 172.21.15.27:0/136695820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565086106330 msgr2=0x56508610f0c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T03:31:18.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.024+0000 7f183e5f7700 1 --2- 172.21.15.27:0/136695820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565086106330 0x56508610f0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:18.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.024+0000 7f183e5f7700 1 -- 172.21.15.27:0/136695820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x565086105970 msgr2=0x565086111dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T03:31:18.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.024+0000 7f183e5f7700 1 --2- 172.21.15.27:0/136695820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x565086105970 0x565086111dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:18.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.024+0000 7f183e5f7700 1 -- 172.21.15.27:0/136695820 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565085ff3820 con 0x56508610b340 2024-05-21T03:31:18.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.025+0000 7f183e5f7700 1 --2- 172.21.15.27:0/136695820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56508610b340 0x56508610f600 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f18280aa210 tx=0x7f18280227d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa880c38e9bff41 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:31:18.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.025+0000 7f184a8f0b80 1 -- 172.21.15.27:0/136695820 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18300043c0 con 0x56508610b340 2024-05-21T03:31:18.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.025+0000 7f182d7fa700 10 client.15816 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:31:18.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.025+0000 7f182d7fa700 1 -- 172.21.15.27:0/136695820 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18280aac10 con 0x56508610b340 2024-05-21T03:31:18.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.026+0000 7f182d7fa700 1 -- 172.21.15.27:0/136695820 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8571+0+0 (secure 0 0 0) 0x7f18280b3060 con 0x56508610b340 2024-05-21T03:31:18.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.026+0000 7f182d7fa700 1 -- 172.21.15.27:0/136695820 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f18280aa510 con 0x56508610b340 2024-05-21T03:31:18.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.026+0000 7f184a8f0b80 10 client.15816 fetch_fsmap learned FSMap version 39 2024-05-21T03:31:18.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.026+0000 7f184a8f0b80 1 -- 172.21.15.27:0/136695820 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({fsmap=39}) -- 0x56508611cdb0 con 0x56508610b340 2024-05-21T03:31:18.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.030+0000 7f183ddf6700 1 --2- 172.21.15.27:0/1364027092 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f18240b2ba0 0x7f18240b5060 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1838076b60 tx=0x7f1820009230 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:31:18.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.030+0000 7f182d7fa700 1 -- 172.21.15.27:0/136695820 <== mon.0 v2:172.21.15.27:3300/0 4 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f1828003bc0 con 0x56508610b340 2024-05-21T03:31:18.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.030+0000 7f184a8f0b80 10 client.15816 fetch_fsmap finished waiting for FSMap version 39 2024-05-21T03:31:18.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.030+0000 7f184a8f0b80 10 client.15816 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-21T03:31:18.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.030+0000 7f184a8f0b80 20 client.15816 populate_metadata read hostname 'smithi027' 2024-05-21T03:31:18.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.031+0000 7f184a8f0b80 1 --2- 172.21.15.27:0/136695820 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x56508611f450 0x565086121890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:18.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.031+0000 7f184a8f0b80 4 client.15816 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:31:18.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.031+0000 7f184a8f0b80 1 -- 172.21.15.27:0/136695820 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f18300043c0 con 0x56508611f450 2024-05-21T03:31:18.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.038+0000 7f183ddf6700 1 --2- 172.21.15.27:0/136695820 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x56508611f450 0x565086121890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:18.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.050+0000 7f182d7fa700 10 client.15816 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:31:18.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.050+0000 7f183ddf6700 1 --2- 172.21.15.27:0/136695820 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x56508611f450 0x565086121890 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45de3076e507f404 server_cookie=67b25b573e4d9ec2 in_seq=0 out_seq=0 2024-05-21T03:31:18.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.055+0000 7f182d7fa700 1 -- 172.21.15.27:0/136695820 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f18300043c0 con 0x56508611f450 2024-05-21T03:31:18.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.055+0000 7f182d7fa700 10 client.15816 handle_command_reply: tid=0 2024-05-21T03:31:18.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.055+0000 7f184a8f0b80 1 client.15816 shutdown 2024-05-21T03:31:18.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.055+0000 7f184a8f0b80 1 -- 172.21.15.27:0/136695820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56508610b340 msgr2=0x56508610f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:31:18.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.056+0000 7f184a8f0b80 1 --2- 172.21.15.27:0/136695820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56508610b340 0x56508610f600 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f18280aa210 tx=0x7f18280227d0 comp rx=0 tx=0).stop 2024-05-21T03:31:18.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.056+0000 7f184a8f0b80 1 -- 172.21.15.27:0/136695820 shutdown_connections 2024-05-21T03:31:18.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.056+0000 7f184a8f0b80 1 --2- 172.21.15.27:0/136695820 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x56508611f450 0x565086121890 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:18.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.056+0000 7f184a8f0b80 1 --2- 172.21.15.27:0/136695820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56508610b340 0x56508610f600 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:18.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.056+0000 7f184a8f0b80 1 --2- 172.21.15.27:0/136695820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x565086105970 0x565086111dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:18.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.056+0000 7f184a8f0b80 1 --2- 172.21.15.27:0/136695820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565086106330 0x56508610f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:18.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.056+0000 7f184a8f0b80 1 -- 172.21.15.27:0/136695820 >> 172.21.15.27:0/136695820 conn(0x565086033210 msgr2=0x5650860fad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:31:18.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.056+0000 7f182d7fa700 0 client.15816 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:31:18.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.056+0000 7f184a8f0b80 1 -- 172.21.15.27:0/136695820 shutdown_connections 2024-05-21T03:31:18.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.056+0000 7f184a8f0b80 1 -- 172.21.15.27:0/136695820 wait complete. 2024-05-21T03:31:18.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.057+0000 7f184a8f0b80 20 client.15816 trim_cache size 0 max 16384 2024-05-21T03:31:18.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.112+0000 7f184a8f0b80 1 Processor -- start 2024-05-21T03:31:18.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.112+0000 7f184a8f0b80 1 -- start start 2024-05-21T03:31:18.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.113+0000 7f184a8f0b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x565086105970 0x565086109530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:18.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.113+0000 7f184a8f0b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565086106330 0x565086127160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:18.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.113+0000 7f184a8f0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56508610b340 0x5650861276a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:18.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.113+0000 7f183ddf6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x565086105970 0x565086109530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:18.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.113+0000 7f183ddf6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x565086105970 0x565086109530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.27:39468/0 (socket says 172.21.15.27:39468) 2024-05-21T03:31:18.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.114+0000 7f183d5f5700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565086106330 0x565086127160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:18.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.114+0000 7f183d5f5700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565086106330 0x565086127160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.27:37434/0 (socket says 172.21.15.27:37434) 2024-05-21T03:31:18.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.114+0000 7f183ddf6700 1 -- 172.21.15.27:0/1834861389 learned_addr learned my addr 172.21.15.27:0/1834861389 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:31:18.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.114+0000 7f184a8f0b80 1 -- 172.21.15.27:0/1834861389 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f181800d660 con 0x56508610b340 2024-05-21T03:31:18.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.114+0000 7f184a8f0b80 1 -- 172.21.15.27:0/1834861389 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f181800f260 con 0x565086105970 2024-05-21T03:31:18.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.114+0000 7f184a8f0b80 1 -- 172.21.15.27:0/1834861389 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f181800dd30 con 0x565086106330 2024-05-21T03:31:18.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.115+0000 7f183ddf6700 1 -- 172.21.15.27:0/1834861389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565086106330 msgr2=0x565086127160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:31:18.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.115+0000 7f183ddf6700 1 --2- 172.21.15.27:0/1834861389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565086106330 0x565086127160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:18.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.115+0000 7f183ddf6700 1 -- 172.21.15.27:0/1834861389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56508610b340 msgr2=0x5650861276a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:31:18.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.115+0000 7f183ddf6700 1 --2- 172.21.15.27:0/1834861389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56508610b340 0x5650861276a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:18.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.115+0000 7f183ddf6700 1 -- 172.21.15.27:0/1834861389 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565086102420 con 0x565086105970 2024-05-21T03:31:18.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.115+0000 7f183ddf6700 1 --2- 172.21.15.27:0/1834861389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x565086105970 0x565086109530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f182000a420 tx=0x7f182000f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e968e55c25b35d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:31:18.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.118+0000 7f182f7fe700 1 -- 172.21.15.27:0/1834861389 <== mon.2 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1820013070 con 0x565086105970 2024-05-21T03:31:18.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.118+0000 7f182f7fe700 1 -- 172.21.15.27:0/1834861389 <== mon.2 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1820027070 con 0x565086105970 2024-05-21T03:31:18.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.118+0000 7f182f7fe700 1 -- 172.21.15.27:0/1834861389 <== mon.2 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1820021970 con 0x565086105970 2024-05-21T03:31:18.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.119+0000 7f184a8f0b80 1 -- 172.21.15.27:0/1834861389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x565086105970 msgr2=0x565086109530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:31:18.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.119+0000 7f184a8f0b80 1 --2- 172.21.15.27:0/1834861389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x565086105970 0x565086109530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f182000a420 tx=0x7f182000f000 comp rx=0 tx=0).stop 2024-05-21T03:31:18.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.119+0000 7f184a8f0b80 1 -- 172.21.15.27:0/1834861389 shutdown_connections 2024-05-21T03:31:18.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.119+0000 7f184a8f0b80 1 --2- 172.21.15.27:0/1834861389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56508610b340 0x5650861276a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:18.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.119+0000 7f184a8f0b80 1 --2- 172.21.15.27:0/1834861389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x565086105970 0x565086109530 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:18.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.119+0000 7f184a8f0b80 1 --2- 172.21.15.27:0/1834861389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565086106330 0x565086127160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:18.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.119+0000 7f184a8f0b80 1 -- 172.21.15.27:0/1834861389 >> 172.21.15.27:0/1834861389 conn(0x565086033210 msgr2=0x565086108e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:31:18.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.119+0000 7f184a8f0b80 1 -- 172.21.15.27:0/1834861389 shutdown_connections 2024-05-21T03:31:18.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.119+0000 7f184a8f0b80 1 -- 172.21.15.27:0/1834861389 wait complete. 2024-05-21T03:31:18.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.120+0000 7f184a8f0b80 1 -- start start 2024-05-21T03:31:18.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.120+0000 7f184a8f0b80 1 Processor -- start 2024-05-21T03:31:18.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.121+0000 7f184a8f0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565086105970 0x565086117ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:18.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.121+0000 7f184a8f0b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565086106330 0x565086115290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:18.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.121+0000 7f184a8f0b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56508610b340 0x5650861157d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:18.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.121+0000 7f184a8f0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f181800eb90 con 0x565086105970 2024-05-21T03:31:18.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.121+0000 7f184a8f0b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f181800e4c0 con 0x56508610b340 2024-05-21T03:31:18.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.121+0000 7f184a8f0b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f181800b620 con 0x565086106330 2024-05-21T03:31:18.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.121+0000 7f183d5f5700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565086106330 0x565086115290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:18.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.121+0000 7f183d5f5700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565086106330 0x565086115290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.27:37442/0 (socket says 172.21.15.27:37442) 2024-05-21T03:31:18.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.121+0000 7f183d5f5700 1 -- 172.21.15.27:0/993249801 learned_addr learned my addr 172.21.15.27:0/993249801 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:31:18.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.121+0000 7f183e5f7700 1 --2- 172.21.15.27:0/993249801 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56508610b340 0x5650861157d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:18.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.121+0000 7f184a8f0b80 10 client.0 _ll_register_callbacks cb 0x565085fefab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:31:18.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.124+0000 7f183e5f7700 1 -- 172.21.15.27:0/993249801 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565086106330 msgr2=0x565086115290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:31:18.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.124+0000 7f183e5f7700 1 --2- 172.21.15.27:0/993249801 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565086106330 0x565086115290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:18.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.124+0000 7f183e5f7700 1 -- 172.21.15.27:0/993249801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565086105970 msgr2=0x565086117ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:31:18.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.124+0000 7f183e5f7700 1 --2- 172.21.15.27:0/993249801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565086105970 0x565086117ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:18.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.124+0000 7f183e5f7700 1 -- 172.21.15.27:0/993249801 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565086115d10 con 0x56508610b340 2024-05-21T03:31:18.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.124+0000 7f183e5f7700 1 --2- 172.21.15.27:0/993249801 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56508610b340 0x5650861157d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f18280aa4a0 tx=0x7f1828092040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d597e9b27aa4656 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:31:18.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.124+0000 7f182d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-21T03:31:18.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.125+0000 7f182d7fa700 1 -- 172.21.15.27:0/993249801 <== mon.2 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18280aac10 con 0x56508610b340 2024-05-21T03:31:18.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.125+0000 7f182d7fa700 1 -- 172.21.15.27:0/993249801 <== mon.2 v2:172.21.15.39:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8571+0+0 (secure 0 0 0) 0x7f1828005300 con 0x56508610b340 2024-05-21T03:31:18.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.125+0000 7f184a8f0b80 1 -- 172.21.15.27:0/993249801 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1820027070 con 0x56508610b340 2024-05-21T03:31:18.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.125+0000 7f182d7fa700 1 -- 172.21.15.27:0/993249801 <== mon.2 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f18280ae8c0 con 0x56508610b340 2024-05-21T03:31:18.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.125+0000 7f184a8f0b80 10 client.25355 fetch_fsmap learned FSMap version 39 2024-05-21T03:31:18.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.125+0000 7f184a8f0b80 1 -- 172.21.15.27:0/993249801 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56508618bd00 con 0x56508610b340 2024-05-21T03:31:18.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.126+0000 7f182d7fa700 1 -- 172.21.15.27:0/993249801 <== mon.2 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18280b32e0 con 0x56508610b340 2024-05-21T03:31:18.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.126+0000 7f182d7fa700 1 -- 172.21.15.27:0/993249801 <== mon.2 v2:172.21.15.39:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f18280a5e80 con 0x56508610b340 2024-05-21T03:31:18.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.126+0000 7f184a8f0b80 10 client.25355 fetch_fsmap finished waiting for FSMap version 39 2024-05-21T03:31:18.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.126+0000 7f184a8f0b80 10 client.25355 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-21T03:31:18.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.126+0000 7f184a8f0b80 20 client.25355 populate_metadata read hostname 'smithi027' 2024-05-21T03:31:18.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.126+0000 7f184a8f0b80 1 --2- 172.21.15.27:0/993249801 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x56508618c2e0 0x56508618e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:18.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.126+0000 7f184a8f0b80 4 client.25355 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.3, "await_for": 45.0}] 2024-05-21T03:31:18.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.126+0000 7f184a8f0b80 1 -- 172.21.15.27:0/993249801 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.3, "await_for": 45.0}) -- 0x7f1820027070 con 0x56508618c2e0 2024-05-21T03:31:18.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.126+0000 7f183ddf6700 1 --2- 172.21.15.27:0/993249801 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x56508618c2e0 0x56508618e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:18.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.127+0000 7f182d7fa700 10 client.25355 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:31:18.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:18.127+0000 7f183ddf6700 1 --2- 172.21.15.27:0/993249801 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x56508618c2e0 0x56508618e700 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=666f67d31e56ad4f server_cookie=26e4a2663d31cead in_seq=0 out_seq=0 2024-05-21T03:31:19.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:18 smithi039 ceph-mon[36429]: pgmap v223: 97 pgs: 97 active+clean; 17 GiB data, 59 GiB used, 1013 GiB / 1.0 TiB avail; 44 KiB/s rd, 42 MiB/s wr, 8.68k op/s 2024-05-21T03:31:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:18 smithi165 ceph-mon[36269]: pgmap v223: 97 pgs: 97 active+clean; 17 GiB data, 59 GiB used, 1013 GiB / 1.0 TiB avail; 44 KiB/s rd, 42 MiB/s wr, 8.68k op/s 2024-05-21T03:31:19.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:18 smithi027 ceph-mon[32179]: pgmap v223: 97 pgs: 97 active+clean; 17 GiB data, 59 GiB used, 1013 GiB / 1.0 TiB avail; 44 KiB/s rd, 42 MiB/s wr, 8.68k op/s 2024-05-21T03:31:21.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:20 smithi027 ceph-mon[32179]: pgmap v224: 97 pgs: 97 active+clean; 17 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 22 KiB/s rd, 47 MiB/s wr, 9.86k op/s 2024-05-21T03:31:21.268 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:20 smithi039 ceph-mon[36429]: pgmap v224: 97 pgs: 97 active+clean; 17 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 22 KiB/s rd, 47 MiB/s wr, 9.86k op/s 2024-05-21T03:31:21.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:20 smithi165 ceph-mon[36269]: pgmap v224: 97 pgs: 97 active+clean; 17 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 22 KiB/s rd, 47 MiB/s wr, 9.86k op/s 2024-05-21T03:31:23.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:22 smithi027 ceph-mon[32179]: pgmap v225: 97 pgs: 97 active+clean; 18 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 32 KiB/s rd, 78 MiB/s wr, 16.58k op/s 2024-05-21T03:31:23.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:22 smithi039 ceph-mon[36429]: pgmap v225: 97 pgs: 97 active+clean; 18 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 32 KiB/s rd, 78 MiB/s wr, 16.58k op/s 2024-05-21T03:31:23.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:22 smithi165 ceph-mon[36269]: pgmap v225: 97 pgs: 97 active+clean; 18 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 32 KiB/s rd, 78 MiB/s wr, 16.58k op/s 2024-05-21T03:31:25.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:24 smithi027 ceph-mon[32179]: pgmap v226: 97 pgs: 97 active+clean; 18 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 32 KiB/s rd, 77 MiB/s wr, 16.41k op/s 2024-05-21T03:31:25.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:24 smithi039 ceph-mon[36429]: pgmap v226: 97 pgs: 97 active+clean; 18 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 32 KiB/s rd, 77 MiB/s wr, 16.41k op/s 2024-05-21T03:31:25.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:24 smithi165 ceph-mon[36269]: pgmap v226: 97 pgs: 97 active+clean; 18 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 32 KiB/s rd, 77 MiB/s wr, 16.41k op/s 2024-05-21T03:31:27.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:26 smithi027 ceph-mon[32179]: pgmap v227: 97 pgs: 97 active+clean; 18 GiB data, 62 GiB used, 1010 GiB / 1.0 TiB avail; 32 KiB/s rd, 90 MiB/s wr, 19.18k op/s 2024-05-21T03:31:27.268 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:26 smithi039 ceph-mon[36429]: pgmap v227: 97 pgs: 97 active+clean; 18 GiB data, 62 GiB used, 1010 GiB / 1.0 TiB avail; 32 KiB/s rd, 90 MiB/s wr, 19.18k op/s 2024-05-21T03:31:27.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:26 smithi165 ceph-mon[36269]: pgmap v227: 97 pgs: 97 active+clean; 18 GiB data, 62 GiB used, 1010 GiB / 1.0 TiB avail; 32 KiB/s rd, 90 MiB/s wr, 19.18k op/s 2024-05-21T03:31:29.070 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:28 smithi039 ceph-mon[36429]: pgmap v228: 97 pgs: 97 active+clean; 18 GiB data, 63 GiB used, 1010 GiB / 1.0 TiB avail; 28 KiB/s rd, 80 MiB/s wr, 17.29k op/s 2024-05-21T03:31:29.072 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:28 smithi165 ceph-mon[36269]: pgmap v228: 97 pgs: 97 active+clean; 18 GiB data, 63 GiB used, 1010 GiB / 1.0 TiB avail; 28 KiB/s rd, 80 MiB/s wr, 17.29k op/s 2024-05-21T03:31:29.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:28 smithi027 ceph-mon[32179]: pgmap v228: 97 pgs: 97 active+clean; 18 GiB data, 63 GiB used, 1010 GiB / 1.0 TiB avail; 28 KiB/s rd, 80 MiB/s wr, 17.29k op/s 2024-05-21T03:31:31.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:30 smithi027 ceph-mon[32179]: pgmap v229: 97 pgs: 97 active+clean; 18 GiB data, 63 GiB used, 1010 GiB / 1.0 TiB avail; 41 KiB/s rd, 79 MiB/s wr, 17.27k op/s 2024-05-21T03:31:31.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:30 smithi039 ceph-mon[36429]: pgmap v229: 97 pgs: 97 active+clean; 18 GiB data, 63 GiB used, 1010 GiB / 1.0 TiB avail; 41 KiB/s rd, 79 MiB/s wr, 17.27k op/s 2024-05-21T03:31:31.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:30 smithi165 ceph-mon[36269]: pgmap v229: 97 pgs: 97 active+clean; 18 GiB data, 63 GiB used, 1010 GiB / 1.0 TiB avail; 41 KiB/s rd, 79 MiB/s wr, 17.27k op/s 2024-05-21T03:31:31.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:31.619+0000 7f182d7fa700 1 -- 172.21.15.27:0/993249801 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f1820027070 con 0x56508618c2e0 2024-05-21T03:31:31.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:31.619+0000 7f182d7fa700 10 client.25355 handle_command_reply: tid=0 2024-05-21T03:31:31.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:31.620+0000 7f184a8f0b80 1 client.25355 shutdown 2024-05-21T03:31:31.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:31.620+0000 7f184a8f0b80 1 -- 172.21.15.27:0/993249801 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56508610b340 msgr2=0x5650861157d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:31:31.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:31.620+0000 7f184a8f0b80 1 --2- 172.21.15.27:0/993249801 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56508610b340 0x5650861157d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f18280aa4a0 tx=0x7f1828092040 comp rx=0 tx=0).stop 2024-05-21T03:31:31.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:31.620+0000 7f184a8f0b80 1 -- 172.21.15.27:0/993249801 shutdown_connections 2024-05-21T03:31:31.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:31.620+0000 7f184a8f0b80 1 --2- 172.21.15.27:0/993249801 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x56508618c2e0 0x56508618e700 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:31.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:31.620+0000 7f184a8f0b80 1 --2- 172.21.15.27:0/993249801 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56508610b340 0x5650861157d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:31.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:31.620+0000 7f184a8f0b80 1 --2- 172.21.15.27:0/993249801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565086105970 0x565086117ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:31.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:31.620+0000 7f184a8f0b80 1 --2- 172.21.15.27:0/993249801 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565086106330 0x565086115290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:31.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:31.620+0000 7f182d7fa700 0 client.25355 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:31:31.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:31.620+0000 7f184a8f0b80 1 -- 172.21.15.27:0/993249801 >> 172.21.15.27:0/993249801 conn(0x565086033210 msgr2=0x5650861096b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:31:31.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:31.620+0000 7f184a8f0b80 1 -- 172.21.15.27:0/993249801 shutdown_connections 2024-05-21T03:31:31.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:31.620+0000 7f184a8f0b80 1 -- 172.21.15.27:0/993249801 wait complete. 2024-05-21T03:31:31.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:31.620+0000 7f184a8f0b80 20 client.25355 trim_cache size 0 max 16384 2024-05-21T03:31:31.623 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:31:31.623 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 39, 2024-05-21T03:31:31.623 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24476, 2024-05-21T03:31:31.623 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 19, 2024-05-21T03:31:31.623 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-05-21T03:31:31.623 INFO:teuthology.orchestra.run.smithi027.stdout: "d4642da5": { 2024-05-21T03:31:31.623 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 19, 2024-05-21T03:31:31.623 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-05-21T03:31:31.623 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-05-21T03:31:31.623 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-05-21T03:31:31.623 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-05-21T03:31:31.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:31:31.623 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-05-21T03:31:31.624 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 147.3, 2024-05-21T03:31:31.624 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-05-21T03:31:31.624 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-05-21T03:31:31.624 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-05-21T03:31:31.624 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-05-21T03:31:31.624 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-05-21T03:31:31.624 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-05-21T03:31:31.624 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:31:31.624 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:31:31.624 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:31:31.624 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:31:31.624 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:31:31.624 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:31:31.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:31.621+0000 7f184005a700 1 -- 172.21.15.27:0/1364027092 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f18240b2ba0 msgr2=0x7f18240b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:31:31.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:31.621+0000 7f184005a700 1 --2- 172.21.15.27:0/1364027092 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f18240b2ba0 0x7f18240b5060 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1838076b60 tx=0x7f1820009230 comp rx=0 tx=0).stop 2024-05-21T03:31:31.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:31.621+0000 7f184005a700 1 -- 172.21.15.27:0/1364027092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f183806e2f0 msgr2=0x7f183806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:31:31.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:31.621+0000 7f184005a700 1 --2- 172.21.15.27:0/1364027092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f183806e2f0 0x7f183806e770 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f182800b5c0 tx=0x7f1828005eb0 comp rx=0 tx=0).stop 2024-05-21T03:31:31.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:31.621+0000 7f184005a700 1 -- 172.21.15.27:0/1364027092 shutdown_connections 2024-05-21T03:31:31.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:31.621+0000 7f184005a700 1 --2- 172.21.15.27:0/1364027092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f183806e2f0 0x7f183806e770 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:31.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:31.621+0000 7f184005a700 1 --2- 172.21.15.27:0/1364027092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1838075cb0 0x7f1838074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:31.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:31.621+0000 7f184005a700 1 --2- 172.21.15.27:0/1364027092 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f18240b2ba0 0x7f18240b5060 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:31.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:31.621+0000 7f184005a700 1 --2- 172.21.15.27:0/1364027092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1838114230 0x7f183806ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:31.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:31.621+0000 7f184005a700 1 -- 172.21.15.27:0/1364027092 >> 172.21.15.27:0/1364027092 conn(0x7f183807a610 msgr2=0x7f1838112f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:31:31.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:31.621+0000 7f184005a700 1 -- 172.21.15.27:0/1364027092 shutdown_connections 2024-05-21T03:31:31.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:31.621+0000 7f184005a700 1 -- 172.21.15.27:0/1364027092 wait complete. 2024-05-21T03:31:32.661 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd4642da5', quiesce took 23.5 seconds. Will release after: 3.8000000000000007 2024-05-21T03:31:32.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:32 smithi027 ceph-mon[32179]: pgmap v230: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 59 KiB/s rd, 72 MiB/s wr, 15.81k op/s 2024-05-21T03:31:33.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:32 smithi039 ceph-mon[36429]: pgmap v230: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 59 KiB/s rd, 72 MiB/s wr, 15.81k op/s 2024-05-21T03:31:33.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:32 smithi165 ceph-mon[36269]: pgmap v230: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 59 KiB/s rd, 72 MiB/s wr, 15.81k op/s 2024-05-21T03:31:34.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:33 smithi027 ceph-mon[32179]: pgmap v231: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 51 KiB/s rd, 46 MiB/s wr, 10.21k op/s 2024-05-21T03:31:34.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:33 smithi039 ceph-mon[36429]: pgmap v231: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 51 KiB/s rd, 46 MiB/s wr, 10.21k op/s 2024-05-21T03:31:34.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:33 smithi165 ceph-mon[36269]: pgmap v231: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 51 KiB/s rd, 46 MiB/s wr, 10.21k op/s 2024-05-21T03:31:36.463 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:31:36.729 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:31:36.851 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:36 smithi027 ceph-mon[32179]: pgmap v232: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 58 KiB/s rd, 46 MiB/s wr, 10.21k op/s 2024-05-21T03:31:37.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:36 smithi039 ceph-mon[36429]: pgmap v232: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 58 KiB/s rd, 46 MiB/s wr, 10.21k op/s 2024-05-21T03:31:37.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:36 smithi165 ceph-mon[36269]: pgmap v232: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 58 KiB/s rd, 46 MiB/s wr, 10.21k op/s 2024-05-21T03:31:38.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.521+0000 7f69fe79e700 1 -- 172.21.15.27:0/3209214144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69f8108840 msgr2=0x7f69f81103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:31:38.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.521+0000 7f69fe79e700 1 --2- 172.21.15.27:0/3209214144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69f8108840 0x7f69f81103f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f69f000b0a0 tx=0x7f69f000b3b0 comp rx=0 tx=0).stop 2024-05-21T03:31:38.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.521+0000 7f69fe79e700 1 -- 172.21.15.27:0/3209214144 shutdown_connections 2024-05-21T03:31:38.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.521+0000 7f69fe79e700 1 --2- 172.21.15.27:0/3209214144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69f8108840 0x7f69f81103f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:38.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.521+0000 7f69fe79e700 1 --2- 172.21.15.27:0/3209214144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69f8101fd0 0x7f69f81023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:38.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.521+0000 7f69fe79e700 1 --2- 172.21.15.27:0/3209214144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69f8102980 0x7f69f8108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:38.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.521+0000 7f69fe79e700 1 -- 172.21.15.27:0/3209214144 >> 172.21.15.27:0/3209214144 conn(0x7f69f80776f0 msgr2=0x7f69f8077b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:31:38.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.521+0000 7f69fe79e700 1 -- 172.21.15.27:0/3209214144 shutdown_connections 2024-05-21T03:31:38.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.521+0000 7f69fe79e700 1 -- 172.21.15.27:0/3209214144 wait complete. 2024-05-21T03:31:38.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.522+0000 7f69fe79e700 1 Processor -- start 2024-05-21T03:31:38.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.522+0000 7f69fe79e700 1 -- start start 2024-05-21T03:31:38.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.522+0000 7f69fe79e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69f8101fd0 0x7f69f81a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:38.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.522+0000 7f69fe79e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69f8102980 0x7f69f81a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:38.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.522+0000 7f69fe79e700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69f8108840 0x7f69f819bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:38.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.522+0000 7f69fe79e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f69f81136a0 con 0x7f69f8101fd0 2024-05-21T03:31:38.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.522+0000 7f69fe79e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f69f8113520 con 0x7f69f8102980 2024-05-21T03:31:38.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.522+0000 7f69fe79e700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f69f8113820 con 0x7f69f8108840 2024-05-21T03:31:38.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.522+0000 7f69fd79c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69f8101fd0 0x7f69f81a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:38.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.522+0000 7f69fd79c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69f8101fd0 0x7f69f81a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58664/0 (socket says 172.21.15.27:58664) 2024-05-21T03:31:38.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.522+0000 7f69fd79c700 1 -- 172.21.15.27:0/1629439974 learned_addr learned my addr 172.21.15.27:0/1629439974 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:31:38.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.522+0000 7f69fcf9b700 1 --2- 172.21.15.27:0/1629439974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69f8102980 0x7f69f81a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:38.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.522+0000 7f69fdf9d700 1 --2- 172.21.15.27:0/1629439974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69f8108840 0x7f69f819bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:38.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.522+0000 7f69fd79c700 1 -- 172.21.15.27:0/1629439974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69f8108840 msgr2=0x7f69f819bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:31:38.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.522+0000 7f69fd79c700 1 --2- 172.21.15.27:0/1629439974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69f8108840 0x7f69f819bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:38.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.522+0000 7f69fd79c700 1 -- 172.21.15.27:0/1629439974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69f8102980 msgr2=0x7f69f81a1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:31:38.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.522+0000 7f69fd79c700 1 --2- 172.21.15.27:0/1629439974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69f8102980 0x7f69f81a1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:38.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.522+0000 7f69fd79c700 1 -- 172.21.15.27:0/1629439974 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f819c410 con 0x7f69f8101fd0 2024-05-21T03:31:38.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.523+0000 7f69fcf9b700 1 --2- 172.21.15.27:0/1629439974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69f8102980 0x7f69f81a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:31:38.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.523+0000 7f69fd79c700 1 --2- 172.21.15.27:0/1629439974 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69f8101fd0 0x7f69f81a1a60 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f69f400d950 tx=0x7f69f400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:31:38.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.523+0000 7f69ee7fc700 1 -- 172.21.15.27:0/1629439974 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f4016070 con 0x7f69f8101fd0 2024-05-21T03:31:38.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.523+0000 7f69ee7fc700 1 -- 172.21.15.27:0/1629439974 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69f4010460 con 0x7f69f8101fd0 2024-05-21T03:31:38.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.523+0000 7f69fe79e700 1 -- 172.21.15.27:0/1629439974 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f819c700 con 0x7f69f8101fd0 2024-05-21T03:31:38.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.523+0000 7f69ee7fc700 1 -- 172.21.15.27:0/1629439974 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f400f620 con 0x7f69f8101fd0 2024-05-21T03:31:38.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.523+0000 7f69fe79e700 1 -- 172.21.15.27:0/1629439974 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f81a89e0 con 0x7f69f8101fd0 2024-05-21T03:31:38.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.524+0000 7f69fe79e700 1 -- 172.21.15.27:0/1629439974 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69f806b640 con 0x7f69f8101fd0 2024-05-21T03:31:38.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.525+0000 7f69ee7fc700 1 -- 172.21.15.27:0/1629439974 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7f69f4019070 con 0x7f69f8101fd0 2024-05-21T03:31:38.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.526+0000 7f69ee7fc700 1 --2- 172.21.15.27:0/1629439974 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f69e40b29f0 0x7f69e40b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:38.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.527+0000 7f69fcf9b700 1 --2- 172.21.15.27:0/1629439974 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f69e40b29f0 0x7f69e40b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:38.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.527+0000 7f69ee7fc700 1 -- 172.21.15.27:0/1629439974 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8571+0+0 (secure 0 0 0) 0x7f69f40e22e0 con 0x7f69f8101fd0 2024-05-21T03:31:38.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.527+0000 7f69fcf9b700 1 --2- 172.21.15.27:0/1629439974 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f69e40b29f0 0x7f69e40b4eb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f69e8005fd0 tx=0x7f69e8005ee0 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:31:38.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.531+0000 7f69ee7fc700 1 -- 172.21.15.27:0/1629439974 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f69f40a9c90 con 0x7f69f8101fd0 2024-05-21T03:31:38.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:38 smithi027 ceph-mon[32179]: pgmap v233: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 58 KiB/s rd, 22 MiB/s wr, 4.85k op/s 2024-05-21T03:31:38.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.802+0000 7f69fe79e700 1 -- 172.21.15.27:0/1629439974 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69f819cff0 con 0x7f69f8101fd0 2024-05-21T03:31:38.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.803+0000 7f69ee7fc700 1 -- 172.21.15.27:0/1629439974 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11931 (secure 0 0 0) 0x7f69f40a9590 con 0x7f69f8101fd0 2024-05-21T03:31:38.802 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:31:38.803 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/148854869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":148854869},{"type":"v1","addr":"172.21.15.27:6835","nonce":148854869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/4145544730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":4145544730},{"type":"v1","addr":"172.21.15.27:6837","nonce":4145544730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T03:20:34.055985+0000","modified":"2024-05-21T03:30:04.352448+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24472,"mds_2":24457,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/745486764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":745486764},{"type":"v1","addr":"172.21.15.27:6839","nonce":745486764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6841/3982752351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3982752351},{"type":"v1","addr":"172.21.15.27:6841","nonce":3982752351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/3497521608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3497521608},{"type":"v1","addr":"172.21.15.39:6835","nonce":3497521608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.165:6835/3576910218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3576910218},{"type":"v1","addr":"172.21.15.165:6835","nonce":3576910218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/3673846852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3673846852},{"type":"v1","addr":"172.21.15.39:6837","nonce":3673846852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6837/691356139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":691356139},{"type":"v1","addr":"172.21.15.165:6837","nonce":691356139}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6839/4158170814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4158170814},{"type":"v1","addr":"172.21.15.165:6839","nonce":4158170814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6839/287607819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":287607819},{"type":"v1","addr":"172.21.15.39:6839","nonce":287607819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24457,24476,24452,24472,24451]},"id":1}]} 2024-05-21T03:31:38.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.805+0000 7f69fe79e700 1 -- 172.21.15.27:0/1629439974 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f69e40b29f0 msgr2=0x7f69e40b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:31:38.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.805+0000 7f69fe79e700 1 --2- 172.21.15.27:0/1629439974 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f69e40b29f0 0x7f69e40b4eb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f69e8005fd0 tx=0x7f69e8005ee0 comp rx=0 tx=0).stop 2024-05-21T03:31:38.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.805+0000 7f69fe79e700 1 -- 172.21.15.27:0/1629439974 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69f8101fd0 msgr2=0x7f69f81a1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:31:38.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.805+0000 7f69fe79e700 1 --2- 172.21.15.27:0/1629439974 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69f8101fd0 0x7f69f81a1a60 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f69f400d950 tx=0x7f69f400dd10 comp rx=0 tx=0).stop 2024-05-21T03:31:38.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.805+0000 7f69fe79e700 1 -- 172.21.15.27:0/1629439974 shutdown_connections 2024-05-21T03:31:38.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.805+0000 7f69fe79e700 1 --2- 172.21.15.27:0/1629439974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69f8108840 0x7f69f819bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:38.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.805+0000 7f69fe79e700 1 --2- 172.21.15.27:0/1629439974 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69f8101fd0 0x7f69f81a1a60 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:38.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.805+0000 7f69fe79e700 1 --2- 172.21.15.27:0/1629439974 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f69e40b29f0 0x7f69e40b4eb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:38.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.805+0000 7f69fe79e700 1 --2- 172.21.15.27:0/1629439974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69f8102980 0x7f69f81a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:38.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.805+0000 7f69fe79e700 1 -- 172.21.15.27:0/1629439974 >> 172.21.15.27:0/1629439974 conn(0x7f69f80776f0 msgr2=0x7f69f810eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:31:38.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.805+0000 7f69fe79e700 1 -- 172.21.15.27:0/1629439974 shutdown_connections 2024-05-21T03:31:38.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:38.805+0000 7f69fe79e700 1 -- 172.21.15.27:0/1629439974 wait complete. 2024-05-21T03:31:38.806 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 39 2024-05-21T03:31:39.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:38 smithi039 ceph-mon[36429]: pgmap v233: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 58 KiB/s rd, 22 MiB/s wr, 4.85k op/s 2024-05-21T03:31:39.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:38 smithi165 ceph-mon[36269]: pgmap v233: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 58 KiB/s rd, 22 MiB/s wr, 4.85k op/s 2024-05-21T03:31:39.379 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id d4642da5 --release --await' 2024-05-21T03:31:39.379 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id d4642da5 --release --await 2024-05-21T03:31:39.632 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:31:39.676 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:39 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/1629439974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:31:40.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:39 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/1629439974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:31:40.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:39 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/1629439974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:31:40.860 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:40 smithi027 ceph-mon[32179]: pgmap v234: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 48 KiB/s rd, 12 MiB/s wr, 2.75k op/s 2024-05-21T03:31:41.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:40 smithi039 ceph-mon[36429]: pgmap v234: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 48 KiB/s rd, 12 MiB/s wr, 2.75k op/s 2024-05-21T03:31:41.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:40 smithi165 ceph-mon[36269]: pgmap v234: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 48 KiB/s rd, 12 MiB/s wr, 2.75k op/s 2024-05-21T03:31:41.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.262+0000 7f1bc9238700 1 -- 172.21.15.27:0/489969158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1bc406bd10 msgr2=0x7f1bc406c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:31:41.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.262+0000 7f1bc9238700 1 --2- 172.21.15.27:0/489969158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1bc406bd10 0x7f1bc406c190 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1bac0099c0 tx=0x7f1bac009cd0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.262+0000 7f1bc9238700 1 -- 172.21.15.27:0/489969158 shutdown_connections 2024-05-21T03:31:41.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.262+0000 7f1bc9238700 1 --2- 172.21.15.27:0/489969158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1bc406c6d0 0x7f1bc4110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.262+0000 7f1bc9238700 1 --2- 172.21.15.27:0/489969158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1bc4109630 0x7f1bc4109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.262+0000 7f1bc9238700 1 --2- 172.21.15.27:0/489969158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1bc406bd10 0x7f1bc406c190 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.262+0000 7f1bc9238700 1 -- 172.21.15.27:0/489969158 >> 172.21.15.27:0/489969158 conn(0x7f1bc40794e0 msgr2=0x7f1bc40798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:31:41.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.262+0000 7f1bc9238700 1 -- 172.21.15.27:0/489969158 shutdown_connections 2024-05-21T03:31:41.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.262+0000 7f1bc9238700 1 -- 172.21.15.27:0/489969158 wait complete. 2024-05-21T03:31:41.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.263+0000 7f1bc9238700 1 Processor -- start 2024-05-21T03:31:41.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.263+0000 7f1bc9238700 1 -- start start 2024-05-21T03:31:41.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.263+0000 7f1bc9238700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1bc406bd10 0x7f1bc419f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:41.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.263+0000 7f1bc9238700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1bc406c6d0 0x7f1bc419fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:41.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.264+0000 7f1bc9238700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1bc4109630 0x7f1bc4199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:41.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.264+0000 7f1bc9238700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc41137b0 con 0x7f1bc406bd10 2024-05-21T03:31:41.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.264+0000 7f1bc9238700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc4113630 con 0x7f1bc406c6d0 2024-05-21T03:31:41.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.264+0000 7f1bc9238700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc4113930 con 0x7f1bc4109630 2024-05-21T03:31:41.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.264+0000 7f1bc2d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1bc406bd10 0x7f1bc419f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:41.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.264+0000 7f1bc2d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1bc406bd10 0x7f1bc419f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58680/0 (socket says 172.21.15.27:58680) 2024-05-21T03:31:41.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.264+0000 7f1bc2d9d700 1 -- 172.21.15.27:0/396340867 learned_addr learned my addr 172.21.15.27:0/396340867 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:31:41.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.264+0000 7f1bc359e700 1 --2- 172.21.15.27:0/396340867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1bc4109630 0x7f1bc4199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:41.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.264+0000 7f1bc259c700 1 --2- 172.21.15.27:0/396340867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1bc406c6d0 0x7f1bc419fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:41.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.264+0000 7f1bc2d9d700 1 -- 172.21.15.27:0/396340867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1bc4109630 msgr2=0x7f1bc4199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:31:41.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.264+0000 7f1bc2d9d700 1 --2- 172.21.15.27:0/396340867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1bc4109630 0x7f1bc4199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.264+0000 7f1bc2d9d700 1 -- 172.21.15.27:0/396340867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1bc406c6d0 msgr2=0x7f1bc419fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:31:41.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.264+0000 7f1bc2d9d700 1 --2- 172.21.15.27:0/396340867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1bc406c6d0 0x7f1bc419fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.264+0000 7f1bc2d9d700 1 -- 172.21.15.27:0/396340867 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc419a190 con 0x7f1bc406bd10 2024-05-21T03:31:41.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.264+0000 7f1bc2d9d700 1 --2- 172.21.15.27:0/396340867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1bc406bd10 0x7f1bc419f870 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4007f10 tx=0x7f1bb400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:31:41.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.264+0000 7f1bbbfff700 1 -- 172.21.15.27:0/396340867 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb4004bb0 con 0x7f1bc406bd10 2024-05-21T03:31:41.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.264+0000 7f1bbbfff700 1 -- 172.21.15.27:0/396340867 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bb4004d50 con 0x7f1bc406bd10 2024-05-21T03:31:41.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.264+0000 7f1bc9238700 1 -- 172.21.15.27:0/396340867 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bc419a480 con 0x7f1bc406bd10 2024-05-21T03:31:41.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.264+0000 7f1bbbfff700 1 -- 172.21.15.27:0/396340867 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb40059b0 con 0x7f1bc406bd10 2024-05-21T03:31:41.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.265+0000 7f1bc9238700 1 -- 172.21.15.27:0/396340867 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bc41a6740 con 0x7f1bc406bd10 2024-05-21T03:31:41.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.267+0000 7f1bbbfff700 1 -- 172.21.15.27:0/396340867 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7f1bb401a020 con 0x7f1bc406bd10 2024-05-21T03:31:41.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.268+0000 7f1bbbfff700 1 --2- 172.21.15.27:0/396340867 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f1bb00b29f0 0x7f1bb00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:41.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.268+0000 7f1bc259c700 1 --2- 172.21.15.27:0/396340867 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f1bb00b29f0 0x7f1bb00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:41.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.268+0000 7f1bbbfff700 1 -- 172.21.15.27:0/396340867 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8571+0+0 (secure 0 0 0) 0x7f1bb40e1d80 con 0x7f1bc406bd10 2024-05-21T03:31:41.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.268+0000 7f1bc259c700 1 --2- 172.21.15.27:0/396340867 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f1bb00b29f0 0x7f1bb00b4eb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1bac009990 tx=0x7f1bac005a90 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:31:41.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.268+0000 7f1bd3aceb80 1 Processor -- start 2024-05-21T03:31:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.268+0000 7f1bd3aceb80 1 -- start start 2024-05-21T03:31:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.269+0000 7f1bd3aceb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641356792b0 0x564135679690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.269+0000 7f1bd3aceb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641356738e0 0x564135673d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.269+0000 7f1bd3aceb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641356742a0 0x5641356800e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.269+0000 7f1bd3aceb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5641355d2d90 con 0x5641356742a0 2024-05-21T03:31:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.269+0000 7f1bd3aceb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5641355c2930 con 0x5641356792b0 2024-05-21T03:31:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.269+0000 7f1bd3aceb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5641355c6970 con 0x5641356738e0 2024-05-21T03:31:41.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.269+0000 7f1bc359e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641356742a0 0x5641356800e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:41.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.269+0000 7f1bc359e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641356742a0 0x5641356800e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58690/0 (socket says 172.21.15.27:58690) 2024-05-21T03:31:41.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.269+0000 7f1bc359e700 1 -- 172.21.15.27:0/4099199208 learned_addr learned my addr 172.21.15.27:0/4099199208 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:31:41.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.269+0000 7f1bc2d9d700 1 --2- 172.21.15.27:0/4099199208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641356792b0 0x564135679690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:41.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.269+0000 7f1bc259c700 1 --2- 172.21.15.27:0/4099199208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641356738e0 0x564135673d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:41.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.269+0000 7f1bc359e700 1 -- 172.21.15.27:0/4099199208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641356738e0 msgr2=0x564135673d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:31:41.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.269+0000 7f1bc359e700 1 --2- 172.21.15.27:0/4099199208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641356738e0 0x564135673d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.269+0000 7f1bc359e700 1 -- 172.21.15.27:0/4099199208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641356792b0 msgr2=0x564135679690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:31:41.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.269+0000 7f1bc359e700 1 --2- 172.21.15.27:0/4099199208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641356792b0 0x564135679690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.269+0000 7f1bc359e700 1 -- 172.21.15.27:0/4099199208 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641355129d0 con 0x5641356742a0 2024-05-21T03:31:41.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.269+0000 7f1bc2d9d700 1 --2- 172.21.15.27:0/4099199208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641356792b0 0x564135679690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:31:41.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.270+0000 7f1bc359e700 1 --2- 172.21.15.27:0/4099199208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641356742a0 0x5641356800e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc009a70 tx=0x7f1bbc009d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dea5b4af2771b33 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:31:41.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.270+0000 7f1bb97fa700 1 -- 172.21.15.27:0/4099199208 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc017070 con 0x5641356742a0 2024-05-21T03:31:41.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.270+0000 7f1bb97fa700 1 -- 172.21.15.27:0/4099199208 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bbc00ed20 con 0x5641356742a0 2024-05-21T03:31:41.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.270+0000 7f1bb97fa700 1 -- 172.21.15.27:0/4099199208 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc013db0 con 0x5641356742a0 2024-05-21T03:31:41.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.270+0000 7f1bd3aceb80 1 -- 172.21.15.27:0/4099199208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641356742a0 msgr2=0x5641356800e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:31:41.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.270+0000 7f1bd3aceb80 1 --2- 172.21.15.27:0/4099199208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641356742a0 0x5641356800e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc009a70 tx=0x7f1bbc009d80 comp rx=0 tx=0).stop 2024-05-21T03:31:41.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.271+0000 7f1bd3aceb80 1 -- 172.21.15.27:0/4099199208 shutdown_connections 2024-05-21T03:31:41.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.271+0000 7f1bd3aceb80 1 --2- 172.21.15.27:0/4099199208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641356742a0 0x5641356800e0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.271+0000 7f1bd3aceb80 1 --2- 172.21.15.27:0/4099199208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641356792b0 0x564135679690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.271+0000 7f1bd3aceb80 1 --2- 172.21.15.27:0/4099199208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641356738e0 0x564135673d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.271+0000 7f1bd3aceb80 1 -- 172.21.15.27:0/4099199208 >> 172.21.15.27:0/4099199208 conn(0x5641355a1180 msgr2=0x56413566e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:31:41.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.271+0000 7f1bd3aceb80 1 -- 172.21.15.27:0/4099199208 shutdown_connections 2024-05-21T03:31:41.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.271+0000 7f1bd3aceb80 1 -- 172.21.15.27:0/4099199208 wait complete. 2024-05-21T03:31:41.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.271+0000 7f1bd3aceb80 1 -- start start 2024-05-21T03:31:41.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.271+0000 7f1bd3aceb80 1 Processor -- start 2024-05-21T03:31:41.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.271+0000 7f1bd3aceb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641356738e0 0x56413567fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:41.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.271+0000 7f1bd3aceb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641356742a0 0x56413567d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:41.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.271+0000 7f1bd3aceb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641356792b0 0x56413567d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:41.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.271+0000 7f1bd3aceb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5641354c90a0 con 0x5641356792b0 2024-05-21T03:31:41.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.271+0000 7f1bd3aceb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5641355648c0 con 0x5641356742a0 2024-05-21T03:31:41.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.271+0000 7f1bd3aceb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5641356806e0 con 0x5641356738e0 2024-05-21T03:31:41.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.272+0000 7f1bd3aceb80 10 client.0 _ll_register_callbacks cb 0x56413555da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:31:41.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.272+0000 7f1bc359e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641356792b0 0x56413567d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:41.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.272+0000 7f1bc359e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641356792b0 0x56413567d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58702/0 (socket says 172.21.15.27:58702) 2024-05-21T03:31:41.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.272+0000 7f1bc359e700 1 -- 172.21.15.27:0/2604424948 learned_addr learned my addr 172.21.15.27:0/2604424948 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:31:41.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.272+0000 7f1bc259c700 1 --2- 172.21.15.27:0/2604424948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641356742a0 0x56413567d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:41.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.272+0000 7f1bc2d9d700 1 --2- 172.21.15.27:0/2604424948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641356738e0 0x56413567fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:41.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.272+0000 7f1bc359e700 1 -- 172.21.15.27:0/2604424948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641356738e0 msgr2=0x56413567fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:31:41.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.272+0000 7f1bc359e700 1 --2- 172.21.15.27:0/2604424948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641356738e0 0x56413567fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.272+0000 7f1bc359e700 1 -- 172.21.15.27:0/2604424948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641356742a0 msgr2=0x56413567d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:31:41.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.272+0000 7f1bc359e700 1 --2- 172.21.15.27:0/2604424948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641356742a0 0x56413567d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.272+0000 7f1bc359e700 1 -- 172.21.15.27:0/2604424948 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564135561790 con 0x5641356792b0 2024-05-21T03:31:41.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.272+0000 7f1bc359e700 1 --2- 172.21.15.27:0/2604424948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641356792b0 0x56413567d590 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0048a0 tx=0x7f1bbc006e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b456436648e19b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:31:41.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.272+0000 7f1baa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:31:41.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.272+0000 7f1baa7fc700 1 -- 172.21.15.27:0/2604424948 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc017040 con 0x5641356792b0 2024-05-21T03:31:41.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.272+0000 7f1baa7fc700 1 -- 172.21.15.27:0/2604424948 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8571+0+0 (secure 0 0 0) 0x7f1bbc01c450 con 0x5641356792b0 2024-05-21T03:31:41.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.272+0000 7f1bd3aceb80 1 -- 172.21.15.27:0/2604424948 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bbc00ed20 con 0x5641356792b0 2024-05-21T03:31:41.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.273+0000 7f1baa7fc700 1 -- 172.21.15.27:0/2604424948 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1bbc01c650 con 0x5641356792b0 2024-05-21T03:31:41.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.273+0000 7f1bd3aceb80 10 client.15834 fetch_fsmap learned FSMap version 39 2024-05-21T03:31:41.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.273+0000 7f1bd3aceb80 1 -- 172.21.15.27:0/2604424948 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56413568adc0 con 0x5641356792b0 2024-05-21T03:31:41.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.273+0000 7f1baa7fc700 1 -- 172.21.15.27:0/2604424948 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bbc00ed20 con 0x5641356792b0 2024-05-21T03:31:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.273+0000 7f1baa7fc700 1 -- 172.21.15.27:0/2604424948 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f1bbc023070 con 0x5641356792b0 2024-05-21T03:31:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.273+0000 7f1bd3aceb80 10 client.15834 fetch_fsmap finished waiting for FSMap version 39 2024-05-21T03:31:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.273+0000 7f1bd3aceb80 10 client.15834 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-21T03:31:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.273+0000 7f1bd3aceb80 20 client.15834 populate_metadata read hostname 'smithi027' 2024-05-21T03:31:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.273+0000 7f1bd3aceb80 1 --2- 172.21.15.27:0/2604424948 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x56413568d290 0x56413568f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.273+0000 7f1bd3aceb80 4 client.15834 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:31:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.273+0000 7f1bd3aceb80 1 -- 172.21.15.27:0/2604424948 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56413563fae0 con 0x56413568d290 2024-05-21T03:31:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.273+0000 7f1bc2d9d700 1 --2- 172.21.15.27:0/2604424948 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x56413568d290 0x56413568f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.274+0000 7f1baa7fc700 10 client.15834 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:31:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.274+0000 7f1bc2d9d700 1 --2- 172.21.15.27:0/2604424948 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x56413568d290 0x56413568f680 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9d2a933237d6140 server_cookie=a9ef267c7fd590fd in_seq=0 out_seq=0 2024-05-21T03:31:41.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.274+0000 7f1baa7fc700 1 -- 172.21.15.27:0/2604424948 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56413563fae0 con 0x56413568d290 2024-05-21T03:31:41.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.274+0000 7f1baa7fc700 10 client.15834 handle_command_reply: tid=0 2024-05-21T03:31:41.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.275+0000 7f1bd3aceb80 1 client.15834 shutdown 2024-05-21T03:31:41.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.275+0000 7f1bd3aceb80 1 -- 172.21.15.27:0/2604424948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641356792b0 msgr2=0x56413567d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:31:41.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.275+0000 7f1bd3aceb80 1 --2- 172.21.15.27:0/2604424948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641356792b0 0x56413567d590 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0048a0 tx=0x7f1bbc006e70 comp rx=0 tx=0).stop 2024-05-21T03:31:41.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.275+0000 7f1bd3aceb80 1 -- 172.21.15.27:0/2604424948 shutdown_connections 2024-05-21T03:31:41.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.275+0000 7f1bd3aceb80 1 --2- 172.21.15.27:0/2604424948 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x56413568d290 0x56413568f680 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.275+0000 7f1bd3aceb80 1 --2- 172.21.15.27:0/2604424948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641356792b0 0x56413567d590 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.275+0000 7f1bd3aceb80 1 --2- 172.21.15.27:0/2604424948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641356738e0 0x56413567fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.275+0000 7f1baa7fc700 0 client.15834 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:31:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.275+0000 7f1bd3aceb80 1 --2- 172.21.15.27:0/2604424948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641356742a0 0x56413567d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.275+0000 7f1bd3aceb80 1 -- 172.21.15.27:0/2604424948 >> 172.21.15.27:0/2604424948 conn(0x5641355a1180 msgr2=0x564135668ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:31:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.275+0000 7f1bd3aceb80 1 -- 172.21.15.27:0/2604424948 shutdown_connections 2024-05-21T03:31:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.275+0000 7f1bd3aceb80 1 -- 172.21.15.27:0/2604424948 wait complete. 2024-05-21T03:31:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.275+0000 7f1bd3aceb80 20 client.15834 trim_cache size 0 max 16384 2024-05-21T03:31:41.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.295+0000 7f1bd3aceb80 1 Processor -- start 2024-05-21T03:31:41.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.295+0000 7f1bd3aceb80 1 -- start start 2024-05-21T03:31:41.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.296+0000 7f1bd3aceb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641356738e0 0x56413567f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:41.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.296+0000 7f1bd3aceb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641356742a0 0x564135694e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:41.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.296+0000 7f1bd3aceb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641356792b0 0x564135695350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:41.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.296+0000 7f1bd3aceb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc41adf50 con 0x5641356738e0 2024-05-21T03:31:41.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.296+0000 7f1bd3aceb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc41afb50 con 0x5641356742a0 2024-05-21T03:31:41.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.296+0000 7f1bd3aceb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc41ae620 con 0x5641356792b0 2024-05-21T03:31:41.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.296+0000 7f1bc2d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641356738e0 0x56413567f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:41.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.296+0000 7f1bc2d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641356738e0 0x56413567f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58710/0 (socket says 172.21.15.27:58710) 2024-05-21T03:31:41.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.296+0000 7f1bc2d9d700 1 -- 172.21.15.27:0/1454029845 learned_addr learned my addr 172.21.15.27:0/1454029845 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:31:41.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.296+0000 7f1bc2d9d700 1 -- 172.21.15.27:0/1454029845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641356792b0 msgr2=0x564135695350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:31:41.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.296+0000 7f1bc259c700 1 --2- 172.21.15.27:0/1454029845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641356742a0 0x564135694e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:41.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.296+0000 7f1bc2d9d700 1 --2- 172.21.15.27:0/1454029845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641356792b0 0x564135695350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.296+0000 7f1bc2d9d700 1 -- 172.21.15.27:0/1454029845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641356742a0 msgr2=0x564135694e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:31:41.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.296+0000 7f1bc2d9d700 1 --2- 172.21.15.27:0/1454029845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641356742a0 0x564135694e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.296+0000 7f1bc2d9d700 1 -- 172.21.15.27:0/1454029845 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56413567b5c0 con 0x5641356738e0 2024-05-21T03:31:41.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.296+0000 7f1bc2d9d700 1 --2- 172.21.15.27:0/1454029845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641356738e0 0x56413567f890 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1bb40a8d30 tx=0x7f1bb40a4f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142ec799c5a9d6df server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:31:41.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.296+0000 7f1bb8ff9700 1 -- 172.21.15.27:0/1454029845 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb406c020 con 0x5641356738e0 2024-05-21T03:31:41.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.297+0000 7f1bb8ff9700 1 -- 172.21.15.27:0/1454029845 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bb40ad3e0 con 0x5641356738e0 2024-05-21T03:31:41.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.297+0000 7f1bb8ff9700 1 -- 172.21.15.27:0/1454029845 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb40a94f0 con 0x5641356738e0 2024-05-21T03:31:41.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.297+0000 7f1bd3aceb80 1 -- 172.21.15.27:0/1454029845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641356738e0 msgr2=0x56413567f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:31:41.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.297+0000 7f1bd3aceb80 1 --2- 172.21.15.27:0/1454029845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641356738e0 0x56413567f890 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1bb40a8d30 tx=0x7f1bb40a4f70 comp rx=0 tx=0).stop 2024-05-21T03:31:41.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.297+0000 7f1bd3aceb80 1 -- 172.21.15.27:0/1454029845 shutdown_connections 2024-05-21T03:31:41.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.297+0000 7f1bd3aceb80 1 --2- 172.21.15.27:0/1454029845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641356792b0 0x564135695350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.297+0000 7f1bd3aceb80 1 --2- 172.21.15.27:0/1454029845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641356738e0 0x56413567f890 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.297+0000 7f1bd3aceb80 1 --2- 172.21.15.27:0/1454029845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641356742a0 0x564135694e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.297+0000 7f1bd3aceb80 1 -- 172.21.15.27:0/1454029845 >> 172.21.15.27:0/1454029845 conn(0x5641355a1180 msgr2=0x564135673160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:31:41.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.297+0000 7f1bd3aceb80 1 -- 172.21.15.27:0/1454029845 shutdown_connections 2024-05-21T03:31:41.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.297+0000 7f1bd3aceb80 1 -- 172.21.15.27:0/1454029845 wait complete. 2024-05-21T03:31:41.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.298+0000 7f1bd3aceb80 1 -- start start 2024-05-21T03:31:41.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.298+0000 7f1bd3aceb80 1 Processor -- start 2024-05-21T03:31:41.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.298+0000 7f1bd3aceb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641356738e0 0x564135682d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:41.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.298+0000 7f1bd3aceb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641356742a0 0x564135683260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:41.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.298+0000 7f1bd3aceb80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641356792b0 0x5641356f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:41.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.298+0000 7f1bd3aceb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc41af480 con 0x5641356742a0 2024-05-21T03:31:41.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.298+0000 7f1bd3aceb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc41aedb0 con 0x5641356738e0 2024-05-21T03:31:41.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.298+0000 7f1bd3aceb80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc41abf10 con 0x5641356792b0 2024-05-21T03:31:41.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.298+0000 7f1bd3aceb80 10 client.0 _ll_register_callbacks cb 0x56413555da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:31:41.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.298+0000 7f1bc259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641356742a0 0x564135683260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:41.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.298+0000 7f1bc259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641356742a0 0x564135683260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58712/0 (socket says 172.21.15.27:58712) 2024-05-21T03:31:41.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.298+0000 7f1bc259c700 1 -- 172.21.15.27:0/2005631011 learned_addr learned my addr 172.21.15.27:0/2005631011 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:31:41.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.298+0000 7f1bc359e700 1 --2- 172.21.15.27:0/2005631011 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641356792b0 0x5641356f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:41.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.298+0000 7f1bc2d9d700 1 --2- 172.21.15.27:0/2005631011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641356738e0 0x564135682d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:41.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.298+0000 7f1bc259c700 1 -- 172.21.15.27:0/2005631011 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641356792b0 msgr2=0x5641356f7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:31:41.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.298+0000 7f1bc259c700 1 --2- 172.21.15.27:0/2005631011 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641356792b0 0x5641356f7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.298+0000 7f1bc259c700 1 -- 172.21.15.27:0/2005631011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641356738e0 msgr2=0x564135682d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:31:41.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.298+0000 7f1bc259c700 1 --2- 172.21.15.27:0/2005631011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641356738e0 0x564135682d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.298+0000 7f1bc259c700 1 -- 172.21.15.27:0/2005631011 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641356f8230 con 0x5641356742a0 2024-05-21T03:31:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.299+0000 7f1bc259c700 1 --2- 172.21.15.27:0/2005631011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641356742a0 0x564135683260 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1bac0098e0 tx=0x7f1bac009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e23cc6587f2aea6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:31:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.299+0000 7f1baa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:31:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.299+0000 7f1baa7fc700 1 -- 172.21.15.27:0/2005631011 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac036060 con 0x5641356742a0 2024-05-21T03:31:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.299+0000 7f1baa7fc700 1 -- 172.21.15.27:0/2005631011 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8571+0+0 (secure 0 0 0) 0x7f1bac031670 con 0x5641356742a0 2024-05-21T03:31:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.299+0000 7f1bd3aceb80 1 -- 172.21.15.27:0/2005631011 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bb40ad3e0 con 0x5641356742a0 2024-05-21T03:31:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.299+0000 7f1baa7fc700 1 -- 172.21.15.27:0/2005631011 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1bac031870 con 0x5641356742a0 2024-05-21T03:31:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.299+0000 7f1bd3aceb80 10 client.15840 fetch_fsmap learned FSMap version 39 2024-05-21T03:31:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.299+0000 7f1bd3aceb80 1 -- 172.21.15.27:0/2005631011 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5641356f94e0 con 0x5641356742a0 2024-05-21T03:31:41.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.299+0000 7f1baa7fc700 1 -- 172.21.15.27:0/2005631011 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bb40ad3e0 con 0x5641356742a0 2024-05-21T03:31:41.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.299+0000 7f1baa7fc700 1 -- 172.21.15.27:0/2005631011 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f1bac00f960 con 0x5641356742a0 2024-05-21T03:31:41.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.299+0000 7f1bd3aceb80 10 client.15840 fetch_fsmap finished waiting for FSMap version 39 2024-05-21T03:31:41.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.299+0000 7f1bd3aceb80 10 client.15840 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-21T03:31:41.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.299+0000 7f1bd3aceb80 20 client.15840 populate_metadata read hostname 'smithi027' 2024-05-21T03:31:41.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.299+0000 7f1bd3aceb80 1 --2- 172.21.15.27:0/2005631011 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x5641356f9b90 0x5641356fbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:31:41.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.299+0000 7f1bd3aceb80 4 client.15840 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d4642da5", "release": true, "await": true}] 2024-05-21T03:31:41.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.300+0000 7f1bd3aceb80 1 -- 172.21.15.27:0/2005631011 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d4642da5", "release": true, "await": true}) -- 0x5641355d4ce0 con 0x5641356f9b90 2024-05-21T03:31:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.300+0000 7f1bc2d9d700 1 --2- 172.21.15.27:0/2005631011 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x5641356f9b90 0x5641356fbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:31:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.300+0000 7f1bc2d9d700 1 --2- 172.21.15.27:0/2005631011 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x5641356f9b90 0x5641356fbf80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8748bc7753a4890 server_cookie=14a4da34d3508926 in_seq=0 out_seq=0 2024-05-21T03:31:41.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.300+0000 7f1baa7fc700 10 client.15840 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:31:41.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.314+0000 7f1baa7fc700 1 -- 172.21.15.27:0/2005631011 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5641355d4ce0 con 0x5641356f9b90 2024-05-21T03:31:41.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.314+0000 7f1baa7fc700 10 client.15840 handle_command_reply: tid=0 2024-05-21T03:31:41.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.314+0000 7f1bd3aceb80 1 client.15840 shutdown 2024-05-21T03:31:41.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.314+0000 7f1bd3aceb80 1 -- 172.21.15.27:0/2005631011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641356742a0 msgr2=0x564135683260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:31:41.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.314+0000 7f1bd3aceb80 1 --2- 172.21.15.27:0/2005631011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641356742a0 0x564135683260 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1bac0098e0 tx=0x7f1bac009cd0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.314+0000 7f1bd3aceb80 1 -- 172.21.15.27:0/2005631011 shutdown_connections 2024-05-21T03:31:41.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.314+0000 7f1bd3aceb80 1 --2- 172.21.15.27:0/2005631011 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x5641356f9b90 0x5641356fbf80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.314+0000 7f1bd3aceb80 1 --2- 172.21.15.27:0/2005631011 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641356792b0 0x5641356f7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.314+0000 7f1bd3aceb80 1 --2- 172.21.15.27:0/2005631011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5641356738e0 0x564135682d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.314+0000 7f1bd3aceb80 1 --2- 172.21.15.27:0/2005631011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5641356742a0 0x564135683260 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.314+0000 7f1bd3aceb80 1 -- 172.21.15.27:0/2005631011 >> 172.21.15.27:0/2005631011 conn(0x5641355a1180 msgr2=0x564135668410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:31:41.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.314+0000 7f1baa7fc700 0 client.15840 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:31:41.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.314+0000 7f1bd3aceb80 1 -- 172.21.15.27:0/2005631011 shutdown_connections 2024-05-21T03:31:41.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.314+0000 7f1bd3aceb80 1 -- 172.21.15.27:0/2005631011 wait complete. 2024-05-21T03:31:41.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.314+0000 7f1bd3aceb80 20 client.15840 trim_cache size 0 max 16384 2024-05-21T03:31:41.315 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:31:41.316 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 39, 2024-05-21T03:31:41.316 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24476, 2024-05-21T03:31:41.316 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 21, 2024-05-21T03:31:41.316 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-05-21T03:31:41.316 INFO:teuthology.orchestra.run.smithi027.stdout: "d4642da5": { 2024-05-21T03:31:41.316 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 21, 2024-05-21T03:31:41.316 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-05-21T03:31:41.316 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-05-21T03:31:41.316 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-05-21T03:31:41.316 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-05-21T03:31:41.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:31:41.316 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-05-21T03:31:41.316 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 147.3, 2024-05-21T03:31:41.317 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-05-21T03:31:41.317 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-05-21T03:31:41.317 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-05-21T03:31:41.317 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-05-21T03:31:41.317 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-05-21T03:31:41.317 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-05-21T03:31:41.317 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:31:41.317 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:31:41.317 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:31:41.317 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:31:41.317 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:31:41.317 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:31:41.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.315+0000 7f1bc9238700 1 -- 172.21.15.27:0/396340867 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f1bb00b29f0 msgr2=0x7f1bb00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:31:41.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.315+0000 7f1bc9238700 1 --2- 172.21.15.27:0/396340867 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f1bb00b29f0 0x7f1bb00b4eb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1bac009990 tx=0x7f1bac005a90 comp rx=0 tx=0).stop 2024-05-21T03:31:41.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.315+0000 7f1bc9238700 1 -- 172.21.15.27:0/396340867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1bc406bd10 msgr2=0x7f1bc419f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:31:41.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.315+0000 7f1bc9238700 1 --2- 172.21.15.27:0/396340867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1bc406bd10 0x7f1bc419f870 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4007f10 tx=0x7f1bb400d3b0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.315+0000 7f1bc9238700 1 -- 172.21.15.27:0/396340867 shutdown_connections 2024-05-21T03:31:41.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.315+0000 7f1bc9238700 1 --2- 172.21.15.27:0/396340867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1bc4109630 0x7f1bc4199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.315+0000 7f1bc9238700 1 --2- 172.21.15.27:0/396340867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1bc406bd10 0x7f1bc419f870 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.315+0000 7f1bc9238700 1 --2- 172.21.15.27:0/396340867 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f1bb00b29f0 0x7f1bb00b4eb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.315+0000 7f1bc9238700 1 --2- 172.21.15.27:0/396340867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1bc406c6d0 0x7f1bc419fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:31:41.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.315+0000 7f1bc9238700 1 -- 172.21.15.27:0/396340867 >> 172.21.15.27:0/396340867 conn(0x7f1bc40794e0 msgr2=0x7f1bc4100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:31:41.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.315+0000 7f1bc9238700 1 -- 172.21.15.27:0/396340867 shutdown_connections 2024-05-21T03:31:41.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:31:41.315+0000 7f1bc9238700 1 -- 172.21.15.27:0/396340867 wait complete. 2024-05-21T03:31:42.097 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd4642da5', total seconds elapsed: 32.9 2024-05-21T03:31:42.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:42 smithi027 ceph-mon[32179]: pgmap v235: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 39 KiB/s rd, 9.3 MiB/s wr, 2.16k op/s 2024-05-21T03:31:43.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:42 smithi039 ceph-mon[36429]: pgmap v235: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 39 KiB/s rd, 9.3 MiB/s wr, 2.16k op/s 2024-05-21T03:31:43.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:42 smithi165 ceph-mon[36269]: pgmap v235: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 39 KiB/s rd, 9.3 MiB/s wr, 2.16k op/s 2024-05-21T03:31:44.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:44 smithi027 ceph-mon[32179]: pgmap v236: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-05-21T03:31:45.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:44 smithi039 ceph-mon[36429]: pgmap v236: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-05-21T03:31:45.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:44 smithi165 ceph-mon[36269]: pgmap v236: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-05-21T03:31:46.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:46 smithi027 ceph-mon[32179]: pgmap v237: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 19 KiB/s rd, 874 KiB/s wr, 229 op/s 2024-05-21T03:31:47.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:46 smithi039 ceph-mon[36429]: pgmap v237: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 19 KiB/s rd, 874 KiB/s wr, 229 op/s 2024-05-21T03:31:47.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:46 smithi165 ceph-mon[36269]: pgmap v237: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 19 KiB/s rd, 874 KiB/s wr, 229 op/s 2024-05-21T03:31:48.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:48 smithi027 ceph-mon[32179]: pgmap v238: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 15 KiB/s rd, 1.4 MiB/s wr, 357 op/s 2024-05-21T03:31:49.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:48 smithi039 ceph-mon[36429]: pgmap v238: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 15 KiB/s rd, 1.4 MiB/s wr, 357 op/s 2024-05-21T03:31:49.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:48 smithi165 ceph-mon[36269]: pgmap v238: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 15 KiB/s rd, 1.4 MiB/s wr, 357 op/s 2024-05-21T03:31:50.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:50 smithi027 ceph-mon[32179]: pgmap v239: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 11 KiB/s rd, 1.7 MiB/s wr, 450 op/s 2024-05-21T03:31:51.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:50 smithi039 ceph-mon[36429]: pgmap v239: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 11 KiB/s rd, 1.7 MiB/s wr, 450 op/s 2024-05-21T03:31:51.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:50 smithi165 ceph-mon[36269]: pgmap v239: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 11 KiB/s rd, 1.7 MiB/s wr, 450 op/s 2024-05-21T03:31:52.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:52 smithi027 ceph-mon[32179]: pgmap v240: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 11 KiB/s rd, 2.6 MiB/s wr, 665 op/s 2024-05-21T03:31:53.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:52 smithi039 ceph-mon[36429]: pgmap v240: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 11 KiB/s rd, 2.6 MiB/s wr, 665 op/s 2024-05-21T03:31:53.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:52 smithi165 ceph-mon[36269]: pgmap v240: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 11 KiB/s rd, 2.6 MiB/s wr, 665 op/s 2024-05-21T03:31:54.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:54 smithi027 ceph-mon[32179]: pgmap v241: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 8.4 KiB/s rd, 2.6 MiB/s wr, 662 op/s 2024-05-21T03:31:55.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:54 smithi039 ceph-mon[36429]: pgmap v241: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 8.4 KiB/s rd, 2.6 MiB/s wr, 662 op/s 2024-05-21T03:31:55.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:54 smithi165 ceph-mon[36269]: pgmap v241: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 8.4 KiB/s rd, 2.6 MiB/s wr, 662 op/s 2024-05-21T03:31:56.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:56 smithi027 ceph-mon[32179]: pgmap v242: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 10 KiB/s rd, 3.3 MiB/s wr, 860 op/s 2024-05-21T03:31:57.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:56 smithi039 ceph-mon[36429]: pgmap v242: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 10 KiB/s rd, 3.3 MiB/s wr, 860 op/s 2024-05-21T03:31:57.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:56 smithi165 ceph-mon[36269]: pgmap v242: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 10 KiB/s rd, 3.3 MiB/s wr, 860 op/s 2024-05-21T03:31:58.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:31:58 smithi027 ceph-mon[32179]: pgmap v243: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 11 KiB/s rd, 2.6 MiB/s wr, 685 op/s 2024-05-21T03:31:59.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:31:58 smithi039 ceph-mon[36429]: pgmap v243: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 11 KiB/s rd, 2.6 MiB/s wr, 685 op/s 2024-05-21T03:31:59.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:31:58 smithi165 ceph-mon[36269]: pgmap v243: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 11 KiB/s rd, 2.6 MiB/s wr, 685 op/s 2024-05-21T03:31:59.996 INFO:tasks.workunit.client.0.smithi027.stdout:Syncing()...0 sec 2024-05-21T03:31:59.996 INFO:tasks.workunit.client.0.smithi027.stdout:FFSB benchmark finished at: Tue May 21 03:31:59 2024 2024-05-21T03:31:59.996 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-05-21T03:31:59.997 INFO:tasks.workunit.client.0.smithi027.stdout:Results: 2024-05-21T03:31:59.997 INFO:tasks.workunit.client.0.smithi027.stdout:Benchmark took 321.82 sec 2024-05-21T03:31:59.997 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-05-21T03:31:59.997 INFO:tasks.workunit.client.0.smithi027.stdout:Total Results 2024-05-21T03:31:59.997 INFO:tasks.workunit.client.0.smithi027.stdout:=============== 2024-05-21T03:31:59.997 INFO:tasks.workunit.client.0.smithi027.stdout: Op Name Transactions Trans/sec % Trans % Op Weight Throughput 2024-05-21T03:31:59.997 INFO:tasks.workunit.client.0.smithi027.stdout: ======= ============ ========= ======= =========== ========== 2024-05-21T03:31:59.997 INFO:tasks.workunit.client.0.smithi027.stdout: write : 4428800 13761.90 100.000% 100.000% 53.8MB/sec 2024-05-21T03:31:59.997 INFO:tasks.workunit.client.0.smithi027.stdout:- 2024-05-21T03:31:59.997 INFO:tasks.workunit.client.0.smithi027.stdout:13761.90 Transactions per Second 2024-05-21T03:31:59.997 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-05-21T03:31:59.997 INFO:tasks.workunit.client.0.smithi027.stdout:Throughput Results 2024-05-21T03:31:59.997 INFO:tasks.workunit.client.0.smithi027.stdout:=================== 2024-05-21T03:31:59.998 INFO:tasks.workunit.client.0.smithi027.stdout:Write Throughput: 53.8MB/sec 2024-05-21T03:32:00.017 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-05-21T03:32:00.017 INFO:tasks.workunit.client.0.smithi027.stdout:System Call Latency statistics in millisecs 2024-05-21T03:32:00.017 INFO:tasks.workunit.client.0.smithi027.stdout:===== 2024-05-21T03:32:00.017 INFO:tasks.workunit.client.0.smithi027.stdout: Min Avg Max Total Calls 2024-05-21T03:32:00.017 INFO:tasks.workunit.client.0.smithi027.stdout: ======== ======== ======== ============ 2024-05-21T03:32:00.017 INFO:tasks.workunit.client.0.smithi027.stdout:[ open] 0.006000 96.276308 9481.172852 3460 2024-05-21T03:32:00.017 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[0] 5000.000000 - 10000.000000 : 26 2024-05-21T03:32:00.018 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[1] 2000.000000 - 5000.000000 : 35 2024-05-21T03:32:00.018 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[2] 1000.000000 - 2000.000000 : 4 2024-05-21T03:32:00.018 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-05-21T03:32:00.018 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-05-21T03:32:00.018 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-05-21T03:32:00.018 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[6] 50.000000 - 100.000000 : 4 2024-05-21T03:32:00.018 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[7] 20.000000 - 50.000000 : 24 2024-05-21T03:32:00.018 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[8] 10.000000 - 20.000000 : 31 2024-05-21T03:32:00.018 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[9] 5.000000 - 10.000000 : 48 2024-05-21T03:32:00.018 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[10] 2.000000 - 5.000000 : 84 2024-05-21T03:32:00.019 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[11] 1.000000 - 2.000000 : 165 2024-05-21T03:32:00.019 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[12] 0.500000 - 1.000000 : 158 2024-05-21T03:32:00.019 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[13] 0.200000 - 0.500000 : 3 2024-05-21T03:32:00.019 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[14] 0.100000 - 0.200000 : 6 2024-05-21T03:32:00.019 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[15] 0.050000 - 0.100000 : 117 2024-05-21T03:32:00.019 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[16] 0.020000 - 0.050000 : 463 2024-05-21T03:32:00.019 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[17] 0.010000 - 0.020000 : 1851 2024-05-21T03:32:00.019 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[18] 0.000000 - 0.010000 : 441 2024-05-21T03:32:00.019 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-05-21T03:32:00.019 INFO:tasks.workunit.client.0.smithi027.stdout:[ write] 0.001000 2.164415 107220.726562 4428800 2024-05-21T03:32:00.019 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[0] 5000.000000 - 10000.000000 : 81 2024-05-21T03:32:00.019 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[1] 2000.000000 - 5000.000000 : 269 2024-05-21T03:32:00.019 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[2] 1000.000000 - 2000.000000 : 430 2024-05-21T03:32:00.019 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[3] 500.000000 - 1000.000000 : 272 2024-05-21T03:32:00.020 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[4] 200.000000 - 500.000000 : 959 2024-05-21T03:32:00.020 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[5] 100.000000 - 200.000000 : 3431 2024-05-21T03:32:00.020 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[6] 50.000000 - 100.000000 : 11334 2024-05-21T03:32:00.020 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[7] 20.000000 - 50.000000 : 24137 2024-05-21T03:32:00.020 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[8] 10.000000 - 20.000000 : 17259 2024-05-21T03:32:00.020 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[9] 5.000000 - 10.000000 : 17107 2024-05-21T03:32:00.020 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[10] 2.000000 - 5.000000 : 41350 2024-05-21T03:32:00.020 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[11] 1.000000 - 2.000000 : 647 2024-05-21T03:32:00.020 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[12] 0.500000 - 1.000000 : 337 2024-05-21T03:32:00.020 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[13] 0.200000 - 0.500000 : 375 2024-05-21T03:32:00.020 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[14] 0.100000 - 0.200000 : 399 2024-05-21T03:32:00.020 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[15] 0.050000 - 0.100000 : 1436 2024-05-21T03:32:00.020 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[16] 0.020000 - 0.050000 : 6022 2024-05-21T03:32:00.020 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[17] 0.010000 - 0.020000 : 36338 2024-05-21T03:32:00.021 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[18] 0.000000 - 0.010000 : 4266470 2024-05-21T03:32:00.021 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-05-21T03:32:00.021 INFO:tasks.workunit.client.0.smithi027.stdout:[ lseek] 0.000000 0.001128 58.133999 4428800 2024-05-21T03:32:00.021 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-05-21T03:32:00.021 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-05-21T03:32:00.021 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-05-21T03:32:00.021 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-05-21T03:32:00.021 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-05-21T03:32:00.021 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-05-21T03:32:00.021 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[6] 50.000000 - 100.000000 : 1 2024-05-21T03:32:00.022 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[7] 20.000000 - 50.000000 : 29 2024-05-21T03:32:00.022 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[8] 10.000000 - 20.000000 : 48 2024-05-21T03:32:00.022 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[9] 5.000000 - 10.000000 : 61 2024-05-21T03:32:00.022 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[10] 2.000000 - 5.000000 : 57 2024-05-21T03:32:00.022 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[11] 1.000000 - 2.000000 : 30 2024-05-21T03:32:00.022 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[12] 0.500000 - 1.000000 : 43 2024-05-21T03:32:00.022 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[13] 0.200000 - 0.500000 : 55 2024-05-21T03:32:00.022 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[14] 0.100000 - 0.200000 : 39 2024-05-21T03:32:00.022 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[15] 0.050000 - 0.100000 : 108 2024-05-21T03:32:00.022 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[16] 0.020000 - 0.050000 : 560 2024-05-21T03:32:00.022 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[17] 0.010000 - 0.020000 : 1486 2024-05-21T03:32:00.023 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[18] 0.000000 - 0.010000 : 4426283 2024-05-21T03:32:00.023 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-05-21T03:32:00.023 INFO:tasks.workunit.client.0.smithi027.stdout:[ close] 0.002000 0.011206 22.014000 3460 2024-05-21T03:32:00.023 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-05-21T03:32:00.023 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-05-21T03:32:00.023 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-05-21T03:32:00.023 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-05-21T03:32:00.023 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-05-21T03:32:00.023 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-05-21T03:32:00.023 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2024-05-21T03:32:00.023 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[7] 20.000000 - 50.000000 : 1 2024-05-21T03:32:00.024 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[8] 10.000000 - 20.000000 : 0 2024-05-21T03:32:00.024 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[9] 5.000000 - 10.000000 : 0 2024-05-21T03:32:00.024 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[10] 2.000000 - 5.000000 : 0 2024-05-21T03:32:00.024 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[11] 1.000000 - 2.000000 : 0 2024-05-21T03:32:00.024 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[12] 0.500000 - 1.000000 : 0 2024-05-21T03:32:00.024 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[13] 0.200000 - 0.500000 : 0 2024-05-21T03:32:00.024 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[14] 0.100000 - 0.200000 : 0 2024-05-21T03:32:00.024 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[15] 0.050000 - 0.100000 : 2 2024-05-21T03:32:00.024 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[16] 0.020000 - 0.050000 : 4 2024-05-21T03:32:00.024 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[17] 0.010000 - 0.020000 : 26 2024-05-21T03:32:00.024 INFO:tasks.workunit.client.0.smithi027.stdout: msec_range[18] 0.000000 - 0.010000 : 3427 2024-05-21T03:32:00.024 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-05-21T03:32:00.024 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-05-21T03:32:00.024 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-05-21T03:32:00.025 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-05-21T03:32:00.025 INFO:tasks.workunit.client.0.smithi027.stdout:1.3% User Time 2024-05-21T03:32:00.025 INFO:tasks.workunit.client.0.smithi027.stdout:6.3% System Time 2024-05-21T03:32:00.025 INFO:tasks.workunit.client.0.smithi027.stdout:7.6% CPU Utilization 2024-05-21T03:32:00.025 INFO:tasks.workunit.client.0.smithi027.stderr:+ cd .. 2024-05-21T03:32:00.026 INFO:tasks.workunit.client.0.smithi027.stderr:+ rm -r tmp ffsb 2024-05-21T03:32:00.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:00 smithi027 ceph-mon[32179]: pgmap v244: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 9.7 KiB/s rd, 2.4 MiB/s wr, 620 op/s 2024-05-21T03:32:01.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:00 smithi039 ceph-mon[36429]: pgmap v244: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 9.7 KiB/s rd, 2.4 MiB/s wr, 620 op/s 2024-05-21T03:32:01.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:00 smithi165 ceph-mon[36269]: pgmap v244: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 9.7 KiB/s rd, 2.4 MiB/s wr, 620 op/s 2024-05-21T03:32:02.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:02 smithi027 ceph-mon[32179]: pgmap v245: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1008 GiB / 1.0 TiB avail; 28 KiB/s rd, 3.2 MiB/s wr, 823 op/s 2024-05-21T03:32:03.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:02 smithi039 ceph-mon[36429]: pgmap v245: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1008 GiB / 1.0 TiB avail; 28 KiB/s rd, 3.2 MiB/s wr, 823 op/s 2024-05-21T03:32:03.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:02 smithi165 ceph-mon[36269]: pgmap v245: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1008 GiB / 1.0 TiB avail; 28 KiB/s rd, 3.2 MiB/s wr, 823 op/s 2024-05-21T03:32:03.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:03 smithi027 ceph-mon[32179]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:32:04.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:03 smithi039 ceph-mon[36429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:32:04.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:03 smithi165 ceph-mon[36269]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:32:04.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:04 smithi027 ceph-mon[32179]: pgmap v246: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1008 GiB / 1.0 TiB avail; 25 KiB/s rd, 2.4 MiB/s wr, 605 op/s 2024-05-21T03:32:05.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:04 smithi039 ceph-mon[36429]: pgmap v246: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1008 GiB / 1.0 TiB avail; 25 KiB/s rd, 2.4 MiB/s wr, 605 op/s 2024-05-21T03:32:05.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:04 smithi165 ceph-mon[36269]: pgmap v246: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1008 GiB / 1.0 TiB avail; 25 KiB/s rd, 2.4 MiB/s wr, 605 op/s 2024-05-21T03:32:06.930 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:06 smithi027 ceph-mon[32179]: pgmap v247: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 74 KiB/s rd, 3.0 MiB/s wr, 860 op/s 2024-05-21T03:32:06.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:06 smithi027 ceph-mon[32179]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:32:07.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:06 smithi039 ceph-mon[36429]: pgmap v247: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 74 KiB/s rd, 3.0 MiB/s wr, 860 op/s 2024-05-21T03:32:07.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:06 smithi039 ceph-mon[36429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:32:07.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:06 smithi165 ceph-mon[36269]: pgmap v247: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 74 KiB/s rd, 3.0 MiB/s wr, 860 op/s 2024-05-21T03:32:07.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:06 smithi165 ceph-mon[36269]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:32:07.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:07 smithi027 ceph-mon[32179]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:32:08.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:07 smithi039 ceph-mon[36429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:32:08.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:07 smithi165 ceph-mon[36269]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:32:08.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:08 smithi027 ceph-mon[32179]: pgmap v248: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 94 KiB/s rd, 2.3 MiB/s wr, 700 op/s 2024-05-21T03:32:08.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:08 smithi027 ceph-mon[32179]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:32:09.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:08 smithi039 ceph-mon[36429]: pgmap v248: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 94 KiB/s rd, 2.3 MiB/s wr, 700 op/s 2024-05-21T03:32:09.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:08 smithi039 ceph-mon[36429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:32:09.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:08 smithi165 ceph-mon[36269]: pgmap v248: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 94 KiB/s rd, 2.3 MiB/s wr, 700 op/s 2024-05-21T03:32:09.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:08 smithi165 ceph-mon[36269]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:32:10.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:10 smithi027 ceph-mon[32179]: pgmap v249: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 115 KiB/s rd, 2.2 MiB/s wr, 690 op/s 2024-05-21T03:32:11.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:10 smithi039 ceph-mon[36429]: pgmap v249: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 115 KiB/s rd, 2.2 MiB/s wr, 690 op/s 2024-05-21T03:32:11.020 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-21T03:32:11.020 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-21T03:32:11.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:10 smithi165 ceph-mon[36269]: pgmap v249: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 115 KiB/s rd, 2.2 MiB/s wr, 690 op/s 2024-05-21T03:32:11.058 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0... 2024-05-21T03:32:11.059 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-21T03:32:11.389 DEBUG:teuthology.parallel:result is None 2024-05-21T03:32:11.389 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-21T03:32:11.429 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-21T03:32:11.429 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-05-21T03:32:12.022 INFO:tasks.quiescer:joining Quiescers 2024-05-21T03:32:12.022 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-05-21T03:32:12.022 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:32:12.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:11 smithi165 ceph-mon[36269]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:32:12.024 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:11 smithi039 ceph-mon[36429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:32:12.046 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:11 smithi027 ceph-mon[32179]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:32:12.274 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:32:13.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:12 smithi039 ceph-mon[36429]: pgmap v250: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 137 KiB/s rd, 2.0 MiB/s wr, 750 op/s 2024-05-21T03:32:13.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:12 smithi165 ceph-mon[36269]: pgmap v250: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 137 KiB/s rd, 2.0 MiB/s wr, 750 op/s 2024-05-21T03:32:13.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:12 smithi027 ceph-mon[32179]: pgmap v250: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 137 KiB/s rd, 2.0 MiB/s wr, 750 op/s 2024-05-21T03:32:13.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.809+0000 7f1ddcf4b700 1 -- 172.21.15.27:0/2345900376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dd8102e00 msgr2=0x7f1dd810f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:32:13.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.809+0000 7f1ddcf4b700 1 --2- 172.21.15.27:0/2345900376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dd8102e00 0x7f1dd810f2f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1dd000b0a0 tx=0x7f1dd000b3b0 comp rx=0 tx=0).stop 2024-05-21T03:32:13.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.809+0000 7f1ddcf4b700 1 -- 172.21.15.27:0/2345900376 shutdown_connections 2024-05-21T03:32:13.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.809+0000 7f1ddcf4b700 1 --2- 172.21.15.27:0/2345900376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dd8102e00 0x7f1dd810f2f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:13.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.809+0000 7f1ddcf4b700 1 --2- 172.21.15.27:0/2345900376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dd8108520 0x7f1dd8108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:13.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.809+0000 7f1ddcf4b700 1 --2- 172.21.15.27:0/2345900376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1dd8102440 0x7f1dd81028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:13.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.809+0000 7f1ddcf4b700 1 -- 172.21.15.27:0/2345900376 >> 172.21.15.27:0/2345900376 conn(0x7f1dd80793f0 msgr2=0x7f1dd8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:32:13.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.809+0000 7f1ddcf4b700 1 -- 172.21.15.27:0/2345900376 shutdown_connections 2024-05-21T03:32:13.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.809+0000 7f1ddcf4b700 1 -- 172.21.15.27:0/2345900376 wait complete. 2024-05-21T03:32:13.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.810+0000 7f1ddcf4b700 1 Processor -- start 2024-05-21T03:32:13.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.810+0000 7f1ddcf4b700 1 -- start start 2024-05-21T03:32:13.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.810+0000 7f1ddcf4b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1dd8102440 0x7f1dd819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:32:13.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.810+0000 7f1ddcf4b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dd8102e00 0x7f1dd819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:32:13.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.810+0000 7f1ddcf4b700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dd8108520 0x7f1dd8199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:32:13.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.810+0000 7f1ddcf4b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd81125d0 con 0x7f1dd8102e00 2024-05-21T03:32:13.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.810+0000 7f1ddcf4b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd8112450 con 0x7f1dd8102440 2024-05-21T03:32:13.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.810+0000 7f1ddcf4b700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd8112750 con 0x7f1dd8108520 2024-05-21T03:32:13.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.811+0000 7f1dd5d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dd8102e00 0x7f1dd819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:32:13.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.811+0000 7f1dd5d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dd8102e00 0x7f1dd819fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58040/0 (socket says 172.21.15.27:58040) 2024-05-21T03:32:13.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.811+0000 7f1dd5d9b700 1 -- 172.21.15.27:0/286197404 learned_addr learned my addr 172.21.15.27:0/286197404 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:32:13.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.811+0000 7f1dd659c700 1 --2- 172.21.15.27:0/286197404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1dd8102440 0x7f1dd819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:32:13.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.811+0000 7f1dd6d9d700 1 --2- 172.21.15.27:0/286197404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dd8108520 0x7f1dd8199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:32:13.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.811+0000 7f1dd5d9b700 1 -- 172.21.15.27:0/286197404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dd8108520 msgr2=0x7f1dd8199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:32:13.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.811+0000 7f1dd5d9b700 1 --2- 172.21.15.27:0/286197404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dd8108520 0x7f1dd8199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:13.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.811+0000 7f1dd5d9b700 1 -- 172.21.15.27:0/286197404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1dd8102440 msgr2=0x7f1dd819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:32:13.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.811+0000 7f1dd5d9b700 1 --2- 172.21.15.27:0/286197404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1dd8102440 0x7f1dd819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:13.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.811+0000 7f1dd5d9b700 1 -- 172.21.15.27:0/286197404 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd819a0e0 con 0x7f1dd8102e00 2024-05-21T03:32:13.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.811+0000 7f1dd6d9d700 1 --2- 172.21.15.27:0/286197404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dd8108520 0x7f1dd8199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:32:13.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.811+0000 7f1dd5d9b700 1 --2- 172.21.15.27:0/286197404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dd8102e00 0x7f1dd819fd00 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00d980 tx=0x7f1dcc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:32:13.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.811+0000 7f1dc77fe700 1 -- 172.21.15.27:0/286197404 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc016070 con 0x7f1dd8102e00 2024-05-21T03:32:13.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.811+0000 7f1ddcf4b700 1 -- 172.21.15.27:0/286197404 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dd819a3d0 con 0x7f1dd8102e00 2024-05-21T03:32:13.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.811+0000 7f1dc77fe700 1 -- 172.21.15.27:0/286197404 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dcc004830 con 0x7f1dd8102e00 2024-05-21T03:32:13.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.811+0000 7f1dc77fe700 1 -- 172.21.15.27:0/286197404 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc0054c0 con 0x7f1dd8102e00 2024-05-21T03:32:13.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.811+0000 7f1ddcf4b700 1 -- 172.21.15.27:0/286197404 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dd81a6690 con 0x7f1dd8102e00 2024-05-21T03:32:13.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.812+0000 7f1ddcf4b700 1 -- 172.21.15.27:0/286197404 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dd806a230 con 0x7f1dd8102e00 2024-05-21T03:32:13.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.814+0000 7f1dc77fe700 1 -- 172.21.15.27:0/286197404 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7f1dcc004220 con 0x7f1dd8102e00 2024-05-21T03:32:13.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.814+0000 7f1dc77fe700 1 --2- 172.21.15.27:0/286197404 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f1dc00b29f0 0x7f1dc00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:32:13.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.814+0000 7f1dd659c700 1 --2- 172.21.15.27:0/286197404 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f1dc00b29f0 0x7f1dc00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:32:13.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.815+0000 7f1dc77fe700 1 -- 172.21.15.27:0/286197404 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8571+0+0 (secure 0 0 0) 0x7f1dcc0e2970 con 0x7f1dd8102e00 2024-05-21T03:32:13.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.815+0000 7f1dd659c700 1 --2- 172.21.15.27:0/286197404 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f1dc00b29f0 0x7f1dc00b4eb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1dd819aac0 tx=0x7f1dc800b410 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:32:13.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:13.818+0000 7f1dc77fe700 1 -- 172.21.15.27:0/286197404 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1dcc01f020 con 0x7f1dd8102e00 2024-05-21T03:32:14.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:14.078+0000 7f1ddcf4b700 1 -- 172.21.15.27:0/286197404 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1dd802e560 con 0x7f1dd8102e00 2024-05-21T03:32:14.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:14.079+0000 7f1dc77fe700 1 -- 172.21.15.27:0/286197404 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11952 (secure 0 0 0) 0x7f1dcc0aa320 con 0x7f1dd8102e00 2024-05-21T03:32:14.079 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:32:14.080 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/148854869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":148854869},{"type":"v1","addr":"172.21.15.27:6835","nonce":148854869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/4145544730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":4145544730},{"type":"v1","addr":"172.21.15.27:6837","nonce":4145544730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T03:20:34.055985+0000","modified":"2024-05-21T03:32:09.917875+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24472,"mds_2":24457,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/745486764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":745486764},{"type":"v1","addr":"172.21.15.27:6839","nonce":745486764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6841/3982752351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3982752351},{"type":"v1","addr":"172.21.15.27:6841","nonce":3982752351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/3497521608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3497521608},{"type":"v1","addr":"172.21.15.39:6835","nonce":3497521608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.165:6835/3576910218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3576910218},{"type":"v1","addr":"172.21.15.165:6835","nonce":3576910218}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/3673846852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3673846852},{"type":"v1","addr":"172.21.15.39:6837","nonce":3673846852}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6837/691356139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":691356139},{"type":"v1","addr":"172.21.15.165:6837","nonce":691356139}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6839/4158170814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4158170814},{"type":"v1","addr":"172.21.15.165:6839","nonce":4158170814}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6839/287607819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":287607819},{"type":"v1","addr":"172.21.15.39:6839","nonce":287607819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24451,24472,24452,24457,24476]},"id":1}]} 2024-05-21T03:32:14.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:14.081+0000 7f1ddcf4b700 1 -- 172.21.15.27:0/286197404 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f1dc00b29f0 msgr2=0x7f1dc00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:32:14.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:14.081+0000 7f1ddcf4b700 1 --2- 172.21.15.27:0/286197404 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f1dc00b29f0 0x7f1dc00b4eb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1dd819aac0 tx=0x7f1dc800b410 comp rx=0 tx=0).stop 2024-05-21T03:32:14.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:14.081+0000 7f1ddcf4b700 1 -- 172.21.15.27:0/286197404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dd8102e00 msgr2=0x7f1dd819fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:32:14.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:14.081+0000 7f1ddcf4b700 1 --2- 172.21.15.27:0/286197404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dd8102e00 0x7f1dd819fd00 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00d980 tx=0x7f1dcc00dc90 comp rx=0 tx=0).stop 2024-05-21T03:32:14.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:14.082+0000 7f1ddcf4b700 1 -- 172.21.15.27:0/286197404 shutdown_connections 2024-05-21T03:32:14.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:14.082+0000 7f1ddcf4b700 1 --2- 172.21.15.27:0/286197404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dd8108520 0x7f1dd8199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:14.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:14.082+0000 7f1ddcf4b700 1 --2- 172.21.15.27:0/286197404 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f1dc00b29f0 0x7f1dc00b4eb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:14.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:14.082+0000 7f1ddcf4b700 1 --2- 172.21.15.27:0/286197404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1dd8102440 0x7f1dd819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:14.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:14.082+0000 7f1ddcf4b700 1 --2- 172.21.15.27:0/286197404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1dd8102e00 0x7f1dd819fd00 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:14.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:14.082+0000 7f1ddcf4b700 1 -- 172.21.15.27:0/286197404 >> 172.21.15.27:0/286197404 conn(0x7f1dd80793f0 msgr2=0x7f1dd806b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:32:14.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:14.082+0000 7f1ddcf4b700 1 -- 172.21.15.27:0/286197404 shutdown_connections 2024-05-21T03:32:14.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:14.082+0000 7f1ddcf4b700 1 -- 172.21.15.27:0/286197404 wait complete. 2024-05-21T03:32:14.083 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 44 2024-05-21T03:32:14.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:14 smithi027 ceph-mon[32179]: pgmap v251: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 118 KiB/s rd, 775 KiB/s wr, 453 op/s 2024-05-21T03:32:14.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:14 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/286197404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:32:14.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:14 smithi039 ceph-mon[36429]: pgmap v251: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 118 KiB/s rd, 775 KiB/s wr, 453 op/s 2024-05-21T03:32:14.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:14 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/286197404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:32:14.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:14 smithi165 ceph-mon[36269]: pgmap v251: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 118 KiB/s rd, 775 KiB/s wr, 453 op/s 2024-05-21T03:32:14.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:14 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/286197404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:32:14.794 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --cancel --all' 2024-05-21T03:32:14.794 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --cancel --all 2024-05-21T03:32:15.119 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:32:15.511 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:15 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:32:15.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:15 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:32:15.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:15 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:32:16.557 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:16 smithi027 ceph-mon[32179]: osdmap e86: 12 total, 12 up, 12 in 2024-05-21T03:32:16.557 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:16 smithi027 ceph-mon[32179]: pgmap v253: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 225 KiB/s rd, 268 KiB/s wr, 267 op/s 2024-05-21T03:32:16.557 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:16 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:32:16.557 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:16 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:32:16.557 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:16 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:32:16.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:16 smithi039 ceph-mon[36429]: osdmap e86: 12 total, 12 up, 12 in 2024-05-21T03:32:16.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:16 smithi039 ceph-mon[36429]: pgmap v253: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 225 KiB/s rd, 268 KiB/s wr, 267 op/s 2024-05-21T03:32:16.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:16 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:32:16.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:16 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:32:16.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:16 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:32:16.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:16 smithi165 ceph-mon[36269]: osdmap e86: 12 total, 12 up, 12 in 2024-05-21T03:32:16.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:16 smithi165 ceph-mon[36269]: pgmap v253: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 225 KiB/s rd, 268 KiB/s wr, 267 op/s 2024-05-21T03:32:16.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:16 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:32:16.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:16 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:32:16.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:16 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:32:16.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.824+0000 7f2d879cd700 1 -- 172.21.15.27:0/449289645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d801095b0 msgr2=0x7f2d80109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:32:16.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.824+0000 7f2d879cd700 1 --2- 172.21.15.27:0/449289645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d801095b0 0x7f2d80109990 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c009990 tx=0x7f2d7c009ca0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.824+0000 7f2d879cd700 1 -- 172.21.15.27:0/449289645 shutdown_connections 2024-05-21T03:32:16.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.824+0000 7f2d879cd700 1 --2- 172.21.15.27:0/449289645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d8006c630 0x7f2d801103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.824+0000 7f2d879cd700 1 --2- 172.21.15.27:0/449289645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d801095b0 0x7f2d80109990 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.824+0000 7f2d879cd700 1 --2- 172.21.15.27:0/449289645 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d8006bc70 0x7f2d8006c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.824+0000 7f2d879cd700 1 -- 172.21.15.27:0/449289645 >> 172.21.15.27:0/449289645 conn(0x7f2d800793f0 msgr2=0x7f2d80079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:32:16.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.824+0000 7f2d879cd700 1 -- 172.21.15.27:0/449289645 shutdown_connections 2024-05-21T03:32:16.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.824+0000 7f2d879cd700 1 -- 172.21.15.27:0/449289645 wait complete. 2024-05-21T03:32:16.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.825+0000 7f2d879cd700 1 Processor -- start 2024-05-21T03:32:16.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.825+0000 7f2d879cd700 1 -- start start 2024-05-21T03:32:16.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.825+0000 7f2d879cd700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d8006bc70 0x7f2d80102540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:32:16.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.825+0000 7f2d879cd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d8006c630 0x7f2d80102a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:32:16.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.825+0000 7f2d879cd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d801095b0 0x7f2d80102fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:32:16.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.825+0000 7f2d879cd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2d80113740 con 0x7f2d801095b0 2024-05-21T03:32:16.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.825+0000 7f2d879cd700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2d801135c0 con 0x7f2d8006c630 2024-05-21T03:32:16.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.825+0000 7f2d879cd700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2d801138c0 con 0x7f2d8006bc70 2024-05-21T03:32:16.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.825+0000 7f2d85f6a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d801095b0 0x7f2d80102fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:32:16.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.825+0000 7f2d85f6a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d801095b0 0x7f2d80102fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58054/0 (socket says 172.21.15.27:58054) 2024-05-21T03:32:16.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.825+0000 7f2d85f6a700 1 -- 172.21.15.27:0/935197288 learned_addr learned my addr 172.21.15.27:0/935197288 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:32:16.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.825+0000 7f2d85769700 1 --2- 172.21.15.27:0/935197288 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d8006bc70 0x7f2d80102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:32:16.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.826+0000 7f2d84f68700 1 --2- 172.21.15.27:0/935197288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d8006c630 0x7f2d80102a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:32:16.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.826+0000 7f2d85f6a700 1 -- 172.21.15.27:0/935197288 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d8006bc70 msgr2=0x7f2d80102540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:32:16.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.826+0000 7f2d85f6a700 1 --2- 172.21.15.27:0/935197288 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d8006bc70 0x7f2d80102540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.826+0000 7f2d85f6a700 1 -- 172.21.15.27:0/935197288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d8006c630 msgr2=0x7f2d80102a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:32:16.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.826+0000 7f2d85f6a700 1 --2- 172.21.15.27:0/935197288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d8006c630 0x7f2d80102a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.826+0000 7f2d85f6a700 1 -- 172.21.15.27:0/935197288 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d801a81e0 con 0x7f2d801095b0 2024-05-21T03:32:16.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.826+0000 7f2d84f68700 1 --2- 172.21.15.27:0/935197288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d8006c630 0x7f2d80102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:32:16.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.826+0000 7f2d85f6a700 1 --2- 172.21.15.27:0/935197288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d801095b0 0x7f2d80102fc0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2d78007ee0 tx=0x7f2d7800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:32:16.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.826+0000 7f2d767fc700 1 -- 172.21.15.27:0/935197288 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7800dbd0 con 0x7f2d801095b0 2024-05-21T03:32:16.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.826+0000 7f2d879cd700 1 -- 172.21.15.27:0/935197288 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d801a84d0 con 0x7f2d801095b0 2024-05-21T03:32:16.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.826+0000 7f2d767fc700 1 -- 172.21.15.27:0/935197288 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d7800dd70 con 0x7f2d801095b0 2024-05-21T03:32:16.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.826+0000 7f2d767fc700 1 -- 172.21.15.27:0/935197288 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d78005990 con 0x7f2d801095b0 2024-05-21T03:32:16.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.826+0000 7f2d879cd700 1 -- 172.21.15.27:0/935197288 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d801a89c0 con 0x7f2d801095b0 2024-05-21T03:32:16.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.829+0000 7f2d767fc700 1 -- 172.21.15.27:0/935197288 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7f2d7801a020 con 0x7f2d801095b0 2024-05-21T03:32:16.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.830+0000 7f2d767fc700 1 --2- 172.21.15.27:0/935197288 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f2d6c0b29f0 0x7f2d6c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:32:16.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.830+0000 7f2d92263b80 1 Processor -- start 2024-05-21T03:32:16.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.830+0000 7f2d85769700 1 --2- 172.21.15.27:0/935197288 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f2d6c0b29f0 0x7f2d6c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:32:16.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.830+0000 7f2d92263b80 1 -- start start 2024-05-21T03:32:16.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.830+0000 7f2d767fc700 1 -- 172.21.15.27:0/935197288 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8615+0+0 (secure 0 0 0) 0x7f2d780e1e20 con 0x7f2d801095b0 2024-05-21T03:32:16.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.830+0000 7f2d92263b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f343c75240 0x55f343c75620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:32:16.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.830+0000 7f2d92263b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f343c75bf0 0x55f343c79610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:32:16.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.830+0000 7f2d92263b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f343c79c30 0x55f343c7a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:32:16.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.830+0000 7f2d92263b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f343bcbd00 con 0x55f343c79c30 2024-05-21T03:32:16.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.830+0000 7f2d92263b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f343bbb8a0 con 0x55f343c75240 2024-05-21T03:32:16.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.830+0000 7f2d92263b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f343bbf8e0 con 0x55f343c75bf0 2024-05-21T03:32:16.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.830+0000 7f2d85f6a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f343c79c30 0x55f343c7a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:32:16.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.830+0000 7f2d85f6a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f343c79c30 0x55f343c7a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58066/0 (socket says 172.21.15.27:58066) 2024-05-21T03:32:16.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.830+0000 7f2d85f6a700 1 -- 172.21.15.27:0/1796078604 learned_addr learned my addr 172.21.15.27:0/1796078604 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:32:16.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.830+0000 7f2d85769700 1 --2- 172.21.15.27:0/935197288 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f2d6c0b29f0 0x7f2d6c0b4eb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00b5c0 tx=0x7f2d7c005960 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:32:16.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.831+0000 7f2d84f68700 1 --2- 172.21.15.27:0/1796078604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f343c75bf0 0x55f343c79610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:32:16.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.831+0000 7f2d85769700 1 --2- 172.21.15.27:0/1796078604 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f343c75240 0x55f343c75620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:32:16.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.831+0000 7f2d85f6a700 1 -- 172.21.15.27:0/1796078604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f343c75bf0 msgr2=0x55f343c79610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:32:16.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.831+0000 7f2d85f6a700 1 --2- 172.21.15.27:0/1796078604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f343c75bf0 0x55f343c79610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.831+0000 7f2d85f6a700 1 -- 172.21.15.27:0/1796078604 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f343c75240 msgr2=0x55f343c75620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:32:16.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.831+0000 7f2d85f6a700 1 --2- 172.21.15.27:0/1796078604 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f343c75240 0x55f343c75620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.831+0000 7f2d85f6a700 1 -- 172.21.15.27:0/1796078604 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f343b0b940 con 0x55f343c79c30 2024-05-21T03:32:16.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.831+0000 7f2d84f68700 1 --2- 172.21.15.27:0/1796078604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f343c75bf0 0x55f343c79610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:32:16.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.831+0000 7f2d85f6a700 1 --2- 172.21.15.27:0/1796078604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f343c79c30 0x55f343c7a090 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2d780a8da0 tx=0x7f2d78007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b01e55ca6ecfc5e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:32:16.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.831+0000 7f2d6b7fe700 1 -- 172.21.15.27:0/1796078604 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d78020160 con 0x55f343c79c30 2024-05-21T03:32:16.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.831+0000 7f2d6b7fe700 1 -- 172.21.15.27:0/1796078604 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d780a90a0 con 0x55f343c79c30 2024-05-21T03:32:16.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.831+0000 7f2d6b7fe700 1 -- 172.21.15.27:0/1796078604 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d780a9560 con 0x55f343c79c30 2024-05-21T03:32:16.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.832+0000 7f2d92263b80 1 -- 172.21.15.27:0/1796078604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f343c79c30 msgr2=0x55f343c7a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:32:16.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.832+0000 7f2d92263b80 1 --2- 172.21.15.27:0/1796078604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f343c79c30 0x55f343c7a090 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2d780a8da0 tx=0x7f2d78007f90 comp rx=0 tx=0).stop 2024-05-21T03:32:16.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.832+0000 7f2d92263b80 1 -- 172.21.15.27:0/1796078604 shutdown_connections 2024-05-21T03:32:16.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.832+0000 7f2d92263b80 1 --2- 172.21.15.27:0/1796078604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f343c79c30 0x55f343c7a090 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.832+0000 7f2d92263b80 1 --2- 172.21.15.27:0/1796078604 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f343c75240 0x55f343c75620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.832+0000 7f2d92263b80 1 --2- 172.21.15.27:0/1796078604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f343c75bf0 0x55f343c79610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.832+0000 7f2d92263b80 1 -- 172.21.15.27:0/1796078604 >> 172.21.15.27:0/1796078604 conn(0x55f343b9a0f0 msgr2=0x55f343c67160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:32:16.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.832+0000 7f2d92263b80 1 -- 172.21.15.27:0/1796078604 shutdown_connections 2024-05-21T03:32:16.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.832+0000 7f2d92263b80 1 -- 172.21.15.27:0/1796078604 wait complete. 2024-05-21T03:32:16.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.832+0000 7f2d92263b80 1 -- start start 2024-05-21T03:32:16.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.832+0000 7f2d92263b80 1 Processor -- start 2024-05-21T03:32:16.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.833+0000 7f2d92263b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f343c75240 0x55f343c81150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:32:16.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.833+0000 7f2d92263b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f343c75bf0 0x55f343c7e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:32:16.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.833+0000 7f2d92263b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f343c79c30 0x55f343c7e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:32:16.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.833+0000 7f2d92263b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f343ac2010 con 0x55f343c79c30 2024-05-21T03:32:16.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.833+0000 7f2d92263b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f343b5d830 con 0x55f343c75bf0 2024-05-21T03:32:16.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.833+0000 7f2d92263b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f343c81e40 con 0x55f343c75240 2024-05-21T03:32:16.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.833+0000 7f2d92263b80 10 client.0 _ll_register_callbacks cb 0x55f343b56990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:32:16.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.833+0000 7f2d85f6a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f343c79c30 0x55f343c7e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:32:16.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.833+0000 7f2d85f6a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f343c79c30 0x55f343c7e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58072/0 (socket says 172.21.15.27:58072) 2024-05-21T03:32:16.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.833+0000 7f2d85f6a700 1 -- 172.21.15.27:0/1041527185 learned_addr learned my addr 172.21.15.27:0/1041527185 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:32:16.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.833+0000 7f2d84f68700 1 --2- 172.21.15.27:0/1041527185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f343c75bf0 0x55f343c7e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:32:16.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.833+0000 7f2d85769700 1 --2- 172.21.15.27:0/1041527185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f343c75240 0x55f343c81150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:32:16.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.833+0000 7f2d85f6a700 1 -- 172.21.15.27:0/1041527185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f343c75240 msgr2=0x55f343c81150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:32:16.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.833+0000 7f2d85f6a700 1 --2- 172.21.15.27:0/1041527185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f343c75240 0x55f343c81150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.833+0000 7f2d85f6a700 1 -- 172.21.15.27:0/1041527185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f343c75bf0 msgr2=0x55f343c7e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:32:16.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.833+0000 7f2d85f6a700 1 --2- 172.21.15.27:0/1041527185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f343c75bf0 0x55f343c7e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.833+0000 7f2d85f6a700 1 -- 172.21.15.27:0/1041527185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f343b5a700 con 0x55f343c79c30 2024-05-21T03:32:16.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.834+0000 7f2d85f6a700 1 --2- 172.21.15.27:0/1041527185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f343c79c30 0x55f343c7e990 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2d780a8fe0 tx=0x7f2d7800df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57eced6da829682 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:32:16.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.834+0000 7f2d68ff9700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:32:16.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.834+0000 7f2d68ff9700 1 -- 172.21.15.27:0/1041527185 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d780c2070 con 0x55f343c79c30 2024-05-21T03:32:16.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.834+0000 7f2d68ff9700 1 -- 172.21.15.27:0/1041527185 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8615+0+0 (secure 0 0 0) 0x7f2d780a9300 con 0x55f343c79c30 2024-05-21T03:32:16.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.834+0000 7f2d92263b80 1 -- 172.21.15.27:0/1041527185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d780a90a0 con 0x55f343c79c30 2024-05-21T03:32:16.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.834+0000 7f2d68ff9700 1 -- 172.21.15.27:0/1041527185 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f2d780ad450 con 0x55f343c79c30 2024-05-21T03:32:16.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.834+0000 7f2d92263b80 10 client.15858 fetch_fsmap learned FSMap version 44 2024-05-21T03:32:16.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.834+0000 7f2d92263b80 1 -- 172.21.15.27:0/1041527185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55f343c8c5f0 con 0x55f343c79c30 2024-05-21T03:32:16.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.834+0000 7f2d68ff9700 1 -- 172.21.15.27:0/1041527185 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d780a90a0 con 0x55f343c79c30 2024-05-21T03:32:16.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.834+0000 7f2d68ff9700 1 -- 172.21.15.27:0/1041527185 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 44) ==== 8418+0+0 (secure 0 0 0) 0x7f2d78017bb0 con 0x55f343c79c30 2024-05-21T03:32:16.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.834+0000 7f2d92263b80 10 client.15858 fetch_fsmap finished waiting for FSMap version 44 2024-05-21T03:32:16.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.834+0000 7f2d92263b80 10 client.15858 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-21T03:32:16.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.835+0000 7f2d92263b80 20 client.15858 populate_metadata read hostname 'smithi027' 2024-05-21T03:32:16.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.835+0000 7f2d92263b80 1 --2- 172.21.15.27:0/1041527185 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55f343c8eac0 0x55f343c90eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:32:16.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.835+0000 7f2d92263b80 4 client.15858 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T03:32:16.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.835+0000 7f2d92263b80 1 -- 172.21.15.27:0/1041527185 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f343c38a50 con 0x55f343c8eac0 2024-05-21T03:32:16.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.835+0000 7f2d85769700 1 --2- 172.21.15.27:0/1041527185 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55f343c8eac0 0x55f343c90eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:32:16.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.835+0000 7f2d68ff9700 10 client.15858 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:32:16.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.835+0000 7f2d85769700 1 --2- 172.21.15.27:0/1041527185 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55f343c8eac0 0x55f343c90eb0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1af4d92795e69ade server_cookie=1e30ac912260b72e in_seq=0 out_seq=0 2024-05-21T03:32:16.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.836+0000 7f2d68ff9700 1 -- 172.21.15.27:0/1041527185 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f343c38a50 con 0x55f343c8eac0 2024-05-21T03:32:16.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.836+0000 7f2d68ff9700 10 client.15858 handle_command_reply: tid=0 2024-05-21T03:32:16.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.836+0000 7f2d92263b80 1 client.15858 shutdown 2024-05-21T03:32:16.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.836+0000 7f2d92263b80 1 -- 172.21.15.27:0/1041527185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f343c79c30 msgr2=0x55f343c7e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:32:16.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.836+0000 7f2d92263b80 1 --2- 172.21.15.27:0/1041527185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f343c79c30 0x55f343c7e990 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2d780a8fe0 tx=0x7f2d7800df90 comp rx=0 tx=0).stop 2024-05-21T03:32:16.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.836+0000 7f2d92263b80 1 -- 172.21.15.27:0/1041527185 shutdown_connections 2024-05-21T03:32:16.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.836+0000 7f2d92263b80 1 --2- 172.21.15.27:0/1041527185 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55f343c8eac0 0x55f343c90eb0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.836+0000 7f2d92263b80 1 --2- 172.21.15.27:0/1041527185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f343c79c30 0x55f343c7e990 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.836+0000 7f2d92263b80 1 --2- 172.21.15.27:0/1041527185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f343c75240 0x55f343c81150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.836+0000 7f2d68ff9700 0 client.15858 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:32:16.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.836+0000 7f2d92263b80 1 --2- 172.21.15.27:0/1041527185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f343c75bf0 0x55f343c7e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.836+0000 7f2d92263b80 1 -- 172.21.15.27:0/1041527185 >> 172.21.15.27:0/1041527185 conn(0x55f343b9a0f0 msgr2=0x55f343c7afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:32:16.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.836+0000 7f2d92263b80 1 -- 172.21.15.27:0/1041527185 shutdown_connections 2024-05-21T03:32:16.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.836+0000 7f2d92263b80 1 -- 172.21.15.27:0/1041527185 wait complete. 2024-05-21T03:32:16.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.836+0000 7f2d92263b80 20 client.15858 trim_cache size 0 max 16384 2024-05-21T03:32:16.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.857+0000 7f2d92263b80 1 Processor -- start 2024-05-21T03:32:16.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.857+0000 7f2d92263b80 1 -- start start 2024-05-21T03:32:16.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.857+0000 7f2d92263b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f343c75240 0x55f343c7d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:32:16.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.857+0000 7f2d92263b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f343c75bf0 0x55f343c6b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:32:16.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.857+0000 7f2d92263b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f343c79c30 0x55f343c6bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:32:16.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.857+0000 7f2d92263b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2d801abe00 con 0x55f343c79c30 2024-05-21T03:32:16.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.857+0000 7f2d92263b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2d801adb80 con 0x55f343c75240 2024-05-21T03:32:16.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.857+0000 7f2d92263b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2d801ac590 con 0x55f343c75bf0 2024-05-21T03:32:16.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.857+0000 7f2d85f6a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f343c79c30 0x55f343c6bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:32:16.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.857+0000 7f2d85f6a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f343c79c30 0x55f343c6bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58074/0 (socket says 172.21.15.27:58074) 2024-05-21T03:32:16.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.857+0000 7f2d85f6a700 1 -- 172.21.15.27:0/1327867455 learned_addr learned my addr 172.21.15.27:0/1327867455 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:32:16.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.857+0000 7f2d85769700 1 --2- 172.21.15.27:0/1327867455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f343c75240 0x55f343c7d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:32:16.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.857+0000 7f2d84f68700 1 --2- 172.21.15.27:0/1327867455 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f343c75bf0 0x55f343c6b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:32:16.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.857+0000 7f2d85f6a700 1 -- 172.21.15.27:0/1327867455 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f343c75bf0 msgr2=0x55f343c6b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:32:16.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.857+0000 7f2d85f6a700 1 --2- 172.21.15.27:0/1327867455 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f343c75bf0 0x55f343c6b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.857+0000 7f2d85f6a700 1 -- 172.21.15.27:0/1327867455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f343c75240 msgr2=0x55f343c7d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:32:16.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.857+0000 7f2d85f6a700 1 --2- 172.21.15.27:0/1327867455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f343c75240 0x55f343c7d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.858+0000 7f2d85f6a700 1 -- 172.21.15.27:0/1327867455 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f343c7c9f0 con 0x55f343c79c30 2024-05-21T03:32:16.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.858+0000 7f2d85769700 1 --2- 172.21.15.27:0/1327867455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f343c75240 0x55f343c7d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:32:16.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.858+0000 7f2d85f6a700 1 --2- 172.21.15.27:0/1327867455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f343c79c30 0x55f343c6bd80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2d780a8da0 tx=0x7f2d780df5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e033e6485ddb20ec server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:32:16.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.858+0000 7f2d6affd700 1 -- 172.21.15.27:0/1327867455 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d780c2070 con 0x55f343c79c30 2024-05-21T03:32:16.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.858+0000 7f2d6affd700 1 -- 172.21.15.27:0/1327867455 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d780b1f00 con 0x55f343c79c30 2024-05-21T03:32:16.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.858+0000 7f2d6affd700 1 -- 172.21.15.27:0/1327867455 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d780a8560 con 0x55f343c79c30 2024-05-21T03:32:16.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.858+0000 7f2d92263b80 1 -- 172.21.15.27:0/1327867455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f343c79c30 msgr2=0x55f343c6bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:32:16.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.858+0000 7f2d92263b80 1 --2- 172.21.15.27:0/1327867455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f343c79c30 0x55f343c6bd80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2d780a8da0 tx=0x7f2d780df5c0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.858+0000 7f2d92263b80 1 -- 172.21.15.27:0/1327867455 shutdown_connections 2024-05-21T03:32:16.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.858+0000 7f2d92263b80 1 --2- 172.21.15.27:0/1327867455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f343c79c30 0x55f343c6bd80 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.858+0000 7f2d92263b80 1 --2- 172.21.15.27:0/1327867455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f343c75240 0x55f343c7d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.858+0000 7f2d92263b80 1 --2- 172.21.15.27:0/1327867455 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f343c75bf0 0x55f343c6b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.858+0000 7f2d92263b80 1 -- 172.21.15.27:0/1327867455 >> 172.21.15.27:0/1327867455 conn(0x55f343b9a0f0 msgr2=0x55f343c7e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:32:16.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.859+0000 7f2d92263b80 1 -- 172.21.15.27:0/1327867455 shutdown_connections 2024-05-21T03:32:16.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.859+0000 7f2d92263b80 1 -- 172.21.15.27:0/1327867455 wait complete. 2024-05-21T03:32:16.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.859+0000 7f2d92263b80 1 -- start start 2024-05-21T03:32:16.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.859+0000 7f2d92263b80 1 Processor -- start 2024-05-21T03:32:16.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.859+0000 7f2d92263b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f343c75240 0x55f343cf2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:32:16.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.859+0000 7f2d92263b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f343c75bf0 0x55f343cf3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:32:16.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.859+0000 7f2d92263b80 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f343c79c30 0x55f343ced290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:32:16.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.859+0000 7f2d92263b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2d801ad4b0 con 0x55f343c75bf0 2024-05-21T03:32:16.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.859+0000 7f2d92263b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2d801acd20 con 0x55f343c75240 2024-05-21T03:32:16.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.859+0000 7f2d92263b80 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2d801a9e30 con 0x55f343c79c30 2024-05-21T03:32:16.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.859+0000 7f2d92263b80 10 client.0 _ll_register_callbacks cb 0x55f343b56990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T03:32:16.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.860+0000 7f2d84f68700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f343c75bf0 0x55f343cf3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:32:16.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.860+0000 7f2d84f68700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f343c75bf0 0x55f343cf3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58088/0 (socket says 172.21.15.27:58088) 2024-05-21T03:32:16.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.860+0000 7f2d84f68700 1 -- 172.21.15.27:0/4081846348 learned_addr learned my addr 172.21.15.27:0/4081846348 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:32:16.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.860+0000 7f2d85769700 1 --2- 172.21.15.27:0/4081846348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f343c75240 0x55f343cf2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:32:16.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.860+0000 7f2d85f6a700 1 --2- 172.21.15.27:0/4081846348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f343c79c30 0x55f343ced290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:32:16.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.860+0000 7f2d84f68700 1 -- 172.21.15.27:0/4081846348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f343c79c30 msgr2=0x55f343ced290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:32:16.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.860+0000 7f2d84f68700 1 --2- 172.21.15.27:0/4081846348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f343c79c30 0x55f343ced290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.860+0000 7f2d84f68700 1 -- 172.21.15.27:0/4081846348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f343c75240 msgr2=0x55f343cf2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T03:32:16.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.860+0000 7f2d84f68700 1 --2- 172.21.15.27:0/4081846348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f343c75240 0x55f343cf2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.860+0000 7f2d84f68700 1 -- 172.21.15.27:0/4081846348 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f343cedad0 con 0x55f343c75bf0 2024-05-21T03:32:16.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.860+0000 7f2d85f6a700 1 --2- 172.21.15.27:0/4081846348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f343c79c30 0x55f343ced290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:32:16.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.860+0000 7f2d85769700 1 --2- 172.21.15.27:0/4081846348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f343c75240 0x55f343cf2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:32:16.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.860+0000 7f2d84f68700 1 --2- 172.21.15.27:0/4081846348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f343c75bf0 0x55f343cf3370 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2d80079a60 tx=0x7f2d70013620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14add78aef8f14f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:32:16.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.860+0000 7f2d68ff9700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-21T03:32:16.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.860+0000 7f2d68ff9700 1 -- 172.21.15.27:0/4081846348 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d70013e50 con 0x55f343c75bf0 2024-05-21T03:32:16.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.860+0000 7f2d68ff9700 1 -- 172.21.15.27:0/4081846348 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8615+0+0 (secure 0 0 0) 0x7f2d7001e480 con 0x55f343c75bf0 2024-05-21T03:32:16.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.860+0000 7f2d92263b80 1 -- 172.21.15.27:0/4081846348 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d780b1f00 con 0x55f343c75bf0 2024-05-21T03:32:16.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.860+0000 7f2d68ff9700 1 -- 172.21.15.27:0/4081846348 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f2d7001e680 con 0x55f343c75bf0 2024-05-21T03:32:16.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.860+0000 7f2d92263b80 10 client.15864 fetch_fsmap learned FSMap version 44 2024-05-21T03:32:16.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.860+0000 7f2d92263b80 1 -- 172.21.15.27:0/4081846348 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55f343cfa740 con 0x55f343c75bf0 2024-05-21T03:32:16.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.861+0000 7f2d68ff9700 1 -- 172.21.15.27:0/4081846348 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d780b1f00 con 0x55f343c75bf0 2024-05-21T03:32:16.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.861+0000 7f2d68ff9700 1 -- 172.21.15.27:0/4081846348 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 44) ==== 8418+0+0 (secure 0 0 0) 0x7f2d70015990 con 0x55f343c75bf0 2024-05-21T03:32:16.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.861+0000 7f2d92263b80 10 client.15864 fetch_fsmap finished waiting for FSMap version 44 2024-05-21T03:32:16.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.861+0000 7f2d92263b80 10 client.15864 resolve_mds: validated gid 24476 aka daemon mds.b 2024-05-21T03:32:16.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.861+0000 7f2d92263b80 20 client.15864 populate_metadata read hostname 'smithi027' 2024-05-21T03:32:16.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.861+0000 7f2d92263b80 1 --2- 172.21.15.27:0/4081846348 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55f343cfae80 0x55f343cfd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:32:16.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.861+0000 7f2d92263b80 4 client.15864 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-05-21T03:32:16.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.861+0000 7f2d92263b80 1 -- 172.21.15.27:0/4081846348 --> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55f343bcdc50 con 0x55f343cfae80 2024-05-21T03:32:16.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.861+0000 7f2d85769700 1 --2- 172.21.15.27:0/4081846348 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55f343cfae80 0x55f343cfd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:32:16.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.862+0000 7f2d85769700 1 --2- 172.21.15.27:0/4081846348 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55f343cfae80 0x55f343cfd2a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b1025af15f5173d server_cookie=7e62338748c3058d in_seq=0 out_seq=0 2024-05-21T03:32:16.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.862+0000 7f2d68ff9700 10 client.15864 ms_handle_connect on v2:172.21.15.39:6838/287607819 2024-05-21T03:32:16.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.862+0000 7f2d68ff9700 1 -- 172.21.15.27:0/4081846348 <== mds.0 v2:172.21.15.39:6838/287607819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55f343bcdc50 con 0x55f343cfae80 2024-05-21T03:32:16.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.862+0000 7f2d68ff9700 10 client.15864 handle_command_reply: tid=0 2024-05-21T03:32:16.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.862+0000 7f2d92263b80 1 client.15864 shutdown 2024-05-21T03:32:16.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.862+0000 7f2d92263b80 1 -- 172.21.15.27:0/4081846348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f343c75bf0 msgr2=0x55f343cf3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:32:16.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.862+0000 7f2d92263b80 1 --2- 172.21.15.27:0/4081846348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f343c75bf0 0x55f343cf3370 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2d80079a60 tx=0x7f2d70013620 comp rx=0 tx=0).stop 2024-05-21T03:32:16.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.862+0000 7f2d92263b80 1 -- 172.21.15.27:0/4081846348 shutdown_connections 2024-05-21T03:32:16.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.862+0000 7f2d92263b80 1 --2- 172.21.15.27:0/4081846348 >> [v2:172.21.15.39:6838/287607819,v1:172.21.15.39:6839/287607819] conn(0x55f343cfae80 0x55f343cfd2a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.862+0000 7f2d92263b80 1 --2- 172.21.15.27:0/4081846348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f343c79c30 0x55f343ced290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.862+0000 7f2d92263b80 1 --2- 172.21.15.27:0/4081846348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f343c75240 0x55f343cf2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.862+0000 7f2d92263b80 1 --2- 172.21.15.27:0/4081846348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f343c75bf0 0x55f343cf3370 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.862+0000 7f2d92263b80 1 -- 172.21.15.27:0/4081846348 >> 172.21.15.27:0/4081846348 conn(0x55f343b9a0f0 msgr2=0x55f343c6b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:32:16.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.862+0000 7f2d68ff9700 0 client.15864 ms_handle_reset on v2:172.21.15.39:6838/287607819 2024-05-21T03:32:16.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.862+0000 7f2d92263b80 1 -- 172.21.15.27:0/4081846348 shutdown_connections 2024-05-21T03:32:16.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.862+0000 7f2d92263b80 1 -- 172.21.15.27:0/4081846348 wait complete. 2024-05-21T03:32:16.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.862+0000 7f2d92263b80 20 client.15864 trim_cache size 0 max 16384 2024-05-21T03:32:16.867 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:32:16.867 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 44, 2024-05-21T03:32:16.867 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24476, 2024-05-21T03:32:16.867 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 21, 2024-05-21T03:32:16.868 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": {} 2024-05-21T03:32:16.868 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:32:16.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.863+0000 7f2d879cd700 1 -- 172.21.15.27:0/935197288 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f2d6c0b29f0 msgr2=0x7f2d6c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:32:16.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.863+0000 7f2d879cd700 1 --2- 172.21.15.27:0/935197288 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f2d6c0b29f0 0x7f2d6c0b4eb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00b5c0 tx=0x7f2d7c005960 comp rx=0 tx=0).stop 2024-05-21T03:32:16.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.863+0000 7f2d879cd700 1 -- 172.21.15.27:0/935197288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d801095b0 msgr2=0x7f2d80102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:32:16.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.863+0000 7f2d879cd700 1 --2- 172.21.15.27:0/935197288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d801095b0 0x7f2d80102fc0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2d78007ee0 tx=0x7f2d7800d3b0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.863+0000 7f2d879cd700 1 -- 172.21.15.27:0/935197288 shutdown_connections 2024-05-21T03:32:16.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.863+0000 7f2d879cd700 1 --2- 172.21.15.27:0/935197288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d801095b0 0x7f2d80102fc0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.863+0000 7f2d879cd700 1 --2- 172.21.15.27:0/935197288 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d8006bc70 0x7f2d80102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.863+0000 7f2d879cd700 1 --2- 172.21.15.27:0/935197288 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f2d6c0b29f0 0x7f2d6c0b4eb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.863+0000 7f2d879cd700 1 --2- 172.21.15.27:0/935197288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d8006c630 0x7f2d80102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:32:16.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.863+0000 7f2d879cd700 1 -- 172.21.15.27:0/935197288 >> 172.21.15.27:0/935197288 conn(0x7f2d800793f0 msgr2=0x7f2d801011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:32:16.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.863+0000 7f2d879cd700 1 -- 172.21.15.27:0/935197288 shutdown_connections 2024-05-21T03:32:16.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:32:16.863+0000 7f2d879cd700 1 -- 172.21.15.27:0/935197288 wait complete. 2024-05-21T03:32:17.606 INFO:tasks.quiescer:done joining Quiescers 2024-05-21T03:32:17.606 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-05-21T03:32:17.616 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-05-21T03:32:17.617 INFO:teuthology.task.exec:Executing custom commands... 2024-05-21T03:32:17.617 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-05-21T03:32:17.617 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-05-21T03:32:17.649 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-05-21T03:32:18.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:18 smithi027 ceph-mon[32179]: pgmap v254: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 211 KiB/s rd, 253 KiB/s wr, 226 op/s 2024-05-21T03:32:19.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:18 smithi039 ceph-mon[36429]: pgmap v254: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 211 KiB/s rd, 253 KiB/s wr, 226 op/s 2024-05-21T03:32:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:18 smithi165 ceph-mon[36269]: pgmap v254: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 211 KiB/s rd, 253 KiB/s wr, 226 op/s 2024-05-21T03:32:20.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:20 smithi027 ceph-mon[32179]: pgmap v255: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 186 KiB/s rd, 215 KiB/s wr, 191 op/s 2024-05-21T03:32:21.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:20 smithi165 ceph-mon[36269]: pgmap v255: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 186 KiB/s rd, 215 KiB/s wr, 191 op/s 2024-05-21T03:32:21.022 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:20 smithi039 ceph-mon[36429]: pgmap v255: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 186 KiB/s rd, 215 KiB/s wr, 191 op/s 2024-05-21T03:32:22.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:22 smithi027 ceph-mon[32179]: pgmap v256: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 164 KiB/s rd, 191 KiB/s wr, 41 op/s 2024-05-21T03:32:23.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:22 smithi039 ceph-mon[36429]: pgmap v256: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 164 KiB/s rd, 191 KiB/s wr, 41 op/s 2024-05-21T03:32:23.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:22 smithi165 ceph-mon[36269]: pgmap v256: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 164 KiB/s rd, 191 KiB/s wr, 41 op/s 2024-05-21T03:32:24.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:24 smithi039 ceph-mon[36429]: osdmap e87: 12 total, 12 up, 12 in 2024-05-21T03:32:24.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:24 smithi039 ceph-mon[36429]: pgmap v258: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 14 op/s 2024-05-21T03:32:24.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:24 smithi165 ceph-mon[36269]: osdmap e87: 12 total, 12 up, 12 in 2024-05-21T03:32:24.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:24 smithi165 ceph-mon[36269]: pgmap v258: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 14 op/s 2024-05-21T03:32:24.851 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:24 smithi027 ceph-mon[32179]: osdmap e87: 12 total, 12 up, 12 in 2024-05-21T03:32:24.851 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:24 smithi027 ceph-mon[32179]: pgmap v258: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 14 op/s 2024-05-21T03:32:26.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:26 smithi027 ceph-mon[32179]: pgmap v259: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-05-21T03:32:27.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:26 smithi039 ceph-mon[36429]: pgmap v259: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-05-21T03:32:27.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:26 smithi165 ceph-mon[36269]: pgmap v259: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-05-21T03:32:28.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:28 smithi027 ceph-mon[32179]: pgmap v260: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 14 KiB/s rd, 9.5 KiB/s wr, 11 op/s 2024-05-21T03:32:29.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:28 smithi039 ceph-mon[36429]: pgmap v260: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 14 KiB/s rd, 9.5 KiB/s wr, 11 op/s 2024-05-21T03:32:29.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:28 smithi165 ceph-mon[36269]: pgmap v260: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 14 KiB/s rd, 9.5 KiB/s wr, 11 op/s 2024-05-21T03:32:30.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:30 smithi027 ceph-mon[32179]: pgmap v261: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-05-21T03:32:31.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:30 smithi039 ceph-mon[36429]: pgmap v261: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-05-21T03:32:31.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:30 smithi165 ceph-mon[36269]: pgmap v261: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-05-21T03:32:32.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:32 smithi027 ceph-mon[32179]: pgmap v262: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.8 KiB/s wr, 10 op/s 2024-05-21T03:32:33.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:32 smithi039 ceph-mon[36429]: pgmap v262: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.8 KiB/s wr, 10 op/s 2024-05-21T03:32:33.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:32 smithi165 ceph-mon[36269]: pgmap v262: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.8 KiB/s wr, 10 op/s 2024-05-21T03:32:34.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:34 smithi027 ceph-mon[32179]: pgmap v263: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-05-21T03:32:35.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:34 smithi039 ceph-mon[36429]: pgmap v263: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-05-21T03:32:35.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:34 smithi165 ceph-mon[36269]: pgmap v263: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-05-21T03:32:36.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:36 smithi027 ceph-mon[32179]: pgmap v264: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 10 KiB/s rd, 4.0 KiB/s wr, 12 op/s 2024-05-21T03:32:37.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:36 smithi039 ceph-mon[36429]: pgmap v264: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 10 KiB/s rd, 4.0 KiB/s wr, 12 op/s 2024-05-21T03:32:37.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:36 smithi165 ceph-mon[36269]: pgmap v264: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 10 KiB/s rd, 4.0 KiB/s wr, 12 op/s 2024-05-21T03:32:38.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:38 smithi027 ceph-mon[32179]: pgmap v265: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-21T03:32:39.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:38 smithi039 ceph-mon[36429]: pgmap v265: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-21T03:32:39.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:38 smithi165 ceph-mon[36269]: pgmap v265: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-21T03:32:40.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:40 smithi027 ceph-mon[32179]: pgmap v266: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-05-21T03:32:41.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:40 smithi039 ceph-mon[36429]: pgmap v266: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-05-21T03:32:41.095 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:40 smithi165 ceph-mon[36269]: pgmap v266: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-05-21T03:32:42.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:42 smithi027 ceph-mon[32179]: pgmap v267: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-05-21T03:32:43.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:42 smithi039 ceph-mon[36429]: pgmap v267: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-05-21T03:32:43.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:42 smithi165 ceph-mon[36269]: pgmap v267: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-05-21T03:32:44.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:44 smithi027 ceph-mon[32179]: pgmap v268: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:32:45.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:44 smithi039 ceph-mon[36429]: pgmap v268: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:32:45.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:44 smithi165 ceph-mon[36269]: pgmap v268: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:32:46.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:46 smithi027 ceph-mon[32179]: pgmap v269: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T03:32:47.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:46 smithi039 ceph-mon[36429]: pgmap v269: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T03:32:47.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:46 smithi165 ceph-mon[36269]: pgmap v269: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T03:32:48.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:48 smithi027 ceph-mon[32179]: pgmap v270: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:32:49.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:48 smithi039 ceph-mon[36429]: pgmap v270: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:32:49.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:48 smithi165 ceph-mon[36269]: pgmap v270: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:32:50.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:50 smithi027 ceph-mon[32179]: pgmap v271: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:32:51.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:50 smithi039 ceph-mon[36429]: pgmap v271: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:32:51.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:50 smithi165 ceph-mon[36269]: pgmap v271: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:32:52.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:52 smithi027 ceph-mon[32179]: pgmap v272: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-21T03:32:53.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:52 smithi039 ceph-mon[36429]: pgmap v272: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-21T03:32:53.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:52 smithi165 ceph-mon[36269]: pgmap v272: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-21T03:32:54.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:54 smithi027 ceph-mon[32179]: pgmap v273: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:32:55.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:54 smithi039 ceph-mon[36429]: pgmap v273: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:32:55.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:54 smithi165 ceph-mon[36269]: pgmap v273: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:32:56.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:56 smithi027 ceph-mon[32179]: pgmap v274: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T03:32:57.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:56 smithi039 ceph-mon[36429]: pgmap v274: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T03:32:57.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:56 smithi165 ceph-mon[36269]: pgmap v274: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T03:32:58.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:32:58 smithi027 ceph-mon[32179]: pgmap v275: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:32:59.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:32:58 smithi039 ceph-mon[36429]: pgmap v275: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:32:59.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:32:58 smithi165 ceph-mon[36269]: pgmap v275: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:33:00.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:00 smithi027 ceph-mon[32179]: pgmap v276: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:33:01.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:00 smithi039 ceph-mon[36429]: pgmap v276: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:33:01.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:00 smithi165 ceph-mon[36269]: pgmap v276: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:33:02.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:02 smithi027 ceph-mon[32179]: pgmap v277: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-05-21T03:33:03.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:02 smithi039 ceph-mon[36429]: pgmap v277: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-05-21T03:33:03.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:02 smithi165 ceph-mon[36269]: pgmap v277: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-05-21T03:33:04.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:04 smithi027 ceph-mon[32179]: pgmap v278: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-05-21T03:33:05.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:04 smithi039 ceph-mon[36429]: pgmap v278: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-05-21T03:33:05.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:04 smithi165 ceph-mon[36269]: pgmap v278: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-05-21T03:33:06.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:06 smithi027 ceph-mon[32179]: pgmap v279: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-05-21T03:33:07.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:06 smithi039 ceph-mon[36429]: pgmap v279: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-05-21T03:33:07.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:06 smithi165 ceph-mon[36269]: pgmap v279: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-05-21T03:33:08.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:08 smithi027 ceph-mon[32179]: pgmap v280: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-05-21T03:33:09.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:08 smithi039 ceph-mon[36429]: pgmap v280: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-05-21T03:33:09.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:08 smithi165 ceph-mon[36269]: pgmap v280: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-05-21T03:33:10.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:10 smithi027 ceph-mon[32179]: pgmap v281: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-05-21T03:33:11.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:10 smithi039 ceph-mon[36429]: pgmap v281: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-05-21T03:33:11.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:10 smithi165 ceph-mon[36269]: pgmap v281: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-05-21T03:33:12.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:11 smithi039 ceph-mon[36429]: osdmap e88: 12 total, 12 up, 12 in 2024-05-21T03:33:12.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:11 smithi165 ceph-mon[36269]: osdmap e88: 12 total, 12 up, 12 in 2024-05-21T03:33:12.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:11 smithi027 ceph-mon[32179]: osdmap e88: 12 total, 12 up, 12 in 2024-05-21T03:33:13.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:12 smithi039 ceph-mon[36429]: pgmap v283: 97 pgs: 2 active+clean+snaptrim, 89 active+clean, 6 active+clean+snaptrim_wait; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-21T03:33:13.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:12 smithi165 ceph-mon[36269]: pgmap v283: 97 pgs: 2 active+clean+snaptrim, 89 active+clean, 6 active+clean+snaptrim_wait; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-21T03:33:13.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:12 smithi027 ceph-mon[32179]: pgmap v283: 97 pgs: 2 active+clean+snaptrim, 89 active+clean, 6 active+clean+snaptrim_wait; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-21T03:33:15.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:14 smithi039 ceph-mon[36429]: pgmap v284: 97 pgs: 2 active+clean+snaptrim, 89 active+clean, 6 active+clean+snaptrim_wait; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-21T03:33:15.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:14 smithi165 ceph-mon[36269]: pgmap v284: 97 pgs: 2 active+clean+snaptrim, 89 active+clean, 6 active+clean+snaptrim_wait; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-21T03:33:15.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:14 smithi027 ceph-mon[32179]: pgmap v284: 97 pgs: 2 active+clean+snaptrim, 89 active+clean, 6 active+clean+snaptrim_wait; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-21T03:33:17.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:16 smithi039 ceph-mon[36429]: pgmap v285: 97 pgs: 14 active+clean+snaptrim, 58 active+clean, 25 active+clean+snaptrim_wait; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T03:33:17.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:16 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:33:17.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:16 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:33:17.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:16 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:33:17.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:16 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:33:17.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:16 smithi165 ceph-mon[36269]: pgmap v285: 97 pgs: 14 active+clean+snaptrim, 58 active+clean, 25 active+clean+snaptrim_wait; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T03:33:17.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:16 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:33:17.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:16 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:33:17.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:16 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:33:17.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:16 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:33:17.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:16 smithi027 ceph-mon[32179]: pgmap v285: 97 pgs: 14 active+clean+snaptrim, 58 active+clean, 25 active+clean+snaptrim_wait; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T03:33:17.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:16 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:33:17.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:16 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:33:17.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:16 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:33:17.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:16 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:33:19.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:18 smithi039 ceph-mon[36429]: pgmap v286: 97 pgs: 16 active+clean+snaptrim, 50 active+clean, 31 active+clean+snaptrim_wait; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T03:33:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:18 smithi165 ceph-mon[36269]: pgmap v286: 97 pgs: 16 active+clean+snaptrim, 50 active+clean, 31 active+clean+snaptrim_wait; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T03:33:19.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:18 smithi027 ceph-mon[32179]: pgmap v286: 97 pgs: 16 active+clean+snaptrim, 50 active+clean, 31 active+clean+snaptrim_wait; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T03:33:21.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:20 smithi039 ceph-mon[36429]: pgmap v287: 97 pgs: 13 active+clean+snaptrim, 55 active+clean, 29 active+clean+snaptrim_wait; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-05-21T03:33:21.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:20 smithi165 ceph-mon[36269]: pgmap v287: 97 pgs: 13 active+clean+snaptrim, 55 active+clean, 29 active+clean+snaptrim_wait; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-05-21T03:33:21.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:20 smithi027 ceph-mon[32179]: pgmap v287: 97 pgs: 13 active+clean+snaptrim, 55 active+clean, 29 active+clean+snaptrim_wait; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-05-21T03:33:23.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:22 smithi039 ceph-mon[36429]: pgmap v288: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.7 KiB/s rd, 188 B/s wr, 9 op/s 2024-05-21T03:33:23.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:22 smithi165 ceph-mon[36269]: pgmap v288: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.7 KiB/s rd, 188 B/s wr, 9 op/s 2024-05-21T03:33:23.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:22 smithi027 ceph-mon[32179]: pgmap v288: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.7 KiB/s rd, 188 B/s wr, 9 op/s 2024-05-21T03:33:24.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:24 smithi039 ceph-mon[36429]: osdmap e89: 12 total, 12 up, 12 in 2024-05-21T03:33:24.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:24 smithi039 ceph-mon[36429]: pgmap v290: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T03:33:24.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:24 smithi165 ceph-mon[36269]: osdmap e89: 12 total, 12 up, 12 in 2024-05-21T03:33:24.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:24 smithi165 ceph-mon[36269]: pgmap v290: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T03:33:24.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:24 smithi027 ceph-mon[32179]: osdmap e89: 12 total, 12 up, 12 in 2024-05-21T03:33:24.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:24 smithi027 ceph-mon[32179]: pgmap v290: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T03:33:26.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:26 smithi027 ceph-mon[32179]: pgmap v291: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T03:33:27.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:26 smithi039 ceph-mon[36429]: pgmap v291: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T03:33:27.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:26 smithi165 ceph-mon[36269]: pgmap v291: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T03:33:28.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:28 smithi027 ceph-mon[32179]: pgmap v292: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T03:33:29.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:28 smithi039 ceph-mon[36429]: pgmap v292: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T03:33:29.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:28 smithi165 ceph-mon[36269]: pgmap v292: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T03:33:30.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:30 smithi027 ceph-mon[32179]: pgmap v293: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T03:33:31.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:30 smithi039 ceph-mon[36429]: pgmap v293: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T03:33:31.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:30 smithi165 ceph-mon[36269]: pgmap v293: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T03:33:32.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:32 smithi027 ceph-mon[32179]: pgmap v294: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T03:33:33.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:32 smithi039 ceph-mon[36429]: pgmap v294: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T03:33:33.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:32 smithi165 ceph-mon[36269]: pgmap v294: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T03:33:34.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:34 smithi027 ceph-mon[32179]: pgmap v295: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-21T03:33:35.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:34 smithi039 ceph-mon[36429]: pgmap v295: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-21T03:33:35.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:34 smithi165 ceph-mon[36269]: pgmap v295: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-21T03:33:36.929 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:36 smithi027 ceph-mon[32179]: pgmap v296: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T03:33:37.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:36 smithi039 ceph-mon[36429]: pgmap v296: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T03:33:37.024 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:36 smithi165 ceph-mon[36269]: pgmap v296: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T03:33:38.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:38 smithi027 ceph-mon[32179]: pgmap v297: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:33:39.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:38 smithi039 ceph-mon[36429]: pgmap v297: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:33:39.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:38 smithi165 ceph-mon[36269]: pgmap v297: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:33:40.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:40 smithi027 ceph-mon[32179]: pgmap v298: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:33:41.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:40 smithi039 ceph-mon[36429]: pgmap v298: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:33:41.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:40 smithi165 ceph-mon[36269]: pgmap v298: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:33:42.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:42 smithi027 ceph-mon[32179]: pgmap v299: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-21T03:33:43.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:42 smithi039 ceph-mon[36429]: pgmap v299: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-21T03:33:43.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:42 smithi165 ceph-mon[36269]: pgmap v299: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-21T03:33:44.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:44 smithi027 ceph-mon[32179]: pgmap v300: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:33:45.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:44 smithi039 ceph-mon[36429]: pgmap v300: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:33:45.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:44 smithi165 ceph-mon[36269]: pgmap v300: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:33:46.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:46 smithi027 ceph-mon[32179]: pgmap v301: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T03:33:47.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:46 smithi039 ceph-mon[36429]: pgmap v301: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T03:33:47.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:46 smithi165 ceph-mon[36269]: pgmap v301: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T03:33:48.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:48 smithi027 ceph-mon[32179]: pgmap v302: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:33:49.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:48 smithi039 ceph-mon[36429]: pgmap v302: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:33:49.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:48 smithi165 ceph-mon[36269]: pgmap v302: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:33:50.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:50 smithi027 ceph-mon[32179]: pgmap v303: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:33:51.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:50 smithi039 ceph-mon[36429]: pgmap v303: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:33:51.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:50 smithi165 ceph-mon[36269]: pgmap v303: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:33:52.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:52 smithi027 ceph-mon[32179]: pgmap v304: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-21T03:33:53.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:52 smithi039 ceph-mon[36429]: pgmap v304: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-21T03:33:53.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:52 smithi165 ceph-mon[36269]: pgmap v304: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-21T03:33:54.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:54 smithi027 ceph-mon[32179]: pgmap v305: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:33:55.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:54 smithi039 ceph-mon[36429]: pgmap v305: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:33:55.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:54 smithi165 ceph-mon[36269]: pgmap v305: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:33:56.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:56 smithi027 ceph-mon[32179]: pgmap v306: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T03:33:57.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:56 smithi039 ceph-mon[36429]: pgmap v306: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T03:33:57.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:56 smithi165 ceph-mon[36269]: pgmap v306: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T03:33:58.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:33:58 smithi027 ceph-mon[32179]: pgmap v307: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:33:59.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:33:58 smithi039 ceph-mon[36429]: pgmap v307: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:33:59.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:33:58 smithi165 ceph-mon[36269]: pgmap v307: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:34:00.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:00 smithi027 ceph-mon[32179]: pgmap v308: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:34:01.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:00 smithi039 ceph-mon[36429]: pgmap v308: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:34:01.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:00 smithi165 ceph-mon[36269]: pgmap v308: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:34:02.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:02 smithi027 ceph-mon[32179]: pgmap v309: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-05-21T03:34:03.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:02 smithi039 ceph-mon[36429]: pgmap v309: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-05-21T03:34:03.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:02 smithi165 ceph-mon[36269]: pgmap v309: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-05-21T03:34:03.721 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-21T03:34:03.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.841+0000 7ffa2f0a0640 1 -- 172.21.15.27:0/993145652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa28108a40 msgr2=0x7ffa28108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:03.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.841+0000 7ffa2f0a0640 1 --2- 172.21.15.27:0/993145652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa28108a40 0x7ffa28108e20 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c009900 tx=0x7ffa1c02a410 comp rx=0 tx=0).stop 2024-05-21T03:34:03.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.842+0000 7ffa2f0a0640 1 -- 172.21.15.27:0/993145652 shutdown_connections 2024-05-21T03:34:03.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.842+0000 7ffa2f0a0640 1 --2- 172.21.15.27:0/993145652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa2806bc60 0x7ffa2810f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:03.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.842+0000 7ffa2f0a0640 1 --2- 172.21.15.27:0/993145652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa2806b2c0 0x7ffa2806b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:03.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.842+0000 7ffa2f0a0640 1 --2- 172.21.15.27:0/993145652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa28108a40 0x7ffa28108e20 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:03.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.842+0000 7ffa2f0a0640 1 -- 172.21.15.27:0/993145652 >> 172.21.15.27:0/993145652 conn(0x7ffa28077750 msgr2=0x7ffa28077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:34:03.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.842+0000 7ffa2f0a0640 1 -- 172.21.15.27:0/993145652 shutdown_connections 2024-05-21T03:34:03.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.842+0000 7ffa2f0a0640 1 -- 172.21.15.27:0/993145652 wait complete. 2024-05-21T03:34:03.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.842+0000 7ffa2f0a0640 1 Processor -- start 2024-05-21T03:34:03.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.842+0000 7ffa2f0a0640 1 -- start start 2024-05-21T03:34:03.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.842+0000 7ffa2f0a0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa2806b2c0 0x7ffa281990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:03.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.842+0000 7ffa2f0a0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa2806bc60 0x7ffa281995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:03.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.843+0000 7ffa2f0a0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa28108a40 0x7ffa2819d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:03.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.843+0000 7ffa2f0a0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ffa28112cb0 con 0x7ffa2806b2c0 2024-05-21T03:34:03.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.843+0000 7ffa2f0a0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ffa28112b30 con 0x7ffa28108a40 2024-05-21T03:34:03.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.843+0000 7ffa2f0a0640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ffa28112e30 con 0x7ffa2806bc60 2024-05-21T03:34:03.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.843+0000 7ffa2ce3c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa2806b2c0 0x7ffa281990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:03.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.843+0000 7ffa2ce3c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa2806b2c0 0x7ffa281990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38550/0 (socket says 172.21.15.27:38550) 2024-05-21T03:34:03.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.843+0000 7ffa2ce3c640 1 -- 172.21.15.27:0/1739468472 learned_addr learned my addr 172.21.15.27:0/1739468472 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:34:03.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.843+0000 7ffa2d63d640 1 --2- 172.21.15.27:0/1739468472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa28108a40 0x7ffa2819d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:03.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.843+0000 7ffa2ce3c640 1 -- 172.21.15.27:0/1739468472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa2806bc60 msgr2=0x7ffa281995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:03.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.843+0000 7ffa27fff640 1 --2- 172.21.15.27:0/1739468472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa2806bc60 0x7ffa281995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:03.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.843+0000 7ffa2ce3c640 1 --2- 172.21.15.27:0/1739468472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa2806bc60 0x7ffa281995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:03.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.843+0000 7ffa2ce3c640 1 -- 172.21.15.27:0/1739468472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa28108a40 msgr2=0x7ffa2819d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:03.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.843+0000 7ffa2ce3c640 1 --2- 172.21.15.27:0/1739468472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa28108a40 0x7ffa2819d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:03.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.843+0000 7ffa2ce3c640 1 -- 172.21.15.27:0/1739468472 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa2819e050 con 0x7ffa2806b2c0 2024-05-21T03:34:03.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.843+0000 7ffa2d63d640 1 --2- 172.21.15.27:0/1739468472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa28108a40 0x7ffa2819d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:34:03.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.843+0000 7ffa2ce3c640 1 --2- 172.21.15.27:0/1739468472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa2806b2c0 0x7ffa281990b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c006010 tx=0x7ffa1c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:34:03.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.843+0000 7ffa25ffb640 1 -- 172.21.15.27:0/1739468472 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c03f030 con 0x7ffa2806b2c0 2024-05-21T03:34:03.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.843+0000 7ffa2f0a0640 1 -- 172.21.15.27:0/1739468472 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa2819e2e0 con 0x7ffa2806b2c0 2024-05-21T03:34:03.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.843+0000 7ffa25ffb640 1 -- 172.21.15.27:0/1739468472 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa1c003800 con 0x7ffa2806b2c0 2024-05-21T03:34:03.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.843+0000 7ffa25ffb640 1 -- 172.21.15.27:0/1739468472 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c035070 con 0x7ffa2806b2c0 2024-05-21T03:34:03.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.844+0000 7ffa2f0a0640 1 -- 172.21.15.27:0/1739468472 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa281a5bd0 con 0x7ffa2806b2c0 2024-05-21T03:34:03.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.845+0000 7ffa2f0a0640 1 -- 172.21.15.27:0/1739468472 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa28104160 con 0x7ffa2806b2c0 2024-05-21T03:34:03.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.846+0000 7ffa25ffb640 1 -- 172.21.15.27:0/1739468472 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7ffa1c003cb0 con 0x7ffa2806b2c0 2024-05-21T03:34:03.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.847+0000 7ffa25ffb640 1 --2- 172.21.15.27:0/1739468472 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7ffa180b2a10 0x7ffa180b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:03.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.847+0000 7ffa27fff640 1 --2- 172.21.15.27:0/1739468472 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7ffa180b2a10 0x7ffa180b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:03.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.847+0000 7ffa25ffb640 1 -- 172.21.15.27:0/1739468472 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8571+0+0 (secure 0 0 0) 0x7ffa1c1032b0 con 0x7ffa2806b2c0 2024-05-21T03:34:03.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.847+0000 7ffa27fff640 1 --2- 172.21.15.27:0/1739468472 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7ffa180b2a10 0x7ffa180b4ed0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffa28108010 tx=0x7ffa14005cc0 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:34:03.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:03.850+0000 7ffa25ffb640 1 -- 172.21.15.27:0/1739468472 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffa1c0cac60 con 0x7ffa2806b2c0 2024-05-21T03:34:04.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.064+0000 7ffa2f0a0640 1 -- 172.21.15.27:0/1739468472 --> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7ffa28100e80 con 0x7ffa180b2a10 2024-05-21T03:34:04.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.072+0000 7ffa25ffb640 1 -- 172.21.15.27:0/1739468472 <== mgr.24913 v2:172.21.15.27:6800/1477999924 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7ffa28100e80 con 0x7ffa180b2a10 2024-05-21T03:34:04.073 INFO:teuthology.orchestra.run.smithi027.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-21T00:00:00", "created": "2024-05-21T03:24:02", "first": "2024-05-21T03:25:00", "last": "2024-05-21T03:34:00", "last_pruned": "2024-05-21T03:34:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-05-21T03:34:04.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.075+0000 7ffa2f0a0640 1 -- 172.21.15.27:0/1739468472 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7ffa180b2a10 msgr2=0x7ffa180b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:04.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.075+0000 7ffa2f0a0640 1 --2- 172.21.15.27:0/1739468472 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7ffa180b2a10 0x7ffa180b4ed0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffa28108010 tx=0x7ffa14005cc0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.075+0000 7ffa2f0a0640 1 -- 172.21.15.27:0/1739468472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa2806b2c0 msgr2=0x7ffa281990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:04.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.075+0000 7ffa2f0a0640 1 --2- 172.21.15.27:0/1739468472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa2806b2c0 0x7ffa281990b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c006010 tx=0x7ffa1c0042f0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.075+0000 7ffa2f0a0640 1 -- 172.21.15.27:0/1739468472 shutdown_connections 2024-05-21T03:34:04.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.075+0000 7ffa2f0a0640 1 --2- 172.21.15.27:0/1739468472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa28108a40 0x7ffa2819d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.075+0000 7ffa2f0a0640 1 --2- 172.21.15.27:0/1739468472 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7ffa180b2a10 0x7ffa180b4ed0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.075+0000 7ffa2f0a0640 1 --2- 172.21.15.27:0/1739468472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa2806bc60 0x7ffa281995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.075+0000 7ffa2f0a0640 1 --2- 172.21.15.27:0/1739468472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa2806b2c0 0x7ffa281990b0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.075+0000 7ffa2f0a0640 1 -- 172.21.15.27:0/1739468472 >> 172.21.15.27:0/1739468472 conn(0x7ffa28077750 msgr2=0x7ffa28100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:34:04.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.075+0000 7ffa2f0a0640 1 -- 172.21.15.27:0/1739468472 shutdown_connections 2024-05-21T03:34:04.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.075+0000 7ffa2f0a0640 1 -- 172.21.15.27:0/1739468472 wait complete. 2024-05-21T03:34:04.086 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-21T03:34:04.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.193+0000 7f9f61b46640 1 -- 172.21.15.27:0/3750556228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c102340 msgr2=0x7f9f5c10e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:04.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.193+0000 7f9f61b46640 1 --2- 172.21.15.27:0/3750556228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c102340 0x7f9f5c10e800 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9f54009cd0 tx=0x7f9f5402a8e0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.193+0000 7f9f61b46640 1 -- 172.21.15.27:0/3750556228 shutdown_connections 2024-05-21T03:34:04.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.193+0000 7f9f61b46640 1 --2- 172.21.15.27:0/3750556228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c102340 0x7f9f5c10e800 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.193+0000 7f9f61b46640 1 --2- 172.21.15.27:0/3750556228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f5c101980 0x7f9f5c101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.193+0000 7f9f61b46640 1 --2- 172.21.15.27:0/3750556228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f5c107940 0x7f9f5c107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.193+0000 7f9f61b46640 1 -- 172.21.15.27:0/3750556228 >> 172.21.15.27:0/3750556228 conn(0x7f9f5c078ad0 msgr2=0x7f9f5c078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:34:04.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.193+0000 7f9f61b46640 1 -- 172.21.15.27:0/3750556228 shutdown_connections 2024-05-21T03:34:04.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.193+0000 7f9f61b46640 1 -- 172.21.15.27:0/3750556228 wait complete. 2024-05-21T03:34:04.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.194+0000 7f9f61b46640 1 Processor -- start 2024-05-21T03:34:04.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.194+0000 7f9f61b46640 1 -- start start 2024-05-21T03:34:04.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.194+0000 7f9f61b46640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c101980 0x7f9f5c1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:04.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.194+0000 7f9f61b46640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f5c102340 0x7f9f5c199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:04.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.194+0000 7f9f61b46640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f5c107940 0x7f9f5c19da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:04.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.194+0000 7f9f61b46640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c111c00 con 0x7f9f5c101980 2024-05-21T03:34:04.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.194+0000 7f9f61b46640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c111a80 con 0x7f9f5c107940 2024-05-21T03:34:04.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.194+0000 7f9f61b46640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c111d80 con 0x7f9f5c102340 2024-05-21T03:34:04.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.194+0000 7f9f5b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c101980 0x7f9f5c1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:04.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.194+0000 7f9f5b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c101980 0x7f9f5c1991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38572/0 (socket says 172.21.15.27:38572) 2024-05-21T03:34:04.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.194+0000 7f9f5b7fe640 1 -- 172.21.15.27:0/2198348790 learned_addr learned my addr 172.21.15.27:0/2198348790 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:34:04.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.194+0000 7f9f5bfff640 1 --2- 172.21.15.27:0/2198348790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f5c107940 0x7f9f5c19da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:04.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.194+0000 7f9f5affd640 1 --2- 172.21.15.27:0/2198348790 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f5c102340 0x7f9f5c199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:04.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.194+0000 7f9f5b7fe640 1 -- 172.21.15.27:0/2198348790 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f5c102340 msgr2=0x7f9f5c199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:04.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.194+0000 7f9f5b7fe640 1 --2- 172.21.15.27:0/2198348790 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f5c102340 0x7f9f5c199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.194+0000 7f9f5b7fe640 1 -- 172.21.15.27:0/2198348790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f5c107940 msgr2=0x7f9f5c19da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:04.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.194+0000 7f9f5b7fe640 1 --2- 172.21.15.27:0/2198348790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f5c107940 0x7f9f5c19da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.194+0000 7f9f5b7fe640 1 -- 172.21.15.27:0/2198348790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5c19e160 con 0x7f9f5c101980 2024-05-21T03:34:04.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.194+0000 7f9f5bfff640 1 --2- 172.21.15.27:0/2198348790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f5c107940 0x7f9f5c19da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:34:04.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.195+0000 7f9f5b7fe640 1 --2- 172.21.15.27:0/2198348790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c101980 0x7f9f5c1991c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00c970 tx=0x7f9f4c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:34:04.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.195+0000 7f9f58ff9640 1 -- 172.21.15.27:0/2198348790 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c007bb0 con 0x7f9f5c101980 2024-05-21T03:34:04.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.195+0000 7f9f61b46640 1 -- 172.21.15.27:0/2198348790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f5c19e450 con 0x7f9f5c101980 2024-05-21T03:34:04.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.195+0000 7f9f58ff9640 1 -- 172.21.15.27:0/2198348790 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f4c007d50 con 0x7f9f5c101980 2024-05-21T03:34:04.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.195+0000 7f9f58ff9640 1 -- 172.21.15.27:0/2198348790 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c00f450 con 0x7f9f5c101980 2024-05-21T03:34:04.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.195+0000 7f9f61b46640 1 -- 172.21.15.27:0/2198348790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f5c1a5ce0 con 0x7f9f5c101980 2024-05-21T03:34:04.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.196+0000 7f9f61b46640 1 -- 172.21.15.27:0/2198348790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f5c0507e0 con 0x7f9f5c101980 2024-05-21T03:34:04.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.197+0000 7f9f58ff9640 1 -- 172.21.15.27:0/2198348790 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7f9f4c004f20 con 0x7f9f5c101980 2024-05-21T03:34:04.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.198+0000 7f9f58ff9640 1 --2- 172.21.15.27:0/2198348790 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f9f480b2a10 0x7f9f480b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:04.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.198+0000 7f9f5affd640 1 --2- 172.21.15.27:0/2198348790 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f9f480b2a10 0x7f9f480b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:04.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.198+0000 7f9f58ff9640 1 -- 172.21.15.27:0/2198348790 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8571+0+0 (secure 0 0 0) 0x7f9f4c0e23d0 con 0x7f9f5c101980 2024-05-21T03:34:04.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.199+0000 7f9f5affd640 1 --2- 172.21.15.27:0/2198348790 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f9f480b2a10 0x7f9f480b4ed0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c06b330 tx=0x7f9f44005c60 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:34:04.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.202+0000 7f9f58ff9640 1 -- 172.21.15.27:0/2198348790 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9f4c014030 con 0x7f9f5c101980 2024-05-21T03:34:04.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.417+0000 7f9f61b46640 1 -- 172.21.15.27:0/2198348790 --> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f9f5c103b40 con 0x7f9f480b2a10 2024-05-21T03:34:04.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.424+0000 7f9f58ff9640 1 -- 172.21.15.27:0/2198348790 <== mgr.24913 v2:172.21.15.27:6800/1477999924 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f9f5c103b40 con 0x7f9f480b2a10 2024-05-21T03:34:04.424 INFO:teuthology.orchestra.run.smithi027.stdout:/ 1m 6m3h 2024-05-21T03:34:04.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.426+0000 7f9f61b46640 1 -- 172.21.15.27:0/2198348790 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f9f480b2a10 msgr2=0x7f9f480b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:04.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.426+0000 7f9f61b46640 1 --2- 172.21.15.27:0/2198348790 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f9f480b2a10 0x7f9f480b4ed0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c06b330 tx=0x7f9f44005c60 comp rx=0 tx=0).stop 2024-05-21T03:34:04.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.426+0000 7f9f61b46640 1 -- 172.21.15.27:0/2198348790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c101980 msgr2=0x7f9f5c1991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:04.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.426+0000 7f9f61b46640 1 --2- 172.21.15.27:0/2198348790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c101980 0x7f9f5c1991c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00c970 tx=0x7f9f4c00ce30 comp rx=0 tx=0).stop 2024-05-21T03:34:04.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.426+0000 7f9f61b46640 1 -- 172.21.15.27:0/2198348790 shutdown_connections 2024-05-21T03:34:04.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.426+0000 7f9f61b46640 1 --2- 172.21.15.27:0/2198348790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f5c107940 0x7f9f5c19da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.426+0000 7f9f61b46640 1 --2- 172.21.15.27:0/2198348790 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f9f480b2a10 0x7f9f480b4ed0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.426+0000 7f9f61b46640 1 --2- 172.21.15.27:0/2198348790 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f5c102340 0x7f9f5c199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.426+0000 7f9f61b46640 1 --2- 172.21.15.27:0/2198348790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c101980 0x7f9f5c1991c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.426+0000 7f9f61b46640 1 -- 172.21.15.27:0/2198348790 >> 172.21.15.27:0/2198348790 conn(0x7f9f5c078ad0 msgr2=0x7f9f5c103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:34:04.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.426+0000 7f9f61b46640 1 -- 172.21.15.27:0/2198348790 shutdown_connections 2024-05-21T03:34:04.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.426+0000 7f9f61b46640 1 -- 172.21.15.27:0/2198348790 wait complete. 2024-05-21T03:34:04.437 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-21T03:34:04.522 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T03:34:04.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.628+0000 7f945b9af640 1 -- 172.21.15.27:0/2642442126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f945410f460 msgr2=0x7f9454111850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:04.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.628+0000 7f945b9af640 1 --2- 172.21.15.27:0/2642442126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f945410f460 0x7f9454111850 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f944c009ca0 tx=0x7f944c02a8c0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.628+0000 7f945b9af640 1 -- 172.21.15.27:0/2642442126 shutdown_connections 2024-05-21T03:34:04.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.628+0000 7f945b9af640 1 --2- 172.21.15.27:0/2642442126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f945410f460 0x7f9454111850 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.628+0000 7f945b9af640 1 --2- 172.21.15.27:0/2642442126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94541021d0 0x7f945410ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.628+0000 7f945b9af640 1 --2- 172.21.15.27:0/2642442126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94541018b0 0x7f9454101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.628+0000 7f945b9af640 1 -- 172.21.15.27:0/2642442126 >> 172.21.15.27:0/2642442126 conn(0x7f9454078ab0 msgr2=0x7f9454078ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:34:04.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.628+0000 7f945b9af640 1 -- 172.21.15.27:0/2642442126 shutdown_connections 2024-05-21T03:34:04.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.628+0000 7f945b9af640 1 -- 172.21.15.27:0/2642442126 wait complete. 2024-05-21T03:34:04.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.628+0000 7f945b9af640 1 Processor -- start 2024-05-21T03:34:04.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.628+0000 7f945b9af640 1 -- start start 2024-05-21T03:34:04.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.629+0000 7f945b9af640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94541018b0 0x7f945419b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:04.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.629+0000 7f945b9af640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94541021d0 0x7f945419ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:04.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.629+0000 7f945b9af640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f945410f460 0x7f945419fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:04.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.629+0000 7f945b9af640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9454114f20 con 0x7f94541018b0 2024-05-21T03:34:04.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.629+0000 7f945b9af640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9454114da0 con 0x7f94541021d0 2024-05-21T03:34:04.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.629+0000 7f945b9af640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f94541150a0 con 0x7f945410f460 2024-05-21T03:34:04.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.629+0000 7f945974b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94541018b0 0x7f945419b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:04.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.629+0000 7f945974b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94541018b0 0x7f945419b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38590/0 (socket says 172.21.15.27:38590) 2024-05-21T03:34:04.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.629+0000 7f945974b640 1 -- 172.21.15.27:0/3621301769 learned_addr learned my addr 172.21.15.27:0/3621301769 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:34:04.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.629+0000 7f9458f4a640 1 --2- 172.21.15.27:0/3621301769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94541021d0 0x7f945419ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:04.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.629+0000 7f9459f4c640 1 --2- 172.21.15.27:0/3621301769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f945410f460 0x7f945419fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:04.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.629+0000 7f945974b640 1 -- 172.21.15.27:0/3621301769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f945410f460 msgr2=0x7f945419fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:04.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.629+0000 7f945974b640 1 --2- 172.21.15.27:0/3621301769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f945410f460 0x7f945419fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.629+0000 7f945974b640 1 -- 172.21.15.27:0/3621301769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94541021d0 msgr2=0x7f945419ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:04.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.629+0000 7f945974b640 1 --2- 172.21.15.27:0/3621301769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94541021d0 0x7f945419ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.629+0000 7f945974b640 1 -- 172.21.15.27:0/3621301769 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94541a0540 con 0x7f94541018b0 2024-05-21T03:34:04.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.629+0000 7f9458f4a640 1 --2- 172.21.15.27:0/3621301769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94541021d0 0x7f945419ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:34:04.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.629+0000 7f945974b640 1 --2- 172.21.15.27:0/3621301769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94541018b0 0x7f945419b550 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f945000d770 tx=0x7f945000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:34:04.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.629+0000 7f944a7fc640 1 -- 172.21.15.27:0/3621301769 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450017070 con 0x7f94541018b0 2024-05-21T03:34:04.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.629+0000 7f944a7fc640 1 -- 172.21.15.27:0/3621301769 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94500054d0 con 0x7f94541018b0 2024-05-21T03:34:04.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.629+0000 7f945b9af640 1 -- 172.21.15.27:0/3621301769 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94541a0830 con 0x7f94541018b0 2024-05-21T03:34:04.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.629+0000 7f944a7fc640 1 -- 172.21.15.27:0/3621301769 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450004c30 con 0x7f94541018b0 2024-05-21T03:34:04.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.629+0000 7f945b9af640 1 -- 172.21.15.27:0/3621301769 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94541a80c0 con 0x7f94541018b0 2024-05-21T03:34:04.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.630+0000 7f945b9af640 1 -- 172.21.15.27:0/3621301769 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9454069cc0 con 0x7f94541018b0 2024-05-21T03:34:04.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.632+0000 7f944a7fc640 1 -- 172.21.15.27:0/3621301769 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7f9450004dd0 con 0x7f94541018b0 2024-05-21T03:34:04.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.633+0000 7f944a7fc640 1 --2- 172.21.15.27:0/3621301769 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f94400b29e0 0x7f94400b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:04.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.633+0000 7f9458f4a640 1 --2- 172.21.15.27:0/3621301769 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f94400b29e0 0x7f94400b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:04.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.633+0000 7f944a7fc640 1 -- 172.21.15.27:0/3621301769 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8571+0+0 (secure 0 0 0) 0x7f94500e2530 con 0x7f94541018b0 2024-05-21T03:34:04.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.633+0000 7f9458f4a640 1 --2- 172.21.15.27:0/3621301769 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f94400b29e0 0x7f94400b4ea0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9454103320 tx=0x7f9444005f90 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:34:04.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.636+0000 7f944a7fc640 1 -- 172.21.15.27:0/3621301769 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f94500a9f60 con 0x7f94541018b0 2024-05-21T03:34:04.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.830+0000 7f945b9af640 1 -- 172.21.15.27:0/3621301769 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f945410a5f0 con 0x7f94541018b0 2024-05-21T03:34:04.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.831+0000 7f944a7fc640 1 -- 172.21.15.27:0/3621301769 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23150 (secure 0 0 0) 0x7f94500a9860 con 0x7f94541018b0 2024-05-21T03:34:04.832 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:34:04.832 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":89,"fsid":"c9ca94be-171f-11ef-bc9a-c7b262605968","created":"2024-05-21T03:12:18.217236+0000","modified":"2024-05-21T03:33:23.436025+0000","last_up_change":"2024-05-21T03:19:30.382150+0000","last_in_change":"2024-05-21T03:19:09.058249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T03:15:41.340194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T03:20:27.507346+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T03:20:30.830169+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":13,"snap_epoch":88,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c745c09f-9a76-4d05-af75-c7554f6d5e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6803","nonce":1399996449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6805","nonce":1399996449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6809","nonce":1399996449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6807","nonce":1399996449}]},"public_addr":"172.21.15.27:6803/1399996449","cluster_addr":"172.21.15.27:6805/1399996449","heartbeat_back_addr":"172.21.15.27:6809/1399996449","heartbeat_front_addr":"172.21.15.27:6807/1399996449","state":["exists","up"]},{"osd":1,"uuid":"8b73aa00-4364-438f-9722-53caa4e9bec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6801","nonce":1776318847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6803","nonce":1776318847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6807","nonce":1776318847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6805","nonce":1776318847}]},"public_addr":"172.21.15.39:6801/1776318847","cluster_addr":"172.21.15.39:6803/1776318847","heartbeat_back_addr":"172.21.15.39:6807/1776318847","heartbeat_front_addr":"172.21.15.39:6805/1776318847","state":["exists","up"]},{"osd":2,"uuid":"0822043e-7d3c-4e98-9dc0-94b98b4d562a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6801","nonce":703322625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6803","nonce":703322625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6807","nonce":703322625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6805","nonce":703322625}]},"public_addr":"172.21.15.165:6801/703322625","cluster_addr":"172.21.15.165:6803/703322625","heartbeat_back_addr":"172.21.15.165:6807/703322625","heartbeat_front_addr":"172.21.15.165:6805/703322625","state":["exists","up"]},{"osd":3,"uuid":"c563b1b2-efcb-4f8c-8121-c1938f83131e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6811","nonce":1933403135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6813","nonce":1933403135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6817","nonce":1933403135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6815","nonce":1933403135}]},"public_addr":"172.21.15.27:6811/1933403135","cluster_addr":"172.21.15.27:6813/1933403135","heartbeat_back_addr":"172.21.15.27:6817/1933403135","heartbeat_front_addr":"172.21.15.27:6815/1933403135","state":["exists","up"]},{"osd":4,"uuid":"0c793b87-b045-4b4a-89e7-87ec36deeb0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6809","nonce":155579331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6811","nonce":155579331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6815","nonce":155579331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6813","nonce":155579331}]},"public_addr":"172.21.15.39:6809/155579331","cluster_addr":"172.21.15.39:6811/155579331","heartbeat_back_addr":"172.21.15.39:6815/155579331","heartbeat_front_addr":"172.21.15.39:6813/155579331","state":["exists","up"]},{"osd":5,"uuid":"367eeb5a-9d75-4644-bf46-317c03f33d8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6809","nonce":386850244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6811","nonce":386850244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6815","nonce":386850244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6813","nonce":386850244}]},"public_addr":"172.21.15.165:6809/386850244","cluster_addr":"172.21.15.165:6811/386850244","heartbeat_back_addr":"172.21.15.165:6815/386850244","heartbeat_front_addr":"172.21.15.165:6813/386850244","state":["exists","up"]},{"osd":6,"uuid":"e000167f-24fd-4115-83f0-8afbfa8d8991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6819","nonce":3531376679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6821","nonce":3531376679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6825","nonce":3531376679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6823","nonce":3531376679}]},"public_addr":"172.21.15.27:6819/3531376679","cluster_addr":"172.21.15.27:6821/3531376679","heartbeat_back_addr":"172.21.15.27:6825/3531376679","heartbeat_front_addr":"172.21.15.27:6823/3531376679","state":["exists","up"]},{"osd":7,"uuid":"4b8d486b-868e-4f33-98a1-b99686adeaaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6817","nonce":1550035852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6819","nonce":1550035852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6823","nonce":1550035852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6821","nonce":1550035852}]},"public_addr":"172.21.15.39:6817/1550035852","cluster_addr":"172.21.15.39:6819/1550035852","heartbeat_back_addr":"172.21.15.39:6823/1550035852","heartbeat_front_addr":"172.21.15.39:6821/1550035852","state":["exists","up"]},{"osd":8,"uuid":"2259f35c-2edd-4331-9a63-8c0c1acbdc71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6817","nonce":3184704971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6819","nonce":3184704971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6823","nonce":3184704971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6821","nonce":3184704971}]},"public_addr":"172.21.15.165:6817/3184704971","cluster_addr":"172.21.15.165:6819/3184704971","heartbeat_back_addr":"172.21.15.165:6823/3184704971","heartbeat_front_addr":"172.21.15.165:6821/3184704971","state":["exists","up"]},{"osd":9,"uuid":"10307ecc-8270-4be2-b62b-9708e09bb207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6827","nonce":2800048793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6829","nonce":2800048793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6833","nonce":2800048793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6831","nonce":2800048793}]},"public_addr":"172.21.15.27:6827/2800048793","cluster_addr":"172.21.15.27:6829/2800048793","heartbeat_back_addr":"172.21.15.27:6833/2800048793","heartbeat_front_addr":"172.21.15.27:6831/2800048793","state":["exists","up"]},{"osd":10,"uuid":"dfe1b8ca-ff8a-4b7f-ade5-ad4f00cf60cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6825","nonce":363500811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6827","nonce":363500811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6831","nonce":363500811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6829","nonce":363500811}]},"public_addr":"172.21.15.39:6825/363500811","cluster_addr":"172.21.15.39:6827/363500811","heartbeat_back_addr":"172.21.15.39:6831/363500811","heartbeat_front_addr":"172.21.15.39:6829/363500811","state":["exists","up"]},{"osd":11,"uuid":"f9531145-5d09-4dbd-a050-3159ee09db24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6825","nonce":2274332243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6827","nonce":2274332243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6831","nonce":2274332243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6829","nonce":2274332243}]},"public_addr":"172.21.15.165:6825/2274332243","cluster_addr":"172.21.15.165:6827/2274332243","heartbeat_back_addr":"172.21.15.165:6831/2274332243","heartbeat_front_addr":"172.21.15.165:6829/2274332243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:14:47.077860+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:15:13.272879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:15:38.046651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:03.376483+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:29.179280+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:53.391261+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:17:18.472829+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:17:44.349925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:18:08.652193+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:18:35.282669+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:19:01.637151+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:19:27.461566+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/744303480":"2024-05-22T03:24:01.382214+0000","172.21.15.27:0/3254504266":"2024-05-22T03:24:01.382214+0000","172.21.15.27:0/4135246381":"2024-05-22T03:24:01.382214+0000","172.21.15.27:6800/1185518061":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/4189166571":"2024-05-22T03:24:01.382214+0000","172.21.15.27:6801/1185518061":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/4194431212":"2024-05-22T03:13:13.249220+0000","172.21.15.27:6801/3683408243":"2024-05-22T03:24:01.382214+0000","172.21.15.27:0/1890032101":"2024-05-22T03:13:13.249220+0000","172.21.15.27:6800/175229478":"2024-05-22T03:12:45.739195+0000","172.21.15.27:0/999325706":"2024-05-22T03:12:45.739195+0000","172.21.15.27:6801/175229478":"2024-05-22T03:12:45.739195+0000","172.21.15.27:6800/3683408243":"2024-05-22T03:24:01.382214+0000","172.21.15.27:0/4184128736":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/3000197506":"2024-05-22T03:12:45.739195+0000","172.21.15.27:0/972677319":"2024-05-22T03:12:45.739195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":4,"length":1},{"begin":13,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T03:34:04.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.834+0000 7f945b9af640 1 -- 172.21.15.27:0/3621301769 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f94400b29e0 msgr2=0x7f94400b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:04.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.834+0000 7f945b9af640 1 --2- 172.21.15.27:0/3621301769 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f94400b29e0 0x7f94400b4ea0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9454103320 tx=0x7f9444005f90 comp rx=0 tx=0).stop 2024-05-21T03:34:04.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.834+0000 7f945b9af640 1 -- 172.21.15.27:0/3621301769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94541018b0 msgr2=0x7f945419b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:04.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.834+0000 7f945b9af640 1 --2- 172.21.15.27:0/3621301769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94541018b0 0x7f945419b550 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f945000d770 tx=0x7f945000dc30 comp rx=0 tx=0).stop 2024-05-21T03:34:04.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.834+0000 7f945b9af640 1 -- 172.21.15.27:0/3621301769 shutdown_connections 2024-05-21T03:34:04.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.834+0000 7f945b9af640 1 --2- 172.21.15.27:0/3621301769 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f94400b29e0 0x7f94400b4ea0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.834+0000 7f945b9af640 1 --2- 172.21.15.27:0/3621301769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f945410f460 0x7f945419fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.834+0000 7f945b9af640 1 --2- 172.21.15.27:0/3621301769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94541021d0 0x7f945419ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.834+0000 7f945b9af640 1 --2- 172.21.15.27:0/3621301769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94541018b0 0x7f945419b550 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.834+0000 7f945b9af640 1 -- 172.21.15.27:0/3621301769 >> 172.21.15.27:0/3621301769 conn(0x7f9454078ab0 msgr2=0x7f9454110a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:34:04.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.834+0000 7f945b9af640 1 -- 172.21.15.27:0/3621301769 shutdown_connections 2024-05-21T03:34:04.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.834+0000 7f945b9af640 1 -- 172.21.15.27:0/3621301769 wait complete. 2024-05-21T03:34:04.844 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:04 smithi027 ceph-mon[32179]: pgmap v310: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.8 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-21T03:34:04.850 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-21T03:15:41.340194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-21T03:20:27.507346+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-21T03:20:30.830169+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 13, 'snap_epoch': 88, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-21T03:34:04.851 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-21T03:34:04.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.959+0000 7f190eae8640 1 -- 172.21.15.27:0/808029958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1908103cd0 msgr2=0x7f19081040b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:04.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.960+0000 7f190eae8640 1 --2- 172.21.15.27:0/808029958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1908103cd0 0x7f19081040b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1900009900 tx=0x7f190002a510 comp rx=0 tx=0).stop 2024-05-21T03:34:04.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.960+0000 7f190eae8640 1 -- 172.21.15.27:0/808029958 shutdown_connections 2024-05-21T03:34:04.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.960+0000 7f190eae8640 1 --2- 172.21.15.27:0/808029958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f190810d010 0x7f190810f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.960+0000 7f190eae8640 1 --2- 172.21.15.27:0/808029958 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19081045f0 0x7f190810cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.960+0000 7f190eae8640 1 --2- 172.21.15.27:0/808029958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1908103cd0 0x7f19081040b0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.960+0000 7f190eae8640 1 -- 172.21.15.27:0/808029958 >> 172.21.15.27:0/808029958 conn(0x7f1908078a50 msgr2=0x7f1908078e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:34:04.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.960+0000 7f190eae8640 1 -- 172.21.15.27:0/808029958 shutdown_connections 2024-05-21T03:34:04.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.960+0000 7f190eae8640 1 -- 172.21.15.27:0/808029958 wait complete. 2024-05-21T03:34:04.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.960+0000 7f190eae8640 1 Processor -- start 2024-05-21T03:34:04.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.960+0000 7f190eae8640 1 -- start start 2024-05-21T03:34:04.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.960+0000 7f190eae8640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1908103cd0 0x7f190819b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:04.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.961+0000 7f190eae8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19081045f0 0x7f190819ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:04.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.961+0000 7f190eae8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f190810d010 0x7f190819fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:04.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.961+0000 7f190eae8640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1908112cc0 con 0x7f190810d010 2024-05-21T03:34:04.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.961+0000 7f190eae8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1908112b40 con 0x7f19081045f0 2024-05-21T03:34:04.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.961+0000 7f190eae8640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1908112e40 con 0x7f1908103cd0 2024-05-21T03:34:04.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.961+0000 7f190d085640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f190810d010 0x7f190819fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:04.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.961+0000 7f190d085640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f190810d010 0x7f190819fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38622/0 (socket says 172.21.15.27:38622) 2024-05-21T03:34:04.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.961+0000 7f190d085640 1 -- 172.21.15.27:0/4155294914 learned_addr learned my addr 172.21.15.27:0/4155294914 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:34:04.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.961+0000 7f1907fff640 1 --2- 172.21.15.27:0/4155294914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19081045f0 0x7f190819ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:04.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.961+0000 7f190c884640 1 --2- 172.21.15.27:0/4155294914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1908103cd0 0x7f190819b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:04.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.961+0000 7f190d085640 1 -- 172.21.15.27:0/4155294914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1908103cd0 msgr2=0x7f190819b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:04.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.961+0000 7f190d085640 1 --2- 172.21.15.27:0/4155294914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1908103cd0 0x7f190819b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.961+0000 7f190d085640 1 -- 172.21.15.27:0/4155294914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19081045f0 msgr2=0x7f190819ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:04.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.961+0000 7f190d085640 1 --2- 172.21.15.27:0/4155294914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19081045f0 0x7f190819ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:04.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.961+0000 7f190d085640 1 -- 172.21.15.27:0/4155294914 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19081a04f0 con 0x7f190810d010 2024-05-21T03:34:04.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.961+0000 7f1907fff640 1 --2- 172.21.15.27:0/4155294914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19081045f0 0x7f190819ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:34:04.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.961+0000 7f190d085640 1 --2- 172.21.15.27:0/4155294914 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f190810d010 0x7f190819fdc0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f18f800cce0 tx=0x7f18f800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:34:04.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.961+0000 7f1905ffb640 1 -- 172.21.15.27:0/4155294914 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f800f450 con 0x7f190810d010 2024-05-21T03:34:04.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.961+0000 7f1905ffb640 1 -- 172.21.15.27:0/4155294914 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18f800faa0 con 0x7f190810d010 2024-05-21T03:34:04.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.961+0000 7f190eae8640 1 -- 172.21.15.27:0/4155294914 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19081a07e0 con 0x7f190810d010 2024-05-21T03:34:04.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.961+0000 7f1905ffb640 1 -- 172.21.15.27:0/4155294914 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f8018b70 con 0x7f190810d010 2024-05-21T03:34:04.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.962+0000 7f190eae8640 1 -- 172.21.15.27:0/4155294914 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19081a8070 con 0x7f190810d010 2024-05-21T03:34:04.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.963+0000 7f190eae8640 1 -- 172.21.15.27:0/4155294914 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1908069870 con 0x7f190810d010 2024-05-21T03:34:04.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.964+0000 7f1905ffb640 1 -- 172.21.15.27:0/4155294914 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7f18f8018d10 con 0x7f190810d010 2024-05-21T03:34:04.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.964+0000 7f1905ffb640 1 --2- 172.21.15.27:0/4155294914 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f18f40b29e0 0x7f18f40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:04.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.965+0000 7f190c884640 1 --2- 172.21.15.27:0/4155294914 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f18f40b29e0 0x7f18f40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:04.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.965+0000 7f1905ffb640 1 -- 172.21.15.27:0/4155294914 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8571+0+0 (secure 0 0 0) 0x7f18f80e2ae0 con 0x7f190810d010 2024-05-21T03:34:04.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.965+0000 7f190c884640 1 --2- 172.21.15.27:0/4155294914 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f18f40b29e0 0x7f18f40b4ea0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1900031440 tx=0x7f19000095f0 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:34:04.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:04.969+0000 7f1905ffb640 1 -- 172.21.15.27:0/4155294914 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f18f8014030 con 0x7f190810d010 2024-05-21T03:34:05.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:04 smithi165 ceph-mon[36269]: pgmap v310: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.8 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-21T03:34:05.023 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:04 smithi039 ceph-mon[36429]: pgmap v310: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.8 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-21T03:34:05.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.161+0000 7f190eae8640 1 -- 172.21.15.27:0/4155294914 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f19081a82a0 con 0x7f190810d010 2024-05-21T03:34:05.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.162+0000 7f1905ffb640 1 -- 172.21.15.27:0/4155294914 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f18f80aa430 con 0x7f190810d010 2024-05-21T03:34:05.162 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-05-21T03:34:05.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.164+0000 7f190eae8640 1 -- 172.21.15.27:0/4155294914 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f18f40b29e0 msgr2=0x7f18f40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:05.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.164+0000 7f190eae8640 1 --2- 172.21.15.27:0/4155294914 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f18f40b29e0 0x7f18f40b4ea0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1900031440 tx=0x7f19000095f0 comp rx=0 tx=0).stop 2024-05-21T03:34:05.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.164+0000 7f190eae8640 1 -- 172.21.15.27:0/4155294914 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f190810d010 msgr2=0x7f190819fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:05.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.164+0000 7f190eae8640 1 --2- 172.21.15.27:0/4155294914 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f190810d010 0x7f190819fdc0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f18f800cce0 tx=0x7f18f800a620 comp rx=0 tx=0).stop 2024-05-21T03:34:05.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.164+0000 7f190eae8640 1 -- 172.21.15.27:0/4155294914 shutdown_connections 2024-05-21T03:34:05.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.164+0000 7f190eae8640 1 --2- 172.21.15.27:0/4155294914 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f190810d010 0x7f190819fdc0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:05.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.164+0000 7f190eae8640 1 --2- 172.21.15.27:0/4155294914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19081045f0 0x7f190819ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:05.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.164+0000 7f190eae8640 1 --2- 172.21.15.27:0/4155294914 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f18f40b29e0 0x7f18f40b4ea0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:05.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.164+0000 7f190eae8640 1 --2- 172.21.15.27:0/4155294914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1908103cd0 0x7f190819b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:05.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.164+0000 7f190eae8640 1 -- 172.21.15.27:0/4155294914 >> 172.21.15.27:0/4155294914 conn(0x7f1908078a50 msgr2=0x7f1908108fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:34:05.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.164+0000 7f190eae8640 1 -- 172.21.15.27:0/4155294914 shutdown_connections 2024-05-21T03:34:05.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.164+0000 7f190eae8640 1 -- 172.21.15.27:0/4155294914 wait complete. 2024-05-21T03:34:05.176 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-21T03:34:05.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.285+0000 7f8466c75640 1 -- 172.21.15.27:0/417447952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8460108a40 msgr2=0x7f8460108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:05.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.285+0000 7f8466c75640 1 --2- 172.21.15.27:0/417447952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8460108a40 0x7f8460108e20 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8450009930 tx=0x7f845002a420 comp rx=0 tx=0).stop 2024-05-21T03:34:05.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.285+0000 7f8466c75640 1 -- 172.21.15.27:0/417447952 shutdown_connections 2024-05-21T03:34:05.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.285+0000 7f8466c75640 1 --2- 172.21.15.27:0/417447952 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f846006ba90 0x7f846010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:05.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.285+0000 7f8466c75640 1 --2- 172.21.15.27:0/417447952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f846006b0d0 0x7f846006b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:05.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.285+0000 7f8466c75640 1 --2- 172.21.15.27:0/417447952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8460108a40 0x7f8460108e20 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:05.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.285+0000 7f8466c75640 1 -- 172.21.15.27:0/417447952 >> 172.21.15.27:0/417447952 conn(0x7f84600789b0 msgr2=0x7f8460078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:34:05.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.285+0000 7f8466c75640 1 -- 172.21.15.27:0/417447952 shutdown_connections 2024-05-21T03:34:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.285+0000 7f8466c75640 1 -- 172.21.15.27:0/417447952 wait complete. 2024-05-21T03:34:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.286+0000 7f8466c75640 1 Processor -- start 2024-05-21T03:34:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.286+0000 7f8466c75640 1 -- start start 2024-05-21T03:34:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.286+0000 7f8466c75640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f846006b0d0 0x7f8460101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.286+0000 7f8466c75640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f846006ba90 0x7f8460101ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.286+0000 7f8466c75640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8460108a40 0x7f8460102400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.286+0000 7f8466c75640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8460112d50 con 0x7f8460108a40 2024-05-21T03:34:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.286+0000 7f8466c75640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8460112bd0 con 0x7f846006b0d0 2024-05-21T03:34:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.286+0000 7f8466c75640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8460112ed0 con 0x7f846006ba90 2024-05-21T03:34:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.286+0000 7f8465212640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8460108a40 0x7f8460102400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.286+0000 7f8465212640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8460108a40 0x7f8460102400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38648/0 (socket says 172.21.15.27:38648) 2024-05-21T03:34:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.286+0000 7f8465212640 1 -- 172.21.15.27:0/961530602 learned_addr learned my addr 172.21.15.27:0/961530602 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:34:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.287+0000 7f8464a11640 1 --2- 172.21.15.27:0/961530602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f846006b0d0 0x7f8460101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:05.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.287+0000 7f845ffff640 1 --2- 172.21.15.27:0/961530602 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f846006ba90 0x7f8460101ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:05.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.287+0000 7f8465212640 1 -- 172.21.15.27:0/961530602 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f846006ba90 msgr2=0x7f8460101ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:05.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.287+0000 7f8465212640 1 --2- 172.21.15.27:0/961530602 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f846006ba90 0x7f8460101ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:05.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.287+0000 7f8465212640 1 -- 172.21.15.27:0/961530602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f846006b0d0 msgr2=0x7f8460101980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:05.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.287+0000 7f8465212640 1 --2- 172.21.15.27:0/961530602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f846006b0d0 0x7f8460101980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:05.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.287+0000 7f8465212640 1 -- 172.21.15.27:0/961530602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84601a7690 con 0x7f8460108a40 2024-05-21T03:34:05.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.287+0000 7f845ffff640 1 --2- 172.21.15.27:0/961530602 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f846006ba90 0x7f8460101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:34:05.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.287+0000 7f8464a11640 1 --2- 172.21.15.27:0/961530602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f846006b0d0 0x7f8460101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T03:34:05.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.287+0000 7f8465212640 1 --2- 172.21.15.27:0/961530602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8460108a40 0x7f8460102400 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f845800eaa0 tx=0x7f845800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:34:05.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.287+0000 7f845dffb640 1 -- 172.21.15.27:0/961530602 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845800cdb0 con 0x7f8460108a40 2024-05-21T03:34:05.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.287+0000 7f8466c75640 1 -- 172.21.15.27:0/961530602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84601a7980 con 0x7f8460108a40 2024-05-21T03:34:05.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.287+0000 7f845dffb640 1 -- 172.21.15.27:0/961530602 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8458005020 con 0x7f8460108a40 2024-05-21T03:34:05.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.287+0000 7f845dffb640 1 -- 172.21.15.27:0/961530602 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84580116d0 con 0x7f8460108a40 2024-05-21T03:34:05.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.287+0000 7f8466c75640 1 -- 172.21.15.27:0/961530602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84601a7f20 con 0x7f8460108a40 2024-05-21T03:34:05.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.288+0000 7f8466c75640 1 -- 172.21.15.27:0/961530602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8460069cc0 con 0x7f8460108a40 2024-05-21T03:34:05.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.289+0000 7f845dffb640 1 -- 172.21.15.27:0/961530602 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7f84580054a0 con 0x7f8460108a40 2024-05-21T03:34:05.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.291+0000 7f845dffb640 1 --2- 172.21.15.27:0/961530602 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f84480b2960 0x7f84480b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:05.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.291+0000 7f8464a11640 1 --2- 172.21.15.27:0/961530602 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f84480b2960 0x7f84480b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:05.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.291+0000 7f845dffb640 1 -- 172.21.15.27:0/961530602 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8571+0+0 (secure 0 0 0) 0x7f84580e2870 con 0x7f8460108a40 2024-05-21T03:34:05.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.291+0000 7f8464a11640 1 --2- 172.21.15.27:0/961530602 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f84480b2960 0x7f84480b4e20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f845002a940 tx=0x7f8450005fd0 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:34:05.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.295+0000 7f845dffb640 1 -- 172.21.15.27:0/961530602 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8458016030 con 0x7f8460108a40 2024-05-21T03:34:05.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.487+0000 7f8466c75640 1 -- 172.21.15.27:0/961530602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f846002dba0 con 0x7f8460108a40 2024-05-21T03:34:05.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.488+0000 7f845dffb640 1 -- 172.21.15.27:0/961530602 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f84580aa1c0 con 0x7f8460108a40 2024-05-21T03:34:05.488 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-05-21T03:34:05.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.490+0000 7f8466c75640 1 -- 172.21.15.27:0/961530602 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f84480b2960 msgr2=0x7f84480b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:05.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.490+0000 7f8466c75640 1 --2- 172.21.15.27:0/961530602 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f84480b2960 0x7f84480b4e20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f845002a940 tx=0x7f8450005fd0 comp rx=0 tx=0).stop 2024-05-21T03:34:05.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.490+0000 7f8466c75640 1 -- 172.21.15.27:0/961530602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8460108a40 msgr2=0x7f8460102400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:05.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.490+0000 7f8466c75640 1 --2- 172.21.15.27:0/961530602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8460108a40 0x7f8460102400 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f845800eaa0 tx=0x7f845800ef60 comp rx=0 tx=0).stop 2024-05-21T03:34:05.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.490+0000 7f8466c75640 1 -- 172.21.15.27:0/961530602 shutdown_connections 2024-05-21T03:34:05.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.490+0000 7f8466c75640 1 --2- 172.21.15.27:0/961530602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8460108a40 0x7f8460102400 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:05.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.490+0000 7f8466c75640 1 --2- 172.21.15.27:0/961530602 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f84480b2960 0x7f84480b4e20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:05.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.490+0000 7f8466c75640 1 --2- 172.21.15.27:0/961530602 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f846006ba90 0x7f8460101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:05.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.490+0000 7f8466c75640 1 --2- 172.21.15.27:0/961530602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f846006b0d0 0x7f8460101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:05.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.490+0000 7f8466c75640 1 -- 172.21.15.27:0/961530602 >> 172.21.15.27:0/961530602 conn(0x7f84600789b0 msgr2=0x7f84600ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:34:05.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.490+0000 7f8466c75640 1 -- 172.21.15.27:0/961530602 shutdown_connections 2024-05-21T03:34:05.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.490+0000 7f8466c75640 1 -- 172.21.15.27:0/961530602 wait complete. 2024-05-21T03:34:05.502 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-21T03:34:05.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.610+0000 7f9ef0a59640 1 -- 172.21.15.27:0/2583321563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eec06b0d0 msgr2=0x7f9eec06b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:05.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.610+0000 7f9ef0a59640 1 --2- 172.21.15.27:0/2583321563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eec06b0d0 0x7f9eec06b550 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0009930 tx=0x7f9ee002a420 comp rx=0 tx=0).stop 2024-05-21T03:34:05.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.610+0000 7f9ef0a59640 1 -- 172.21.15.27:0/2583321563 shutdown_connections 2024-05-21T03:34:05.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.610+0000 7f9ef0a59640 1 --2- 172.21.15.27:0/2583321563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9eec06ba90 0x7f9eec10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:05.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.610+0000 7f9ef0a59640 1 --2- 172.21.15.27:0/2583321563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eec06b0d0 0x7f9eec06b550 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:05.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.610+0000 7f9ef0a59640 1 --2- 172.21.15.27:0/2583321563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9eec108a40 0x7f9eec108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:05.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.610+0000 7f9ef0a59640 1 -- 172.21.15.27:0/2583321563 >> 172.21.15.27:0/2583321563 conn(0x7f9eec0789b0 msgr2=0x7f9eec078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:34:05.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.610+0000 7f9ef0a59640 1 -- 172.21.15.27:0/2583321563 shutdown_connections 2024-05-21T03:34:05.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.610+0000 7f9ef0a59640 1 -- 172.21.15.27:0/2583321563 wait complete. 2024-05-21T03:34:05.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.611+0000 7f9ef0a59640 1 Processor -- start 2024-05-21T03:34:05.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.611+0000 7f9ef0a59640 1 -- start start 2024-05-21T03:34:05.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.611+0000 7f9ef0a59640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9eec06b0d0 0x7f9eec199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:05.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.611+0000 7f9ef0a59640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eec06ba90 0x7f9eec1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:05.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.611+0000 7f9ef0a59640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9eec108a40 0x7f9eec19db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:05.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.611+0000 7f9ef0a59640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec112d50 con 0x7f9eec06ba90 2024-05-21T03:34:05.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.611+0000 7f9ef0a59640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec112bd0 con 0x7f9eec108a40 2024-05-21T03:34:05.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.611+0000 7f9ef0a59640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec112ed0 con 0x7f9eec06b0d0 2024-05-21T03:34:05.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.611+0000 7f9ee9d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eec06ba90 0x7f9eec1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:05.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.611+0000 7f9ee9d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eec06ba90 0x7f9eec1997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38664/0 (socket says 172.21.15.27:38664) 2024-05-21T03:34:05.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.611+0000 7f9ee9d9b640 1 -- 172.21.15.27:0/784942473 learned_addr learned my addr 172.21.15.27:0/784942473 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:34:05.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.612+0000 7f9eead9d640 1 --2- 172.21.15.27:0/784942473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9eec108a40 0x7f9eec19db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:05.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.612+0000 7f9eea59c640 1 --2- 172.21.15.27:0/784942473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9eec06b0d0 0x7f9eec199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:05.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.612+0000 7f9ee9d9b640 1 -- 172.21.15.27:0/784942473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9eec06b0d0 msgr2=0x7f9eec199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:05.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.612+0000 7f9ee9d9b640 1 --2- 172.21.15.27:0/784942473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9eec06b0d0 0x7f9eec199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:05.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.612+0000 7f9ee9d9b640 1 -- 172.21.15.27:0/784942473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9eec108a40 msgr2=0x7f9eec19db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:05.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.612+0000 7f9ee9d9b640 1 --2- 172.21.15.27:0/784942473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9eec108a40 0x7f9eec19db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:05.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.612+0000 7f9ee9d9b640 1 -- 172.21.15.27:0/784942473 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eec19e280 con 0x7f9eec06ba90 2024-05-21T03:34:05.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.612+0000 7f9eead9d640 1 --2- 172.21.15.27:0/784942473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9eec108a40 0x7f9eec19db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:34:05.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.612+0000 7f9ee9d9b640 1 --2- 172.21.15.27:0/784942473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eec06ba90 0x7f9eec1997d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0005ab0 tx=0x7f9ee0003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:34:05.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.612+0000 7f9edb7fe640 1 -- 172.21.15.27:0/784942473 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee003f070 con 0x7f9eec06ba90 2024-05-21T03:34:05.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.612+0000 7f9edb7fe640 1 -- 172.21.15.27:0/784942473 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ee0031d60 con 0x7f9eec06ba90 2024-05-21T03:34:05.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.612+0000 7f9ef0a59640 1 -- 172.21.15.27:0/784942473 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eec19e510 con 0x7f9eec06ba90 2024-05-21T03:34:05.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.612+0000 7f9edb7fe640 1 -- 172.21.15.27:0/784942473 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0035030 con 0x7f9eec06ba90 2024-05-21T03:34:05.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.612+0000 7f9ef0a59640 1 -- 172.21.15.27:0/784942473 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eec1a5e00 con 0x7f9eec06ba90 2024-05-21T03:34:05.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.613+0000 7f9ef0a59640 1 -- 172.21.15.27:0/784942473 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eec069cc0 con 0x7f9eec06ba90 2024-05-21T03:34:05.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.615+0000 7f9edb7fe640 1 -- 172.21.15.27:0/784942473 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7f9ee0042070 con 0x7f9eec06ba90 2024-05-21T03:34:05.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.615+0000 7f9edb7fe640 1 --2- 172.21.15.27:0/784942473 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f9ed40b2960 0x7f9ed40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:05.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.616+0000 7f9eea59c640 1 --2- 172.21.15.27:0/784942473 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f9ed40b2960 0x7f9ed40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:05.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.616+0000 7f9edb7fe640 1 -- 172.21.15.27:0/784942473 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8571+0+0 (secure 0 0 0) 0x7f9ee0103590 con 0x7f9eec06ba90 2024-05-21T03:34:05.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.616+0000 7f9eea59c640 1 --2- 172.21.15.27:0/784942473 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f9ed40b2960 0x7f9ed40b4e20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9edc005e00 tx=0x7f9edc005c50 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:34:05.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.618+0000 7f9edb7fe640 1 -- 172.21.15.27:0/784942473 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9ee00caf40 con 0x7f9eec06ba90 2024-05-21T03:34:05.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.809+0000 7f9ef0a59640 1 -- 172.21.15.27:0/784942473 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9eec10aff0 con 0x7f9eec06ba90 2024-05-21T03:34:05.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.810+0000 7f9edb7fe640 1 -- 172.21.15.27:0/784942473 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f9ee00ca840 con 0x7f9eec06ba90 2024-05-21T03:34:05.810 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2024-05-21T03:34:05.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.812+0000 7f9ef0a59640 1 -- 172.21.15.27:0/784942473 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f9ed40b2960 msgr2=0x7f9ed40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:05.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.812+0000 7f9ef0a59640 1 --2- 172.21.15.27:0/784942473 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f9ed40b2960 0x7f9ed40b4e20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9edc005e00 tx=0x7f9edc005c50 comp rx=0 tx=0).stop 2024-05-21T03:34:05.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.812+0000 7f9ef0a59640 1 -- 172.21.15.27:0/784942473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eec06ba90 msgr2=0x7f9eec1997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:05.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.812+0000 7f9ef0a59640 1 --2- 172.21.15.27:0/784942473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eec06ba90 0x7f9eec1997d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0005ab0 tx=0x7f9ee0003cd0 comp rx=0 tx=0).stop 2024-05-21T03:34:05.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.812+0000 7f9ef0a59640 1 -- 172.21.15.27:0/784942473 shutdown_connections 2024-05-21T03:34:05.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.812+0000 7f9ef0a59640 1 --2- 172.21.15.27:0/784942473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9eec108a40 0x7f9eec19db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:05.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.812+0000 7f9ef0a59640 1 --2- 172.21.15.27:0/784942473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eec06ba90 0x7f9eec1997d0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:05.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.812+0000 7f9ef0a59640 1 --2- 172.21.15.27:0/784942473 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f9ed40b2960 0x7f9ed40b4e20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:05.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.812+0000 7f9ef0a59640 1 --2- 172.21.15.27:0/784942473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9eec06b0d0 0x7f9eec199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:05.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.812+0000 7f9ef0a59640 1 -- 172.21.15.27:0/784942473 >> 172.21.15.27:0/784942473 conn(0x7f9eec0789b0 msgr2=0x7f9eec0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:34:05.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.812+0000 7f9ef0a59640 1 -- 172.21.15.27:0/784942473 shutdown_connections 2024-05-21T03:34:05.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.812+0000 7f9ef0a59640 1 -- 172.21.15.27:0/784942473 wait complete. 2024-05-21T03:34:05.822 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:05 smithi027 ceph-mon[32179]: from='client.15870 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:34:05.822 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:05 smithi027 ceph-mon[32179]: from='client.15876 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:34:05.823 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:05 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/3621301769' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:34:05.823 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:05 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/4155294914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T03:34:05.823 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:05 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/961530602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T03:34:05.824 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-21T03:34:05.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.929+0000 7f6d52f19640 1 -- 172.21.15.27:0/2353662949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d4c104650 msgr2=0x7f6d4c10cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:05.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.929+0000 7f6d52f19640 1 --2- 172.21.15.27:0/2353662949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d4c104650 0x7f6d4c10cb30 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6d34009900 tx=0x7f6d3402a480 comp rx=0 tx=0).stop 2024-05-21T03:34:05.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.929+0000 7f6d52f19640 1 -- 172.21.15.27:0/2353662949 shutdown_connections 2024-05-21T03:34:05.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.929+0000 7f6d52f19640 1 --2- 172.21.15.27:0/2353662949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d4c10d070 0x7f6d4c10f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:05.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.929+0000 7f6d52f19640 1 --2- 172.21.15.27:0/2353662949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d4c104650 0x7f6d4c10cb30 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:05.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.929+0000 7f6d52f19640 1 --2- 172.21.15.27:0/2353662949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d4c103d30 0x7f6d4c104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:05.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.929+0000 7f6d52f19640 1 -- 172.21.15.27:0/2353662949 >> 172.21.15.27:0/2353662949 conn(0x7f6d4c078ab0 msgr2=0x7f6d4c078ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:34:05.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.929+0000 7f6d52f19640 1 -- 172.21.15.27:0/2353662949 shutdown_connections 2024-05-21T03:34:05.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.929+0000 7f6d52f19640 1 -- 172.21.15.27:0/2353662949 wait complete. 2024-05-21T03:34:05.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.929+0000 7f6d52f19640 1 Processor -- start 2024-05-21T03:34:05.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.929+0000 7f6d52f19640 1 -- start start 2024-05-21T03:34:05.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.930+0000 7f6d52f19640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d4c103d30 0x7f6d4c1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:05.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.930+0000 7f6d52f19640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d4c104650 0x7f6d4c1997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:05.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.930+0000 7f6d52f19640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d4c10d070 0x7f6d4c19db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:05.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.930+0000 7f6d52f19640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c112cf0 con 0x7f6d4c103d30 2024-05-21T03:34:05.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.930+0000 7f6d52f19640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c112b70 con 0x7f6d4c10d070 2024-05-21T03:34:05.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.930+0000 7f6d52f19640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c112e70 con 0x7f6d4c104650 2024-05-21T03:34:05.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.930+0000 7f6d50cb5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d4c103d30 0x7f6d4c1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:05.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.930+0000 7f6d50cb5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d4c103d30 0x7f6d4c1992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38684/0 (socket says 172.21.15.27:38684) 2024-05-21T03:34:05.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.930+0000 7f6d50cb5640 1 -- 172.21.15.27:0/824687682 learned_addr learned my addr 172.21.15.27:0/824687682 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:34:05.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.930+0000 7f6d514b6640 1 --2- 172.21.15.27:0/824687682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d4c10d070 0x7f6d4c19db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:05.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.930+0000 7f6d4bfff640 1 --2- 172.21.15.27:0/824687682 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d4c104650 0x7f6d4c1997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:05.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.930+0000 7f6d50cb5640 1 -- 172.21.15.27:0/824687682 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d4c104650 msgr2=0x7f6d4c1997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:05.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.930+0000 7f6d50cb5640 1 --2- 172.21.15.27:0/824687682 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d4c104650 0x7f6d4c1997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:05.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.930+0000 7f6d50cb5640 1 -- 172.21.15.27:0/824687682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d4c10d070 msgr2=0x7f6d4c19db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:05.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.930+0000 7f6d50cb5640 1 --2- 172.21.15.27:0/824687682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d4c10d070 0x7f6d4c19db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:05.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.930+0000 7f6d50cb5640 1 -- 172.21.15.27:0/824687682 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4c19e250 con 0x7f6d4c103d30 2024-05-21T03:34:05.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.930+0000 7f6d4bfff640 1 --2- 172.21.15.27:0/824687682 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d4c104650 0x7f6d4c1997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:34:05.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.930+0000 7f6d50cb5640 1 --2- 172.21.15.27:0/824687682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d4c103d30 0x7f6d4c1992b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00e9f0 tx=0x7f6d3c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:34:05.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.930+0000 7f6d49ffb640 1 -- 172.21.15.27:0/824687682 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c00cd80 con 0x7f6d4c103d30 2024-05-21T03:34:05.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.930+0000 7f6d49ffb640 1 -- 172.21.15.27:0/824687682 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d3c0108e0 con 0x7f6d4c103d30 2024-05-21T03:34:05.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.930+0000 7f6d52f19640 1 -- 172.21.15.27:0/824687682 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d4c19e540 con 0x7f6d4c103d30 2024-05-21T03:34:05.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.930+0000 7f6d49ffb640 1 -- 172.21.15.27:0/824687682 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c018980 con 0x7f6d4c103d30 2024-05-21T03:34:05.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.931+0000 7f6d52f19640 1 -- 172.21.15.27:0/824687682 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d4c1a5dd0 con 0x7f6d4c103d30 2024-05-21T03:34:05.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.932+0000 7f6d52f19640 1 -- 172.21.15.27:0/824687682 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d4c069cc0 con 0x7f6d4c103d30 2024-05-21T03:34:05.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.933+0000 7f6d49ffb640 1 -- 172.21.15.27:0/824687682 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7f6d3c010430 con 0x7f6d4c103d30 2024-05-21T03:34:05.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.934+0000 7f6d49ffb640 1 --2- 172.21.15.27:0/824687682 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f6d380b2960 0x7f6d380b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:05.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.934+0000 7f6d4bfff640 1 --2- 172.21.15.27:0/824687682 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f6d380b2960 0x7f6d380b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:05.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.934+0000 7f6d49ffb640 1 -- 172.21.15.27:0/824687682 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8571+0+0 (secure 0 0 0) 0x7f6d3c0e2f00 con 0x7f6d4c103d30 2024-05-21T03:34:05.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.935+0000 7f6d4bfff640 1 --2- 172.21.15.27:0/824687682 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f6d380b2960 0x7f6d380b4e20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6d34031440 tx=0x7f6d34005a20 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:34:05.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:05.938+0000 7f6d49ffb640 1 -- 172.21.15.27:0/824687682 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6d3c0aa850 con 0x7f6d4c103d30 2024-05-21T03:34:06.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:05 smithi039 ceph-mon[36429]: from='client.15870 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:34:06.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:05 smithi039 ceph-mon[36429]: from='client.15876 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:34:06.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:05 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/3621301769' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:34:06.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:05 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/4155294914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T03:34:06.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:05 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/961530602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T03:34:06.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:05 smithi165 ceph-mon[36269]: from='client.15870 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:34:06.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:05 smithi165 ceph-mon[36269]: from='client.15876 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T03:34:06.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:05 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/3621301769' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:34:06.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:05 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/4155294914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T03:34:06.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:05 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/961530602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T03:34:06.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:06.181+0000 7f6d52f19640 1 -- 172.21.15.27:0/824687682 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f6d4c19a5b0 con 0x7f6d4c103d30 2024-05-21T03:34:06.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:06.190+0000 7f6d49ffb640 1 -- 172.21.15.27:0/824687682 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v28) ==== 81+0+816174 (secure 0 0 0) 0x7f6d3c0aa150 con 0x7f6d4c103d30 2024-05-21T03:34:06.191 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:34:06.191 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:34:06.191 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 28, 2024-05-21T03:34:06.191 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.191 INFO:teuthology.orchestra.run.smithi027.stdout: "active_gid": 24913, 2024-05-21T03:34:06.191 INFO:teuthology.orchestra.run.smithi027.stdout: "active_name": "x", 2024-05-21T03:34:06.191 INFO:teuthology.orchestra.run.smithi027.stdout: "active_addrs": { 2024-05-21T03:34:06.192 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-05-21T03:34:06.192 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.192 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-05-21T03:34:06.192 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2024-05-21T03:34:06.192 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1477999924 2024-05-21T03:34:06.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.192 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.192 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-05-21T03:34:06.192 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2024-05-21T03:34:06.192 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1477999924 2024-05-21T03:34:06.192 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.192 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-05-21T03:34:06.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.193 INFO:teuthology.orchestra.run.smithi027.stdout: "active_addr": "172.21.15.27:6801/1477999924", 2024-05-21T03:34:06.193 INFO:teuthology.orchestra.run.smithi027.stdout: "active_change": "2024-05-21T03:24:01.382437+0000", 2024-05-21T03:34:06.193 INFO:teuthology.orchestra.run.smithi027.stdout: "active_mgr_features": 4540701547738038271, 2024-05-21T03:34:06.193 INFO:teuthology.orchestra.run.smithi027.stdout: "available": true, 2024-05-21T03:34:06.193 INFO:teuthology.orchestra.run.smithi027.stdout: "standbys": [ 2024-05-21T03:34:06.193 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.193 INFO:teuthology.orchestra.run.smithi027.stdout: "gid": 24910, 2024-05-21T03:34:06.193 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "z", 2024-05-21T03:34:06.193 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_features": 4540701547738038271, 2024-05-21T03:34:06.193 INFO:teuthology.orchestra.run.smithi027.stdout: "available_modules": [ 2024-05-21T03:34:06.193 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.193 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "alerts", 2024-05-21T03:34:06.193 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.193 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.194 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.194 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-05-21T03:34:06.194 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-05-21T03:34:06.194 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.194 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.194 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.194 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-05-21T03:34:06.194 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.194 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.194 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.194 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How frequently to reexamine health status", 2024-05-21T03:34:06.194 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.194 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.195 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.195 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.195 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.195 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.195 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.195 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.195 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.195 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.195 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.195 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.195 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.195 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.196 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.196 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.196 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.196 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.196 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.196 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.196 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.196 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.196 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.196 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.196 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.196 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.196 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.197 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.197 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.197 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.197 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.197 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.197 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.197 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.197 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.197 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.197 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.197 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.197 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.197 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.197 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.198 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.198 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.198 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.198 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.198 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.198 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.198 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.198 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.198 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.198 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.198 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.198 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.198 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.198 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.199 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.199 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.199 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.199 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.199 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.199 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.199 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.199 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.199 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.199 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.199 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.199 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.200 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.200 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.200 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.200 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_destination": { 2024-05-21T03:34:06.200 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_destination", 2024-05-21T03:34:06.200 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.200 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.200 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.200 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.200 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.200 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.200 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.200 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email address to send alerts to", 2024-05-21T03:34:06.200 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.200 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.201 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.201 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_from_name": { 2024-05-21T03:34:06.201 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_from_name", 2024-05-21T03:34:06.201 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.201 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.201 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.201 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "Ceph", 2024-05-21T03:34:06.201 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.201 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.201 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.201 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email From: name", 2024-05-21T03:34:06.201 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.201 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.201 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.202 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_host": { 2024-05-21T03:34:06.202 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_host", 2024-05-21T03:34:06.202 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.202 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.202 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.202 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.202 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.202 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.202 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.202 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP server", 2024-05-21T03:34:06.202 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.202 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.202 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.203 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_password": { 2024-05-21T03:34:06.203 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_password", 2024-05-21T03:34:06.203 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.203 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.203 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.203 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.203 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.203 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.203 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.203 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Password to authenticate with", 2024-05-21T03:34:06.203 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.203 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.203 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.204 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_port": { 2024-05-21T03:34:06.204 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_port", 2024-05-21T03:34:06.204 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.204 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.204 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.204 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "465", 2024-05-21T03:34:06.204 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.204 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.204 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.205 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP port", 2024-05-21T03:34:06.205 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.205 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.205 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.205 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_sender": { 2024-05-21T03:34:06.205 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_sender", 2024-05-21T03:34:06.205 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.205 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.205 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.205 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.205 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.205 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.205 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.205 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP envelope sender", 2024-05-21T03:34:06.206 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.206 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.206 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.206 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_ssl": { 2024-05-21T03:34:06.206 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_ssl", 2024-05-21T03:34:06.206 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.206 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.206 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.206 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.206 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.206 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.206 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.207 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-21T03:34:06.207 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.207 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.207 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.207 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_user": { 2024-05-21T03:34:06.207 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_user", 2024-05-21T03:34:06.207 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.207 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.207 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.207 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.207 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.207 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.210 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.210 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "User to authenticate as", 2024-05-21T03:34:06.211 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.211 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.211 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.211 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.211 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.211 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.211 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.211 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.211 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.211 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.211 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.211 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.212 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.212 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.212 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.212 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.212 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.212 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.212 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.212 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "balancer", 2024-05-21T03:34:06.212 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.212 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.212 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.212 INFO:teuthology.orchestra.run.smithi027.stdout: "active": { 2024-05-21T03:34:06.213 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "active", 2024-05-21T03:34:06.213 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.213 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.213 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.213 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.213 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.213 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.213 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.213 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically balance PGs across cluster", 2024-05-21T03:34:06.213 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.213 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.213 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.214 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_time": { 2024-05-21T03:34:06.214 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_time", 2024-05-21T03:34:06.214 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.214 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.214 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.214 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0000", 2024-05-21T03:34:06.214 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.214 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.214 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.214 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "beginning time of day to automatically balance", 2024-05-21T03:34:06.214 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-21T03:34:06.214 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.214 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.215 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_weekday": { 2024-05-21T03:34:06.215 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_weekday", 2024-05-21T03:34:06.215 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-05-21T03:34:06.215 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.215 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.215 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.215 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-05-21T03:34:06.215 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-05-21T03:34:06.215 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.215 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-21T03:34:06.215 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-21T03:34:06.215 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.215 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.216 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_max_iterations": { 2024-05-21T03:34:06.216 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_max_iterations", 2024-05-21T03:34:06.216 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-05-21T03:34:06.216 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.216 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.216 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "25", 2024-05-21T03:34:06.216 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-05-21T03:34:06.216 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "250", 2024-05-21T03:34:06.216 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.216 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-21T03:34:06.216 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.216 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.216 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.217 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_metrics": { 2024-05-21T03:34:06.217 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_metrics", 2024-05-21T03:34:06.217 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.217 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.217 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.217 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "pgs,objects,bytes", 2024-05-21T03:34:06.217 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.217 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.217 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.217 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-21T03:34:06.217 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-21T03:34:06.217 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.217 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.217 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_step": { 2024-05-21T03:34:06.218 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_step", 2024-05-21T03:34:06.218 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-05-21T03:34:06.218 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.218 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.218 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.5", 2024-05-21T03:34:06.218 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0.001", 2024-05-21T03:34:06.218 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "0.999", 2024-05-21T03:34:06.218 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.218 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "aggressiveness of optimization", 2024-05-21T03:34:06.218 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-21T03:34:06.218 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.218 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.218 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.218 INFO:teuthology.orchestra.run.smithi027.stdout: "end_time": { 2024-05-21T03:34:06.218 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_time", 2024-05-21T03:34:06.219 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.219 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.219 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.219 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2359", 2024-05-21T03:34:06.219 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.219 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.219 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.219 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ending time of day to automatically balance", 2024-05-21T03:34:06.219 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-21T03:34:06.219 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.219 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.219 INFO:teuthology.orchestra.run.smithi027.stdout: "end_weekday": { 2024-05-21T03:34:06.219 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_weekday", 2024-05-21T03:34:06.219 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-05-21T03:34:06.220 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.220 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.220 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.220 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-05-21T03:34:06.220 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-05-21T03:34:06.220 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.220 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-21T03:34:06.220 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-21T03:34:06.220 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.220 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.220 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.220 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.220 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.220 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.221 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.221 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.221 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.221 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.221 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.221 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.221 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.221 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.228 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.228 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.228 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.228 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.228 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.228 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.228 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.228 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.228 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.228 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.228 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.228 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.229 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.229 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.229 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.229 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.229 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.229 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.229 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.229 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.229 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.229 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.229 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.229 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.229 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.230 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.230 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.230 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.230 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.230 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.230 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.230 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.230 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.230 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.230 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.230 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.230 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.230 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.230 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.230 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.231 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.231 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.231 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.231 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.231 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.231 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.231 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.231 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.231 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.231 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.231 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.231 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.231 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.231 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.232 INFO:teuthology.orchestra.run.smithi027.stdout: "min_score": { 2024-05-21T03:34:06.232 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_score", 2024-05-21T03:34:06.232 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-05-21T03:34:06.232 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.232 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.232 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.232 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.232 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.232 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.232 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-21T03:34:06.232 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.232 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.233 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.233 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-05-21T03:34:06.233 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-05-21T03:34:06.233 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.233 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.233 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.233 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "upmap", 2024-05-21T03:34:06.233 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.233 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.233 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.233 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-compat", 2024-05-21T03:34:06.233 INFO:teuthology.orchestra.run.smithi027.stdout: "none", 2024-05-21T03:34:06.233 INFO:teuthology.orchestra.run.smithi027.stdout: "read", 2024-05-21T03:34:06.233 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap", 2024-05-21T03:34:06.234 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap-read" 2024-05-21T03:34:06.234 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.234 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Balancer mode", 2024-05-21T03:34:06.234 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.234 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.234 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.234 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_ids": { 2024-05-21T03:34:06.234 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_ids", 2024-05-21T03:34:06.234 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.234 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.234 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.235 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.235 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.235 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.235 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.235 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-21T03:34:06.235 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.235 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.235 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.235 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-05-21T03:34:06.235 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-05-21T03:34:06.235 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.235 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.235 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.235 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-05-21T03:34:06.236 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.236 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.236 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.236 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-21T03:34:06.236 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.236 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.236 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.236 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.236 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.236 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.236 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.236 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.236 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.236 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.237 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.237 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.237 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.237 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.237 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.237 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.237 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_deviation": { 2024-05-21T03:34:06.237 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_deviation", 2024-05-21T03:34:06.237 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.237 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.237 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.237 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-05-21T03:34:06.237 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-05-21T03:34:06.238 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.238 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.238 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-21T03:34:06.238 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-21T03:34:06.238 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.238 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.238 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_optimizations": { 2024-05-21T03:34:06.238 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_optimizations", 2024-05-21T03:34:06.238 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-05-21T03:34:06.238 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.238 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.238 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-05-21T03:34:06.238 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.238 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.239 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.239 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-21T03:34:06.239 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.239 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.239 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.239 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.239 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.239 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.239 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm", 2024-05-21T03:34:06.239 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.239 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.239 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.239 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_down_multiplier": { 2024-05-21T03:34:06.240 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_down_multiplier", 2024-05-21T03:34:06.240 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-05-21T03:34:06.240 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.240 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.240 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-05-21T03:34:06.240 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.240 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.240 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.240 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-21T03:34:06.240 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.240 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.240 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.240 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_refresh_rate": { 2024-05-21T03:34:06.241 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_refresh_rate", 2024-05-21T03:34:06.241 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.241 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.241 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.241 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "20", 2024-05-21T03:34:06.241 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.241 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.241 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.241 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-21T03:34:06.241 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.241 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.241 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.241 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_starting_port": { 2024-05-21T03:34:06.242 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_starting_port", 2024-05-21T03:34:06.242 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.242 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.242 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.242 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4721", 2024-05-21T03:34:06.242 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.242 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.242 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.242 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-21T03:34:06.242 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.242 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.242 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.242 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_ptrace": { 2024-05-21T03:34:06.243 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_ptrace", 2024-05-21T03:34:06.243 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.243 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.243 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.243 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.243 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.243 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.243 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.243 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-21T03:34:06.243 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-21T03:34:06.243 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.243 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.243 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_interval": { 2024-05-21T03:34:06.243 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_interval", 2024-05-21T03:34:06.244 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.244 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.244 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.244 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-05-21T03:34:06.244 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.244 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.244 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.244 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-21T03:34:06.244 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.244 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.244 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.244 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_memory_target_ratio": { 2024-05-21T03:34:06.244 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_memory_target_ratio", 2024-05-21T03:34:06.244 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-05-21T03:34:06.245 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.245 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.245 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.7", 2024-05-21T03:34:06.245 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.245 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.245 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.245 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-21T03:34:06.245 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.245 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.245 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.245 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm_log_destination": { 2024-05-21T03:34:06.245 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm_log_destination", 2024-05-21T03:34:06.246 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.246 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.246 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.246 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.246 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.246 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.246 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.246 INFO:teuthology.orchestra.run.smithi027.stdout: "file", 2024-05-21T03:34:06.246 INFO:teuthology.orchestra.run.smithi027.stdout: "file,syslog", 2024-05-21T03:34:06.246 INFO:teuthology.orchestra.run.smithi027.stdout: "syslog" 2024-05-21T03:34:06.246 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.246 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-21T03:34:06.246 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.247 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.247 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.247 INFO:teuthology.orchestra.run.smithi027.stdout: "cgroups_split": { 2024-05-21T03:34:06.247 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cgroups_split", 2024-05-21T03:34:06.247 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.247 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.247 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.247 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.247 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.247 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.247 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.247 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-21T03:34:06.247 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.247 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.247 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.248 INFO:teuthology.orchestra.run.smithi027.stdout: "config_checks_enabled": { 2024-05-21T03:34:06.248 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_checks_enabled", 2024-05-21T03:34:06.248 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.248 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.248 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.248 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.248 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.248 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.248 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.248 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-21T03:34:06.248 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.248 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.248 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.249 INFO:teuthology.orchestra.run.smithi027.stdout: "config_dashboard": { 2024-05-21T03:34:06.249 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_dashboard", 2024-05-21T03:34:06.249 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.249 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.249 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.249 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.249 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.249 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.249 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.249 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-21T03:34:06.249 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.249 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.249 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.250 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_alertmanager": { 2024-05-21T03:34:06.250 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_alertmanager", 2024-05-21T03:34:06.250 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.250 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.250 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.250 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-21T03:34:06.250 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.250 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.250 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.250 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-05-21T03:34:06.250 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.250 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.250 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.251 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_base": { 2024-05-21T03:34:06.251 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_base", 2024-05-21T03:34:06.251 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.251 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.251 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.251 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-21T03:34:06.251 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.251 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.251 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.251 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Container image name, without the tag", 2024-05-21T03:34:06.251 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.251 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.251 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.252 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_elasticsearch": { 2024-05-21T03:34:06.252 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_elasticsearch", 2024-05-21T03:34:06.252 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.252 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.252 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.252 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-21T03:34:06.252 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.252 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.252 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.252 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "elasticsearch container image", 2024-05-21T03:34:06.252 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.252 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.252 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.253 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_grafana": { 2024-05-21T03:34:06.253 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_grafana", 2024-05-21T03:34:06.253 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.253 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.253 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.253 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-21T03:34:06.253 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.253 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.253 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.253 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-05-21T03:34:06.253 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.253 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.253 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.254 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_haproxy": { 2024-05-21T03:34:06.254 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_haproxy", 2024-05-21T03:34:06.254 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.254 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.254 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.254 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-21T03:34:06.254 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.254 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.254 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.254 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "HAproxy container image", 2024-05-21T03:34:06.254 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.254 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.254 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.255 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_agent": { 2024-05-21T03:34:06.255 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_agent", 2024-05-21T03:34:06.255 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.255 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.255 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.255 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-21T03:34:06.255 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.255 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.255 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.255 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger agent container image", 2024-05-21T03:34:06.255 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.255 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.255 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.256 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_collector": { 2024-05-21T03:34:06.256 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_collector", 2024-05-21T03:34:06.256 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.256 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.256 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.256 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-21T03:34:06.256 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.256 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.256 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.256 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger collector container image", 2024-05-21T03:34:06.256 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.256 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.256 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.256 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_query": { 2024-05-21T03:34:06.256 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_query", 2024-05-21T03:34:06.257 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.257 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.257 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.257 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-21T03:34:06.257 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.257 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.257 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.257 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger query container image", 2024-05-21T03:34:06.257 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.257 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.257 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.257 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_keepalived": { 2024-05-21T03:34:06.257 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_keepalived", 2024-05-21T03:34:06.257 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.258 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.258 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.258 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-21T03:34:06.258 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.258 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.258 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.258 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Keepalived container image", 2024-05-21T03:34:06.258 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.258 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.258 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.258 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_loki": { 2024-05-21T03:34:06.258 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_loki", 2024-05-21T03:34:06.259 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.259 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.259 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.259 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-21T03:34:06.259 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.259 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.259 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.259 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Loki container image", 2024-05-21T03:34:06.259 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.259 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.259 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.259 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_node_exporter": { 2024-05-21T03:34:06.259 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_node_exporter", 2024-05-21T03:34:06.259 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.260 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.260 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.260 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-21T03:34:06.260 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.260 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.260 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.260 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-05-21T03:34:06.260 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.260 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.260 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.260 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nvmeof": { 2024-05-21T03:34:06.260 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nvmeof", 2024-05-21T03:34:06.260 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.260 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.261 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.261 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-21T03:34:06.261 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.261 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.261 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.261 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nvme-of container image", 2024-05-21T03:34:06.261 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.261 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.261 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.261 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_prometheus": { 2024-05-21T03:34:06.261 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_prometheus", 2024-05-21T03:34:06.261 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.261 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.261 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.261 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-21T03:34:06.262 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.262 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.262 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.262 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-05-21T03:34:06.262 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.262 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.262 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.262 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_promtail": { 2024-05-21T03:34:06.262 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_promtail", 2024-05-21T03:34:06.262 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.262 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.262 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.262 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-21T03:34:06.263 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.263 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.263 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.263 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Promtail container image", 2024-05-21T03:34:06.263 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.263 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.263 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.263 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba": { 2024-05-21T03:34:06.263 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba", 2024-05-21T03:34:06.263 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.263 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.263 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.263 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-21T03:34:06.263 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.264 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.264 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.264 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB container image", 2024-05-21T03:34:06.264 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.264 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.264 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.264 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_snmp_gateway": { 2024-05-21T03:34:06.264 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_snmp_gateway", 2024-05-21T03:34:06.264 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.264 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.264 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.264 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-21T03:34:06.265 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.265 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.265 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.265 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SNMP Gateway container image", 2024-05-21T03:34:06.265 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.265 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.265 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.265 INFO:teuthology.orchestra.run.smithi027.stdout: "container_init": { 2024-05-21T03:34:06.265 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_init", 2024-05-21T03:34:06.265 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.265 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.265 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.265 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.265 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.266 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.266 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.266 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Run podman/docker with `--init`", 2024-05-21T03:34:06.266 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.266 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.266 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.266 INFO:teuthology.orchestra.run.smithi027.stdout: "daemon_cache_timeout": { 2024-05-21T03:34:06.266 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "daemon_cache_timeout", 2024-05-21T03:34:06.266 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.266 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.266 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.266 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-05-21T03:34:06.266 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.267 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.267 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.267 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-21T03:34:06.267 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.267 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.267 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.267 INFO:teuthology.orchestra.run.smithi027.stdout: "default_cephadm_command_timeout": { 2024-05-21T03:34:06.267 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_cephadm_command_timeout", 2024-05-21T03:34:06.267 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.267 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.267 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.267 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "900", 2024-05-21T03:34:06.267 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.268 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.268 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.268 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-21T03:34:06.268 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.268 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.268 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.268 INFO:teuthology.orchestra.run.smithi027.stdout: "default_registry": { 2024-05-21T03:34:06.268 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_registry", 2024-05-21T03:34:06.268 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.268 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.268 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.268 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io", 2024-05-21T03:34:06.268 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.269 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.269 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.269 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-21T03:34:06.269 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.269 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.269 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.269 INFO:teuthology.orchestra.run.smithi027.stdout: "device_cache_timeout": { 2024-05-21T03:34:06.269 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_cache_timeout", 2024-05-21T03:34:06.269 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.269 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.269 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.269 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1800", 2024-05-21T03:34:06.269 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.269 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.270 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.270 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache device inventory", 2024-05-21T03:34:06.270 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.270 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.270 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.270 INFO:teuthology.orchestra.run.smithi027.stdout: "device_enhanced_scan": { 2024-05-21T03:34:06.270 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_enhanced_scan", 2024-05-21T03:34:06.270 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.270 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.270 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.270 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.270 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.270 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.270 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.271 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-21T03:34:06.271 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.271 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.271 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.271 INFO:teuthology.orchestra.run.smithi027.stdout: "facts_cache_timeout": { 2024-05-21T03:34:06.271 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "facts_cache_timeout", 2024-05-21T03:34:06.271 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.271 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.271 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.271 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-05-21T03:34:06.271 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.271 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.271 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.271 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache host facts data", 2024-05-21T03:34:06.271 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.272 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.272 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.272 INFO:teuthology.orchestra.run.smithi027.stdout: "grafana_dashboards_path": { 2024-05-21T03:34:06.272 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "grafana_dashboards_path", 2024-05-21T03:34:06.272 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.272 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.272 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.272 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-21T03:34:06.272 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.272 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.272 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.272 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-21T03:34:06.272 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.273 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.273 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.273 INFO:teuthology.orchestra.run.smithi027.stdout: "host_check_interval": { 2024-05-21T03:34:06.273 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "host_check_interval", 2024-05-21T03:34:06.273 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.273 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.273 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.273 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-05-21T03:34:06.273 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.273 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.273 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.273 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to perform a host check", 2024-05-21T03:34:06.273 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.273 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.274 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.274 INFO:teuthology.orchestra.run.smithi027.stdout: "hw_monitoring": { 2024-05-21T03:34:06.274 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hw_monitoring", 2024-05-21T03:34:06.274 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.274 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.274 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.274 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.274 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.274 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.274 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.274 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-21T03:34:06.274 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.275 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.275 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.275 INFO:teuthology.orchestra.run.smithi027.stdout: "inventory_list_all": { 2024-05-21T03:34:06.275 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "inventory_list_all", 2024-05-21T03:34:06.275 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.275 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.275 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.275 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.275 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.275 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.275 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.275 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-21T03:34:06.276 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.276 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.276 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.276 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.276 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.276 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.276 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.276 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.276 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.276 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.276 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.276 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.277 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.277 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.277 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.277 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.277 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.277 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.277 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.277 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.277 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.277 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.277 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.277 INFO:teuthology.orchestra.run.smithi027.stdout: "log_refresh_metadata": { 2024-05-21T03:34:06.277 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_refresh_metadata", 2024-05-21T03:34:06.278 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.278 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.278 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.278 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.278 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.278 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.278 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.278 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-21T03:34:06.278 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.278 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.278 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.278 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.278 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.279 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.279 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.279 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.279 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.279 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.279 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.279 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.279 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-21T03:34:06.279 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.279 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.279 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.279 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.280 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.280 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.280 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.280 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.280 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.280 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.280 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.280 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.280 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.280 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.280 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.280 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.280 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.280 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.280 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.281 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.281 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.281 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.281 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.281 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.281 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.281 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.281 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.281 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.281 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.281 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.281 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.281 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.281 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.282 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.282 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.282 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.282 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-21T03:34:06.282 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-21T03:34:06.282 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.282 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.282 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.282 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.282 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.282 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.282 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.282 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-21T03:34:06.283 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.283 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.283 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.283 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-21T03:34:06.283 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-21T03:34:06.283 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.283 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.283 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.283 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "*", 2024-05-21T03:34:06.283 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.283 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.283 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.283 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-21T03:34:06.284 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.284 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.284 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.284 INFO:teuthology.orchestra.run.smithi027.stdout: "max_count_per_host": { 2024-05-21T03:34:06.284 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_count_per_host", 2024-05-21T03:34:06.284 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.284 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.284 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.284 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-05-21T03:34:06.284 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.284 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.284 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.284 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of daemons per service per host", 2024-05-21T03:34:06.284 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.285 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.285 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.285 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd_draining_count": { 2024-05-21T03:34:06.285 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_osd_draining_count", 2024-05-21T03:34:06.285 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.285 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.285 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.285 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-05-21T03:34:06.285 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.285 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.285 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.285 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-21T03:34:06.285 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.286 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.286 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.286 INFO:teuthology.orchestra.run.smithi027.stdout: "migration_current": { 2024-05-21T03:34:06.286 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "migration_current", 2024-05-21T03:34:06.286 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.286 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.286 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.286 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.286 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.286 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.286 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.286 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "internal - do not modify", 2024-05-21T03:34:06.287 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.287 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.287 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.287 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-05-21T03:34:06.287 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-05-21T03:34:06.287 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.287 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.287 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.287 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "root", 2024-05-21T03:34:06.287 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.287 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.287 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.287 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm-package", 2024-05-21T03:34:06.287 INFO:teuthology.orchestra.run.smithi027.stdout: "root" 2024-05-21T03:34:06.287 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.288 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "mode for remote execution of cephadm", 2024-05-21T03:34:06.288 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.288 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.288 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.288 INFO:teuthology.orchestra.run.smithi027.stdout: "oob_default_addr": { 2024-05-21T03:34:06.288 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "oob_default_addr", 2024-05-21T03:34:06.288 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.288 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.288 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.288 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "169.254.1.1", 2024-05-21T03:34:06.288 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.288 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.288 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.289 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-21T03:34:06.289 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.289 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.289 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.289 INFO:teuthology.orchestra.run.smithi027.stdout: "prometheus_alerts_path": { 2024-05-21T03:34:06.289 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus_alerts_path", 2024-05-21T03:34:06.289 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.289 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.289 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.289 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-21T03:34:06.289 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.289 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.289 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.289 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-21T03:34:06.290 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.290 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.290 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.290 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_insecure": { 2024-05-21T03:34:06.290 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_insecure", 2024-05-21T03:34:06.290 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.290 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.290 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.290 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.290 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.290 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.290 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.290 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-21T03:34:06.290 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.291 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.291 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.291 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_password": { 2024-05-21T03:34:06.291 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_password", 2024-05-21T03:34:06.291 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.291 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.291 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.291 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.291 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.291 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.291 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.291 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-21T03:34:06.291 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.291 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.292 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.292 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_url": { 2024-05-21T03:34:06.292 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_url", 2024-05-21T03:34:06.292 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.292 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.292 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.292 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.292 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.292 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.292 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.292 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-21T03:34:06.292 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.292 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.292 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.293 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_username": { 2024-05-21T03:34:06.293 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_username", 2024-05-21T03:34:06.293 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.293 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.293 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.293 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.293 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.293 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.293 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.293 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-21T03:34:06.293 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.293 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.293 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.294 INFO:teuthology.orchestra.run.smithi027.stdout: "secure_monitoring_stack": { 2024-05-21T03:34:06.294 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secure_monitoring_stack", 2024-05-21T03:34:06.294 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.294 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.294 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.294 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.294 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.294 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.294 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.294 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-21T03:34:06.294 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.294 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.294 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.295 INFO:teuthology.orchestra.run.smithi027.stdout: "service_discovery_port": { 2024-05-21T03:34:06.295 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "service_discovery_port", 2024-05-21T03:34:06.295 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.295 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.295 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.295 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8765", 2024-05-21T03:34:06.295 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.295 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.295 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.295 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "cephadm service discovery port", 2024-05-21T03:34:06.295 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.295 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.295 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.295 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.296 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.296 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.296 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.296 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.296 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.296 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.296 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.296 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.296 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.296 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.296 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.296 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.296 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_config_file": { 2024-05-21T03:34:06.297 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_config_file", 2024-05-21T03:34:06.297 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.297 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.297 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.297 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.297 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.297 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.297 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.297 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-21T03:34:06.297 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.297 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.297 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.297 INFO:teuthology.orchestra.run.smithi027.stdout: "use_agent": { 2024-05-21T03:34:06.297 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_agent", 2024-05-21T03:34:06.298 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.298 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.298 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.298 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.298 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.298 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.298 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.298 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-21T03:34:06.298 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.298 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.298 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.298 INFO:teuthology.orchestra.run.smithi027.stdout: "use_repo_digest": { 2024-05-21T03:34:06.298 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_repo_digest", 2024-05-21T03:34:06.298 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.299 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.299 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.299 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.299 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.299 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.299 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.299 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-21T03:34:06.299 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.299 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.299 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.299 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_failed_host_check": { 2024-05-21T03:34:06.299 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_failed_host_check", 2024-05-21T03:34:06.299 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.299 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.300 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.300 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.300 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.300 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.300 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.300 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if the host check fails", 2024-05-21T03:34:06.300 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.300 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.300 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.300 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_daemons": { 2024-05-21T03:34:06.300 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_daemons", 2024-05-21T03:34:06.300 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.300 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.300 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.301 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.301 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.301 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.301 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.301 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-21T03:34:06.301 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.301 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.301 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.301 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_hosts": { 2024-05-21T03:34:06.301 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_hosts", 2024-05-21T03:34:06.301 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.301 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.301 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.301 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.302 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.302 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.302 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.302 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-21T03:34:06.302 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.302 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.302 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.302 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.302 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.302 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crash", 2024-05-21T03:34:06.302 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.302 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.303 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.303 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.303 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.303 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.303 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.303 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.303 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.303 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.303 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.303 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.303 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.303 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.303 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.303 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.303 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.304 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.304 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.304 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.304 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.304 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.304 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.304 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.304 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.304 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.305 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.305 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.305 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.305 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.305 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.305 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.305 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.305 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.305 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.305 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.305 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.305 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.306 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.306 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.306 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.306 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.306 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.306 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.306 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.306 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.306 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.306 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.306 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.306 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.306 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.306 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.306 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.307 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.307 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.307 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.307 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.307 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.307 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.307 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.307 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.307 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.307 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.307 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.307 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.307 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.307 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.308 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.308 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.308 INFO:teuthology.orchestra.run.smithi027.stdout: "retain_interval": { 2024-05-21T03:34:06.308 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retain_interval", 2024-05-21T03:34:06.308 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.308 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.308 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.308 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "31536000", 2024-05-21T03:34:06.308 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.308 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.308 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.308 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-21T03:34:06.308 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.308 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.309 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.309 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.309 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.309 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.309 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.309 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.309 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.309 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.309 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.309 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.309 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.309 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.309 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.310 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.310 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_recent_interval": { 2024-05-21T03:34:06.310 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_recent_interval", 2024-05-21T03:34:06.310 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.310 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.310 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.310 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1209600", 2024-05-21T03:34:06.310 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.310 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.310 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.310 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-21T03:34:06.310 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.310 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.310 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.310 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.311 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.311 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dashboard", 2024-05-21T03:34:06.311 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.311 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.311 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.311 INFO:teuthology.orchestra.run.smithi027.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-21T03:34:06.311 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-21T03:34:06.311 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.311 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.311 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.311 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-05-21T03:34:06.311 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.311 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.311 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.312 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.312 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.312 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.312 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.312 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_HOST": { 2024-05-21T03:34:06.312 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-21T03:34:06.312 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.312 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.312 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.312 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.312 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.312 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.312 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.313 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.313 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.313 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.313 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.313 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-21T03:34:06.313 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-21T03:34:06.313 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.313 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.313 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.313 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.313 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.313 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.313 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.313 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.314 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.314 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.314 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.314 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_ENABLED": { 2024-05-21T03:34:06.314 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_ENABLED", 2024-05-21T03:34:06.314 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.314 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.314 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.314 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.314 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.314 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.314 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.315 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.315 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.315 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.315 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.315 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-21T03:34:06.315 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-21T03:34:06.315 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.315 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.315 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.315 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.315 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.315 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.315 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.315 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.316 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.316 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.316 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.316 INFO:teuthology.orchestra.run.smithi027.stdout: "ENABLE_BROWSABLE_API": { 2024-05-21T03:34:06.316 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-21T03:34:06.316 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.316 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.316 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.316 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.316 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.316 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.316 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.316 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.317 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.317 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.317 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.317 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-21T03:34:06.317 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-21T03:34:06.317 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.317 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.317 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.317 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.317 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.317 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.317 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.317 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.317 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.317 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.318 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.318 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-21T03:34:06.318 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-21T03:34:06.318 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.318 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.318 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.318 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.318 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.318 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.318 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.318 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.318 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.318 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.318 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.319 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-21T03:34:06.319 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-21T03:34:06.319 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.319 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.319 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.319 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.319 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.319 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.319 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.319 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.319 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.319 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.319 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.320 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-21T03:34:06.320 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-21T03:34:06.320 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.320 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.320 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.320 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.320 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.320 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.320 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.320 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.320 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.320 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.320 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.320 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-21T03:34:06.321 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-21T03:34:06.321 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.321 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.321 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.321 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.321 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.321 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.321 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.321 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.321 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.321 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.321 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.321 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-21T03:34:06.322 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-21T03:34:06.322 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.322 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.322 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.322 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.322 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.322 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.322 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.322 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.322 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.322 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.322 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.322 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-21T03:34:06.323 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-21T03:34:06.323 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.323 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.323 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.323 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.323 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.323 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.323 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.323 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.323 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.323 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.323 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.323 INFO:teuthology.orchestra.run.smithi027.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-21T03:34:06.323 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-21T03:34:06.324 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.324 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.324 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.324 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.324 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.324 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.324 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.324 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.324 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.324 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.324 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.324 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_PASSWORD": { 2024-05-21T03:34:06.324 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-21T03:34:06.325 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.325 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.325 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.325 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-05-21T03:34:06.325 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.325 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.325 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.325 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.325 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.325 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.325 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.325 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-21T03:34:06.325 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-21T03:34:06.326 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.326 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.326 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.326 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.326 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.326 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.326 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.326 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.326 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.326 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.326 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.326 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_URL": { 2024-05-21T03:34:06.326 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_URL", 2024-05-21T03:34:06.327 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.327 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.327 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.327 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.327 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.327 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.327 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.327 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.327 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.327 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.327 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.327 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_USERNAME": { 2024-05-21T03:34:06.327 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-21T03:34:06.327 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.328 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.328 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.328 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-05-21T03:34:06.328 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.328 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.328 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.328 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.328 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.328 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.328 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.328 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-21T03:34:06.328 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-21T03:34:06.328 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.328 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.328 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.329 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.329 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.329 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.329 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.329 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.329 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.329 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.329 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.329 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-21T03:34:06.329 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-21T03:34:06.329 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.329 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.330 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.330 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.330 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.330 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.330 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.330 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.330 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.330 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.330 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.330 INFO:teuthology.orchestra.run.smithi027.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-21T03:34:06.330 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-21T03:34:06.330 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.330 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.330 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.331 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.331 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.331 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.331 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.331 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.331 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.331 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.331 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.331 INFO:teuthology.orchestra.run.smithi027.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-21T03:34:06.331 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-21T03:34:06.331 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.331 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.331 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.332 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.332 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.332 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.332 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.332 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.332 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.332 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.332 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.332 INFO:teuthology.orchestra.run.smithi027.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-21T03:34:06.332 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-21T03:34:06.332 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.332 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.332 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.332 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "[]", 2024-05-21T03:34:06.333 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.333 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.333 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.333 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.333 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.333 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.333 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.333 INFO:teuthology.orchestra.run.smithi027.stdout: "MULTICLUSTER_CONFIG": { 2024-05-21T03:34:06.333 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-21T03:34:06.333 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.333 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.333 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.333 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "{}", 2024-05-21T03:34:06.333 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.334 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.334 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.334 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.334 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.334 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.334 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.334 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_HOST": { 2024-05-21T03:34:06.334 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-21T03:34:06.334 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.334 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.334 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.334 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.334 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.335 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.335 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.335 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.335 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.335 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.335 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.335 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-21T03:34:06.335 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-21T03:34:06.335 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.335 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.335 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.335 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.335 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.335 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.336 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.336 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.336 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.336 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.336 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.336 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-21T03:34:06.336 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-21T03:34:06.336 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.336 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.336 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.336 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.336 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.336 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.336 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.337 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.337 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.337 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.337 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.337 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-21T03:34:06.337 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-21T03:34:06.337 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.337 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.337 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.337 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.337 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.337 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.337 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.337 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.337 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.338 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.338 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.338 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-21T03:34:06.338 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-21T03:34:06.338 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.338 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.338 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.338 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.338 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.338 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.338 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.338 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.339 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.339 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.339 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.339 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-21T03:34:06.339 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-21T03:34:06.339 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.339 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.339 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.339 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.339 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.339 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.339 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.339 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.339 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.340 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.340 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.340 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-21T03:34:06.340 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-21T03:34:06.340 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.340 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.340 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.340 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.340 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.340 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.340 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.340 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.340 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.340 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.341 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.341 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-21T03:34:06.341 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-21T03:34:06.341 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.341 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.341 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.341 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.341 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.341 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.341 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.341 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.341 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.341 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.342 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.342 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-21T03:34:06.342 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-21T03:34:06.342 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.342 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.342 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.342 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.342 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.342 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.342 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.342 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.342 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.343 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.343 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.343 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_ENABLED": { 2024-05-21T03:34:06.343 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-21T03:34:06.343 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.343 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.343 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.343 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.343 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.343 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.343 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.343 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.343 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.344 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.344 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.344 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-21T03:34:06.344 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-21T03:34:06.344 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.344 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.344 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.344 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-21T03:34:06.344 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.344 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.344 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.344 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.344 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.344 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.345 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.345 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-21T03:34:06.345 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-21T03:34:06.345 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.345 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.345 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.345 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-05-21T03:34:06.345 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.345 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.345 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.345 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.345 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.345 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.345 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.346 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-21T03:34:06.346 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-21T03:34:06.346 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.346 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.346 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.346 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8", 2024-05-21T03:34:06.346 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.346 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.346 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.346 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.346 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.346 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.347 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.347 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.347 INFO:teuthology.orchestra.run.smithi027.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-21T03:34:06.347 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-21T03:34:06.347 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.347 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.347 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.347 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "45", 2024-05-21T03:34:06.347 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.347 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.347 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.347 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.347 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.347 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.347 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.348 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ACCESS_KEY": { 2024-05-21T03:34:06.348 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-21T03:34:06.348 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.348 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.348 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.348 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.348 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.348 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.348 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.348 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.348 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.348 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.348 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.349 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-21T03:34:06.349 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-21T03:34:06.349 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.349 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.349 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.349 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-05-21T03:34:06.349 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.349 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.349 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.349 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.349 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.349 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.349 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.349 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SECRET_KEY": { 2024-05-21T03:34:06.349 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-21T03:34:06.350 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.350 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.350 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.350 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.350 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.350 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.350 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.350 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.350 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.350 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.350 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.350 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SSL_VERIFY": { 2024-05-21T03:34:06.350 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-21T03:34:06.351 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.351 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.351 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.351 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.351 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.351 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.351 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.351 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.351 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.351 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.351 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.351 INFO:teuthology.orchestra.run.smithi027.stdout: "UNSAFE_TLS_v1_2": { 2024-05-21T03:34:06.352 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-21T03:34:06.352 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.352 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.352 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.352 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.352 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.352 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.352 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.352 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.352 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.352 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.352 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.352 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.352 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-21T03:34:06.352 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-21T03:34:06.353 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.353 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.353 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.353 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.353 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.353 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.353 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.353 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.353 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.353 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.353 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.353 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.353 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-21T03:34:06.353 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-21T03:34:06.353 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.354 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.354 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.354 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-05-21T03:34:06.354 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.354 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.354 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.354 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.354 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.354 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.354 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.354 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-21T03:34:06.354 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-21T03:34:06.355 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.355 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.355 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.355 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-05-21T03:34:06.355 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.355 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.355 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.355 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.355 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.355 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.355 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.355 INFO:teuthology.orchestra.run.smithi027.stdout: "cross_origin_url": { 2024-05-21T03:34:06.355 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cross_origin_url", 2024-05-21T03:34:06.356 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.356 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.356 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.356 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.356 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.356 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.356 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.356 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.356 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.356 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.356 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.356 INFO:teuthology.orchestra.run.smithi027.stdout: "crt_file": { 2024-05-21T03:34:06.356 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crt_file", 2024-05-21T03:34:06.356 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.356 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.357 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.357 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.357 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.357 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.357 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.357 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.357 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.357 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.357 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.357 INFO:teuthology.orchestra.run.smithi027.stdout: "debug": { 2024-05-21T03:34:06.357 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "debug", 2024-05-21T03:34:06.357 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.357 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.358 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.358 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.358 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.358 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.358 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.358 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable/disable debug options", 2024-05-21T03:34:06.358 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.358 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.358 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.358 INFO:teuthology.orchestra.run.smithi027.stdout: "jwt_token_ttl": { 2024-05-21T03:34:06.358 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "jwt_token_ttl", 2024-05-21T03:34:06.358 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.359 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.359 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.359 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "28800", 2024-05-21T03:34:06.359 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.359 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.359 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.359 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.359 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.359 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.359 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.359 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-05-21T03:34:06.359 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-05-21T03:34:06.359 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.360 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.360 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.360 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.360 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.360 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.360 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.360 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.360 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.360 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.360 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.360 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.360 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.360 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.360 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.361 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.361 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.361 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.361 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.361 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.361 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.361 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.361 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.361 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.361 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.361 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.361 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.361 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.361 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.361 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.361 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.362 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.362 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.362 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.362 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.362 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.362 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.362 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.362 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.362 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.362 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.362 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.362 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.362 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.363 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.363 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.363 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.363 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.363 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.363 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.363 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.363 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.363 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.363 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.363 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.363 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.363 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.363 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.364 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.364 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.364 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.364 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.364 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.364 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.364 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.364 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.364 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.364 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.364 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.364 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.365 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.365 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.365 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.365 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.365 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.365 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.365 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.365 INFO:teuthology.orchestra.run.smithi027.stdout: "motd": { 2024-05-21T03:34:06.365 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "motd", 2024-05-21T03:34:06.365 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.365 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.365 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.365 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.365 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.366 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.366 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.366 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "The message of the day", 2024-05-21T03:34:06.366 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.366 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.366 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.366 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect_resolve_ip_addr": { 2024-05-21T03:34:06.366 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "redirect_resolve_ip_addr", 2024-05-21T03:34:06.366 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.366 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.366 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.366 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.367 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.367 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.367 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.367 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.367 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.367 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.367 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.367 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-05-21T03:34:06.367 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-05-21T03:34:06.367 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.367 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.367 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.367 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-05-21T03:34:06.368 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.368 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.368 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.368 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.368 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.368 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.368 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.368 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-05-21T03:34:06.368 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-05-21T03:34:06.368 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.368 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.368 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.369 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8080", 2024-05-21T03:34:06.369 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.369 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.369 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.369 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.369 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.369 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.369 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.369 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.369 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.369 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.369 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.369 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.370 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.370 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.370 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.370 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.370 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.370 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.370 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.370 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.370 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-05-21T03:34:06.370 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-05-21T03:34:06.370 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.371 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.371 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.371 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.371 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.371 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.371 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.371 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.371 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.371 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.371 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.371 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl_server_port": { 2024-05-21T03:34:06.371 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl_server_port", 2024-05-21T03:34:06.371 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.371 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.372 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.372 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8443", 2024-05-21T03:34:06.372 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.372 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.372 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.372 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.372 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.372 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.372 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.372 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-05-21T03:34:06.372 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-05-21T03:34:06.372 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.372 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.373 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.373 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "redirect", 2024-05-21T03:34:06.373 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.373 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.373 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.373 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.373 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect" 2024-05-21T03:34:06.373 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.373 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.373 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.373 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.373 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.373 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-05-21T03:34:06.374 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-05-21T03:34:06.374 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.374 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.374 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.374 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-05-21T03:34:06.374 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-05-21T03:34:06.374 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-05-21T03:34:06.374 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.374 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.374 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.374 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.374 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.375 INFO:teuthology.orchestra.run.smithi027.stdout: "url_prefix": { 2024-05-21T03:34:06.375 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url_prefix", 2024-05-21T03:34:06.375 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.375 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.375 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.375 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.375 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.375 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.375 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.375 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.375 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.375 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.375 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.375 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.376 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.376 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.376 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-05-21T03:34:06.376 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.376 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.376 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.376 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_monitoring": { 2024-05-21T03:34:06.376 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_monitoring", 2024-05-21T03:34:06.376 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.376 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.376 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.376 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.376 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.376 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.377 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.377 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "monitor device health metrics", 2024-05-21T03:34:06.377 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.377 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.377 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.377 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.377 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.377 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.377 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.377 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.377 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.377 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.378 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.378 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.378 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.378 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.378 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.378 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.378 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.378 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.378 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.378 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.378 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.378 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.378 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.379 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.379 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.379 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.379 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.379 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.379 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.379 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.379 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.379 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.379 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.379 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.379 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.379 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.380 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.380 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.380 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.380 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.380 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.380 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.380 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.380 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.380 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.380 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.380 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.380 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.381 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.381 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.381 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.381 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.381 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.381 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.381 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.381 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.381 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.381 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.381 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.381 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.381 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.382 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.382 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.382 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.382 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.382 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.382 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.382 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.382 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.382 INFO:teuthology.orchestra.run.smithi027.stdout: "mark_out_threshold": { 2024-05-21T03:34:06.382 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mark_out_threshold", 2024-05-21T03:34:06.382 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.382 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.383 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.383 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2419200", 2024-05-21T03:34:06.383 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.383 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.383 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.383 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-21T03:34:06.383 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.383 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.383 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.383 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": { 2024-05-21T03:34:06.383 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_name", 2024-05-21T03:34:06.383 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.384 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.384 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.384 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "device_health_metrics", 2024-05-21T03:34:06.384 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.384 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.384 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.384 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-21T03:34:06.384 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.384 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.384 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.384 INFO:teuthology.orchestra.run.smithi027.stdout: "retention_period": { 2024-05-21T03:34:06.384 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retention_period", 2024-05-21T03:34:06.384 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.384 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.385 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.385 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15552000", 2024-05-21T03:34:06.385 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.385 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.385 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.385 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain device health metrics", 2024-05-21T03:34:06.385 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.385 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.385 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.385 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_frequency": { 2024-05-21T03:34:06.385 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_frequency", 2024-05-21T03:34:06.385 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.385 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.385 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.386 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-05-21T03:34:06.386 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.386 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.386 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.386 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-21T03:34:06.386 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.386 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.386 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.386 INFO:teuthology.orchestra.run.smithi027.stdout: "self_heal": { 2024-05-21T03:34:06.386 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "self_heal", 2024-05-21T03:34:06.386 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.386 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.387 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.387 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.387 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.387 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.387 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.387 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-21T03:34:06.387 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.387 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.387 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.387 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-05-21T03:34:06.387 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-05-21T03:34:06.387 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.387 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.388 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.388 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-05-21T03:34:06.388 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.388 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.388 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.388 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and check device health", 2024-05-21T03:34:06.388 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.388 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.388 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.388 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.388 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.388 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.388 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.389 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.389 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.389 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.389 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.389 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.389 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.389 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.389 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.389 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.389 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_threshold": { 2024-05-21T03:34:06.389 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_threshold", 2024-05-21T03:34:06.389 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.389 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.389 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.390 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7257600", 2024-05-21T03:34:06.390 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.390 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.390 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.390 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-21T03:34:06.390 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.390 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.390 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.390 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.390 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.390 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "diskprediction_local", 2024-05-21T03:34:06.390 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.391 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.391 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.391 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.391 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.391 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.391 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.391 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.391 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.391 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.391 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.391 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.391 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.391 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.392 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.392 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.392 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.392 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.392 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.392 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.392 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.392 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.392 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.392 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.392 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.392 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.392 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.393 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.393 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.393 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.393 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.393 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.393 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.393 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.393 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.393 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.393 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.393 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.393 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.393 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.393 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.394 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.394 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.394 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.394 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.394 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.394 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.394 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.394 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.394 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.394 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.394 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.394 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.394 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.394 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.395 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.395 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.395 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.395 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.395 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.395 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.395 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.395 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.395 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.395 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.395 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.395 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.395 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.396 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.396 INFO:teuthology.orchestra.run.smithi027.stdout: "predict_interval": { 2024-05-21T03:34:06.396 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predict_interval", 2024-05-21T03:34:06.396 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.396 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.396 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.396 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-05-21T03:34:06.396 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.396 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.396 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.396 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.396 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.396 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.397 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.397 INFO:teuthology.orchestra.run.smithi027.stdout: "predictor_model": { 2024-05-21T03:34:06.397 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predictor_model", 2024-05-21T03:34:06.397 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.397 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.397 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.397 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "prophetstor", 2024-05-21T03:34:06.397 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.397 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.397 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.397 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.397 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.397 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.397 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.398 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-05-21T03:34:06.398 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-05-21T03:34:06.398 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.398 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.398 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.398 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-05-21T03:34:06.398 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.398 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.398 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.398 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.398 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.398 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.399 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.399 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.399 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.399 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.399 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.399 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.399 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.399 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.399 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.399 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.399 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.399 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.399 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.400 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.400 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.400 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.400 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.400 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "influx", 2024-05-21T03:34:06.400 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": false, 2024-05-21T03:34:06.400 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "influxdb python module not found", 2024-05-21T03:34:06.400 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.400 INFO:teuthology.orchestra.run.smithi027.stdout: "batch_size": { 2024-05-21T03:34:06.400 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "batch_size", 2024-05-21T03:34:06.400 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.400 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.400 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.400 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5000", 2024-05-21T03:34:06.401 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.401 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.401 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.401 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-21T03:34:06.401 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.401 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.401 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.401 INFO:teuthology.orchestra.run.smithi027.stdout: "database": { 2024-05-21T03:34:06.401 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "database", 2024-05-21T03:34:06.401 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.401 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.401 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.401 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "ceph", 2024-05-21T03:34:06.402 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.402 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.402 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.402 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-21T03:34:06.402 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.402 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.402 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.402 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": { 2024-05-21T03:34:06.402 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hostname", 2024-05-21T03:34:06.402 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.402 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.402 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.402 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.403 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.403 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.403 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.403 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server hostname", 2024-05-21T03:34:06.403 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.403 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.403 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.403 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-05-21T03:34:06.403 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-05-21T03:34:06.403 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.403 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.403 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.403 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "30", 2024-05-21T03:34:06.404 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "5", 2024-05-21T03:34:06.404 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.404 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.404 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-21T03:34:06.404 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.404 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.404 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.404 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.404 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.404 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.404 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.404 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.405 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.405 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.405 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.405 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.405 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.405 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.405 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.405 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.405 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.405 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.405 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.405 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.405 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.406 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.406 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.406 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.406 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.406 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.406 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.406 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.406 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.406 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.406 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.406 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.406 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.406 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.407 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.407 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.407 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.407 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.407 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.407 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.407 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.407 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.407 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.407 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.407 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.407 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.407 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.408 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.408 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.408 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.408 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.408 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.408 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.408 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.408 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.408 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.408 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.408 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.408 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.408 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.409 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.409 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.409 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.409 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.409 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.409 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.409 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.409 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.409 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.409 INFO:teuthology.orchestra.run.smithi027.stdout: "password": { 2024-05-21T03:34:06.409 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "password", 2024-05-21T03:34:06.409 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.409 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.410 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.410 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.410 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.410 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.410 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.410 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "password of InfluxDB server user", 2024-05-21T03:34:06.410 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.410 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.410 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.410 INFO:teuthology.orchestra.run.smithi027.stdout: "port": { 2024-05-21T03:34:06.410 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "port", 2024-05-21T03:34:06.410 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.411 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.411 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.411 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8086", 2024-05-21T03:34:06.411 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.411 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.411 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.411 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server port", 2024-05-21T03:34:06.411 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.411 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.411 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.411 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.411 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.411 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.412 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.412 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.412 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.412 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.412 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.412 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.412 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.412 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.412 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.412 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.412 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-05-21T03:34:06.412 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-05-21T03:34:06.412 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.413 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.413 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.413 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "false", 2024-05-21T03:34:06.413 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.413 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.413 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.413 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-21T03:34:06.413 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.413 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.413 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.413 INFO:teuthology.orchestra.run.smithi027.stdout: "threads": { 2024-05-21T03:34:06.413 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threads", 2024-05-21T03:34:06.413 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.414 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.414 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.414 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-05-21T03:34:06.414 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-05-21T03:34:06.414 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "32", 2024-05-21T03:34:06.414 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.414 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-21T03:34:06.414 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.414 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.414 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.414 INFO:teuthology.orchestra.run.smithi027.stdout: "username": { 2024-05-21T03:34:06.414 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "username", 2024-05-21T03:34:06.414 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.415 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.415 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.415 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.415 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.415 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.415 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.415 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "username of InfluxDB server user", 2024-05-21T03:34:06.415 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.415 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.415 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.415 INFO:teuthology.orchestra.run.smithi027.stdout: "verify_ssl": { 2024-05-21T03:34:06.415 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "verify_ssl", 2024-05-21T03:34:06.415 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.416 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.416 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.416 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "true", 2024-05-21T03:34:06.416 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.416 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.416 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.416 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-21T03:34:06.416 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.416 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.416 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.416 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.416 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.416 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.417 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "insights", 2024-05-21T03:34:06.417 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.417 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.417 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.417 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.417 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.417 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.417 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.417 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.417 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.417 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.417 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.417 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.417 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.418 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.418 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.418 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.418 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.418 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.418 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.418 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.418 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.418 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.418 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.418 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.418 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.419 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.419 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.419 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.419 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.419 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.419 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.419 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.419 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.419 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.419 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.419 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.419 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.419 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.420 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.420 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.420 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.420 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.420 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.420 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.420 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.420 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.420 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.420 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.420 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.420 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.420 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.421 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.421 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.421 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.421 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.421 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.421 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.421 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.421 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.421 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.421 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.421 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.421 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.422 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.422 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.422 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.422 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.422 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.422 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.422 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.422 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.422 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.422 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.422 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.422 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.423 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.423 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.423 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.423 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.423 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.423 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.423 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.423 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.423 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.423 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "iostat", 2024-05-21T03:34:06.423 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.423 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.424 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.424 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.424 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.424 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.424 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.424 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.424 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.424 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.424 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.424 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.424 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.424 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.424 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.424 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.425 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.425 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.425 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.425 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.425 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.425 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.425 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.425 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.425 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.425 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.425 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.425 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.425 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.425 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.426 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.426 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.426 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.426 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.426 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.426 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.426 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.426 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.426 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.426 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.426 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.426 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.427 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.427 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.427 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.427 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.427 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.427 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.427 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.427 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.427 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.427 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.427 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.427 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.427 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.427 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.428 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.428 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.428 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.428 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.428 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.428 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.428 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.428 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.428 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.428 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.428 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.428 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.428 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.428 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.429 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.429 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.429 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.429 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.429 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.429 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.429 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.429 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.429 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.429 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.429 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.429 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.429 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.429 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.430 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.430 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "k8sevents", 2024-05-21T03:34:06.430 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.430 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.430 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.430 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_event_retention_days": { 2024-05-21T03:34:06.430 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ceph_event_retention_days", 2024-05-21T03:34:06.430 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.430 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.430 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.430 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7", 2024-05-21T03:34:06.430 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.430 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.431 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.431 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-21T03:34:06.431 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.431 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.431 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.431 INFO:teuthology.orchestra.run.smithi027.stdout: "config_check_secs": { 2024-05-21T03:34:06.431 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_check_secs", 2024-05-21T03:34:06.431 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.431 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.431 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.431 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-05-21T03:34:06.431 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "10", 2024-05-21T03:34:06.431 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.432 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.432 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-21T03:34:06.432 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.432 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.432 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.432 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.432 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.432 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.432 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.432 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.432 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.433 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.433 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.433 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.433 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.433 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.433 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.433 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.433 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.433 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.433 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.433 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.433 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.433 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.433 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.434 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.434 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.434 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.434 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.434 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.434 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.434 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.434 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.434 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.434 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.434 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.434 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.435 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.435 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.435 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.435 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.435 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.435 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.435 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.435 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.435 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.435 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.435 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.435 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.435 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.436 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.436 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.436 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.436 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.436 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.436 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.436 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.436 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.436 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.436 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.436 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.436 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.436 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.437 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.437 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.437 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.437 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.437 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.437 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.437 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.437 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.437 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.437 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.437 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.437 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.437 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.438 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.438 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.438 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.438 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.438 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.438 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.438 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.438 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.438 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.438 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.438 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.438 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "localpool", 2024-05-21T03:34:06.438 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.439 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.439 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.439 INFO:teuthology.orchestra.run.smithi027.stdout: "failure_domain": { 2024-05-21T03:34:06.439 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "failure_domain", 2024-05-21T03:34:06.439 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.439 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.439 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.439 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "host", 2024-05-21T03:34:06.439 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.439 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.439 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.439 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "failure domain for any created local pool", 2024-05-21T03:34:06.439 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-21T03:34:06.440 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.440 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.440 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.440 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.440 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.440 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.440 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.440 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.440 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.440 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.440 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.440 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.440 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.440 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.441 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.441 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.441 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.441 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.441 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.441 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.441 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.441 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.441 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.441 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.441 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.441 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.441 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.442 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.442 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.442 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.442 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.442 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.442 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.442 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.442 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.442 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.442 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.442 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.442 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.443 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.443 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.443 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.443 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.443 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.443 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.443 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.443 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.443 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.443 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.443 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.443 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.443 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.444 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.444 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.444 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.444 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.444 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.444 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.444 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.444 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.444 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.444 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.444 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.444 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.444 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.444 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.445 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.445 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.445 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": { 2024-05-21T03:34:06.445 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_size", 2024-05-21T03:34:06.445 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.445 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.445 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.445 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.445 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.445 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.445 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.445 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default min_size for any created local pool", 2024-05-21T03:34:06.446 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-21T03:34:06.446 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.446 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.446 INFO:teuthology.orchestra.run.smithi027.stdout: "num_rep": { 2024-05-21T03:34:06.446 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "num_rep", 2024-05-21T03:34:06.446 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.446 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.446 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.446 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-05-21T03:34:06.446 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.446 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.446 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.447 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default replica count for any created local pool", 2024-05-21T03:34:06.447 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.447 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.447 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.447 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": { 2024-05-21T03:34:06.447 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_num", 2024-05-21T03:34:06.447 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.447 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.447 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.447 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "128", 2024-05-21T03:34:06.447 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.447 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.447 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.447 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default pg_num for any created local pool", 2024-05-21T03:34:06.448 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.448 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.448 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.448 INFO:teuthology.orchestra.run.smithi027.stdout: "prefix": { 2024-05-21T03:34:06.448 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prefix", 2024-05-21T03:34:06.448 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.448 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.448 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.448 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.448 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.448 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.448 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.448 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name prefix for any created local pool", 2024-05-21T03:34:06.448 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.449 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.449 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.449 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.449 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.449 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.449 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.449 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.449 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.449 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.449 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.449 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.449 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.449 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.449 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.450 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.450 INFO:teuthology.orchestra.run.smithi027.stdout: "subtree": { 2024-05-21T03:34:06.450 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "subtree", 2024-05-21T03:34:06.450 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.450 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.450 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.450 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "rack", 2024-05-21T03:34:06.450 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.450 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.450 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.450 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-21T03:34:06.450 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-21T03:34:06.451 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.451 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.451 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.451 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.451 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mds_autoscaler", 2024-05-21T03:34:06.451 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.451 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.451 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.451 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.451 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.451 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.451 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.452 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.452 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.452 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.452 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.452 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.452 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.452 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.452 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.452 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.452 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.452 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.452 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.452 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.452 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.453 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.453 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.453 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.453 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.453 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.453 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.453 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.453 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.453 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.453 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.453 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.453 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.453 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.453 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.454 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.454 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.454 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.454 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.454 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.454 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.454 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.454 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.454 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.454 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.454 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.454 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.455 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.455 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.455 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.455 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.455 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.455 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.455 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.455 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.455 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.455 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.455 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.455 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.455 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.456 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.456 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.456 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.456 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.456 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.456 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.456 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.456 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.456 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.456 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.456 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.456 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.456 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.456 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.457 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.457 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.457 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.457 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.457 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.457 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.457 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.457 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.457 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.457 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.457 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirroring", 2024-05-21T03:34:06.457 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.458 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.458 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.458 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.458 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.458 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.458 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.458 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.458 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.458 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.458 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.458 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.458 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.458 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.458 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.459 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.459 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.459 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.459 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.459 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.459 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.459 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.459 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.459 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.459 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.459 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.459 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.460 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.460 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.460 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.460 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.460 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.460 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.460 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.460 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.460 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.460 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.460 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.460 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.461 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.461 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.461 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.461 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.461 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.461 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.461 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.461 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.461 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.461 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.461 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.461 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.461 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.461 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.462 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.462 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.462 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.462 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.462 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.462 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.462 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.462 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.462 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.462 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.462 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.462 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.463 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.463 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.463 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.463 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.463 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.463 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.463 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.463 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.463 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.463 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.463 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.463 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.463 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.464 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.464 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.464 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.464 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.464 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.464 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.464 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "nfs", 2024-05-21T03:34:06.464 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.464 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.464 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.464 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.465 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.465 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.465 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.465 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.465 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.465 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.465 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.465 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.465 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.465 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.465 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.465 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.465 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.465 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.466 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.466 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.466 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.466 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.466 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.466 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.466 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.466 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.466 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.466 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.466 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.466 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.467 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.467 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.467 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.467 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.467 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.467 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.467 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.467 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.467 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.467 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.467 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.467 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.467 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.468 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.468 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.468 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.468 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.468 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.468 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.468 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.468 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.468 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.468 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.468 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.468 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.468 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.469 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.469 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.469 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.469 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.469 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.469 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.469 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.469 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.469 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.469 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.469 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.469 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.469 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.470 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.470 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.470 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.470 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.470 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.470 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.470 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.470 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.470 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.470 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.470 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.470 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.471 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.471 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.471 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.471 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.471 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-05-21T03:34:06.471 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.471 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.471 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.471 INFO:teuthology.orchestra.run.smithi027.stdout: "fail_fs": { 2024-05-21T03:34:06.471 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "fail_fs", 2024-05-21T03:34:06.471 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.471 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.471 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.471 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.472 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.472 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.472 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.472 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-21T03:34:06.472 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.472 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.472 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.472 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.472 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.472 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.472 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.472 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.472 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.472 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.473 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.473 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.473 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.473 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.473 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.473 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.473 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.473 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.473 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.473 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.473 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.473 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.473 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.473 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.474 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.474 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.474 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.474 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.474 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.474 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.474 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.474 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.474 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.474 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.474 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.474 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.475 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.475 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.475 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.475 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.475 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.475 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.475 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.475 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.475 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.475 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.475 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.475 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.475 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.475 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.476 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.476 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.476 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.476 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.476 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.476 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.476 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.476 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.476 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.476 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.476 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.476 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.476 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.477 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.477 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.477 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.477 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.477 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.477 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.477 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator": { 2024-05-21T03:34:06.477 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-05-21T03:34:06.477 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.477 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.477 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.477 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.477 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.478 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.478 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.478 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm", 2024-05-21T03:34:06.478 INFO:teuthology.orchestra.run.smithi027.stdout: "rook", 2024-05-21T03:34:06.478 INFO:teuthology.orchestra.run.smithi027.stdout: "test_orchestrator" 2024-05-21T03:34:06.478 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.478 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Orchestrator backend", 2024-05-21T03:34:06.478 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.478 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.478 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.478 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.478 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.478 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.479 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.479 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.479 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.479 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.479 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.479 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.479 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.479 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.479 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.479 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.479 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.479 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.479 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.480 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_perf_query", 2024-05-21T03:34:06.480 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.480 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.480 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.480 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.480 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.480 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.480 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.480 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.480 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.480 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.480 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.480 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.480 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.481 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.481 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.481 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.481 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.481 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.481 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.481 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.481 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.481 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.481 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.481 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.481 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.481 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.482 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.482 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.482 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.482 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.482 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.482 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.482 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.482 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.482 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.482 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.482 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.482 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.483 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.483 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.483 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.483 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.483 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.483 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.483 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.483 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.483 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.483 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.483 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.483 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.483 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.483 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.484 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.484 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.484 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.484 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.484 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.484 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.484 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.484 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.484 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.484 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.484 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.484 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.484 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.485 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.485 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.485 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.485 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.485 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.485 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.485 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.485 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.485 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.485 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.485 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.485 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.485 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.485 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.486 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.486 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.486 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.486 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.486 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.486 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.486 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_support", 2024-05-21T03:34:06.486 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.486 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.486 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.486 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.486 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.487 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.487 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.487 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.487 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.487 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.487 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.487 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.487 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.487 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.487 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.487 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.487 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.487 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.487 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.487 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.487 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.488 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.488 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.488 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.488 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.488 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.488 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.488 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.488 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.488 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.488 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.488 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.488 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.488 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.489 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.489 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.489 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.489 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.489 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.489 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.489 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.489 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.489 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.489 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.489 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.489 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.489 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.489 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.490 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.490 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.490 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.490 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.490 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.490 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.490 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.490 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.490 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.490 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.490 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.490 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.491 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.491 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.491 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.491 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.491 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.491 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.491 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.491 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.491 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.491 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.491 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.491 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.491 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.492 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.492 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.492 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.492 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.492 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.492 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.492 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.492 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.492 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.492 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.492 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.492 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.492 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.492 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_autoscaler", 2024-05-21T03:34:06.493 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.493 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.493 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.493 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.493 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.493 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.493 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.493 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.493 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.493 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.493 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.493 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.493 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.494 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.494 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.494 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.494 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.494 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.494 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.494 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.494 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.494 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.494 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.494 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.494 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.494 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.495 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.495 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.495 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.495 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.495 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.495 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.495 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.495 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.495 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.495 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.495 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.495 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.495 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.496 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.496 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.496 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.496 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.496 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.496 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.496 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.496 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.496 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.496 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.496 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.496 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.496 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.496 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.496 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.496 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.497 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.497 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.497 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.497 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.497 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.497 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.497 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.497 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.497 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.497 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.497 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.497 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.497 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.498 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.498 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-05-21T03:34:06.498 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-05-21T03:34:06.498 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.498 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.498 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.498 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-05-21T03:34:06.498 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.498 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.498 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.498 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.498 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.498 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.498 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.499 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.499 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.499 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.499 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.499 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.499 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.499 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.499 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.499 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.499 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.499 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.499 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.499 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.499 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.500 INFO:teuthology.orchestra.run.smithi027.stdout: "threshold": { 2024-05-21T03:34:06.500 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threshold", 2024-05-21T03:34:06.500 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-05-21T03:34:06.500 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.500 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.500 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-05-21T03:34:06.500 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1.0", 2024-05-21T03:34:06.500 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.500 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.500 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "scaling threshold", 2024-05-21T03:34:06.500 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-21T03:34:06.500 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.500 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.500 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.501 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.501 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.501 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "progress", 2024-05-21T03:34:06.501 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.501 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.501 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.501 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_pg_recovery_event": { 2024-05-21T03:34:06.501 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_pg_recovery_event", 2024-05-21T03:34:06.501 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.501 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.501 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.501 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.501 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.502 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.502 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.502 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-21T03:34:06.502 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.502 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.502 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.502 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-05-21T03:34:06.502 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-05-21T03:34:06.502 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.502 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.502 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.502 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.503 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.504 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.504 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.504 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.504 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.504 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.504 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.505 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.505 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.505 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.505 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.505 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.505 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.505 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.505 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.505 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.505 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.505 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.505 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.505 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.505 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.506 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.506 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.506 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.506 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.506 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.506 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.506 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.506 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.506 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.506 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.506 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.507 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.507 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.507 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.507 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.507 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.507 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.507 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.507 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.507 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.507 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.507 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.507 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.507 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.507 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.508 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.508 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.508 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.508 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.508 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.508 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.508 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.508 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.508 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.508 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.508 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.508 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.508 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.508 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.509 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.509 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.509 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.509 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.509 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.509 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.509 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.509 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.509 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.509 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.509 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.509 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.509 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.509 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.510 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.510 INFO:teuthology.orchestra.run.smithi027.stdout: "max_completed_events": { 2024-05-21T03:34:06.510 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_completed_events", 2024-05-21T03:34:06.510 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.510 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.510 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.510 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "50", 2024-05-21T03:34:06.510 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.510 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.510 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.510 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "number of past completed events to remember", 2024-05-21T03:34:06.510 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.511 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.511 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.511 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-05-21T03:34:06.511 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-05-21T03:34:06.511 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.511 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.511 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.511 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-05-21T03:34:06.511 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.511 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.511 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.511 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long the module is going to sleep", 2024-05-21T03:34:06.511 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.512 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.512 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.512 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.512 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.512 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.512 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.512 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.512 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.512 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.512 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.512 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.512 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.512 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.513 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.513 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.513 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.513 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.513 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.513 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus", 2024-05-21T03:34:06.513 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.513 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.513 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.513 INFO:teuthology.orchestra.run.smithi027.stdout: "cache": { 2024-05-21T03:34:06.513 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cache", 2024-05-21T03:34:06.513 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.513 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.514 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.514 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.514 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.514 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.514 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.514 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.514 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.514 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.514 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.514 INFO:teuthology.orchestra.run.smithi027.stdout: "exclude_perf_counters": { 2024-05-21T03:34:06.514 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "exclude_perf_counters", 2024-05-21T03:34:06.514 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.515 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.515 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.515 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.515 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.515 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.515 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.515 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-21T03:34:06.515 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-21T03:34:06.515 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.515 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.515 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.515 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.515 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.516 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.516 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.516 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.516 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.516 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.516 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.516 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.516 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.516 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.516 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.516 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.517 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.517 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.517 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.517 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.517 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.517 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.517 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.517 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.517 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.517 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.517 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.518 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.518 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.518 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.518 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.518 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.518 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.518 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.518 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.518 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.518 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.518 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.518 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.518 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.519 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.519 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.519 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.519 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.519 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.519 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.519 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.519 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.519 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.519 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.519 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.519 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.519 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.519 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.520 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.520 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.520 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.520 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.520 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.520 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.520 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.520 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.520 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.520 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.520 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.520 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.520 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.521 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.521 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.521 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools": { 2024-05-21T03:34:06.521 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools", 2024-05-21T03:34:06.521 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.521 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.521 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.521 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.521 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.521 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.521 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.521 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.521 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.521 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.522 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.522 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.522 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-21T03:34:06.522 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-21T03:34:06.522 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.522 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.522 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.522 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "300", 2024-05-21T03:34:06.522 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.522 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.522 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.522 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.522 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.523 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.523 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.523 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.523 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_interval": { 2024-05-21T03:34:06.523 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_interval", 2024-05-21T03:34:06.523 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-05-21T03:34:06.523 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.523 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.523 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15.0", 2024-05-21T03:34:06.523 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.523 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.523 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.523 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.523 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.523 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.524 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.524 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.524 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-05-21T03:34:06.524 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-05-21T03:34:06.524 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.524 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.524 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.524 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-05-21T03:34:06.524 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.524 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.524 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.524 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-21T03:34:06.524 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.524 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.525 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.525 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.525 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-05-21T03:34:06.525 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-05-21T03:34:06.525 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.525 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.525 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.525 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "9283", 2024-05-21T03:34:06.525 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.525 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.525 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.525 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-21T03:34:06.525 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.525 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.525 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.526 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.526 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.526 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.526 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.526 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.526 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.526 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.526 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.526 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.526 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.526 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.526 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.526 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.527 INFO:teuthology.orchestra.run.smithi027.stdout: "stale_cache_strategy": { 2024-05-21T03:34:06.527 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stale_cache_strategy", 2024-05-21T03:34:06.527 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.527 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.527 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.527 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "log", 2024-05-21T03:34:06.527 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.527 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.527 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.527 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.527 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.527 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.528 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.528 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-05-21T03:34:06.528 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-05-21T03:34:06.528 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.528 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.528 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.528 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "default", 2024-05-21T03:34:06.528 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.528 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.528 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.528 INFO:teuthology.orchestra.run.smithi027.stdout: "default", 2024-05-21T03:34:06.528 INFO:teuthology.orchestra.run.smithi027.stdout: "error" 2024-05-21T03:34:06.528 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.528 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.529 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.529 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.529 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.529 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.529 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-05-21T03:34:06.529 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-05-21T03:34:06.529 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.529 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.529 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.529 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-05-21T03:34:06.529 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-05-21T03:34:06.529 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-05-21T03:34:06.529 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.529 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.530 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.530 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.530 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.530 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.530 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.530 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.530 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.530 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-05-21T03:34:06.530 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.530 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.530 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.530 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.530 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.531 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.531 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.531 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.531 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.531 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.531 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.531 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.531 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.531 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.531 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.531 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.531 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.531 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.531 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.531 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.532 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.532 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.532 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.532 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.532 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.532 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.532 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.532 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.532 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.532 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.532 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.532 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.533 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.533 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.533 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.533 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.533 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.533 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.533 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.533 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.533 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.533 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.533 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.533 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.533 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.533 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.534 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.534 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.534 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.534 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.534 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.534 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.534 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.534 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.534 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.534 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.534 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.534 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.534 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.535 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.535 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.535 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.535 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.535 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.535 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.535 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.535 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.535 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.535 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.535 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.535 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_snap_create": { 2024-05-21T03:34:06.535 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_snap_create", 2024-05-21T03:34:06.536 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.536 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.536 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.536 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-05-21T03:34:06.536 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.536 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.536 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.536 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.536 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.536 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.536 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.536 INFO:teuthology.orchestra.run.smithi027.stdout: "mirror_snapshot_schedule": { 2024-05-21T03:34:06.536 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirror_snapshot_schedule", 2024-05-21T03:34:06.537 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.537 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.537 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.537 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.537 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.537 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.537 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.537 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.537 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.537 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.537 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.537 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.537 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.538 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.538 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.538 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.538 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.538 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.538 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.538 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.538 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.538 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.538 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.538 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.538 INFO:teuthology.orchestra.run.smithi027.stdout: "trash_purge_schedule": { 2024-05-21T03:34:06.538 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "trash_purge_schedule", 2024-05-21T03:34:06.539 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.539 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.539 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.539 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.539 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.539 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.539 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.539 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.539 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.539 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.539 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.539 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.539 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.540 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.540 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "restful", 2024-05-21T03:34:06.540 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.540 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.540 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.540 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_auth": { 2024-05-21T03:34:06.540 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_auth", 2024-05-21T03:34:06.540 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.540 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.540 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.540 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.540 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.540 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.541 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.541 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.541 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.541 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.541 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.541 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-05-21T03:34:06.541 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-05-21T03:34:06.541 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.541 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.541 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.541 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.541 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.541 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.542 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.542 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.542 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.542 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.542 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.542 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.542 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.542 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.542 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.542 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.542 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.542 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.542 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.543 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.543 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.543 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.543 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.543 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.543 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.543 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.543 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.543 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.543 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.543 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.543 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.543 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.543 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.544 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.544 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.544 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.544 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.544 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.544 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.544 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.544 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.544 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.544 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.544 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.544 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.544 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.544 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.545 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.545 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.545 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.545 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.545 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.545 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.545 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.545 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.545 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.545 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.545 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.545 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.545 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.545 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.546 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.546 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.546 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.546 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.546 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.546 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.546 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.546 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.546 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.546 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.546 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.546 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.546 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.547 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.547 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.547 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.547 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.547 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.547 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.547 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.547 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-05-21T03:34:06.547 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-05-21T03:34:06.547 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.547 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.547 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.547 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.547 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.547 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.547 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.548 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.548 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.548 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.548 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.548 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-05-21T03:34:06.548 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-05-21T03:34:06.548 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.548 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.548 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.548 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.548 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.548 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.548 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.548 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.549 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.549 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.549 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.549 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.549 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.549 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.549 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.549 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.549 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.549 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.549 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.549 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.549 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.549 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.549 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.550 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.550 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.550 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.550 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.550 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.550 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rgw", 2024-05-21T03:34:06.550 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.550 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.550 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.550 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.550 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.550 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.551 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.551 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.551 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.551 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.551 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.551 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.551 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.551 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.551 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.551 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.551 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.551 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.551 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.551 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.552 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.552 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.552 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.552 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.552 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.552 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.552 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.552 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.552 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.552 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.552 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.552 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.552 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.553 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.553 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.553 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.553 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.553 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.553 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.553 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.553 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.553 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.553 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.553 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.553 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.553 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.554 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.554 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.554 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.554 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.554 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.554 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.554 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.554 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.554 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.554 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.554 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.554 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.554 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.555 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.555 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.555 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.555 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.555 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.555 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.555 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.555 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.555 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.555 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.555 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.555 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.555 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.556 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.556 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.556 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.556 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.556 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.556 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.556 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.556 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.556 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.556 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.556 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.556 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.556 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rook", 2024-05-21T03:34:06.557 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.557 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.557 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.557 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.557 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.557 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.557 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.557 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.557 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.557 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.557 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.557 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.557 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.557 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.557 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.558 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.558 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.558 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.558 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.558 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.558 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.558 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.558 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.558 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.558 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.558 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.558 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.559 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.559 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.559 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.559 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.559 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.559 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.559 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.559 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.559 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.559 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.559 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.559 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.560 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.560 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.560 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.560 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.560 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.560 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.560 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.560 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.560 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.560 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.560 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.560 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.560 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.560 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.561 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.561 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.561 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.561 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.561 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.561 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.561 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.561 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.561 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.561 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.561 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.561 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.561 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.561 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.562 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.562 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.562 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.562 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.562 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.562 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.562 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.562 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.562 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.562 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.562 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.562 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.563 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.563 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.563 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.563 INFO:teuthology.orchestra.run.smithi027.stdout: "storage_class": { 2024-05-21T03:34:06.563 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "storage_class", 2024-05-21T03:34:06.563 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.563 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.563 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.563 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "local", 2024-05-21T03:34:06.563 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.563 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.563 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.563 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-21T03:34:06.564 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.564 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.564 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.564 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.564 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.564 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.564 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "selftest", 2024-05-21T03:34:06.564 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.564 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.564 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.564 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.564 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.564 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.565 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.565 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.565 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.565 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.565 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.565 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.565 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.565 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.565 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.565 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.565 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.565 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.565 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.565 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.565 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.566 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.566 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.566 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.566 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.566 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.566 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.566 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.566 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.566 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.566 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.566 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.566 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.567 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.567 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.567 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.567 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.567 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.567 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.567 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.567 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.567 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.567 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.567 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.567 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.567 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.567 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.568 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.568 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.568 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.568 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.568 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.568 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.568 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.568 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.568 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.568 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.568 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.568 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.568 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.568 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.569 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.569 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.569 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.569 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.569 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.569 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.569 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.569 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.569 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.569 INFO:teuthology.orchestra.run.smithi027.stdout: "roption1": { 2024-05-21T03:34:06.569 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption1", 2024-05-21T03:34:06.569 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.569 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.569 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.570 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.570 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.570 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.570 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.570 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.570 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.570 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.570 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.570 INFO:teuthology.orchestra.run.smithi027.stdout: "roption2": { 2024-05-21T03:34:06.570 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption2", 2024-05-21T03:34:06.570 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.570 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.570 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.571 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "xyz", 2024-05-21T03:34:06.571 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.571 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.571 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.571 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.571 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.571 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.571 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.571 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption1": { 2024-05-21T03:34:06.571 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption1", 2024-05-21T03:34:06.571 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.571 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.571 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.571 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.572 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.572 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.572 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.572 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.572 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.572 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.572 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.572 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.572 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption2": { 2024-05-21T03:34:06.572 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption2", 2024-05-21T03:34:06.572 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.572 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.572 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.572 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.573 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.573 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.573 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.573 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.573 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.573 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.573 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.573 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption3": { 2024-05-21T03:34:06.573 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption3", 2024-05-21T03:34:06.573 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-05-21T03:34:06.573 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.573 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.573 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.574 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.574 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.574 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.574 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.574 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.574 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.574 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.574 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.574 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption4": { 2024-05-21T03:34:06.574 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption4", 2024-05-21T03:34:06.574 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.574 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.574 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.575 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.575 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.575 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.575 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.575 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.575 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.575 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.575 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.575 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.575 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption5": { 2024-05-21T03:34:06.575 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption5", 2024-05-21T03:34:06.575 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.575 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.575 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.576 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.576 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.576 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.576 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.576 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.576 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.576 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.576 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.576 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.576 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption6": { 2024-05-21T03:34:06.576 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption6", 2024-05-21T03:34:06.576 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.576 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.576 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.577 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.577 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.577 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.577 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.577 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.577 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.577 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.577 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.577 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.577 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption7": { 2024-05-21T03:34:06.577 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption7", 2024-05-21T03:34:06.577 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.577 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.577 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.578 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.578 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-05-21T03:34:06.578 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "42", 2024-05-21T03:34:06.578 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.578 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.578 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.578 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.578 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.578 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.578 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.578 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.578 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.578 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.579 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.579 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.579 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.579 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.579 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.579 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.579 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.579 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.579 INFO:teuthology.orchestra.run.smithi027.stdout: "testkey": { 2024-05-21T03:34:06.579 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testkey", 2024-05-21T03:34:06.579 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.579 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.579 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.580 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.580 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.580 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.580 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.580 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.580 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.580 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.580 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.580 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.580 INFO:teuthology.orchestra.run.smithi027.stdout: "testlkey": { 2024-05-21T03:34:06.580 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testlkey", 2024-05-21T03:34:06.580 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.580 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.580 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.580 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.581 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.581 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.581 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.581 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.581 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.581 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.581 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.581 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.581 INFO:teuthology.orchestra.run.smithi027.stdout: "testnewline": { 2024-05-21T03:34:06.581 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testnewline", 2024-05-21T03:34:06.581 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.581 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.581 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.581 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.581 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.582 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.582 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.582 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.582 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.582 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.582 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.582 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.582 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.582 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.582 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.582 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smb", 2024-05-21T03:34:06.582 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.582 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.583 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.583 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.583 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.583 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.583 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.583 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.583 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.583 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.583 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.583 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.583 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.583 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.583 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.583 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.584 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.584 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.584 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.584 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.584 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.584 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.584 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.584 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.584 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.584 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.584 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.584 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.584 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.584 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.585 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.585 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.585 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.585 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.585 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.585 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.585 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.585 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.585 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.585 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.585 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.585 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.585 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.585 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.586 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.586 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.586 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.586 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.586 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.586 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.586 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.586 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.586 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.586 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.586 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.586 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.586 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.587 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.587 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.587 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.587 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.587 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.587 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.587 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.587 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.587 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.587 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.587 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.587 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.587 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.587 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.587 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.588 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.588 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.588 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.588 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.588 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.588 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.588 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.588 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.588 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.588 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.588 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.588 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.588 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.588 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.588 INFO:teuthology.orchestra.run.smithi027.stdout: "update_orchestration": { 2024-05-21T03:34:06.589 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "update_orchestration", 2024-05-21T03:34:06.589 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.589 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.589 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.589 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.589 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.589 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.589 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.589 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-21T03:34:06.589 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.589 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.589 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.589 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.589 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.589 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.590 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.590 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-05-21T03:34:06.590 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.590 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.590 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.590 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_m_granularity": { 2024-05-21T03:34:06.590 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_m_granularity", 2024-05-21T03:34:06.590 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.590 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.590 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.590 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.590 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.590 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.591 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.591 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow minute scheduled snapshots", 2024-05-21T03:34:06.591 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.591 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.591 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.591 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.591 INFO:teuthology.orchestra.run.smithi027.stdout: "dump_on_update": { 2024-05-21T03:34:06.591 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dump_on_update", 2024-05-21T03:34:06.591 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.591 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.591 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.591 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.591 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.591 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.591 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.592 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "dump database to debug log on update", 2024-05-21T03:34:06.592 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.592 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.592 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.592 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.592 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.592 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.592 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.592 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.592 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.592 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.592 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.592 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.592 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.593 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.593 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.593 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.593 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.593 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.593 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.593 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.593 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.593 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.593 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.593 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.593 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.593 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.594 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.594 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.594 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.594 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.594 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.594 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.594 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.594 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.594 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.594 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.594 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.594 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.594 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.595 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.595 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.595 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.595 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.595 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.595 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.595 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.595 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.595 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.595 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.595 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.595 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.595 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.595 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.596 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.596 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.596 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.596 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.596 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.596 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.596 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.596 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.596 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.596 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.596 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.596 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.596 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.596 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.597 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.597 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.597 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.597 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.597 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.597 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.597 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.597 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.597 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.597 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.597 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.597 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.597 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.597 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.597 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.598 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.598 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.598 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.598 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.598 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stats", 2024-05-21T03:34:06.598 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.598 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.598 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.598 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.598 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.598 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.598 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.599 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.599 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.599 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.599 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.599 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.599 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.599 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.599 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.599 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.599 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.599 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.599 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.599 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.599 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.599 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.600 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.600 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.600 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.600 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.600 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.600 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.600 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.600 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.600 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.600 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.600 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.600 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.600 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.601 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.601 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.601 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.601 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.601 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.601 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.601 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.601 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.601 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.601 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.601 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.601 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.601 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.602 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.602 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.602 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.602 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.602 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.602 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.602 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.602 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.602 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.602 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.602 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.602 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.602 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.603 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.603 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.603 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.603 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.603 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.603 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.603 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.603 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.603 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.603 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.603 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.603 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.603 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.604 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.604 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.604 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.604 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.604 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.604 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.604 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.604 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.604 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.604 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.604 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.604 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.604 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "status", 2024-05-21T03:34:06.605 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.605 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.605 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.605 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.605 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.605 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.605 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.605 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.605 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.605 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.605 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.605 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.605 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.605 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.605 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.606 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.606 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.606 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.606 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.606 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.606 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.606 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.606 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.606 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.606 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.606 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.607 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.607 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.607 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.607 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.607 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.607 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.607 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.607 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.607 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.607 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.607 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.607 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.607 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.607 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.608 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.608 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.608 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.608 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.608 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.608 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.608 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.608 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.608 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.608 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.608 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.608 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.608 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.608 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.608 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.608 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.609 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.609 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.609 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.609 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.609 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.609 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.609 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.609 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.609 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.609 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.609 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.609 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.609 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.610 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.610 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.610 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.610 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.610 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.610 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.610 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.610 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.610 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.610 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.610 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.610 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.611 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.611 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.611 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.611 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.611 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.611 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telegraf", 2024-05-21T03:34:06.611 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.611 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.611 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.611 INFO:teuthology.orchestra.run.smithi027.stdout: "address": { 2024-05-21T03:34:06.611 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "address", 2024-05-21T03:34:06.611 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.611 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.611 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.611 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-21T03:34:06.612 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.612 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.612 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.612 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.612 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.612 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.612 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.612 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-05-21T03:34:06.612 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-05-21T03:34:06.612 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.612 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.612 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.612 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15", 2024-05-21T03:34:06.612 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.613 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.613 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.613 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.613 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.613 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.613 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.613 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.613 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.613 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.613 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.613 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.613 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.613 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.614 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.614 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.614 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.614 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.614 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.614 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.614 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.614 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.614 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.614 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.614 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.614 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.615 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.615 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.615 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.615 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.615 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.615 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.615 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.615 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.615 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.615 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.615 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.615 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.615 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.615 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.616 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.616 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.616 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.616 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.616 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.616 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.616 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.616 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.616 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.616 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.616 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.616 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.616 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.617 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.617 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.617 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.617 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.617 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.617 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.617 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.617 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.617 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.617 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.617 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.617 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.617 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.618 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.618 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.618 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.618 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.618 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.618 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.618 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.618 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.618 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.618 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.618 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.619 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.619 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.619 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.619 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.619 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.619 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.619 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.619 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.619 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.619 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.619 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.619 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telemetry", 2024-05-21T03:34:06.620 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.620 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.620 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.620 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_basic": { 2024-05-21T03:34:06.620 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_basic", 2024-05-21T03:34:06.620 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.620 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.620 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.620 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.620 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.620 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.620 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.620 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-21T03:34:06.620 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.620 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.621 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.621 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_crash": { 2024-05-21T03:34:06.621 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_crash", 2024-05-21T03:34:06.621 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.621 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.621 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.621 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.621 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.621 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.621 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.621 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-21T03:34:06.621 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.621 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.621 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.622 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_device": { 2024-05-21T03:34:06.622 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_device", 2024-05-21T03:34:06.622 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.622 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.622 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.622 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.622 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.622 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.622 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.622 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-21T03:34:06.622 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.622 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.622 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.623 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_ident": { 2024-05-21T03:34:06.623 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_ident", 2024-05-21T03:34:06.623 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.623 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.623 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.623 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.623 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.623 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.623 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.623 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-21T03:34:06.623 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.623 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.623 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.624 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_perf": { 2024-05-21T03:34:06.624 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_perf", 2024-05-21T03:34:06.624 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.624 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.624 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.624 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.624 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.624 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.624 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.624 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-21T03:34:06.624 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.624 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.624 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.625 INFO:teuthology.orchestra.run.smithi027.stdout: "contact": { 2024-05-21T03:34:06.625 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "contact", 2024-05-21T03:34:06.625 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.625 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.625 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.625 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.625 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.625 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.625 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.625 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.625 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.625 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.625 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.626 INFO:teuthology.orchestra.run.smithi027.stdout: "description": { 2024-05-21T03:34:06.626 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "description", 2024-05-21T03:34:06.626 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.626 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.626 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.626 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.626 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.626 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.626 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.626 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.626 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.626 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.626 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.627 INFO:teuthology.orchestra.run.smithi027.stdout: "device_url": { 2024-05-21T03:34:06.627 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_url", 2024-05-21T03:34:06.627 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.627 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.627 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.627 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-21T03:34:06.627 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.627 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.627 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.627 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.627 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.627 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.627 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.627 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-05-21T03:34:06.628 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-05-21T03:34:06.628 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.628 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.628 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.628 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.628 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.628 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.628 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.628 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.628 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.628 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.628 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.628 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-05-21T03:34:06.628 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-05-21T03:34:06.629 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.629 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.629 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.629 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "24", 2024-05-21T03:34:06.629 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "8", 2024-05-21T03:34:06.629 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.629 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.629 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.629 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.629 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.629 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.629 INFO:teuthology.orchestra.run.smithi027.stdout: "last_opt_revision": { 2024-05-21T03:34:06.629 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "last_opt_revision", 2024-05-21T03:34:06.630 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.630 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.630 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.630 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1", 2024-05-21T03:34:06.630 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.630 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.630 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.630 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.630 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.630 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.630 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.630 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard": { 2024-05-21T03:34:06.630 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard", 2024-05-21T03:34:06.630 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.631 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.631 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.631 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.631 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.631 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.631 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.631 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.631 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.631 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.631 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.631 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard_description": { 2024-05-21T03:34:06.631 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard_description", 2024-05-21T03:34:06.631 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.631 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.632 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.632 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.632 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.632 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.632 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.632 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.632 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.632 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.632 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.632 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.632 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.632 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.632 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.633 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.633 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.633 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.633 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.633 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.633 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.633 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.633 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.633 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.633 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.633 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.633 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.633 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.633 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.633 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.634 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.634 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.634 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.634 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.634 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.634 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.634 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.634 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.634 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.634 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.634 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.634 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.634 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.635 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.635 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.635 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.635 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.635 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.635 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.635 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.635 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.635 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.635 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.635 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.635 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.635 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.635 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.636 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.636 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.636 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.636 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.636 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.636 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.636 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.636 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.636 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.636 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.636 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.636 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.636 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.637 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.637 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.637 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.637 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.637 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.637 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.637 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.637 INFO:teuthology.orchestra.run.smithi027.stdout: "organization": { 2024-05-21T03:34:06.637 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "organization", 2024-05-21T03:34:06.637 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.637 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.637 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.637 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.637 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.638 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.638 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.638 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.638 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.638 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.638 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.638 INFO:teuthology.orchestra.run.smithi027.stdout: "proxy": { 2024-05-21T03:34:06.638 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "proxy", 2024-05-21T03:34:06.638 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.638 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.638 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.638 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.638 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.638 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.639 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.639 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.639 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.639 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.639 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.639 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.639 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.639 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.639 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.639 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.639 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.639 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.639 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.639 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.640 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.640 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.640 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.640 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.640 INFO:teuthology.orchestra.run.smithi027.stdout: "url": { 2024-05-21T03:34:06.640 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url", 2024-05-21T03:34:06.640 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.640 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.640 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.640 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-21T03:34:06.640 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.640 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.640 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.641 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.641 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.641 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.641 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.641 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.641 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.641 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "test_orchestrator", 2024-05-21T03:34:06.641 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.641 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.641 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.641 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.641 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.642 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.642 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.642 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.642 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.642 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.642 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.642 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.642 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.642 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.642 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.642 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.642 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.642 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.643 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.643 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.643 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.643 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.643 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.643 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.643 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.643 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.643 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.643 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.643 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.643 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.643 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.644 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.644 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.644 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.644 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.644 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.644 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.644 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.644 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.644 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.644 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.644 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.644 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.644 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.645 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.645 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.645 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.645 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.645 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.645 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.645 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.645 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.645 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.645 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.645 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.646 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.646 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.646 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.646 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.646 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.646 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.646 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.646 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.647 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.647 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.647 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.647 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.647 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.647 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.647 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.647 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.647 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.647 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.647 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.647 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.647 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.648 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.648 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.648 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.648 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.648 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.648 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.648 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.648 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.648 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-05-21T03:34:06.648 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.648 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.648 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.648 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.649 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.649 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.649 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.649 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.649 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.649 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.649 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.649 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.649 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.649 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.649 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.649 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.649 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.649 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.649 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.650 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.650 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.650 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.650 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.650 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.650 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.650 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.650 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.650 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.650 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.650 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.650 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.650 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.651 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.651 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.651 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.651 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.651 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.651 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.651 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.651 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.651 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.651 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.651 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.651 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.651 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.651 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.652 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.652 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.652 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.652 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.652 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.652 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.652 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.652 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.652 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.652 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.652 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.652 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.652 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.652 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.653 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.653 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.653 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.653 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.653 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.653 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.653 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.653 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.653 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.653 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_clones": { 2024-05-21T03:34:06.653 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_clones", 2024-05-21T03:34:06.653 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.653 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.654 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.654 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4", 2024-05-21T03:34:06.654 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.654 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.654 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.654 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-21T03:34:06.654 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.654 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.654 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.654 INFO:teuthology.orchestra.run.smithi027.stdout: "periodic_async_work": { 2024-05-21T03:34:06.654 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "periodic_async_work", 2024-05-21T03:34:06.654 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.655 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.655 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.655 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.655 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.655 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.655 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.655 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Periodically check for async work", 2024-05-21T03:34:06.655 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.655 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.655 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.655 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_delay": { 2024-05-21T03:34:06.655 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_delay", 2024-05-21T03:34:06.655 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.656 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.656 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.656 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.656 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.656 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.656 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.656 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-21T03:34:06.656 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.656 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.656 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.656 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_no_wait": { 2024-05-21T03:34:06.656 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_no_wait", 2024-05-21T03:34:06.656 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.657 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.657 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.657 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.657 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.657 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.657 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.657 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-21T03:34:06.657 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.657 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.657 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.657 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.657 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.657 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.658 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.658 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.658 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.658 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.658 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.658 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.658 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.658 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.658 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.658 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.658 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.658 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.659 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix", 2024-05-21T03:34:06.659 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.659 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.659 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.659 INFO:teuthology.orchestra.run.smithi027.stdout: "discovery_interval": { 2024-05-21T03:34:06.659 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "discovery_interval", 2024-05-21T03:34:06.659 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-05-21T03:34:06.659 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.659 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.659 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "100", 2024-05-21T03:34:06.659 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.659 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.659 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.660 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.660 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.660 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.660 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.660 INFO:teuthology.orchestra.run.smithi027.stdout: "identifier": { 2024-05-21T03:34:06.660 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "identifier", 2024-05-21T03:34:06.660 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.660 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.660 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.660 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.660 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.660 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.661 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.661 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.661 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.661 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.661 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.661 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-05-21T03:34:06.661 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-05-21T03:34:06.661 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.661 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.661 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.661 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-05-21T03:34:06.661 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.661 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.661 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.662 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.662 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.662 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.662 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.662 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.662 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.662 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.662 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.662 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.662 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.662 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.662 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.663 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.663 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.663 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.663 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.663 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.663 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.663 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.663 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.663 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.663 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.663 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.663 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.663 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.664 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.664 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.664 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.664 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.664 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.664 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.664 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.664 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.664 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.664 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.664 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.664 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.664 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.665 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.665 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.665 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.665 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.665 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.665 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.665 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.665 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.665 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.665 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.665 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.665 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.665 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.665 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.666 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.666 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.666 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.666 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.666 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.666 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.666 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.666 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.666 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.666 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.666 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.666 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.666 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.666 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.667 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.667 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.667 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.667 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.667 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.667 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.667 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.667 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.667 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.667 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.667 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.667 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.667 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.668 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.668 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.668 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.668 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.668 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_host": { 2024-05-21T03:34:06.668 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_host", 2024-05-21T03:34:06.668 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.668 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.668 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.668 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.668 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.668 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.668 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.669 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.669 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.669 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.669 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.669 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_port": { 2024-05-21T03:34:06.669 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_port", 2024-05-21T03:34:06.669 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.669 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.669 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.669 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10051", 2024-05-21T03:34:06.669 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.669 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.669 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.670 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.670 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.670 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.670 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.670 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_sender": { 2024-05-21T03:34:06.670 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_sender", 2024-05-21T03:34:06.670 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.670 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.670 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.670 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-21T03:34:06.670 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.670 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.670 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.671 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.671 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.671 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.671 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.671 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.671 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.671 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.671 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-05-21T03:34:06.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.671 INFO:teuthology.orchestra.run.smithi027.stdout: "gid": 24914, 2024-05-21T03:34:06.671 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "y", 2024-05-21T03:34:06.671 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_features": 4540701547738038271, 2024-05-21T03:34:06.671 INFO:teuthology.orchestra.run.smithi027.stdout: "available_modules": [ 2024-05-21T03:34:06.672 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.672 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "alerts", 2024-05-21T03:34:06.672 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.672 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.672 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.672 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-05-21T03:34:06.672 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-05-21T03:34:06.672 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.672 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.672 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.672 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-05-21T03:34:06.672 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.672 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.672 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.672 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How frequently to reexamine health status", 2024-05-21T03:34:06.673 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.673 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.673 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.673 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.673 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.673 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.673 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.673 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.673 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.673 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.673 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.673 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.673 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.673 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.674 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.674 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.674 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.674 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.674 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.674 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.674 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.674 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.674 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.674 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.674 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.674 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.674 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.675 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.675 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.675 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.675 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.675 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.675 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.675 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.675 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.675 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.675 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.675 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.675 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.675 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.675 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.676 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.676 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.676 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.676 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.676 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.676 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.676 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.676 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.676 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.676 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.676 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.676 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.676 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.676 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.677 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.677 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.677 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.677 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.677 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.677 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.677 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.677 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.677 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.677 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.677 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.677 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.677 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.678 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.678 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_destination": { 2024-05-21T03:34:06.678 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_destination", 2024-05-21T03:34:06.678 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.678 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.678 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.678 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.678 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.678 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.678 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.678 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email address to send alerts to", 2024-05-21T03:34:06.678 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.679 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.679 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.679 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_from_name": { 2024-05-21T03:34:06.679 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_from_name", 2024-05-21T03:34:06.679 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.679 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.679 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.679 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "Ceph", 2024-05-21T03:34:06.679 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.679 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.679 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.679 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email From: name", 2024-05-21T03:34:06.679 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.679 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.680 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.680 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_host": { 2024-05-21T03:34:06.680 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_host", 2024-05-21T03:34:06.680 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.680 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.680 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.680 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.680 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.680 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.680 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.680 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP server", 2024-05-21T03:34:06.680 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.680 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.680 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.681 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_password": { 2024-05-21T03:34:06.681 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_password", 2024-05-21T03:34:06.681 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.681 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.681 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.681 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.681 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.681 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.681 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.681 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Password to authenticate with", 2024-05-21T03:34:06.681 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.681 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.681 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.682 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_port": { 2024-05-21T03:34:06.682 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_port", 2024-05-21T03:34:06.682 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.682 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.682 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.682 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "465", 2024-05-21T03:34:06.682 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.682 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.682 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.682 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP port", 2024-05-21T03:34:06.682 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.682 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.683 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.683 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_sender": { 2024-05-21T03:34:06.683 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_sender", 2024-05-21T03:34:06.683 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.683 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.683 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.683 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.683 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.683 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.683 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.683 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP envelope sender", 2024-05-21T03:34:06.683 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.683 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.683 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.684 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_ssl": { 2024-05-21T03:34:06.684 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_ssl", 2024-05-21T03:34:06.684 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.684 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.684 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.684 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.684 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.684 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.684 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.684 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-21T03:34:06.684 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.684 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.684 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.685 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_user": { 2024-05-21T03:34:06.685 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_user", 2024-05-21T03:34:06.685 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.685 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.685 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.685 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.685 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.685 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.685 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.685 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "User to authenticate as", 2024-05-21T03:34:06.685 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.685 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.685 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.686 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.686 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.686 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.686 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.686 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.686 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.686 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.686 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.686 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.686 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.686 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.686 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.687 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.687 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.687 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.687 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "balancer", 2024-05-21T03:34:06.687 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.687 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.687 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.687 INFO:teuthology.orchestra.run.smithi027.stdout: "active": { 2024-05-21T03:34:06.687 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "active", 2024-05-21T03:34:06.687 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.687 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.687 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.688 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.688 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.688 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.688 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.688 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically balance PGs across cluster", 2024-05-21T03:34:06.688 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.688 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.688 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.688 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_time": { 2024-05-21T03:34:06.688 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_time", 2024-05-21T03:34:06.688 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.688 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.688 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.689 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0000", 2024-05-21T03:34:06.689 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.689 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.689 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.689 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "beginning time of day to automatically balance", 2024-05-21T03:34:06.689 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-21T03:34:06.689 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.689 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.689 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_weekday": { 2024-05-21T03:34:06.689 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_weekday", 2024-05-21T03:34:06.689 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-05-21T03:34:06.689 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.689 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.689 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.689 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-05-21T03:34:06.690 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-05-21T03:34:06.690 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.690 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-21T03:34:06.690 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-21T03:34:06.690 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.690 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.690 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_max_iterations": { 2024-05-21T03:34:06.690 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_max_iterations", 2024-05-21T03:34:06.690 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-05-21T03:34:06.690 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.690 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.690 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "25", 2024-05-21T03:34:06.690 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-05-21T03:34:06.691 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "250", 2024-05-21T03:34:06.691 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.691 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-21T03:34:06.691 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.691 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.691 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.691 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_metrics": { 2024-05-21T03:34:06.691 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_metrics", 2024-05-21T03:34:06.691 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.691 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.691 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.691 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "pgs,objects,bytes", 2024-05-21T03:34:06.691 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.691 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.692 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.692 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-21T03:34:06.692 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-21T03:34:06.692 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.692 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.692 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_step": { 2024-05-21T03:34:06.692 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_step", 2024-05-21T03:34:06.692 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-05-21T03:34:06.692 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.692 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.692 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.5", 2024-05-21T03:34:06.692 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0.001", 2024-05-21T03:34:06.692 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "0.999", 2024-05-21T03:34:06.692 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.693 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "aggressiveness of optimization", 2024-05-21T03:34:06.693 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-21T03:34:06.693 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.693 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.693 INFO:teuthology.orchestra.run.smithi027.stdout: "end_time": { 2024-05-21T03:34:06.693 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_time", 2024-05-21T03:34:06.693 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.693 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.693 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.693 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2359", 2024-05-21T03:34:06.693 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.693 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.693 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.694 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ending time of day to automatically balance", 2024-05-21T03:34:06.694 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-21T03:34:06.694 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.694 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.694 INFO:teuthology.orchestra.run.smithi027.stdout: "end_weekday": { 2024-05-21T03:34:06.694 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_weekday", 2024-05-21T03:34:06.694 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-05-21T03:34:06.694 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.694 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.694 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.694 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-05-21T03:34:06.694 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-05-21T03:34:06.694 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.695 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-21T03:34:06.695 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-21T03:34:06.695 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.695 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.695 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.695 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.695 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.695 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.695 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.695 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.695 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.695 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.695 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.696 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.696 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.696 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.696 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.696 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.696 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.696 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.696 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.696 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.696 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.696 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.696 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.696 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.696 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.697 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.697 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.697 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.697 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.697 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.697 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.697 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.697 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.697 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.697 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.697 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.697 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.697 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.697 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.698 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.698 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.698 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.698 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.698 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.698 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.698 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.698 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.698 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.698 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.698 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.698 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.698 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.699 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.699 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.699 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.699 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.699 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.699 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.699 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.699 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.699 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.699 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.699 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.699 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.699 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.699 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.700 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.700 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.700 INFO:teuthology.orchestra.run.smithi027.stdout: "min_score": { 2024-05-21T03:34:06.700 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_score", 2024-05-21T03:34:06.700 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-05-21T03:34:06.700 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.700 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.700 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.700 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.700 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.700 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.700 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-21T03:34:06.700 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.700 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.701 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.701 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-05-21T03:34:06.701 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-05-21T03:34:06.701 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.701 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.701 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.701 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "upmap", 2024-05-21T03:34:06.701 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.701 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.701 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.701 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-compat", 2024-05-21T03:34:06.701 INFO:teuthology.orchestra.run.smithi027.stdout: "none", 2024-05-21T03:34:06.701 INFO:teuthology.orchestra.run.smithi027.stdout: "read", 2024-05-21T03:34:06.701 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap", 2024-05-21T03:34:06.702 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap-read" 2024-05-21T03:34:06.702 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.702 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Balancer mode", 2024-05-21T03:34:06.702 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.702 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.702 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.702 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_ids": { 2024-05-21T03:34:06.702 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_ids", 2024-05-21T03:34:06.702 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.702 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.702 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.702 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.703 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.703 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.703 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.703 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-21T03:34:06.703 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.703 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.703 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.703 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-05-21T03:34:06.703 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-05-21T03:34:06.703 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.703 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.703 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.703 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-05-21T03:34:06.703 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.704 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.704 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.704 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-21T03:34:06.704 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.704 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.704 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.704 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.704 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.704 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.704 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.704 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.704 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.704 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.704 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.705 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.705 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.705 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.705 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.705 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.705 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_deviation": { 2024-05-21T03:34:06.705 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_deviation", 2024-05-21T03:34:06.705 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.705 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.705 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.705 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-05-21T03:34:06.705 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-05-21T03:34:06.706 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.706 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.706 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-21T03:34:06.706 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-21T03:34:06.706 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.706 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.706 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_optimizations": { 2024-05-21T03:34:06.706 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_optimizations", 2024-05-21T03:34:06.706 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-05-21T03:34:06.706 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.706 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.706 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-05-21T03:34:06.707 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.707 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.707 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.707 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-21T03:34:06.707 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.707 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.707 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.707 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.707 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.707 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm", 2024-05-21T03:34:06.707 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.707 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.708 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.708 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_down_multiplier": { 2024-05-21T03:34:06.708 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_down_multiplier", 2024-05-21T03:34:06.708 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-05-21T03:34:06.708 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.708 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.708 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-05-21T03:34:06.708 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.708 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.708 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.708 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-21T03:34:06.708 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.708 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.708 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.709 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_refresh_rate": { 2024-05-21T03:34:06.709 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_refresh_rate", 2024-05-21T03:34:06.709 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.709 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.709 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.709 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "20", 2024-05-21T03:34:06.709 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.709 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.709 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.709 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-21T03:34:06.709 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.709 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.709 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.710 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_starting_port": { 2024-05-21T03:34:06.710 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_starting_port", 2024-05-21T03:34:06.710 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.710 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.710 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.710 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4721", 2024-05-21T03:34:06.710 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.710 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.710 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.710 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-21T03:34:06.710 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.710 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.710 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.711 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_ptrace": { 2024-05-21T03:34:06.711 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_ptrace", 2024-05-21T03:34:06.711 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.711 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.711 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.711 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.711 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.711 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.711 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.711 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-21T03:34:06.711 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-21T03:34:06.711 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.711 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.711 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_interval": { 2024-05-21T03:34:06.712 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_interval", 2024-05-21T03:34:06.712 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.712 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.712 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.712 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-05-21T03:34:06.712 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.712 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.712 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.712 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-21T03:34:06.712 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.712 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.712 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.712 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_memory_target_ratio": { 2024-05-21T03:34:06.712 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_memory_target_ratio", 2024-05-21T03:34:06.713 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-05-21T03:34:06.713 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.713 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.713 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.7", 2024-05-21T03:34:06.713 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.713 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.713 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.713 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-21T03:34:06.713 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.713 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.713 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.713 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm_log_destination": { 2024-05-21T03:34:06.713 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm_log_destination", 2024-05-21T03:34:06.713 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.713 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.714 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.714 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.714 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.714 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.714 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.714 INFO:teuthology.orchestra.run.smithi027.stdout: "file", 2024-05-21T03:34:06.714 INFO:teuthology.orchestra.run.smithi027.stdout: "file,syslog", 2024-05-21T03:34:06.714 INFO:teuthology.orchestra.run.smithi027.stdout: "syslog" 2024-05-21T03:34:06.714 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.714 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-21T03:34:06.714 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.714 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.715 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.715 INFO:teuthology.orchestra.run.smithi027.stdout: "cgroups_split": { 2024-05-21T03:34:06.715 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cgroups_split", 2024-05-21T03:34:06.715 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.715 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.715 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.715 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.715 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.715 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.715 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.715 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-21T03:34:06.715 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.716 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.716 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.716 INFO:teuthology.orchestra.run.smithi027.stdout: "config_checks_enabled": { 2024-05-21T03:34:06.716 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_checks_enabled", 2024-05-21T03:34:06.716 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.716 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.716 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.716 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.716 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.716 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.716 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.716 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-21T03:34:06.716 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.716 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.717 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.717 INFO:teuthology.orchestra.run.smithi027.stdout: "config_dashboard": { 2024-05-21T03:34:06.717 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_dashboard", 2024-05-21T03:34:06.717 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.717 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.717 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.717 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.717 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.717 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.717 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.717 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-21T03:34:06.717 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.717 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.717 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.718 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_alertmanager": { 2024-05-21T03:34:06.718 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_alertmanager", 2024-05-21T03:34:06.718 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.718 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.718 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.718 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-21T03:34:06.718 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.718 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.718 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.718 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-05-21T03:34:06.718 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.718 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.719 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.719 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_base": { 2024-05-21T03:34:06.719 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_base", 2024-05-21T03:34:06.719 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.719 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.719 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.719 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-21T03:34:06.719 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.719 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.719 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.719 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Container image name, without the tag", 2024-05-21T03:34:06.719 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.720 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.720 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.720 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_elasticsearch": { 2024-05-21T03:34:06.720 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_elasticsearch", 2024-05-21T03:34:06.720 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.720 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.720 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.720 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-21T03:34:06.720 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.720 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.720 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.720 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "elasticsearch container image", 2024-05-21T03:34:06.720 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.720 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.721 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.721 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_grafana": { 2024-05-21T03:34:06.721 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_grafana", 2024-05-21T03:34:06.721 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.721 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.721 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.721 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-21T03:34:06.721 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.721 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.721 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.721 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-05-21T03:34:06.721 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.721 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.721 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.722 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_haproxy": { 2024-05-21T03:34:06.722 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_haproxy", 2024-05-21T03:34:06.722 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.722 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.722 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.722 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-21T03:34:06.722 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.722 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.722 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.722 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "HAproxy container image", 2024-05-21T03:34:06.722 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.722 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.722 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.723 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_agent": { 2024-05-21T03:34:06.723 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_agent", 2024-05-21T03:34:06.723 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.723 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.723 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.723 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-21T03:34:06.723 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.723 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.723 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.723 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger agent container image", 2024-05-21T03:34:06.723 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.723 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.723 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.724 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_collector": { 2024-05-21T03:34:06.724 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_collector", 2024-05-21T03:34:06.724 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.724 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.724 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.724 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-21T03:34:06.724 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.724 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.724 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.724 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger collector container image", 2024-05-21T03:34:06.724 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.724 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.724 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.725 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_query": { 2024-05-21T03:34:06.725 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_query", 2024-05-21T03:34:06.725 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.725 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.725 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.725 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-21T03:34:06.725 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.725 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.725 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.725 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger query container image", 2024-05-21T03:34:06.725 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.725 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.725 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.726 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_keepalived": { 2024-05-21T03:34:06.726 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_keepalived", 2024-05-21T03:34:06.726 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.726 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.726 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.726 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-21T03:34:06.726 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.726 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.726 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.726 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Keepalived container image", 2024-05-21T03:34:06.726 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.726 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.726 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.726 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_loki": { 2024-05-21T03:34:06.727 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_loki", 2024-05-21T03:34:06.727 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.727 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.727 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.727 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-21T03:34:06.727 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.727 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.727 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.727 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Loki container image", 2024-05-21T03:34:06.727 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.727 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.727 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.727 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_node_exporter": { 2024-05-21T03:34:06.727 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_node_exporter", 2024-05-21T03:34:06.728 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.728 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.728 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.728 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-21T03:34:06.728 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.728 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.728 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.728 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-05-21T03:34:06.728 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.728 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.728 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.728 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nvmeof": { 2024-05-21T03:34:06.728 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nvmeof", 2024-05-21T03:34:06.728 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.728 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.729 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.729 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-21T03:34:06.729 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.729 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.729 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.729 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nvme-of container image", 2024-05-21T03:34:06.729 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.729 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.729 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.729 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_prometheus": { 2024-05-21T03:34:06.729 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_prometheus", 2024-05-21T03:34:06.729 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.729 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.730 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.730 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-21T03:34:06.730 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.730 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.730 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.730 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-05-21T03:34:06.730 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.730 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.730 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.730 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_promtail": { 2024-05-21T03:34:06.730 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_promtail", 2024-05-21T03:34:06.730 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.731 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.731 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.731 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-21T03:34:06.731 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.731 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.731 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.731 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Promtail container image", 2024-05-21T03:34:06.731 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.731 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.731 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.731 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba": { 2024-05-21T03:34:06.731 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba", 2024-05-21T03:34:06.731 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.731 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.732 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.732 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-21T03:34:06.732 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.732 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.732 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.732 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB container image", 2024-05-21T03:34:06.732 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.732 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.732 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.732 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_snmp_gateway": { 2024-05-21T03:34:06.732 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_snmp_gateway", 2024-05-21T03:34:06.732 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.732 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.732 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.733 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-21T03:34:06.733 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.733 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.733 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.733 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SNMP Gateway container image", 2024-05-21T03:34:06.733 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.733 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.733 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.733 INFO:teuthology.orchestra.run.smithi027.stdout: "container_init": { 2024-05-21T03:34:06.733 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_init", 2024-05-21T03:34:06.733 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.733 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.733 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.734 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.734 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.734 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.734 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.734 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Run podman/docker with `--init`", 2024-05-21T03:34:06.734 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.734 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.734 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.734 INFO:teuthology.orchestra.run.smithi027.stdout: "daemon_cache_timeout": { 2024-05-21T03:34:06.734 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "daemon_cache_timeout", 2024-05-21T03:34:06.734 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.734 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.734 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.734 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-05-21T03:34:06.735 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.735 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.735 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.735 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-21T03:34:06.735 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.735 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.735 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.735 INFO:teuthology.orchestra.run.smithi027.stdout: "default_cephadm_command_timeout": { 2024-05-21T03:34:06.735 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_cephadm_command_timeout", 2024-05-21T03:34:06.735 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.735 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.735 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.735 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "900", 2024-05-21T03:34:06.735 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.736 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.736 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.736 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-21T03:34:06.736 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.736 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.736 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.736 INFO:teuthology.orchestra.run.smithi027.stdout: "default_registry": { 2024-05-21T03:34:06.736 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_registry", 2024-05-21T03:34:06.736 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.736 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.736 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.736 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io", 2024-05-21T03:34:06.736 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.736 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.737 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.737 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-21T03:34:06.737 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.737 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.737 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.737 INFO:teuthology.orchestra.run.smithi027.stdout: "device_cache_timeout": { 2024-05-21T03:34:06.737 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_cache_timeout", 2024-05-21T03:34:06.737 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.737 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.737 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.737 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1800", 2024-05-21T03:34:06.737 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.737 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.738 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.738 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache device inventory", 2024-05-21T03:34:06.738 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.738 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.738 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.738 INFO:teuthology.orchestra.run.smithi027.stdout: "device_enhanced_scan": { 2024-05-21T03:34:06.738 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_enhanced_scan", 2024-05-21T03:34:06.738 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.738 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.738 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.738 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.738 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.738 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.738 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.739 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-21T03:34:06.739 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.739 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.739 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.739 INFO:teuthology.orchestra.run.smithi027.stdout: "facts_cache_timeout": { 2024-05-21T03:34:06.739 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "facts_cache_timeout", 2024-05-21T03:34:06.739 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.739 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.739 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.739 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-05-21T03:34:06.739 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.739 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.739 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.740 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache host facts data", 2024-05-21T03:34:06.740 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.740 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.740 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.740 INFO:teuthology.orchestra.run.smithi027.stdout: "grafana_dashboards_path": { 2024-05-21T03:34:06.740 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "grafana_dashboards_path", 2024-05-21T03:34:06.740 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.740 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.740 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.740 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-21T03:34:06.740 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.740 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.740 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.740 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-21T03:34:06.741 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.741 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.741 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.741 INFO:teuthology.orchestra.run.smithi027.stdout: "host_check_interval": { 2024-05-21T03:34:06.741 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "host_check_interval", 2024-05-21T03:34:06.741 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.741 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.741 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.741 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-05-21T03:34:06.741 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.741 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.741 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.741 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to perform a host check", 2024-05-21T03:34:06.742 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.742 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.742 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.742 INFO:teuthology.orchestra.run.smithi027.stdout: "hw_monitoring": { 2024-05-21T03:34:06.742 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hw_monitoring", 2024-05-21T03:34:06.742 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.742 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.742 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.742 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.742 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.742 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.742 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.742 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-21T03:34:06.743 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.743 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.743 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.743 INFO:teuthology.orchestra.run.smithi027.stdout: "inventory_list_all": { 2024-05-21T03:34:06.743 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "inventory_list_all", 2024-05-21T03:34:06.743 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.743 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.743 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.743 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.743 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.743 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.743 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.743 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-21T03:34:06.743 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.744 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.744 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.744 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.744 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.744 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.744 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.744 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.744 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.744 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.744 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.744 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.744 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.744 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.744 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.745 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.745 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.745 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.745 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.745 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.745 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.745 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.745 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.745 INFO:teuthology.orchestra.run.smithi027.stdout: "log_refresh_metadata": { 2024-05-21T03:34:06.745 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_refresh_metadata", 2024-05-21T03:34:06.745 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.745 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.745 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.745 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.746 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.746 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.746 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.746 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-21T03:34:06.746 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.746 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.746 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.746 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.746 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.746 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.746 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.747 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.747 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.747 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.747 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.747 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.747 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-21T03:34:06.747 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.747 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.747 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.747 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.747 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.747 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.747 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.748 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.748 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.748 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.748 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.748 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.748 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.748 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.748 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.748 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.748 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.748 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.748 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.748 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.748 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.748 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.749 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.749 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.749 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.749 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.749 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.749 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.749 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.749 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.749 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.749 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.749 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.749 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.749 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.750 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.750 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-21T03:34:06.750 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-21T03:34:06.750 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.750 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.750 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.750 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.750 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.750 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.750 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.750 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-21T03:34:06.750 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.751 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.751 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.751 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-21T03:34:06.751 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-21T03:34:06.751 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.751 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.751 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.751 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "*", 2024-05-21T03:34:06.751 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.751 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.751 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.751 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-21T03:34:06.751 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.751 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.752 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.752 INFO:teuthology.orchestra.run.smithi027.stdout: "max_count_per_host": { 2024-05-21T03:34:06.752 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_count_per_host", 2024-05-21T03:34:06.752 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.752 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.752 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.752 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-05-21T03:34:06.752 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.752 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.752 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.752 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of daemons per service per host", 2024-05-21T03:34:06.752 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.752 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.753 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.753 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd_draining_count": { 2024-05-21T03:34:06.753 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_osd_draining_count", 2024-05-21T03:34:06.753 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.753 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.753 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.753 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-05-21T03:34:06.753 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.753 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.753 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.753 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-21T03:34:06.753 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.753 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.754 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.754 INFO:teuthology.orchestra.run.smithi027.stdout: "migration_current": { 2024-05-21T03:34:06.754 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "migration_current", 2024-05-21T03:34:06.754 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.754 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.754 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.754 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.754 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.754 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.754 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.754 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "internal - do not modify", 2024-05-21T03:34:06.755 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.755 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.755 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.755 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-05-21T03:34:06.755 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-05-21T03:34:06.755 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.755 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.755 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.755 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "root", 2024-05-21T03:34:06.755 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.755 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.755 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.755 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm-package", 2024-05-21T03:34:06.756 INFO:teuthology.orchestra.run.smithi027.stdout: "root" 2024-05-21T03:34:06.756 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.756 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "mode for remote execution of cephadm", 2024-05-21T03:34:06.756 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.756 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.756 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.756 INFO:teuthology.orchestra.run.smithi027.stdout: "oob_default_addr": { 2024-05-21T03:34:06.756 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "oob_default_addr", 2024-05-21T03:34:06.756 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.756 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.756 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.756 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "169.254.1.1", 2024-05-21T03:34:06.756 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.757 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.757 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.757 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-21T03:34:06.757 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.757 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.757 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.757 INFO:teuthology.orchestra.run.smithi027.stdout: "prometheus_alerts_path": { 2024-05-21T03:34:06.757 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus_alerts_path", 2024-05-21T03:34:06.757 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.757 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.757 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.757 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-21T03:34:06.757 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.758 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.758 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.758 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-21T03:34:06.758 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.758 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.758 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.758 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_insecure": { 2024-05-21T03:34:06.758 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_insecure", 2024-05-21T03:34:06.758 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.758 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.758 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.758 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.758 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.759 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.759 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.759 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-21T03:34:06.759 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.759 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.759 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.759 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_password": { 2024-05-21T03:34:06.759 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_password", 2024-05-21T03:34:06.759 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.759 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.759 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.759 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.759 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.760 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.760 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.760 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-21T03:34:06.760 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.760 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.760 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.760 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_url": { 2024-05-21T03:34:06.760 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_url", 2024-05-21T03:34:06.760 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.760 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.760 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.760 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.760 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.761 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.761 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.761 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-21T03:34:06.761 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.761 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.761 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.761 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_username": { 2024-05-21T03:34:06.761 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_username", 2024-05-21T03:34:06.761 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.761 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.761 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.761 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.761 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.762 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.762 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.762 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-21T03:34:06.762 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.762 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.762 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.762 INFO:teuthology.orchestra.run.smithi027.stdout: "secure_monitoring_stack": { 2024-05-21T03:34:06.762 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secure_monitoring_stack", 2024-05-21T03:34:06.762 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.762 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.762 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.762 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.762 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.763 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.763 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.763 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-21T03:34:06.763 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.763 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.763 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.763 INFO:teuthology.orchestra.run.smithi027.stdout: "service_discovery_port": { 2024-05-21T03:34:06.763 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "service_discovery_port", 2024-05-21T03:34:06.763 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.763 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.763 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.763 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8765", 2024-05-21T03:34:06.763 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.764 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.764 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.764 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "cephadm service discovery port", 2024-05-21T03:34:06.764 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.764 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.764 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.764 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.764 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.764 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.764 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.764 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.764 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.764 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.765 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.765 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.765 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.765 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.765 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.765 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.765 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_config_file": { 2024-05-21T03:34:06.765 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_config_file", 2024-05-21T03:34:06.765 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.765 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.765 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.765 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.765 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.766 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.766 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.766 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-21T03:34:06.766 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.766 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.766 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.766 INFO:teuthology.orchestra.run.smithi027.stdout: "use_agent": { 2024-05-21T03:34:06.766 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_agent", 2024-05-21T03:34:06.766 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.766 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.766 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.766 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.766 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.767 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.767 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.767 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-21T03:34:06.767 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.767 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.767 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.767 INFO:teuthology.orchestra.run.smithi027.stdout: "use_repo_digest": { 2024-05-21T03:34:06.767 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_repo_digest", 2024-05-21T03:34:06.767 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.767 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.767 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.767 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.767 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.767 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.768 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.768 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-21T03:34:06.768 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.768 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.768 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.768 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_failed_host_check": { 2024-05-21T03:34:06.768 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_failed_host_check", 2024-05-21T03:34:06.768 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.768 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.768 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.768 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.768 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.768 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.769 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.769 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if the host check fails", 2024-05-21T03:34:06.769 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.769 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.769 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.769 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_daemons": { 2024-05-21T03:34:06.769 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_daemons", 2024-05-21T03:34:06.769 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.769 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.769 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.769 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.769 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.769 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.770 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.770 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-21T03:34:06.770 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.770 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.770 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.770 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_hosts": { 2024-05-21T03:34:06.770 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_hosts", 2024-05-21T03:34:06.770 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.770 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.770 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.770 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.770 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.771 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.771 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.771 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-21T03:34:06.771 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.771 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.771 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.771 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.771 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.771 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crash", 2024-05-21T03:34:06.771 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.771 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.771 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.772 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.772 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.772 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.772 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.772 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.772 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.772 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.772 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.772 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.772 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.772 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.772 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.772 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.772 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.772 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.773 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.773 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.773 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.773 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.773 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.773 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.773 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.773 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.773 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.773 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.773 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.773 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.773 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.773 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.774 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.774 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.774 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.774 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.774 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.774 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.774 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.774 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.774 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.774 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.774 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.774 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.774 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.775 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.775 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.775 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.775 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.775 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.775 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.775 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.775 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.775 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.775 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.775 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.775 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.775 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.775 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.775 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.776 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.776 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.776 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.776 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.776 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.776 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.776 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.776 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.776 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.776 INFO:teuthology.orchestra.run.smithi027.stdout: "retain_interval": { 2024-05-21T03:34:06.776 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retain_interval", 2024-05-21T03:34:06.776 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.777 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.777 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.777 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "31536000", 2024-05-21T03:34:06.777 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.777 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.777 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.777 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-21T03:34:06.777 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.777 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.777 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.777 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.777 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.777 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.777 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.777 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.778 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.778 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.778 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.778 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.778 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.778 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.778 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.778 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.778 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_recent_interval": { 2024-05-21T03:34:06.778 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_recent_interval", 2024-05-21T03:34:06.778 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.778 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.778 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.779 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1209600", 2024-05-21T03:34:06.779 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.779 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.779 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.779 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-21T03:34:06.779 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.779 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.779 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.779 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.779 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.779 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dashboard", 2024-05-21T03:34:06.779 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.780 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.780 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.780 INFO:teuthology.orchestra.run.smithi027.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-21T03:34:06.780 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-21T03:34:06.780 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.780 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.780 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.780 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-05-21T03:34:06.780 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.780 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.780 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.780 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.780 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.780 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.780 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.781 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_HOST": { 2024-05-21T03:34:06.781 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-21T03:34:06.781 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.781 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.781 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.781 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.781 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.781 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.781 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.781 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.781 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.781 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.781 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.782 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-21T03:34:06.782 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-21T03:34:06.782 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.782 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.782 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.782 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.782 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.782 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.782 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.782 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.782 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.782 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.782 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.782 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_ENABLED": { 2024-05-21T03:34:06.783 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_ENABLED", 2024-05-21T03:34:06.783 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.783 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.783 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.783 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.783 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.783 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.783 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.783 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.783 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.783 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.783 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.783 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-21T03:34:06.783 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-21T03:34:06.784 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.784 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.784 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.784 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.784 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.784 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.784 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.784 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.784 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.784 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.784 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.784 INFO:teuthology.orchestra.run.smithi027.stdout: "ENABLE_BROWSABLE_API": { 2024-05-21T03:34:06.785 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-21T03:34:06.785 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.785 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.785 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.785 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.785 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.785 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.785 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.785 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.785 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.785 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.785 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.785 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-21T03:34:06.785 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-21T03:34:06.785 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.786 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.786 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.786 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.786 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.786 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.786 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.786 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.786 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.786 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.786 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.786 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-21T03:34:06.786 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-21T03:34:06.786 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.787 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.787 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.787 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.787 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.787 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.787 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.787 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.787 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.787 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.787 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.787 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-21T03:34:06.787 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-21T03:34:06.787 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.787 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.788 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.788 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.788 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.788 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.788 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.788 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.788 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.788 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.788 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.788 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-21T03:34:06.788 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-21T03:34:06.788 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.788 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.788 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.789 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.789 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.789 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.789 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.789 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.789 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.789 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.789 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.789 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-21T03:34:06.789 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-21T03:34:06.789 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.789 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.789 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.789 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.789 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.790 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.790 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.790 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.790 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.790 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.790 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.790 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-21T03:34:06.790 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-21T03:34:06.790 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.790 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.790 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.790 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.790 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.790 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.791 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.791 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.791 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.791 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.791 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.791 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-21T03:34:06.791 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-21T03:34:06.791 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.791 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.791 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.791 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.791 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.791 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.791 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.792 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.792 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.792 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.792 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.792 INFO:teuthology.orchestra.run.smithi027.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-21T03:34:06.792 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-21T03:34:06.792 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.792 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.792 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.792 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.792 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.792 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.792 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.793 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.793 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.793 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.793 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.793 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_PASSWORD": { 2024-05-21T03:34:06.793 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-21T03:34:06.793 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.793 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.793 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.793 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-05-21T03:34:06.793 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.793 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.793 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.793 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.793 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.794 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.794 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.794 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-21T03:34:06.794 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-21T03:34:06.794 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.794 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.794 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.794 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.794 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.794 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.794 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.794 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.794 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.795 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.795 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.795 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_URL": { 2024-05-21T03:34:06.795 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_URL", 2024-05-21T03:34:06.795 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.795 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.795 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.795 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.795 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.795 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.795 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.795 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.795 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.795 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.796 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.796 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_USERNAME": { 2024-05-21T03:34:06.796 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-21T03:34:06.796 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.796 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.796 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.796 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-05-21T03:34:06.796 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.796 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.796 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.796 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.796 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.796 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.796 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.797 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-21T03:34:06.797 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-21T03:34:06.797 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.797 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.797 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.797 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.797 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.797 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.797 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.797 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.797 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.797 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.797 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.797 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-21T03:34:06.797 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-21T03:34:06.798 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.798 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.798 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.798 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.798 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.798 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.798 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.798 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.798 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.798 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.798 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.798 INFO:teuthology.orchestra.run.smithi027.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-21T03:34:06.798 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-21T03:34:06.798 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.799 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.799 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.799 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.799 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.799 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.799 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.799 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.799 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.799 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.799 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.799 INFO:teuthology.orchestra.run.smithi027.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-21T03:34:06.799 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-21T03:34:06.799 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.799 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.800 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.800 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.800 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.800 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.800 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.800 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.800 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.800 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.800 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.800 INFO:teuthology.orchestra.run.smithi027.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-21T03:34:06.800 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-21T03:34:06.800 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.800 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.800 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.800 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "[]", 2024-05-21T03:34:06.801 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.801 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.801 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.801 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.801 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.801 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.801 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.801 INFO:teuthology.orchestra.run.smithi027.stdout: "MULTICLUSTER_CONFIG": { 2024-05-21T03:34:06.801 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-21T03:34:06.801 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.801 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.801 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.801 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "{}", 2024-05-21T03:34:06.801 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.801 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.802 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.802 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.802 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.802 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.802 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.802 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_HOST": { 2024-05-21T03:34:06.802 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-21T03:34:06.802 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.802 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.802 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.802 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.802 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.802 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.802 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.803 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.803 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.803 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.803 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.803 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-21T03:34:06.803 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-21T03:34:06.803 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.803 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.803 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.803 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.803 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.803 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.803 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.803 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.804 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.804 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.804 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.804 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-21T03:34:06.804 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-21T03:34:06.804 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.804 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.804 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.804 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.804 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.804 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.804 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.804 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.804 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.805 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.805 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.805 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-21T03:34:06.805 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-21T03:34:06.805 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.805 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.805 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.805 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.805 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.805 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.805 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.805 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.805 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.806 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.806 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.806 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-21T03:34:06.806 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-21T03:34:06.806 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.806 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.806 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.806 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.806 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.806 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.806 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.806 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.807 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.807 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.807 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.807 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-21T03:34:06.807 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-21T03:34:06.807 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.807 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.807 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.807 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.807 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.807 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.807 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.807 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.807 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.807 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.808 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.808 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-21T03:34:06.808 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-21T03:34:06.808 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.808 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.808 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.808 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.808 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.808 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.808 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.808 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.808 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.808 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.808 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.809 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-21T03:34:06.809 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-21T03:34:06.809 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.809 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.809 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.809 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.809 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.809 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.809 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.809 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.809 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.809 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.809 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.810 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-21T03:34:06.810 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-21T03:34:06.810 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.810 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.810 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.810 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.810 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.810 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.810 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.810 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.810 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.810 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.810 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.811 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_ENABLED": { 2024-05-21T03:34:06.811 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-21T03:34:06.811 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.811 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.811 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.811 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.811 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.811 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.811 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.811 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.811 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.811 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.811 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.812 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-21T03:34:06.812 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-21T03:34:06.812 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.812 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.812 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.812 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-21T03:34:06.812 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.812 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.812 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.812 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.812 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.812 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.812 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.812 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-21T03:34:06.813 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-21T03:34:06.813 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.813 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.813 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.813 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-05-21T03:34:06.813 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.813 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.813 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.813 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.813 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.813 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.813 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.813 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-21T03:34:06.813 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-21T03:34:06.813 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.814 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.814 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.814 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8", 2024-05-21T03:34:06.814 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.814 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.814 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.814 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.814 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.814 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.814 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.814 INFO:teuthology.orchestra.run.smithi027.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-21T03:34:06.814 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-21T03:34:06.814 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.815 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.815 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.815 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "45", 2024-05-21T03:34:06.815 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.815 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.815 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.815 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.815 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.815 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.815 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.815 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ACCESS_KEY": { 2024-05-21T03:34:06.815 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-21T03:34:06.815 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.816 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.816 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.816 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.816 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.816 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.816 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.816 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.816 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.816 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.816 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.816 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-21T03:34:06.816 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-21T03:34:06.816 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.817 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.817 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.817 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-05-21T03:34:06.817 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.817 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.817 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.817 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.817 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.817 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.817 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.817 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SECRET_KEY": { 2024-05-21T03:34:06.817 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-21T03:34:06.817 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.817 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.818 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.818 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.818 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.818 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.818 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.818 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.818 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.818 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.818 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.818 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SSL_VERIFY": { 2024-05-21T03:34:06.818 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-21T03:34:06.818 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.818 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.819 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.819 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.819 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.819 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.819 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.819 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.819 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.819 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.819 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.819 INFO:teuthology.orchestra.run.smithi027.stdout: "UNSAFE_TLS_v1_2": { 2024-05-21T03:34:06.819 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-21T03:34:06.819 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.819 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.819 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.820 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.820 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.820 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.820 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.820 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.820 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.820 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.820 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.820 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-21T03:34:06.820 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-21T03:34:06.820 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.820 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.820 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.820 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.821 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.821 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.821 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.821 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.821 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.821 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.821 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.821 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-21T03:34:06.821 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-21T03:34:06.821 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.821 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.821 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.821 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-05-21T03:34:06.821 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.821 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.822 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.822 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.822 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.822 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.822 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.822 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-21T03:34:06.822 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-21T03:34:06.822 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.822 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.822 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.822 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-05-21T03:34:06.822 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.822 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.823 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.823 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.823 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.823 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.823 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.823 INFO:teuthology.orchestra.run.smithi027.stdout: "cross_origin_url": { 2024-05-21T03:34:06.823 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cross_origin_url", 2024-05-21T03:34:06.823 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.823 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.823 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.823 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.823 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.823 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.823 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.823 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.824 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.824 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.824 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.824 INFO:teuthology.orchestra.run.smithi027.stdout: "crt_file": { 2024-05-21T03:34:06.824 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crt_file", 2024-05-21T03:34:06.824 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.824 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.824 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.824 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.824 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.824 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.824 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.824 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.824 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.825 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.825 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.825 INFO:teuthology.orchestra.run.smithi027.stdout: "debug": { 2024-05-21T03:34:06.825 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "debug", 2024-05-21T03:34:06.825 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.825 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.825 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.825 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.825 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.825 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.825 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.825 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable/disable debug options", 2024-05-21T03:34:06.825 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.825 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.825 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.826 INFO:teuthology.orchestra.run.smithi027.stdout: "jwt_token_ttl": { 2024-05-21T03:34:06.826 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "jwt_token_ttl", 2024-05-21T03:34:06.826 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.826 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.826 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.826 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "28800", 2024-05-21T03:34:06.826 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.826 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.826 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.826 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.826 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.826 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.826 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.827 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-05-21T03:34:06.827 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-05-21T03:34:06.827 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.827 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.827 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.827 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.827 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.827 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.827 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.827 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.827 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.827 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.827 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.827 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.828 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.828 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.828 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.828 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.828 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.828 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.828 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.828 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.828 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.828 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.828 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.828 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.828 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.828 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.828 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.829 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.829 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.829 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.829 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.829 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.829 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.829 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.829 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.829 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.829 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.829 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.829 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.829 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.829 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.829 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.830 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.830 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.830 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.830 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.830 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.830 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.830 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.830 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.830 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.830 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.830 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.830 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.830 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.830 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.831 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.831 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.831 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.831 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.831 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.831 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.831 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.831 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.831 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.831 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.831 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.831 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.831 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.831 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.831 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.832 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.832 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.832 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.832 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.832 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.832 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.832 INFO:teuthology.orchestra.run.smithi027.stdout: "motd": { 2024-05-21T03:34:06.832 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "motd", 2024-05-21T03:34:06.832 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.832 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.832 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.832 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.832 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.832 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.833 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.833 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "The message of the day", 2024-05-21T03:34:06.833 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.833 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.833 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.833 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect_resolve_ip_addr": { 2024-05-21T03:34:06.833 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "redirect_resolve_ip_addr", 2024-05-21T03:34:06.833 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.833 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.833 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.833 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.833 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.833 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.833 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.834 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.834 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.834 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.834 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.834 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-05-21T03:34:06.834 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-05-21T03:34:06.834 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.834 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.834 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.834 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-05-21T03:34:06.834 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.834 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.834 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.834 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.835 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.835 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.835 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.835 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-05-21T03:34:06.835 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-05-21T03:34:06.835 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.835 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.835 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.835 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8080", 2024-05-21T03:34:06.835 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.835 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.835 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.835 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.835 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.836 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.836 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.836 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.836 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.836 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.836 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.836 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.836 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.836 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.836 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.836 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.836 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.836 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.836 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.836 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.837 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-05-21T03:34:06.837 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-05-21T03:34:06.837 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.837 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.837 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.837 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.837 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.837 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.837 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.837 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.837 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.837 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.837 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.838 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl_server_port": { 2024-05-21T03:34:06.838 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl_server_port", 2024-05-21T03:34:06.838 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.838 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.838 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.838 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8443", 2024-05-21T03:34:06.838 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.838 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.838 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.838 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.838 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.838 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.838 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.839 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-05-21T03:34:06.839 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-05-21T03:34:06.839 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.839 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.839 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.839 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "redirect", 2024-05-21T03:34:06.839 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.839 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.839 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.839 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.839 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect" 2024-05-21T03:34:06.839 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.839 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.839 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.840 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.840 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.840 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-05-21T03:34:06.840 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-05-21T03:34:06.840 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.840 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.840 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.840 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-05-21T03:34:06.840 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-05-21T03:34:06.840 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-05-21T03:34:06.840 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.840 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.840 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.840 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.841 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.841 INFO:teuthology.orchestra.run.smithi027.stdout: "url_prefix": { 2024-05-21T03:34:06.841 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url_prefix", 2024-05-21T03:34:06.841 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.841 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.841 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.841 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.841 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.841 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.841 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.841 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.841 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.841 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.842 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.842 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.842 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.842 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-05-21T03:34:06.842 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.842 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.842 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.842 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_monitoring": { 2024-05-21T03:34:06.842 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_monitoring", 2024-05-21T03:34:06.842 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.843 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.843 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.843 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.843 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.843 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.843 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.843 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "monitor device health metrics", 2024-05-21T03:34:06.843 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.843 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.843 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.843 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.843 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.843 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.843 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.844 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.844 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.844 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.844 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.844 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.844 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.844 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.844 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.844 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.844 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.844 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.844 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.844 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.844 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.845 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.845 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.845 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.845 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.845 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.845 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.845 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.845 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.845 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.845 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.845 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.845 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.846 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.846 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.846 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.846 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.846 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.846 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.846 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.846 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.846 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.846 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.846 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.846 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.847 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.847 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.847 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.847 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.847 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.847 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.847 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.847 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.850 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.850 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.850 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.850 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.850 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.850 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.850 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.851 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.851 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.851 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.851 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.851 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.851 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.851 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.851 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.851 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.851 INFO:teuthology.orchestra.run.smithi027.stdout: "mark_out_threshold": { 2024-05-21T03:34:06.851 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mark_out_threshold", 2024-05-21T03:34:06.851 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.852 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.852 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.852 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2419200", 2024-05-21T03:34:06.852 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.852 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.852 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.852 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-21T03:34:06.852 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.852 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.852 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.852 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": { 2024-05-21T03:34:06.852 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_name", 2024-05-21T03:34:06.852 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.853 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.853 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.853 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "device_health_metrics", 2024-05-21T03:34:06.853 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.853 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.853 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.853 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-21T03:34:06.853 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.853 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.853 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.853 INFO:teuthology.orchestra.run.smithi027.stdout: "retention_period": { 2024-05-21T03:34:06.853 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retention_period", 2024-05-21T03:34:06.854 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.854 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.854 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.854 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15552000", 2024-05-21T03:34:06.854 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.854 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.854 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.854 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain device health metrics", 2024-05-21T03:34:06.854 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.854 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.854 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.854 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_frequency": { 2024-05-21T03:34:06.855 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_frequency", 2024-05-21T03:34:06.855 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.855 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.855 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.855 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-05-21T03:34:06.855 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.855 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.855 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.855 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-21T03:34:06.855 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.855 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.855 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.856 INFO:teuthology.orchestra.run.smithi027.stdout: "self_heal": { 2024-05-21T03:34:06.856 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "self_heal", 2024-05-21T03:34:06.856 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.856 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.856 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.856 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.856 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.856 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.856 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.856 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-21T03:34:06.856 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.856 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.856 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.857 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-05-21T03:34:06.857 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-05-21T03:34:06.857 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.857 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.857 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.857 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-05-21T03:34:06.857 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.857 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.857 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.857 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and check device health", 2024-05-21T03:34:06.857 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.857 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.857 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.858 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.858 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.858 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.858 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.858 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.858 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.858 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.858 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.858 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.859 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.859 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.859 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.859 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.859 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_threshold": { 2024-05-21T03:34:06.859 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_threshold", 2024-05-21T03:34:06.859 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.859 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.859 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.859 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7257600", 2024-05-21T03:34:06.859 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.859 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.860 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.860 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-21T03:34:06.860 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.860 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.860 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.860 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.860 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.860 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "diskprediction_local", 2024-05-21T03:34:06.860 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.860 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.860 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.860 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.861 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.861 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.861 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.861 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.861 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.861 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.861 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.861 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.861 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.861 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.861 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.861 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.861 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.862 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.862 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.862 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.862 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.862 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.862 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.862 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.862 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.862 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.862 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.862 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.862 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.863 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.863 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.863 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.863 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.863 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.863 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.863 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.863 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.863 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.863 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.863 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.863 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.863 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.863 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.864 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.864 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.864 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.864 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.864 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.864 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.864 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.864 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.864 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.864 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.864 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.864 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.864 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.865 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.865 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.865 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.865 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.865 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.865 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.865 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.865 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.865 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.865 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.865 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.865 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.865 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.866 INFO:teuthology.orchestra.run.smithi027.stdout: "predict_interval": { 2024-05-21T03:34:06.866 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predict_interval", 2024-05-21T03:34:06.866 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.866 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.866 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.866 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-05-21T03:34:06.866 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.866 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.866 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.866 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.866 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.866 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.867 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.867 INFO:teuthology.orchestra.run.smithi027.stdout: "predictor_model": { 2024-05-21T03:34:06.867 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predictor_model", 2024-05-21T03:34:06.867 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.867 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.867 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.867 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "prophetstor", 2024-05-21T03:34:06.867 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.867 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.867 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.867 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.867 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.868 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.868 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.868 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.868 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-05-21T03:34:06.868 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-05-21T03:34:06.868 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.868 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.868 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.868 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-05-21T03:34:06.868 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.868 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.868 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.868 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.868 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.868 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.869 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.869 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.869 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.869 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.869 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.869 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.869 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.869 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.869 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.869 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.869 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.869 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.869 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.869 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.869 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.870 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.870 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.870 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.870 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "influx", 2024-05-21T03:34:06.870 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": false, 2024-05-21T03:34:06.870 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "influxdb python module not found", 2024-05-21T03:34:06.870 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.870 INFO:teuthology.orchestra.run.smithi027.stdout: "batch_size": { 2024-05-21T03:34:06.870 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "batch_size", 2024-05-21T03:34:06.870 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.870 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.870 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.871 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5000", 2024-05-21T03:34:06.871 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.871 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.871 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.871 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-21T03:34:06.871 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.871 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.871 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.871 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.871 INFO:teuthology.orchestra.run.smithi027.stdout: "database": { 2024-05-21T03:34:06.871 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "database", 2024-05-21T03:34:06.871 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.871 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.871 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.871 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "ceph", 2024-05-21T03:34:06.872 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.872 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.872 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.872 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-21T03:34:06.872 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.872 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.872 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.872 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": { 2024-05-21T03:34:06.872 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hostname", 2024-05-21T03:34:06.872 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.872 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.872 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.873 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.873 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.873 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.873 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.873 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server hostname", 2024-05-21T03:34:06.873 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.873 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.873 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.873 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.873 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-05-21T03:34:06.873 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-05-21T03:34:06.873 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.873 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.873 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.874 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "30", 2024-05-21T03:34:06.874 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "5", 2024-05-21T03:34:06.874 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.874 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.874 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-21T03:34:06.874 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.874 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.874 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.874 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.874 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.874 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.875 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.875 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.875 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.875 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.875 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.875 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.875 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.875 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.875 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.875 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.875 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.875 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.875 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.875 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.875 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.876 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.876 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.876 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.876 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.876 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.876 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.876 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.876 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.876 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.876 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.876 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.876 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.876 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.876 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.876 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.877 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.877 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.877 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.877 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.877 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.877 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.877 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.877 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.877 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.877 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.877 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.877 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.878 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.878 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.878 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.878 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.878 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.878 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.878 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.878 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.878 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.878 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.878 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.878 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.879 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.879 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.879 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.879 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.879 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.879 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.879 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.879 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.879 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.879 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.879 INFO:teuthology.orchestra.run.smithi027.stdout: "password": { 2024-05-21T03:34:06.879 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "password", 2024-05-21T03:34:06.879 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.879 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.880 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.880 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.880 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.880 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.880 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.880 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "password of InfluxDB server user", 2024-05-21T03:34:06.880 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.880 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.880 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.880 INFO:teuthology.orchestra.run.smithi027.stdout: "port": { 2024-05-21T03:34:06.880 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "port", 2024-05-21T03:34:06.880 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.880 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.880 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.880 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8086", 2024-05-21T03:34:06.881 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.881 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.881 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.881 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server port", 2024-05-21T03:34:06.881 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.881 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.881 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.881 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.881 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.881 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.881 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.881 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.881 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.881 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.881 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.882 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.882 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.882 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.882 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.882 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.882 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-05-21T03:34:06.882 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-05-21T03:34:06.882 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.882 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.882 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.882 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "false", 2024-05-21T03:34:06.882 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.882 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.883 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.883 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-21T03:34:06.883 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.883 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.883 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.883 INFO:teuthology.orchestra.run.smithi027.stdout: "threads": { 2024-05-21T03:34:06.883 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threads", 2024-05-21T03:34:06.883 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.883 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.883 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.883 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-05-21T03:34:06.883 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-05-21T03:34:06.883 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "32", 2024-05-21T03:34:06.884 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.884 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-21T03:34:06.884 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.884 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.884 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.884 INFO:teuthology.orchestra.run.smithi027.stdout: "username": { 2024-05-21T03:34:06.884 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "username", 2024-05-21T03:34:06.884 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.884 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.884 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.884 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.884 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.884 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.884 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.885 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "username of InfluxDB server user", 2024-05-21T03:34:06.885 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.885 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.885 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.885 INFO:teuthology.orchestra.run.smithi027.stdout: "verify_ssl": { 2024-05-21T03:34:06.885 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "verify_ssl", 2024-05-21T03:34:06.885 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.885 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.885 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.885 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "true", 2024-05-21T03:34:06.885 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.885 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.885 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.886 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-21T03:34:06.886 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.886 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.886 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.886 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.886 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.886 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.886 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "insights", 2024-05-21T03:34:06.886 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.886 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.886 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.886 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.886 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.887 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.887 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.887 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.887 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.887 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.887 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.887 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.887 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.887 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.887 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.887 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.887 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.887 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.887 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.887 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.888 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.888 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.888 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.888 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.888 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.888 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.888 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.888 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.888 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.888 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.888 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.888 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.888 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.888 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.888 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.889 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.889 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.889 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.889 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.889 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.889 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.889 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.889 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.889 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.889 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.889 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.889 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.889 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.889 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.890 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.890 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.890 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.890 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.890 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.890 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.890 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.890 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.890 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.890 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.890 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.890 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.890 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.891 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.891 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.891 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.891 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.891 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.891 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.891 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.891 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.891 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.891 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.891 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.891 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.891 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.891 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.891 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.892 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.892 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.892 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.892 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.892 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.892 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.892 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.892 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.892 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.892 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "iostat", 2024-05-21T03:34:06.892 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.892 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.892 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.892 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.892 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.893 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.893 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.893 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.893 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.893 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.893 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.893 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.893 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.893 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.893 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.893 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.893 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.893 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.893 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.893 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.894 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.894 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.894 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.894 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.894 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.894 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.894 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.894 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.894 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.894 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.894 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.894 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.894 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.895 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.895 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.895 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.895 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.895 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.895 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.895 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.895 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.895 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.895 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.895 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.895 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.895 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.895 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.895 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.896 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.896 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.896 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.896 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.896 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.896 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.896 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.896 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.896 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.896 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.896 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.896 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.896 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.896 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.897 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.897 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.897 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.897 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.897 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.897 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.897 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.897 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.897 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.897 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.897 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.897 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.897 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.897 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.897 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.898 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.898 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.898 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.898 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.898 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.898 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.898 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.898 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.898 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "k8sevents", 2024-05-21T03:34:06.898 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.898 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.898 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.899 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_event_retention_days": { 2024-05-21T03:34:06.899 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ceph_event_retention_days", 2024-05-21T03:34:06.899 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.899 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.899 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.899 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7", 2024-05-21T03:34:06.899 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.899 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.899 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.899 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-21T03:34:06.899 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.899 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.899 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.899 INFO:teuthology.orchestra.run.smithi027.stdout: "config_check_secs": { 2024-05-21T03:34:06.899 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_check_secs", 2024-05-21T03:34:06.900 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.900 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.900 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.900 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-05-21T03:34:06.900 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "10", 2024-05-21T03:34:06.900 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.900 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.900 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-21T03:34:06.900 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.900 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.900 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.900 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.900 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.900 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.900 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.901 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.901 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.901 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.901 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.901 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.901 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.901 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.901 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.901 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.901 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.901 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.901 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.901 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.901 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.901 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.902 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.902 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.902 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.902 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.902 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.902 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.902 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.902 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.902 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.902 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.902 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.902 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.902 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.902 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.903 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.903 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.903 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.903 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.903 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.903 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.903 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.903 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.903 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.903 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.903 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.903 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.903 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.903 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.903 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.904 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.904 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.904 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.904 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.904 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.904 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.904 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.904 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.904 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.904 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.904 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.904 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.904 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.904 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.905 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.905 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.905 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.905 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.905 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.905 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.905 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.905 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.905 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.905 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.905 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.905 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.905 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.906 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.906 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.906 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.906 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.906 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.906 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.906 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.906 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.906 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.906 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "localpool", 2024-05-21T03:34:06.906 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.906 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.906 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.906 INFO:teuthology.orchestra.run.smithi027.stdout: "failure_domain": { 2024-05-21T03:34:06.907 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "failure_domain", 2024-05-21T03:34:06.907 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.907 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.907 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.907 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "host", 2024-05-21T03:34:06.907 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.907 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.907 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.907 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "failure domain for any created local pool", 2024-05-21T03:34:06.907 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-21T03:34:06.907 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.907 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.907 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.907 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.907 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.907 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.908 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.908 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.908 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.908 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.908 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.908 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.908 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.908 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.908 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.908 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.908 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.908 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.908 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.908 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.909 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.909 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.909 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.909 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.909 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.909 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.909 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.909 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.909 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.909 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.909 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.909 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.909 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.909 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.909 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.910 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.910 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.910 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.910 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.910 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.910 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.910 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.910 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.910 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.910 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.910 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.910 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.910 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.910 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.911 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.911 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.911 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.911 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.911 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.911 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.911 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.911 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.911 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.911 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.911 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.911 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.911 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.911 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.912 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.912 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.912 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.912 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.912 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.912 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.912 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.912 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": { 2024-05-21T03:34:06.912 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_size", 2024-05-21T03:34:06.912 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.912 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.912 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.912 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.912 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.913 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.913 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.913 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default min_size for any created local pool", 2024-05-21T03:34:06.913 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-21T03:34:06.913 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.913 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.913 INFO:teuthology.orchestra.run.smithi027.stdout: "num_rep": { 2024-05-21T03:34:06.913 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "num_rep", 2024-05-21T03:34:06.913 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.913 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.913 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.913 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-05-21T03:34:06.913 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.913 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.913 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.914 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default replica count for any created local pool", 2024-05-21T03:34:06.914 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.914 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.914 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.914 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.914 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": { 2024-05-21T03:34:06.914 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_num", 2024-05-21T03:34:06.914 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.914 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.914 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.914 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "128", 2024-05-21T03:34:06.914 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.914 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.914 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.915 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default pg_num for any created local pool", 2024-05-21T03:34:06.915 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.915 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.915 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.915 INFO:teuthology.orchestra.run.smithi027.stdout: "prefix": { 2024-05-21T03:34:06.915 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prefix", 2024-05-21T03:34:06.915 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.915 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.915 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.915 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.915 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.915 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.915 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.916 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name prefix for any created local pool", 2024-05-21T03:34:06.916 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.916 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.916 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.916 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.916 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.916 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.916 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.916 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.916 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.916 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.916 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.916 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.916 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.916 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.917 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.917 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.917 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.917 INFO:teuthology.orchestra.run.smithi027.stdout: "subtree": { 2024-05-21T03:34:06.917 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "subtree", 2024-05-21T03:34:06.917 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.917 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.917 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.917 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "rack", 2024-05-21T03:34:06.917 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.917 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.917 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.917 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-21T03:34:06.917 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-21T03:34:06.918 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.918 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.918 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.918 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.918 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.918 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mds_autoscaler", 2024-05-21T03:34:06.918 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.918 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.918 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.918 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.918 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.918 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.918 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.919 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.919 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.919 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.919 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.919 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.919 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.919 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.919 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.919 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.919 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.919 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.919 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.919 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.919 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.919 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.919 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.920 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.920 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.920 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.920 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.920 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.920 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.920 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.920 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.920 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.920 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.920 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.920 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.920 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.921 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.921 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.921 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.921 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.921 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.921 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.921 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.921 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.921 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.921 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.921 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.921 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.921 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.921 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.921 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.922 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.922 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.922 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.922 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.922 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.922 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.922 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.922 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.922 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.922 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.922 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.922 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.922 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.922 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.923 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.923 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.923 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.923 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.923 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.923 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.923 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.923 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.923 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.923 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.923 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.923 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.923 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.923 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.923 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.924 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.924 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.924 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.924 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.924 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.924 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.924 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirroring", 2024-05-21T03:34:06.924 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.924 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.924 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.924 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.924 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.924 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.924 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.925 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.925 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.925 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.925 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.925 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.925 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.925 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.925 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.925 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.925 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.925 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.925 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.925 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.925 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.926 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.926 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.926 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.926 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.926 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.926 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.926 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.926 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.926 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.926 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.926 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.926 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.926 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.927 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.927 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.927 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.927 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.927 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.927 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.927 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.927 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.927 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.927 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.927 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.927 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.927 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.927 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.927 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.928 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.928 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.928 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.928 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.928 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.928 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.928 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.928 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.928 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.928 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.928 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.928 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.928 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.928 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.928 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.928 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.929 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.929 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.929 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.929 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.929 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.929 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.929 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.929 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.929 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.929 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.929 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.929 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.929 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.929 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.929 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.930 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.930 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.930 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.930 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.930 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.930 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.930 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.930 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "nfs", 2024-05-21T03:34:06.930 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.930 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.930 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.930 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.930 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.930 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.931 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.931 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.931 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.931 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.931 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.931 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.931 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.931 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.931 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.931 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.931 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.931 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.931 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.931 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.931 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.932 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.932 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.932 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.932 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.932 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.932 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.932 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.932 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.932 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.932 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.932 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.932 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.932 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.932 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.932 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.932 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.933 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.933 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.933 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.933 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.933 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.933 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.933 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.933 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.933 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.933 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.933 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.933 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.933 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.933 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.933 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.933 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.934 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.934 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.934 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.934 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.934 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.934 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.934 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.934 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.934 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.934 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.934 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.934 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.934 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.934 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.934 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.935 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.935 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.935 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.935 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.935 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.935 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.935 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.935 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.935 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.935 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.935 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.935 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.935 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.935 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.936 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.936 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.936 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.936 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.936 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.936 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.936 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-05-21T03:34:06.936 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.936 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.936 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.936 INFO:teuthology.orchestra.run.smithi027.stdout: "fail_fs": { 2024-05-21T03:34:06.936 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "fail_fs", 2024-05-21T03:34:06.936 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.936 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.936 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.937 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.937 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.937 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.937 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.937 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-21T03:34:06.937 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.937 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.937 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.937 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.937 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.937 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.937 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.937 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.937 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.937 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.938 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.938 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.938 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.938 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.938 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.938 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.938 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.938 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.938 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.938 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.938 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.938 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.938 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.938 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.939 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.939 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.939 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.939 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.939 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.939 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.939 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.939 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.939 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.939 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.939 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.939 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.939 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.940 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.940 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.940 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.940 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.940 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.940 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.940 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.940 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.940 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.940 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.940 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.940 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.940 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.940 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.940 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.941 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.941 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.941 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.941 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.941 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.941 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.941 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.941 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.941 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.941 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.941 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.941 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.941 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.941 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.942 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.942 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.942 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.942 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.942 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.942 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator": { 2024-05-21T03:34:06.942 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-05-21T03:34:06.942 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.942 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.942 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.942 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.942 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.942 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.942 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.943 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm", 2024-05-21T03:34:06.943 INFO:teuthology.orchestra.run.smithi027.stdout: "rook", 2024-05-21T03:34:06.943 INFO:teuthology.orchestra.run.smithi027.stdout: "test_orchestrator" 2024-05-21T03:34:06.943 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.943 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Orchestrator backend", 2024-05-21T03:34:06.943 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.943 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.943 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.943 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.943 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.943 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.943 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.943 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.943 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.943 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.944 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.944 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.944 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.944 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.944 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.944 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.944 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.944 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.944 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.944 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_perf_query", 2024-05-21T03:34:06.944 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.944 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.944 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.944 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.944 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.945 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.945 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.945 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.945 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.945 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.945 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.945 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.945 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.945 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.945 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.945 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.945 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.945 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.945 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.945 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.946 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.946 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.946 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.946 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.946 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.946 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.946 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.946 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.946 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.946 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.946 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.946 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.946 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.947 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.947 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.947 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.947 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.947 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.947 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.947 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.947 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.947 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.947 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.947 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.947 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.947 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.947 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.947 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.948 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.948 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.948 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.948 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.948 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.948 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.948 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.948 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.948 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.948 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.948 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.948 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.948 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.948 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.948 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.949 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.949 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.949 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.949 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.949 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.949 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.949 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.949 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.949 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.949 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.949 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.949 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.949 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.949 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.949 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.950 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.950 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.950 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.950 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.950 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.950 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.950 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.950 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.950 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.950 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_support", 2024-05-21T03:34:06.950 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.950 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.950 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.951 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.951 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.951 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.951 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.951 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.951 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.951 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.951 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.951 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.951 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.951 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.951 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.951 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.952 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.952 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.952 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.952 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.952 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.952 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.952 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.952 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.952 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.952 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.952 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.952 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.952 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.953 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.953 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.953 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.953 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.953 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.953 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.953 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.953 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.953 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.953 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.953 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.953 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.953 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.953 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.954 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.954 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.954 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.954 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.954 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.954 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.954 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.954 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.954 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.954 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.954 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.954 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.954 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.955 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.955 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.955 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.955 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.955 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.955 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.955 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.955 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.955 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.955 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.955 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.955 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.955 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.955 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.956 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.956 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.956 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.956 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.956 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.956 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.956 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.956 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.956 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.956 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.956 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.956 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.956 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.956 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.957 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.957 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_autoscaler", 2024-05-21T03:34:06.957 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.957 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.957 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.957 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.957 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.957 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.957 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.957 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.957 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.957 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.957 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.957 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.957 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.958 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.958 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.958 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.958 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.958 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.958 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.958 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.958 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.958 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.958 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.958 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.958 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.958 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.959 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.959 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.959 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.959 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.959 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.959 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.959 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.959 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.959 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.959 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.959 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.959 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.959 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.959 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.959 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.960 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.960 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.960 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.960 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.960 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.960 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.960 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.960 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.960 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.960 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.960 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.960 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.960 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.960 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.960 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.961 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.961 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.961 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.961 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.961 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.961 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.961 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.961 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.961 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.961 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.961 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.961 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.961 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.961 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-05-21T03:34:06.962 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-05-21T03:34:06.962 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.962 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.962 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.962 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-05-21T03:34:06.962 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.962 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.962 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.962 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.962 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.962 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.962 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.962 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.963 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.963 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.963 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.963 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.963 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.963 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.963 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.963 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.963 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.963 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.963 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.963 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.963 INFO:teuthology.orchestra.run.smithi027.stdout: "threshold": { 2024-05-21T03:34:06.963 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threshold", 2024-05-21T03:34:06.964 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-05-21T03:34:06.964 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.964 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.964 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-05-21T03:34:06.964 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1.0", 2024-05-21T03:34:06.964 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.964 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.964 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "scaling threshold", 2024-05-21T03:34:06.964 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-21T03:34:06.964 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.964 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.964 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.964 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.964 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.964 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.965 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "progress", 2024-05-21T03:34:06.965 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.965 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.965 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.965 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_pg_recovery_event": { 2024-05-21T03:34:06.965 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_pg_recovery_event", 2024-05-21T03:34:06.965 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.965 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.965 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.965 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.965 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.965 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.965 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.965 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-21T03:34:06.965 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.966 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.966 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.966 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.966 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-05-21T03:34:06.966 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-05-21T03:34:06.966 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.966 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.966 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.966 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.966 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.966 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.966 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.966 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.966 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.967 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.967 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.967 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.967 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.967 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.967 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.967 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.967 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.967 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.967 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.967 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.967 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.967 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.967 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.968 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.968 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.968 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.968 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.968 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.968 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.968 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.968 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.968 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.968 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.968 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.968 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.968 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.968 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.968 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.969 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.969 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.969 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.969 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.969 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.969 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.969 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.969 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.969 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.969 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.969 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.969 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.969 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.969 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.969 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.970 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.970 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.970 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.970 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.970 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.970 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.970 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.970 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.970 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.970 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.970 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.970 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.971 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.971 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.971 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.971 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.971 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.971 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.971 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.971 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.971 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.971 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.971 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.971 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.971 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.971 INFO:teuthology.orchestra.run.smithi027.stdout: "max_completed_events": { 2024-05-21T03:34:06.971 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_completed_events", 2024-05-21T03:34:06.972 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.972 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.972 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.972 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "50", 2024-05-21T03:34:06.972 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.972 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.972 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.972 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "number of past completed events to remember", 2024-05-21T03:34:06.972 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.972 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.972 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.972 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-05-21T03:34:06.972 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-05-21T03:34:06.972 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:06.972 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.973 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.973 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-05-21T03:34:06.973 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.973 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.973 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.973 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long the module is going to sleep", 2024-05-21T03:34:06.973 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.973 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.973 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.973 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.973 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.973 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.973 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.973 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.973 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.974 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.974 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.974 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.974 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.974 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.974 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.974 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.974 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.974 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.974 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.974 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus", 2024-05-21T03:34:06.974 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.974 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.974 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.974 INFO:teuthology.orchestra.run.smithi027.stdout: "cache": { 2024-05-21T03:34:06.975 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cache", 2024-05-21T03:34:06.975 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.975 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.975 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.975 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.975 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.975 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.975 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.975 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.975 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.975 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.975 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.975 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.975 INFO:teuthology.orchestra.run.smithi027.stdout: "exclude_perf_counters": { 2024-05-21T03:34:06.975 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "exclude_perf_counters", 2024-05-21T03:34:06.976 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.976 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.976 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.976 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:06.976 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.976 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.976 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.976 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-21T03:34:06.976 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-21T03:34:06.976 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.976 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.976 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.976 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.976 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.977 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.977 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.977 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.977 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.977 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.977 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.977 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.977 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.977 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.977 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.977 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.977 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.977 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.977 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.978 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.978 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.978 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.978 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.978 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.978 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.978 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.978 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.978 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.978 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.978 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.978 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.979 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.979 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.979 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.979 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.979 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.979 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.979 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.979 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.979 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.979 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.979 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.979 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.979 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.979 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.979 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.980 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.980 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.980 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.980 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.980 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.980 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.980 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.980 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.980 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.980 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.980 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.980 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.980 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.980 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.980 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.981 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.981 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.981 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.981 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.981 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.981 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.981 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.981 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.981 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools": { 2024-05-21T03:34:06.981 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools", 2024-05-21T03:34:06.981 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.981 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.981 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.982 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.982 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.982 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.982 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.982 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.982 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.982 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.982 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.982 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.982 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-21T03:34:06.982 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-21T03:34:06.982 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.982 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.982 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.983 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "300", 2024-05-21T03:34:06.983 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.983 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.983 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.983 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.983 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.983 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.983 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.983 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.983 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_interval": { 2024-05-21T03:34:06.983 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_interval", 2024-05-21T03:34:06.983 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-05-21T03:34:06.983 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.983 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.984 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15.0", 2024-05-21T03:34:06.984 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.984 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.984 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.984 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.984 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.984 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.984 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.984 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.984 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-05-21T03:34:06.984 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-05-21T03:34:06.984 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.984 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.984 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.984 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-05-21T03:34:06.985 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.985 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.985 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.985 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-21T03:34:06.985 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.985 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.985 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.985 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.985 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-05-21T03:34:06.985 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-05-21T03:34:06.985 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.985 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.985 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.985 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "9283", 2024-05-21T03:34:06.985 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.986 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.986 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.986 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-21T03:34:06.986 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.986 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.986 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.986 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.986 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.986 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.986 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.986 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.986 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.986 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.987 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.987 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.987 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.987 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.987 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.987 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.987 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.987 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.987 INFO:teuthology.orchestra.run.smithi027.stdout: "stale_cache_strategy": { 2024-05-21T03:34:06.987 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stale_cache_strategy", 2024-05-21T03:34:06.987 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.987 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.987 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.987 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "log", 2024-05-21T03:34:06.987 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.988 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.988 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.988 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.988 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.988 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.988 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.988 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.988 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-05-21T03:34:06.988 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-05-21T03:34:06.988 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.988 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.988 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.988 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "default", 2024-05-21T03:34:06.988 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.989 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.989 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.989 INFO:teuthology.orchestra.run.smithi027.stdout: "default", 2024-05-21T03:34:06.989 INFO:teuthology.orchestra.run.smithi027.stdout: "error" 2024-05-21T03:34:06.989 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.989 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.989 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.989 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.989 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.989 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-05-21T03:34:06.989 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-05-21T03:34:06.989 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.989 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.989 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.990 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-05-21T03:34:06.990 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-05-21T03:34:06.990 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-05-21T03:34:06.990 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.990 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.990 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.990 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.990 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.990 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.990 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.990 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.990 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-05-21T03:34:06.990 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.991 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.991 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:06.991 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:06.991 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:06.991 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.991 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.991 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.991 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.991 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.991 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.991 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.991 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.991 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.991 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.991 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.992 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.992 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.992 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.992 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.992 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.992 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.992 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.992 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.992 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:06.992 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:06.992 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.992 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.992 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.992 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.992 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.993 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.993 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.993 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.993 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.993 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.993 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.993 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:06.993 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:06.993 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.993 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.993 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.993 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:06.993 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.993 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.994 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:06.994 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:06.994 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:06.994 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:06.994 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:06.994 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:06.994 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:06.994 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:06.994 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.994 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.994 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.994 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.994 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:06.995 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:06.995 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:06.995 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.995 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.995 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:06.995 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.995 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.995 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.995 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.995 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.995 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.995 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.995 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.995 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_snap_create": { 2024-05-21T03:34:06.995 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_snap_create", 2024-05-21T03:34:06.996 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.996 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.996 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.996 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-05-21T03:34:06.996 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.996 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.996 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.996 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.996 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.996 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.996 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.996 INFO:teuthology.orchestra.run.smithi027.stdout: "mirror_snapshot_schedule": { 2024-05-21T03:34:06.996 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirror_snapshot_schedule", 2024-05-21T03:34:06.996 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.996 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.997 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.997 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.997 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.997 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.997 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.997 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.997 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.997 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.997 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.997 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:06.997 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:06.997 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:06.997 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:06.997 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:06.998 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:06.998 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.998 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.998 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.998 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.998 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.998 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.998 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.998 INFO:teuthology.orchestra.run.smithi027.stdout: "trash_purge_schedule": { 2024-05-21T03:34:06.998 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "trash_purge_schedule", 2024-05-21T03:34:06.998 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:06.998 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:06.998 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:06.999 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:06.999 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:06.999 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:06.999 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:06.999 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:06.999 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:06.999 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:06.999 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:06.999 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.999 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:06.999 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:06.999 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:06.999 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "restful", 2024-05-21T03:34:06.999 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:06.999 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:06.999 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.000 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_auth": { 2024-05-21T03:34:07.000 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_auth", 2024-05-21T03:34:07.000 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.000 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.000 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.000 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.000 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.000 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.000 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.000 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.000 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.000 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.000 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.000 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-05-21T03:34:07.001 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-05-21T03:34:07.001 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.001 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.001 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.001 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.001 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.001 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.001 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.001 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.001 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.001 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.001 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.001 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.001 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.001 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.002 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.002 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.002 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.002 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.002 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.002 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.002 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.002 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.002 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.002 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.002 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.002 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.002 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.002 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.002 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.003 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.003 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.003 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.003 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.003 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.003 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.003 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.003 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.003 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.003 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.003 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.003 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.003 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.003 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.004 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.004 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.004 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.004 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.004 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.004 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.004 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.004 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.004 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.004 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.004 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.004 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.004 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.004 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.004 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.005 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.005 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.005 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.005 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.005 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.005 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.005 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.005 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.005 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.005 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.005 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.005 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.005 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.006 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.006 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.006 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.006 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.006 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.006 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.006 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-05-21T03:34:07.006 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-05-21T03:34:07.006 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.006 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.006 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.006 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.007 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.007 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.007 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.007 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.007 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.007 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.007 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.007 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-05-21T03:34:07.007 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-05-21T03:34:07.007 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.007 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.007 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.007 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.007 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.007 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.008 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.008 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.008 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.008 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.008 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.008 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.008 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.008 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.008 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.008 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.008 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.008 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.008 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.008 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.008 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.009 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.009 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.009 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.009 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.009 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.009 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.009 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.009 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rgw", 2024-05-21T03:34:07.009 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.009 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.009 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.009 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.009 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.009 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.010 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.010 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.010 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.010 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.010 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.010 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.010 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.010 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.010 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.010 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.010 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.010 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.010 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.010 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.010 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.011 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.011 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.011 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.011 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.011 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.011 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.011 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.011 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.011 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.011 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.011 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.011 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.011 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.011 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.011 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.012 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.012 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.012 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.012 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.012 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.012 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.012 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.012 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.012 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.012 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.012 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.012 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.012 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.012 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.012 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.012 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.013 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.013 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.013 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.013 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.013 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.013 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.013 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.013 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.013 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.013 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.013 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.013 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.013 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.013 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.013 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.014 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.014 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.014 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.014 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.014 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.014 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.014 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.014 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.014 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.014 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.014 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.014 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.014 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.015 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.015 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.015 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.015 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.015 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.015 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.015 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rook", 2024-05-21T03:34:07.015 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.015 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.015 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.015 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.015 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.015 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.015 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.016 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.016 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.016 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.016 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.016 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.016 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.016 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.016 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.016 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.016 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.016 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.016 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.016 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.016 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.017 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.017 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.017 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.017 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.017 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.017 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.017 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.017 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.017 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.017 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.017 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.017 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.017 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.018 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.018 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.018 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.018 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.018 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.018 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.018 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.018 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.018 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.018 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.018 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.018 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.018 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.018 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.019 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.019 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.019 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.019 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.019 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.019 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.019 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.019 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.019 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.019 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.019 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.019 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.019 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.019 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.020 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.020 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.020 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.020 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.020 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.020 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.020 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.020 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.020 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.020 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.020 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.020 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.020 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.020 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.021 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.021 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.021 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.021 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.021 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.021 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.021 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.021 INFO:teuthology.orchestra.run.smithi027.stdout: "storage_class": { 2024-05-21T03:34:07.021 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "storage_class", 2024-05-21T03:34:07.021 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.021 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.021 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.021 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "local", 2024-05-21T03:34:07.021 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.021 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.022 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.022 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-21T03:34:07.022 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.022 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.022 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.022 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.022 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.022 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.022 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "selftest", 2024-05-21T03:34:07.022 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.022 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.022 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.022 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.022 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.023 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.023 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.023 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.023 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.023 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.023 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.023 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.023 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.023 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.023 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.023 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.023 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.023 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.023 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.023 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.023 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.024 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.024 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.024 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.024 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.024 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.024 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.024 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.024 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.024 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.024 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.024 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.024 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.024 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.024 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.025 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.025 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.025 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.025 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.025 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.025 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.025 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.025 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.025 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.025 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.025 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.025 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.025 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.025 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.025 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.025 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.026 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.026 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.026 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.026 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.026 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.026 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.026 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.026 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.026 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.026 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.026 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.026 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.026 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.027 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.027 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.027 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.027 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.027 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.027 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.027 INFO:teuthology.orchestra.run.smithi027.stdout: "roption1": { 2024-05-21T03:34:07.027 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption1", 2024-05-21T03:34:07.027 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.027 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.027 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.027 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.027 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.027 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.028 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.028 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.028 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.028 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.028 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.028 INFO:teuthology.orchestra.run.smithi027.stdout: "roption2": { 2024-05-21T03:34:07.028 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption2", 2024-05-21T03:34:07.028 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.028 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.028 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.028 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "xyz", 2024-05-21T03:34:07.028 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.028 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.028 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.028 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.029 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.029 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.029 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.029 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption1": { 2024-05-21T03:34:07.029 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption1", 2024-05-21T03:34:07.029 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.029 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.029 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.029 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.029 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.029 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.029 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.029 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.029 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.030 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.030 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.030 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.030 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption2": { 2024-05-21T03:34:07.030 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption2", 2024-05-21T03:34:07.030 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.030 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.030 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.030 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.030 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.030 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.030 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.030 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.030 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.030 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.031 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.031 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption3": { 2024-05-21T03:34:07.031 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption3", 2024-05-21T03:34:07.031 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-05-21T03:34:07.031 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.031 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.031 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.031 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.031 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.031 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.031 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.031 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.031 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.031 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.032 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption4": { 2024-05-21T03:34:07.032 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption4", 2024-05-21T03:34:07.032 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.032 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.032 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.032 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.032 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.032 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.032 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.032 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.032 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.032 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.032 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.032 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption5": { 2024-05-21T03:34:07.032 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption5", 2024-05-21T03:34:07.033 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.033 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.033 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.033 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.033 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.033 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.033 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.033 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.033 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.033 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.033 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.033 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption6": { 2024-05-21T03:34:07.033 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption6", 2024-05-21T03:34:07.033 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.034 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.034 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.034 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.034 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.034 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.034 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.034 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.034 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.034 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.034 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.034 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption7": { 2024-05-21T03:34:07.034 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption7", 2024-05-21T03:34:07.034 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.035 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.035 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.035 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.035 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-05-21T03:34:07.035 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "42", 2024-05-21T03:34:07.035 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.035 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.035 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.035 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.035 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.035 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.035 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.035 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.035 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.035 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.036 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.036 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.036 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.036 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.036 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.036 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.036 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.036 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.036 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.036 INFO:teuthology.orchestra.run.smithi027.stdout: "testkey": { 2024-05-21T03:34:07.036 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testkey", 2024-05-21T03:34:07.036 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.036 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.036 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.036 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.037 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.037 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.037 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.037 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.037 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.037 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.037 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.037 INFO:teuthology.orchestra.run.smithi027.stdout: "testlkey": { 2024-05-21T03:34:07.037 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testlkey", 2024-05-21T03:34:07.037 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.037 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.037 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.037 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.037 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.038 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.038 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.038 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.038 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.038 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.038 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.038 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.038 INFO:teuthology.orchestra.run.smithi027.stdout: "testnewline": { 2024-05-21T03:34:07.038 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testnewline", 2024-05-21T03:34:07.038 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.038 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.038 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.038 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.038 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.038 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.039 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.039 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.039 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.039 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.039 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.039 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.039 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.039 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.039 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.039 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smb", 2024-05-21T03:34:07.039 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.039 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.039 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.039 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.039 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.040 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.040 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.040 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.040 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.040 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.040 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.040 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.040 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.040 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.040 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.040 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.040 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.040 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.040 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.040 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.040 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.041 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.041 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.041 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.041 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.041 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.041 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.041 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.041 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.041 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.041 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.041 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.041 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.041 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.042 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.042 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.042 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.042 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.042 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.042 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.042 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.042 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.042 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.042 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.042 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.042 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.043 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.043 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.043 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.043 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.043 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.043 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.043 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.043 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.043 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.043 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.043 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.043 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.043 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.043 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.043 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.043 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.044 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.044 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.044 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.044 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.044 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.044 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.044 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.044 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.044 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.044 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.044 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.044 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.044 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.045 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.045 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.045 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.045 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.045 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.045 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.045 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.045 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.045 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.045 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.045 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.045 INFO:teuthology.orchestra.run.smithi027.stdout: "update_orchestration": { 2024-05-21T03:34:07.045 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "update_orchestration", 2024-05-21T03:34:07.045 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.046 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.046 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.046 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.046 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.046 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.046 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.046 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-21T03:34:07.046 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.046 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.046 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.046 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.046 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.046 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.046 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.047 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-05-21T03:34:07.047 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.047 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.047 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.047 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_m_granularity": { 2024-05-21T03:34:07.047 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_m_granularity", 2024-05-21T03:34:07.047 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.047 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.047 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.047 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.047 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.047 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.047 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.048 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow minute scheduled snapshots", 2024-05-21T03:34:07.048 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.048 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.048 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.048 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.048 INFO:teuthology.orchestra.run.smithi027.stdout: "dump_on_update": { 2024-05-21T03:34:07.048 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dump_on_update", 2024-05-21T03:34:07.048 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.048 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.048 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.048 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.048 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.048 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.048 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.048 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "dump database to debug log on update", 2024-05-21T03:34:07.048 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.049 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.049 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.049 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.049 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.049 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.049 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.049 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.049 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.049 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.049 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.049 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.049 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.049 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.049 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.049 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.050 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.050 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.050 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.050 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.050 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.050 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.050 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.050 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.050 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.050 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.050 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.050 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.050 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.051 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.051 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.051 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.051 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.051 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.051 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.051 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.051 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.051 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.051 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.051 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.051 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.051 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.052 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.052 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.052 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.052 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.052 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.052 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.052 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.052 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.052 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.052 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.052 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.052 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.052 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.052 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.052 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.053 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.053 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.053 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.053 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.053 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.053 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.053 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.053 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.053 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.053 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.053 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.053 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.053 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.053 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.053 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.054 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.054 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.054 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.054 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.054 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.054 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.054 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.054 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.054 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.054 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.054 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.054 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.054 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.054 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.054 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.055 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.055 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stats", 2024-05-21T03:34:07.055 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.055 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.055 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.055 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.055 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.055 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.055 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.055 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.055 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.055 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.055 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.055 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.056 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.056 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.056 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.056 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.056 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.056 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.056 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.056 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.056 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.056 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.056 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.056 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.056 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.056 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.057 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.057 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.057 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.057 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.057 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.057 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.057 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.057 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.057 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.057 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.057 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.057 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.057 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.057 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.057 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.057 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.058 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.058 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.058 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.058 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.058 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.058 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.058 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.058 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.058 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.058 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.058 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.058 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.058 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.059 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.059 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.059 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.059 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.059 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.059 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.059 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.059 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.059 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.059 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.059 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.059 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.059 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.059 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.059 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.060 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.060 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.060 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.060 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.060 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.060 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.060 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.060 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.060 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.060 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.060 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.060 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.060 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.060 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.060 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.061 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.061 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.061 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.061 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "status", 2024-05-21T03:34:07.061 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.061 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.061 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.061 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.061 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.061 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.061 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.061 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.061 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.061 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.061 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.062 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.062 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.062 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.062 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.062 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.062 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.062 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.062 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.062 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.062 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.062 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.062 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.062 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.062 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.063 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.063 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.063 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.063 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.063 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.063 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.063 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.063 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.063 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.063 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.063 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.063 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.063 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.063 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.063 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.064 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.064 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.064 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.064 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.064 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.064 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.064 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.064 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.064 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.064 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.064 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.064 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.064 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.064 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.064 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.064 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.065 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.065 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.065 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.065 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.065 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.065 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.065 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.065 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.065 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.065 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.065 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.065 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.065 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.065 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.065 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.065 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.066 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.066 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.066 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.066 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.066 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.066 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.066 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.066 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.066 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.066 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.066 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.066 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.067 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.067 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.067 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.067 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.067 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.067 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telegraf", 2024-05-21T03:34:07.067 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.067 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.067 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.067 INFO:teuthology.orchestra.run.smithi027.stdout: "address": { 2024-05-21T03:34:07.067 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "address", 2024-05-21T03:34:07.067 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.067 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.067 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.067 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-21T03:34:07.068 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.068 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.068 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.068 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.068 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.068 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.068 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.068 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-05-21T03:34:07.068 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-05-21T03:34:07.068 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:07.068 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.068 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.068 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15", 2024-05-21T03:34:07.068 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.068 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.069 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.069 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.069 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.069 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.069 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.069 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.069 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.069 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.069 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.069 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.069 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.069 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.069 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.069 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.069 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.070 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.070 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.070 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.070 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.070 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.070 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.070 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.070 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.070 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.070 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.070 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.070 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.070 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.070 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.071 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.071 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.071 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.071 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.071 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.071 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.071 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.071 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.071 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.071 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.071 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.071 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.071 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.071 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.071 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.072 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.072 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.072 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.072 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.072 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.072 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.072 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.072 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.072 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.072 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.072 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.072 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.072 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.072 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.072 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.073 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.073 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.073 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.073 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.073 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.073 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.073 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.073 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.073 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.073 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.073 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.073 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.073 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.073 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.074 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.074 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.074 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.074 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.074 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.074 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.074 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.074 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.074 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.074 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.074 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.074 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.074 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.074 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.074 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.075 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.075 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.075 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telemetry", 2024-05-21T03:34:07.075 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.075 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.075 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.075 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_basic": { 2024-05-21T03:34:07.075 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_basic", 2024-05-21T03:34:07.075 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.075 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.075 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.075 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.075 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.075 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.075 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.076 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-21T03:34:07.076 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.076 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.076 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.076 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_crash": { 2024-05-21T03:34:07.076 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_crash", 2024-05-21T03:34:07.076 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.076 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.076 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.076 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.076 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.076 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.076 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.076 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-21T03:34:07.077 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.077 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.077 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.077 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_device": { 2024-05-21T03:34:07.077 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_device", 2024-05-21T03:34:07.077 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.077 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.077 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.077 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.077 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.077 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.077 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.077 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-21T03:34:07.077 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.078 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.078 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.078 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_ident": { 2024-05-21T03:34:07.078 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_ident", 2024-05-21T03:34:07.078 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.078 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.078 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.078 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.078 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.078 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.078 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.078 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-21T03:34:07.078 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.078 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.079 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.079 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_perf": { 2024-05-21T03:34:07.079 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_perf", 2024-05-21T03:34:07.079 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.079 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.079 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.079 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.079 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.079 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.079 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.079 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-21T03:34:07.079 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.079 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.080 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.080 INFO:teuthology.orchestra.run.smithi027.stdout: "contact": { 2024-05-21T03:34:07.080 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "contact", 2024-05-21T03:34:07.080 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.080 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.080 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.080 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.080 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.080 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.080 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.080 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.080 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.080 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.080 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.081 INFO:teuthology.orchestra.run.smithi027.stdout: "description": { 2024-05-21T03:34:07.081 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "description", 2024-05-21T03:34:07.081 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.081 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.081 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.081 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.081 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.081 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.081 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.081 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.081 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.081 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.081 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.081 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.081 INFO:teuthology.orchestra.run.smithi027.stdout: "device_url": { 2024-05-21T03:34:07.082 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_url", 2024-05-21T03:34:07.082 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.082 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.082 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.082 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-21T03:34:07.082 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.082 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.082 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.082 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.082 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.082 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.082 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.082 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.082 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-05-21T03:34:07.082 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-05-21T03:34:07.083 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.083 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.083 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.083 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.083 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.083 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.083 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.083 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.083 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.083 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.083 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.083 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-05-21T03:34:07.083 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-05-21T03:34:07.083 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.084 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.084 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.084 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "24", 2024-05-21T03:34:07.084 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "8", 2024-05-21T03:34:07.084 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.084 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.084 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.084 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.084 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.084 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.084 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.084 INFO:teuthology.orchestra.run.smithi027.stdout: "last_opt_revision": { 2024-05-21T03:34:07.084 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "last_opt_revision", 2024-05-21T03:34:07.084 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.084 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.084 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.085 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1", 2024-05-21T03:34:07.085 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.085 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.085 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.085 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.085 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.085 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.085 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.085 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard": { 2024-05-21T03:34:07.085 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard", 2024-05-21T03:34:07.085 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.085 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.085 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.085 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.086 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.086 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.086 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.086 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.086 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.086 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.086 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.086 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.086 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard_description": { 2024-05-21T03:34:07.086 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard_description", 2024-05-21T03:34:07.086 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.086 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.086 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.086 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.086 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.087 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.087 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.087 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.087 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.087 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.087 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.087 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.087 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.087 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.087 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.087 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.087 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.087 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.087 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.087 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.087 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.088 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.088 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.088 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.088 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.088 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.088 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.088 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.088 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.088 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.088 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.088 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.088 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.088 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.088 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.088 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.089 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.089 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.089 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.089 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.089 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.089 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.089 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.089 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.089 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.089 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.089 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.089 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.089 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.089 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.089 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.090 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.090 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.090 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.090 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.090 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.090 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.090 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.090 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.090 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.090 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.090 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.090 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.090 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.090 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.091 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.091 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.091 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.091 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.091 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.091 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.091 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.091 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.091 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.091 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.091 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.091 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.091 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.092 INFO:teuthology.orchestra.run.smithi027.stdout: "organization": { 2024-05-21T03:34:07.092 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "organization", 2024-05-21T03:34:07.092 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.092 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.092 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.092 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.092 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.092 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.092 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.092 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.092 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.092 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.092 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.092 INFO:teuthology.orchestra.run.smithi027.stdout: "proxy": { 2024-05-21T03:34:07.093 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "proxy", 2024-05-21T03:34:07.093 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.093 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.093 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.093 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.093 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.093 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.093 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.093 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.093 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.093 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.093 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.093 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.093 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.093 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.094 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.094 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.094 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.094 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.094 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.094 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.094 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.094 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.094 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.094 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.094 INFO:teuthology.orchestra.run.smithi027.stdout: "url": { 2024-05-21T03:34:07.094 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url", 2024-05-21T03:34:07.094 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.094 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.095 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.095 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-21T03:34:07.095 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.095 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.095 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.095 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.095 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.095 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.095 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.095 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.095 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.095 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.095 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "test_orchestrator", 2024-05-21T03:34:07.095 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.096 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.096 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.096 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.096 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.096 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.096 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.096 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.096 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.096 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.096 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.096 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.096 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.096 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.096 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.096 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.096 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.097 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.097 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.097 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.097 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.097 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.097 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.097 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.097 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.097 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.097 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.097 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.097 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.097 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.097 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.098 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.098 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.098 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.098 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.098 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.098 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.098 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.098 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.098 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.098 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.098 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.098 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.098 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.098 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.099 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.099 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.099 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.099 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.099 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.099 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.099 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.099 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.099 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.099 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.099 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.099 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.099 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.099 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.100 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.100 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.100 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.100 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.100 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.100 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.100 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.100 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.100 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.100 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.100 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.100 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.100 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.100 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.100 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.101 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.101 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.101 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.101 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.101 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.101 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.101 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.101 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.101 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.101 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.101 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.101 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-05-21T03:34:07.101 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.101 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.101 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.102 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.102 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.102 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.102 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.102 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.102 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.102 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.102 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.102 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.102 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.102 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.102 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.102 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.102 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.103 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.103 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.103 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.103 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.103 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.103 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.103 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.103 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.103 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.103 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.103 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.103 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.103 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.103 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.103 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.104 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.104 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.104 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.104 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.104 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.104 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.104 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.104 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.104 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.104 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.104 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.104 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.104 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.104 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.105 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.105 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.105 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.105 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.105 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.105 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.105 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.105 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.105 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.105 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.105 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.105 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.105 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.106 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.106 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.106 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.106 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.106 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.106 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.106 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.106 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.106 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.106 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.106 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_clones": { 2024-05-21T03:34:07.106 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_clones", 2024-05-21T03:34:07.106 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.107 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.107 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.107 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4", 2024-05-21T03:34:07.107 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.107 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.107 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.107 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-21T03:34:07.107 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.107 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.107 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.107 INFO:teuthology.orchestra.run.smithi027.stdout: "periodic_async_work": { 2024-05-21T03:34:07.107 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "periodic_async_work", 2024-05-21T03:34:07.107 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.107 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.108 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.108 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.108 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.108 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.108 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.108 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Periodically check for async work", 2024-05-21T03:34:07.108 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.108 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.108 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.108 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_delay": { 2024-05-21T03:34:07.108 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_delay", 2024-05-21T03:34:07.108 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.108 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.108 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.109 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.109 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.109 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.109 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.109 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-21T03:34:07.109 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.109 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.109 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.109 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_no_wait": { 2024-05-21T03:34:07.109 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_no_wait", 2024-05-21T03:34:07.109 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.109 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.109 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.110 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.110 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.110 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.110 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.110 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-21T03:34:07.110 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.110 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.110 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.110 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.110 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.110 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.110 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.110 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.111 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.111 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.111 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.111 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.111 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.111 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.111 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.111 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.111 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.111 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.111 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix", 2024-05-21T03:34:07.111 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.111 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.111 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.112 INFO:teuthology.orchestra.run.smithi027.stdout: "discovery_interval": { 2024-05-21T03:34:07.112 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "discovery_interval", 2024-05-21T03:34:07.112 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-05-21T03:34:07.112 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.112 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.112 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "100", 2024-05-21T03:34:07.112 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.112 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.112 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.112 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.112 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.112 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.112 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.112 INFO:teuthology.orchestra.run.smithi027.stdout: "identifier": { 2024-05-21T03:34:07.112 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "identifier", 2024-05-21T03:34:07.113 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.113 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.113 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.113 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.113 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.113 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.113 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.113 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.113 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.113 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.113 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.113 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-05-21T03:34:07.113 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-05-21T03:34:07.113 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:07.114 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.114 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.114 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-05-21T03:34:07.114 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.114 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.114 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.114 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.114 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.114 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.114 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.114 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.114 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.114 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.114 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.114 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.115 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.115 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.115 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.115 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.115 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.115 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.115 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.115 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.115 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.115 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.115 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.115 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.115 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.115 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.115 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.116 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.116 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.116 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.116 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.116 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.116 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.116 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.116 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.116 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.116 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.116 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.116 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.116 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.117 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.117 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.117 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.117 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.117 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.117 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.117 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.117 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.117 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.117 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.117 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.117 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.117 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.117 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.117 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.118 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.118 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.118 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.118 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.118 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.118 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.118 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.118 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.118 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.118 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.118 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.118 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.118 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.119 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.119 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.119 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.119 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.119 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.119 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.119 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.119 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.119 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.119 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.119 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.119 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.119 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.119 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.119 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.120 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.120 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.120 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.120 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_host": { 2024-05-21T03:34:07.120 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_host", 2024-05-21T03:34:07.120 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.120 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.120 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.120 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.120 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.120 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.120 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.120 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.120 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.120 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.121 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.121 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_port": { 2024-05-21T03:34:07.121 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_port", 2024-05-21T03:34:07.121 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.121 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.121 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.121 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10051", 2024-05-21T03:34:07.121 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.121 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.121 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.121 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.121 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.121 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.121 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.122 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_sender": { 2024-05-21T03:34:07.122 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_sender", 2024-05-21T03:34:07.122 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.122 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.122 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.122 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-21T03:34:07.122 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.122 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.122 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.122 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.122 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.122 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.122 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.122 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.122 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.123 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.123 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-05-21T03:34:07.123 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.123 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.123 INFO:teuthology.orchestra.run.smithi027.stdout: "modules": [ 2024-05-21T03:34:07.123 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm", 2024-05-21T03:34:07.123 INFO:teuthology.orchestra.run.smithi027.stdout: "dashboard", 2024-05-21T03:34:07.123 INFO:teuthology.orchestra.run.smithi027.stdout: "iostat", 2024-05-21T03:34:07.123 INFO:teuthology.orchestra.run.smithi027.stdout: "nfs", 2024-05-21T03:34:07.123 INFO:teuthology.orchestra.run.smithi027.stdout: "restful", 2024-05-21T03:34:07.123 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_schedule" 2024-05-21T03:34:07.123 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.123 INFO:teuthology.orchestra.run.smithi027.stdout: "available_modules": [ 2024-05-21T03:34:07.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.123 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "alerts", 2024-05-21T03:34:07.124 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.124 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.124 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.124 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-05-21T03:34:07.124 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-05-21T03:34:07.124 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:07.124 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.124 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.124 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-05-21T03:34:07.124 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.124 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.124 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.124 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How frequently to reexamine health status", 2024-05-21T03:34:07.124 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.124 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.124 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.125 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.125 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.125 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.125 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.125 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.125 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.125 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.125 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.125 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.125 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.125 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.125 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.125 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.125 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.125 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.126 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.126 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.126 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.126 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.126 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.126 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.126 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.126 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.126 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.126 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.126 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.126 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.126 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.127 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.127 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.127 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.127 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.127 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.127 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.127 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.127 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.127 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.127 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.127 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.127 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.127 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.127 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.127 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.127 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.128 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.128 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.128 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.128 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.128 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.128 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.128 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.128 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.128 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.128 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.128 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.128 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.128 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.128 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.129 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.129 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.129 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.129 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.129 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.129 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.129 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.129 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.129 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_destination": { 2024-05-21T03:34:07.129 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_destination", 2024-05-21T03:34:07.129 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.129 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.129 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.129 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.129 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.130 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.130 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.130 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email address to send alerts to", 2024-05-21T03:34:07.130 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.130 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.130 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.130 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_from_name": { 2024-05-21T03:34:07.130 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_from_name", 2024-05-21T03:34:07.130 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.130 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.130 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.130 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "Ceph", 2024-05-21T03:34:07.130 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.130 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.131 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.131 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email From: name", 2024-05-21T03:34:07.131 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.131 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.131 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.131 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_host": { 2024-05-21T03:34:07.131 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_host", 2024-05-21T03:34:07.131 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.131 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.131 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.131 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.131 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.131 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.131 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.132 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP server", 2024-05-21T03:34:07.132 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.132 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.132 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.132 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_password": { 2024-05-21T03:34:07.132 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_password", 2024-05-21T03:34:07.132 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.132 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.132 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.132 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.132 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.132 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.132 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.132 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Password to authenticate with", 2024-05-21T03:34:07.132 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.133 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.133 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.133 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_port": { 2024-05-21T03:34:07.133 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_port", 2024-05-21T03:34:07.133 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.133 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.133 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.133 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "465", 2024-05-21T03:34:07.133 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.133 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.133 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.133 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP port", 2024-05-21T03:34:07.133 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.133 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.133 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.134 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_sender": { 2024-05-21T03:34:07.134 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_sender", 2024-05-21T03:34:07.134 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.134 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.134 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.134 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.134 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.134 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.134 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.134 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP envelope sender", 2024-05-21T03:34:07.134 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.134 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.134 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.135 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_ssl": { 2024-05-21T03:34:07.135 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_ssl", 2024-05-21T03:34:07.135 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.135 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.135 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.135 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.135 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.135 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.135 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.135 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-21T03:34:07.135 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.135 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.135 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.135 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_user": { 2024-05-21T03:34:07.136 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_user", 2024-05-21T03:34:07.136 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.136 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.136 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.136 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.136 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.136 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.136 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.136 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "User to authenticate as", 2024-05-21T03:34:07.136 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.136 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.136 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.136 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.136 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.137 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.137 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.137 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.137 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.137 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.137 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.137 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.137 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.137 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.137 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.137 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.137 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.137 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.138 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.138 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "balancer", 2024-05-21T03:34:07.138 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.138 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.138 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.138 INFO:teuthology.orchestra.run.smithi027.stdout: "active": { 2024-05-21T03:34:07.138 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "active", 2024-05-21T03:34:07.138 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.138 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.138 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.138 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.138 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.138 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.139 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.139 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically balance PGs across cluster", 2024-05-21T03:34:07.139 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.139 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.139 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.139 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_time": { 2024-05-21T03:34:07.139 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_time", 2024-05-21T03:34:07.139 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.139 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.139 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.139 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0000", 2024-05-21T03:34:07.139 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.139 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.140 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.140 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "beginning time of day to automatically balance", 2024-05-21T03:34:07.140 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-21T03:34:07.140 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.140 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.140 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_weekday": { 2024-05-21T03:34:07.140 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_weekday", 2024-05-21T03:34:07.140 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-05-21T03:34:07.140 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.140 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.140 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.140 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-05-21T03:34:07.140 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-05-21T03:34:07.140 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.141 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-21T03:34:07.141 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-21T03:34:07.141 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.141 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.141 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_max_iterations": { 2024-05-21T03:34:07.141 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_max_iterations", 2024-05-21T03:34:07.141 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-05-21T03:34:07.141 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.141 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.141 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "25", 2024-05-21T03:34:07.141 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-05-21T03:34:07.141 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "250", 2024-05-21T03:34:07.141 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.142 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-21T03:34:07.142 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.142 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.142 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.142 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_metrics": { 2024-05-21T03:34:07.142 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_metrics", 2024-05-21T03:34:07.142 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.142 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.142 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.142 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "pgs,objects,bytes", 2024-05-21T03:34:07.142 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.142 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.142 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.142 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-21T03:34:07.143 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-21T03:34:07.143 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.143 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.143 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_step": { 2024-05-21T03:34:07.143 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_step", 2024-05-21T03:34:07.143 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-05-21T03:34:07.143 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.143 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.143 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.5", 2024-05-21T03:34:07.143 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0.001", 2024-05-21T03:34:07.143 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "0.999", 2024-05-21T03:34:07.143 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.143 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "aggressiveness of optimization", 2024-05-21T03:34:07.143 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-21T03:34:07.144 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.144 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.144 INFO:teuthology.orchestra.run.smithi027.stdout: "end_time": { 2024-05-21T03:34:07.144 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_time", 2024-05-21T03:34:07.144 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.144 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.144 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.144 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2359", 2024-05-21T03:34:07.144 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.144 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.144 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.144 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ending time of day to automatically balance", 2024-05-21T03:34:07.144 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-21T03:34:07.144 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.144 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.145 INFO:teuthology.orchestra.run.smithi027.stdout: "end_weekday": { 2024-05-21T03:34:07.145 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_weekday", 2024-05-21T03:34:07.145 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-05-21T03:34:07.145 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.145 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.145 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.145 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-05-21T03:34:07.145 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-05-21T03:34:07.145 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.145 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-21T03:34:07.145 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-21T03:34:07.145 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.145 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.145 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.146 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.146 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.146 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.146 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.146 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.146 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.146 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.146 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.146 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.146 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.146 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.146 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.146 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.146 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.146 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.147 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.147 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.147 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.147 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.147 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.147 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.147 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.147 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.147 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.147 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.147 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.147 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.147 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.147 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.147 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.147 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.148 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.148 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.148 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.148 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.148 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.148 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.148 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.148 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.148 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.148 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.148 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.148 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.148 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.148 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.149 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.149 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.149 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.149 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.149 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.149 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.149 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.149 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.149 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.149 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.149 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.149 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.149 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.149 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.150 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.150 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.150 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.150 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.150 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.150 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.150 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.150 INFO:teuthology.orchestra.run.smithi027.stdout: "min_score": { 2024-05-21T03:34:07.150 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_score", 2024-05-21T03:34:07.150 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-05-21T03:34:07.150 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.150 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.150 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.151 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.151 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.151 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.151 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-21T03:34:07.151 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.151 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.151 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.151 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-05-21T03:34:07.151 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-05-21T03:34:07.151 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.151 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.151 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.151 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "upmap", 2024-05-21T03:34:07.151 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.152 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.152 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.152 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-compat", 2024-05-21T03:34:07.152 INFO:teuthology.orchestra.run.smithi027.stdout: "none", 2024-05-21T03:34:07.152 INFO:teuthology.orchestra.run.smithi027.stdout: "read", 2024-05-21T03:34:07.152 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap", 2024-05-21T03:34:07.152 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap-read" 2024-05-21T03:34:07.152 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.152 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Balancer mode", 2024-05-21T03:34:07.152 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.152 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.152 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.152 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_ids": { 2024-05-21T03:34:07.153 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_ids", 2024-05-21T03:34:07.153 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.153 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.153 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.153 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.153 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.153 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.153 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.153 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-21T03:34:07.153 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.153 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.153 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.154 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-05-21T03:34:07.154 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-05-21T03:34:07.154 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:07.154 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.154 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.154 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-05-21T03:34:07.154 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.154 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.154 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.154 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-21T03:34:07.154 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.154 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.154 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.154 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.155 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.155 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.155 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.155 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.155 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.155 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.155 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.155 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.155 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.155 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.155 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.155 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.155 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_deviation": { 2024-05-21T03:34:07.155 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_deviation", 2024-05-21T03:34:07.155 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.156 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.156 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.156 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-05-21T03:34:07.156 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-05-21T03:34:07.156 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.156 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.156 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-21T03:34:07.156 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-21T03:34:07.156 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.156 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.156 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_optimizations": { 2024-05-21T03:34:07.156 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_optimizations", 2024-05-21T03:34:07.156 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-05-21T03:34:07.156 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.156 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.157 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-05-21T03:34:07.157 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.157 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.157 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.157 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-21T03:34:07.157 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.157 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.157 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.157 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.157 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.157 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.157 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm", 2024-05-21T03:34:07.157 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.157 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.157 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.158 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_down_multiplier": { 2024-05-21T03:34:07.158 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_down_multiplier", 2024-05-21T03:34:07.158 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-05-21T03:34:07.158 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.158 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.158 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-05-21T03:34:07.158 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.158 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.158 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.158 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-21T03:34:07.158 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.158 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.158 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.159 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_refresh_rate": { 2024-05-21T03:34:07.159 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_refresh_rate", 2024-05-21T03:34:07.159 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:07.159 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.159 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.159 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "20", 2024-05-21T03:34:07.159 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.159 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.159 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.159 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-21T03:34:07.159 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.159 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.159 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.159 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_starting_port": { 2024-05-21T03:34:07.160 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_starting_port", 2024-05-21T03:34:07.160 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.160 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.160 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.160 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4721", 2024-05-21T03:34:07.160 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.160 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.160 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.160 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-21T03:34:07.160 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.160 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.160 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.160 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_ptrace": { 2024-05-21T03:34:07.160 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_ptrace", 2024-05-21T03:34:07.161 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.161 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.161 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.161 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.161 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.161 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.161 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.161 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-21T03:34:07.161 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-21T03:34:07.161 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.161 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.161 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_interval": { 2024-05-21T03:34:07.161 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_interval", 2024-05-21T03:34:07.161 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:07.162 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.162 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.162 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-05-21T03:34:07.162 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.162 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.162 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.162 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-21T03:34:07.162 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.162 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.162 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.162 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_memory_target_ratio": { 2024-05-21T03:34:07.162 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_memory_target_ratio", 2024-05-21T03:34:07.162 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-05-21T03:34:07.162 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.163 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.163 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.7", 2024-05-21T03:34:07.163 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.163 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.163 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.163 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-21T03:34:07.163 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.163 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.163 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.163 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm_log_destination": { 2024-05-21T03:34:07.163 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm_log_destination", 2024-05-21T03:34:07.163 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.163 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.164 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.164 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.164 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.164 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.164 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.164 INFO:teuthology.orchestra.run.smithi027.stdout: "file", 2024-05-21T03:34:07.164 INFO:teuthology.orchestra.run.smithi027.stdout: "file,syslog", 2024-05-21T03:34:07.164 INFO:teuthology.orchestra.run.smithi027.stdout: "syslog" 2024-05-21T03:34:07.164 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.164 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-21T03:34:07.164 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.164 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.164 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.165 INFO:teuthology.orchestra.run.smithi027.stdout: "cgroups_split": { 2024-05-21T03:34:07.165 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cgroups_split", 2024-05-21T03:34:07.165 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.165 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.165 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.165 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.165 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.165 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.165 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.165 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-21T03:34:07.165 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.165 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.165 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.165 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.165 INFO:teuthology.orchestra.run.smithi027.stdout: "config_checks_enabled": { 2024-05-21T03:34:07.165 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_checks_enabled", 2024-05-21T03:34:07.166 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.166 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.166 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.166 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.166 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.166 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.166 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.166 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-21T03:34:07.166 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.166 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.166 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.166 INFO:teuthology.orchestra.run.smithi027.stdout: "config_dashboard": { 2024-05-21T03:34:07.166 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_dashboard", 2024-05-21T03:34:07.167 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.167 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.167 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.167 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.167 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.167 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.167 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.167 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-21T03:34:07.167 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.167 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.167 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.167 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_alertmanager": { 2024-05-21T03:34:07.167 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_alertmanager", 2024-05-21T03:34:07.167 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.167 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.168 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.168 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-21T03:34:07.168 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.168 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.168 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.168 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-05-21T03:34:07.168 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.168 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.168 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.168 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_base": { 2024-05-21T03:34:07.168 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_base", 2024-05-21T03:34:07.168 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.168 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.169 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.169 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-21T03:34:07.169 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.169 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.169 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.169 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Container image name, without the tag", 2024-05-21T03:34:07.169 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.169 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.169 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.169 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.169 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_elasticsearch": { 2024-05-21T03:34:07.169 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_elasticsearch", 2024-05-21T03:34:07.169 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.169 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.169 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.169 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-21T03:34:07.170 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.170 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.170 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.170 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "elasticsearch container image", 2024-05-21T03:34:07.170 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.170 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.170 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.170 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.170 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_grafana": { 2024-05-21T03:34:07.170 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_grafana", 2024-05-21T03:34:07.170 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.170 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.170 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.170 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-21T03:34:07.170 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.171 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.171 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.171 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-05-21T03:34:07.171 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.171 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.171 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.171 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_haproxy": { 2024-05-21T03:34:07.171 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_haproxy", 2024-05-21T03:34:07.171 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.171 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.171 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.171 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-21T03:34:07.171 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.172 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.172 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.172 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "HAproxy container image", 2024-05-21T03:34:07.172 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.172 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.172 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.172 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_agent": { 2024-05-21T03:34:07.172 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_agent", 2024-05-21T03:34:07.172 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.172 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.172 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.172 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-21T03:34:07.172 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.172 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.172 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.173 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger agent container image", 2024-05-21T03:34:07.173 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.173 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.173 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.173 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_collector": { 2024-05-21T03:34:07.173 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_collector", 2024-05-21T03:34:07.173 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.173 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.173 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.173 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-21T03:34:07.173 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.173 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.173 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.173 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger collector container image", 2024-05-21T03:34:07.174 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.174 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.174 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.174 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_query": { 2024-05-21T03:34:07.174 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_query", 2024-05-21T03:34:07.174 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.174 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.174 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.174 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-21T03:34:07.174 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.174 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.174 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.174 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger query container image", 2024-05-21T03:34:07.175 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.175 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.175 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.175 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_keepalived": { 2024-05-21T03:34:07.175 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_keepalived", 2024-05-21T03:34:07.175 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.175 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.175 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.175 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-21T03:34:07.175 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.175 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.175 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.175 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Keepalived container image", 2024-05-21T03:34:07.175 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.175 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.176 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.176 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_loki": { 2024-05-21T03:34:07.176 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_loki", 2024-05-21T03:34:07.176 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.176 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.176 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.176 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-21T03:34:07.176 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.176 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.176 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.176 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Loki container image", 2024-05-21T03:34:07.176 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.176 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.176 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.177 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_node_exporter": { 2024-05-21T03:34:07.177 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_node_exporter", 2024-05-21T03:34:07.177 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.177 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.177 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.177 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-21T03:34:07.177 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.177 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.177 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.177 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-05-21T03:34:07.177 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.177 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.177 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.178 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nvmeof": { 2024-05-21T03:34:07.178 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nvmeof", 2024-05-21T03:34:07.178 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.178 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.178 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.178 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-21T03:34:07.178 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.178 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.178 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.178 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nvme-of container image", 2024-05-21T03:34:07.178 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.178 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.178 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.178 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_prometheus": { 2024-05-21T03:34:07.179 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_prometheus", 2024-05-21T03:34:07.179 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.179 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.179 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.179 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-21T03:34:07.179 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.179 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.179 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.179 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-05-21T03:34:07.179 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.179 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.179 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.179 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_promtail": { 2024-05-21T03:34:07.179 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_promtail", 2024-05-21T03:34:07.180 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.180 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.180 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.180 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-21T03:34:07.180 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.180 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.180 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.180 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Promtail container image", 2024-05-21T03:34:07.180 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.180 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.180 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.180 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba": { 2024-05-21T03:34:07.180 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba", 2024-05-21T03:34:07.180 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.181 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.181 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.181 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-21T03:34:07.181 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.181 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.181 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.181 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB container image", 2024-05-21T03:34:07.181 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.181 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.181 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.181 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_snmp_gateway": { 2024-05-21T03:34:07.181 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_snmp_gateway", 2024-05-21T03:34:07.181 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.181 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.182 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.182 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-21T03:34:07.182 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.182 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.182 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.182 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SNMP Gateway container image", 2024-05-21T03:34:07.182 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.182 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.182 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.182 INFO:teuthology.orchestra.run.smithi027.stdout: "container_init": { 2024-05-21T03:34:07.182 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_init", 2024-05-21T03:34:07.182 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.182 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.182 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Run podman/docker with `--init`", 2024-05-21T03:34:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: "daemon_cache_timeout": { 2024-05-21T03:34:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "daemon_cache_timeout", 2024-05-21T03:34:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-05-21T03:34:07.184 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.184 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.184 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.184 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-21T03:34:07.184 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.184 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.184 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.184 INFO:teuthology.orchestra.run.smithi027.stdout: "default_cephadm_command_timeout": { 2024-05-21T03:34:07.184 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_cephadm_command_timeout", 2024-05-21T03:34:07.184 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.184 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.184 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.184 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "900", 2024-05-21T03:34:07.184 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.184 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.185 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.185 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-21T03:34:07.185 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.185 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.185 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.185 INFO:teuthology.orchestra.run.smithi027.stdout: "default_registry": { 2024-05-21T03:34:07.185 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_registry", 2024-05-21T03:34:07.185 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.185 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.185 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.185 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io", 2024-05-21T03:34:07.185 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.185 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.185 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.185 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-21T03:34:07.186 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.186 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.186 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.186 INFO:teuthology.orchestra.run.smithi027.stdout: "device_cache_timeout": { 2024-05-21T03:34:07.186 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_cache_timeout", 2024-05-21T03:34:07.186 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:07.186 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.186 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.186 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1800", 2024-05-21T03:34:07.186 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.186 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.186 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.186 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache device inventory", 2024-05-21T03:34:07.186 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.187 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.187 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.187 INFO:teuthology.orchestra.run.smithi027.stdout: "device_enhanced_scan": { 2024-05-21T03:34:07.187 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_enhanced_scan", 2024-05-21T03:34:07.187 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.187 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.187 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.187 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.187 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.187 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.187 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.187 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-21T03:34:07.187 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.188 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.188 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.188 INFO:teuthology.orchestra.run.smithi027.stdout: "facts_cache_timeout": { 2024-05-21T03:34:07.188 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "facts_cache_timeout", 2024-05-21T03:34:07.188 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:07.188 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.188 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.188 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-05-21T03:34:07.188 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.188 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.188 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.188 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache host facts data", 2024-05-21T03:34:07.188 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.188 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.189 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.189 INFO:teuthology.orchestra.run.smithi027.stdout: "grafana_dashboards_path": { 2024-05-21T03:34:07.189 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "grafana_dashboards_path", 2024-05-21T03:34:07.189 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.189 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.189 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.189 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-21T03:34:07.189 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.189 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.189 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.189 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-21T03:34:07.189 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.189 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.189 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.190 INFO:teuthology.orchestra.run.smithi027.stdout: "host_check_interval": { 2024-05-21T03:34:07.190 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "host_check_interval", 2024-05-21T03:34:07.190 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:07.190 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.190 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.190 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-05-21T03:34:07.190 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.190 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.190 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.190 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to perform a host check", 2024-05-21T03:34:07.190 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.190 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.190 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.190 INFO:teuthology.orchestra.run.smithi027.stdout: "hw_monitoring": { 2024-05-21T03:34:07.191 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hw_monitoring", 2024-05-21T03:34:07.191 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.191 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.191 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.191 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.191 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.191 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.191 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.191 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-21T03:34:07.191 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.191 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.191 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.191 INFO:teuthology.orchestra.run.smithi027.stdout: "inventory_list_all": { 2024-05-21T03:34:07.192 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "inventory_list_all", 2024-05-21T03:34:07.192 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.192 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.192 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.192 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.192 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.192 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.192 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.192 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-21T03:34:07.192 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.192 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.192 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.192 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.192 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.193 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.193 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.193 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.193 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.193 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.193 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.193 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.193 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.193 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.193 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.193 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.193 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.193 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.193 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.194 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.194 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.194 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.194 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.194 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.194 INFO:teuthology.orchestra.run.smithi027.stdout: "log_refresh_metadata": { 2024-05-21T03:34:07.194 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_refresh_metadata", 2024-05-21T03:34:07.194 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.194 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.194 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.194 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.194 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.194 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.194 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.194 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-21T03:34:07.195 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.195 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.195 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.195 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.195 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.195 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.195 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.195 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.195 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.195 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.195 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.196 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.196 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-21T03:34:07.196 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.196 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.196 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.196 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.196 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.196 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.196 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.196 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.196 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.196 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.196 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.196 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.196 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.197 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.197 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.197 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.197 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.197 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.197 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.197 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.197 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.197 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.197 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.197 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.197 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.197 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.197 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.197 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.198 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.198 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.198 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.198 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.198 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.198 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.198 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.198 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.198 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-21T03:34:07.198 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-21T03:34:07.198 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.198 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.198 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.199 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.199 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.199 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.199 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.199 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-21T03:34:07.199 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.199 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.199 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.199 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-21T03:34:07.199 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-21T03:34:07.199 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.199 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.199 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.199 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "*", 2024-05-21T03:34:07.199 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.200 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.200 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.200 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-21T03:34:07.200 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.200 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.200 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.200 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.200 INFO:teuthology.orchestra.run.smithi027.stdout: "max_count_per_host": { 2024-05-21T03:34:07.200 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_count_per_host", 2024-05-21T03:34:07.200 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.200 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.200 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.200 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-05-21T03:34:07.200 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.200 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.201 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.201 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of daemons per service per host", 2024-05-21T03:34:07.201 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.201 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.201 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.201 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd_draining_count": { 2024-05-21T03:34:07.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:06.195+0000 7f6d52f19640 1 -- 172.21.15.27:0/824687682 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f6d380b2960 msgr2=0x7f6d380b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:07.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:06.195+0000 7f6d52f19640 1 --2- 172.21.15.27:0/824687682 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f6d380b2960 0x7f6d380b4e20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6d34031440 tx=0x7f6d34005a20 comp rx=0 tx=0).stop 2024-05-21T03:34:07.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:06.195+0000 7f6d52f19640 1 -- 172.21.15.27:0/824687682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d4c103d30 msgr2=0x7f6d4c1992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:07.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:06.195+0000 7f6d52f19640 1 --2- 172.21.15.27:0/824687682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d4c103d30 0x7f6d4c1992b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00e9f0 tx=0x7f6d3c00eeb0 comp rx=0 tx=0).stop 2024-05-21T03:34:07.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:06.195+0000 7f6d52f19640 1 -- 172.21.15.27:0/824687682 shutdown_connections 2024-05-21T03:34:07.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:06.196+0000 7f6d52f19640 1 --2- 172.21.15.27:0/824687682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d4c10d070 0x7f6d4c19db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:07.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:06.196+0000 7f6d52f19640 1 --2- 172.21.15.27:0/824687682 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f6d380b2960 0x7f6d380b4e20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:07.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:06.196+0000 7f6d52f19640 1 --2- 172.21.15.27:0/824687682 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d4c104650 0x7f6d4c1997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:07.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:06.196+0000 7f6d52f19640 1 --2- 172.21.15.27:0/824687682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d4c103d30 0x7f6d4c1992b0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:07.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:06.196+0000 7f6d52f19640 1 -- 172.21.15.27:0/824687682 >> 172.21.15.27:0/824687682 conn(0x7f6d4c078ab0 msgr2=0x7f6d4c0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:34:07.202 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_osd_draining_count", 2024-05-21T03:34:07.202 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.203 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.203 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.203 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-05-21T03:34:07.203 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.203 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.203 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.203 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-21T03:34:07.203 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.203 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.203 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.203 INFO:teuthology.orchestra.run.smithi027.stdout: "migration_current": { 2024-05-21T03:34:07.203 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "migration_current", 2024-05-21T03:34:07.203 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.204 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.204 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.204 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.204 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.204 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.204 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.204 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "internal - do not modify", 2024-05-21T03:34:07.204 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.204 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.204 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.204 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-05-21T03:34:07.204 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-05-21T03:34:07.204 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.204 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.204 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.205 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "root", 2024-05-21T03:34:07.205 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.205 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.205 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.205 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm-package", 2024-05-21T03:34:07.205 INFO:teuthology.orchestra.run.smithi027.stdout: "root" 2024-05-21T03:34:07.205 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.205 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "mode for remote execution of cephadm", 2024-05-21T03:34:07.205 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.205 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.205 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.205 INFO:teuthology.orchestra.run.smithi027.stdout: "oob_default_addr": { 2024-05-21T03:34:07.206 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "oob_default_addr", 2024-05-21T03:34:07.206 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.206 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.206 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.206 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "169.254.1.1", 2024-05-21T03:34:07.206 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.206 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.206 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.206 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-21T03:34:07.206 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.206 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.206 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.207 INFO:teuthology.orchestra.run.smithi027.stdout: "prometheus_alerts_path": { 2024-05-21T03:34:07.207 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus_alerts_path", 2024-05-21T03:34:07.207 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.207 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.207 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.207 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-21T03:34:07.207 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.207 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.207 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.207 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-21T03:34:07.207 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.207 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.207 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.207 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_insecure": { 2024-05-21T03:34:07.208 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_insecure", 2024-05-21T03:34:07.208 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.208 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.208 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.208 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.208 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.208 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.208 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.208 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-21T03:34:07.208 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.208 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.208 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.208 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_password": { 2024-05-21T03:34:07.208 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_password", 2024-05-21T03:34:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-21T03:34:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_url": { 2024-05-21T03:34:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_url", 2024-05-21T03:34:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.209 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.210 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.210 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.210 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.210 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.210 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.210 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-21T03:34:07.210 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.210 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.210 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.210 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_username": { 2024-05-21T03:34:07.210 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_username", 2024-05-21T03:34:07.211 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.211 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.211 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.211 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.211 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.211 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.211 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.211 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-21T03:34:07.211 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.211 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.211 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.211 INFO:teuthology.orchestra.run.smithi027.stdout: "secure_monitoring_stack": { 2024-05-21T03:34:07.211 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secure_monitoring_stack", 2024-05-21T03:34:07.211 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.211 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.212 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.212 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.212 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.212 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.212 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.212 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-21T03:34:07.212 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.212 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.212 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.212 INFO:teuthology.orchestra.run.smithi027.stdout: "service_discovery_port": { 2024-05-21T03:34:07.212 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "service_discovery_port", 2024-05-21T03:34:07.212 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.212 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.212 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.213 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8765", 2024-05-21T03:34:07.213 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.213 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.213 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.213 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "cephadm service discovery port", 2024-05-21T03:34:07.213 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.213 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.213 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.213 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.213 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.213 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.213 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.213 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.213 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.213 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.214 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.214 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.214 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.214 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.214 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.214 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.214 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_config_file": { 2024-05-21T03:34:07.214 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_config_file", 2024-05-21T03:34:07.214 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.214 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.214 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.214 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.214 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.214 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.215 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.215 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-21T03:34:07.215 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.215 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.215 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.215 INFO:teuthology.orchestra.run.smithi027.stdout: "use_agent": { 2024-05-21T03:34:07.215 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_agent", 2024-05-21T03:34:07.215 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.215 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.215 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.215 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.215 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.215 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.215 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.215 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-21T03:34:07.216 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.216 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.216 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.216 INFO:teuthology.orchestra.run.smithi027.stdout: "use_repo_digest": { 2024-05-21T03:34:07.216 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_repo_digest", 2024-05-21T03:34:07.216 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.216 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.216 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.216 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.216 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.216 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.216 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.216 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-21T03:34:07.217 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.217 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.217 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.217 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_failed_host_check": { 2024-05-21T03:34:07.217 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_failed_host_check", 2024-05-21T03:34:07.217 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.217 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.217 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.217 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.217 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.217 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.217 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.217 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if the host check fails", 2024-05-21T03:34:07.218 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.218 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.218 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.218 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.218 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_daemons": { 2024-05-21T03:34:07.218 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_daemons", 2024-05-21T03:34:07.218 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.218 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.218 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.218 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.218 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.218 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.218 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.219 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-21T03:34:07.219 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.219 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.219 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.219 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_hosts": { 2024-05-21T03:34:07.219 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_hosts", 2024-05-21T03:34:07.219 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.219 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.219 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.219 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.219 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.219 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.219 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.219 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-21T03:34:07.220 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.220 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.220 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.220 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.220 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.220 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.220 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crash", 2024-05-21T03:34:07.220 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.220 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.220 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.220 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.220 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.220 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.221 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.221 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.221 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.221 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.221 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.221 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.221 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.221 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.221 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.221 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.221 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.221 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.221 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.221 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.222 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.223 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.224 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: "retain_interval": { 2024-05-21T03:34:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retain_interval", 2024-05-21T03:34:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.225 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "31536000", 2024-05-21T03:34:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-21T03:34:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.226 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_recent_interval": { 2024-05-21T03:34:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_recent_interval", 2024-05-21T03:34:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:07.227 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1209600", 2024-05-21T03:34:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-21T03:34:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dashboard", 2024-05-21T03:34:07.228 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-21T03:34:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-21T03:34:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-05-21T03:34:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.229 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_HOST": { 2024-05-21T03:34:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-21T03:34:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.230 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-21T03:34:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-21T03:34:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_ENABLED": { 2024-05-21T03:34:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_ENABLED", 2024-05-21T03:34:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.232 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-21T03:34:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-21T03:34:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.233 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "ENABLE_BROWSABLE_API": { 2024-05-21T03:34:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-21T03:34:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.234 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.235 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-21T03:34:07.235 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-21T03:34:07.235 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.235 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.235 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.235 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.235 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.235 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.235 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.235 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.235 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.235 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.235 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.236 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-21T03:34:07.236 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-21T03:34:07.236 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.236 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.236 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.236 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.236 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.236 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.236 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.236 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.236 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.236 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.236 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.237 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-21T03:34:07.237 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-21T03:34:07.237 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.237 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.237 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.237 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.237 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.237 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.237 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.237 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.237 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.237 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.237 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.237 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-21T03:34:07.238 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-21T03:34:07.238 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.238 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.238 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.238 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.238 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.238 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.238 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.238 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.238 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.238 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.238 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.239 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-21T03:34:07.239 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-21T03:34:07.239 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.239 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.239 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.239 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.239 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.239 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.239 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.239 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.239 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.239 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.239 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.239 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-21T03:34:07.240 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-21T03:34:07.240 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.240 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.240 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.240 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.240 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.240 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.240 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.240 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.240 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.240 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.240 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.240 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-21T03:34:07.241 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-21T03:34:07.241 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.241 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.241 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.241 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.241 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.241 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.241 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.241 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.241 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.241 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.241 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.241 INFO:teuthology.orchestra.run.smithi027.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-21T03:34:07.242 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-21T03:34:07.242 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.242 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.242 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.242 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.242 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.242 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.242 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.242 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.242 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.242 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.242 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.243 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_PASSWORD": { 2024-05-21T03:34:07.243 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-21T03:34:07.243 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.243 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.243 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.243 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-05-21T03:34:07.243 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.243 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.243 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.243 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.243 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.243 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.243 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.243 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-21T03:34:07.244 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-21T03:34:07.244 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.244 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.244 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.244 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.244 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.244 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.244 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.244 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.244 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.244 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.244 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.244 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_URL": { 2024-05-21T03:34:07.245 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_URL", 2024-05-21T03:34:07.245 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.245 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.245 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.245 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.245 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.245 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.245 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.245 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.245 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.245 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.245 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.245 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_USERNAME": { 2024-05-21T03:34:07.245 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-21T03:34:07.246 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.246 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.246 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.246 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-05-21T03:34:07.246 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.246 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.246 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.246 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.246 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.246 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.246 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.246 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-21T03:34:07.246 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-21T03:34:07.247 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.247 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.247 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.247 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.247 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.247 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.247 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.247 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.247 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.247 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.247 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.247 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-21T03:34:07.247 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-21T03:34:07.247 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.248 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.248 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.248 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.248 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.248 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.248 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.248 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.248 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.248 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.248 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.248 INFO:teuthology.orchestra.run.smithi027.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-21T03:34:07.248 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-21T03:34:07.248 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.249 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.249 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.249 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.249 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.249 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.249 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.249 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.249 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.249 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.249 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.249 INFO:teuthology.orchestra.run.smithi027.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-21T03:34:07.249 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-21T03:34:07.249 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.249 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-21T03:34:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-21T03:34:07.250 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "[]", 2024-05-21T03:34:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: "MULTICLUSTER_CONFIG": { 2024-05-21T03:34:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-21T03:34:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.251 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.252 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.252 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "{}", 2024-05-21T03:34:07.252 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.252 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.252 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.252 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.252 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.252 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.252 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.252 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_HOST": { 2024-05-21T03:34:07.252 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-21T03:34:07.252 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.252 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.253 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.253 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.253 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.253 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.253 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.253 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.253 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.253 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.253 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.253 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-21T03:34:07.253 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-21T03:34:07.253 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.253 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.253 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.254 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.254 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.254 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.254 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.254 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.254 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.254 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.254 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.254 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-21T03:34:07.254 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-21T03:34:07.254 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.254 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.255 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.255 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.255 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.255 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.258 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.258 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.258 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.258 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.258 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.259 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-21T03:34:07.259 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-21T03:34:07.259 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.259 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.259 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.259 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.259 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.259 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.259 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.259 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.259 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.259 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.259 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.260 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-21T03:34:07.260 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-21T03:34:07.260 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.260 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.260 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.260 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.260 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.260 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.260 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.260 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.260 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.260 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.260 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.261 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-21T03:34:07.261 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-21T03:34:07.261 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.261 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.261 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.261 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.261 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.261 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.261 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.262 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.262 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.262 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.262 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.262 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-21T03:34:07.262 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-21T03:34:07.262 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.262 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.262 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.262 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.262 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.263 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.263 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.263 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.263 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.263 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.263 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.263 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-21T03:34:07.263 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-21T03:34:07.263 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.263 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.263 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.263 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.263 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.264 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.264 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.264 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.264 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.264 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.264 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.264 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-21T03:34:07.264 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-21T03:34:07.264 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.264 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.264 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.264 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.264 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.265 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.265 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.265 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.265 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.265 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.265 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.265 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_ENABLED": { 2024-05-21T03:34:07.265 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-21T03:34:07.265 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.265 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.265 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.265 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.265 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.265 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.265 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.266 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.266 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.266 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.266 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.266 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-21T03:34:07.266 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-21T03:34:07.266 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.266 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.266 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.266 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-21T03:34:07.266 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.266 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.266 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.267 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.267 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.267 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.267 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.267 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-21T03:34:07.267 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-21T03:34:07.267 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.267 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.267 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.267 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-05-21T03:34:07.267 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.267 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.267 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.267 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:06.196+0000 7f6d52f19640 1 -- 172.21.15.27:0/824687682 shutdown_connections 2024-05-21T03:34:07.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:06.196+0000 7f6d52f19640 1 -- 172.21.15.27:0/824687682 wait complete. 2024-05-21T03:34:07.270 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.270 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.270 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.270 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-21T03:34:07.270 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-21T03:34:07.270 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.270 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.271 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.271 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8", 2024-05-21T03:34:07.271 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.271 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.271 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.271 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.271 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.271 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.271 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.271 INFO:teuthology.orchestra.run.smithi027.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-21T03:34:07.271 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-21T03:34:07.271 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.271 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.272 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.272 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "45", 2024-05-21T03:34:07.272 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.272 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.272 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.272 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.272 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.272 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.272 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.272 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ACCESS_KEY": { 2024-05-21T03:34:07.272 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-21T03:34:07.272 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.272 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.273 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.273 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.273 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.273 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.273 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.273 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.273 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.273 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.273 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.273 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-21T03:34:07.273 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-21T03:34:07.273 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.273 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.273 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.274 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-05-21T03:34:07.274 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.274 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.274 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.274 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.274 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.274 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.274 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.274 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SECRET_KEY": { 2024-05-21T03:34:07.274 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-21T03:34:07.274 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.274 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.275 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.275 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.275 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.275 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.275 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.275 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.275 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.275 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.275 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.275 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SSL_VERIFY": { 2024-05-21T03:34:07.275 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-21T03:34:07.275 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.275 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.275 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.276 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.276 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.276 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.276 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.276 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.276 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.276 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.276 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.276 INFO:teuthology.orchestra.run.smithi027.stdout: "UNSAFE_TLS_v1_2": { 2024-05-21T03:34:07.276 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-21T03:34:07.276 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.276 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.276 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.276 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.276 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.277 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.277 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.277 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.277 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.277 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.277 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.277 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-21T03:34:07.277 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-21T03:34:07.277 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.277 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.277 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.277 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.277 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.277 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.278 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.278 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.278 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.278 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.278 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.278 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-21T03:34:07.278 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-21T03:34:07.278 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.278 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.278 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.278 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-05-21T03:34:07.279 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.279 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.279 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.279 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.279 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.279 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.279 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.279 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-21T03:34:07.279 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-21T03:34:07.279 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.279 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.279 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.279 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-05-21T03:34:07.280 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.280 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.280 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.280 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.280 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.280 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.280 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.280 INFO:teuthology.orchestra.run.smithi027.stdout: "cross_origin_url": { 2024-05-21T03:34:07.280 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cross_origin_url", 2024-05-21T03:34:07.280 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.280 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.280 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.280 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.280 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.281 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.281 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.281 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.281 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.281 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.281 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.281 INFO:teuthology.orchestra.run.smithi027.stdout: "crt_file": { 2024-05-21T03:34:07.281 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crt_file", 2024-05-21T03:34:07.281 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.281 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.281 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.281 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.281 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.281 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.282 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.282 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.282 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.282 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.282 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.282 INFO:teuthology.orchestra.run.smithi027.stdout: "debug": { 2024-05-21T03:34:07.282 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "debug", 2024-05-21T03:34:07.282 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.282 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.282 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.282 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.282 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.283 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.283 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.283 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable/disable debug options", 2024-05-21T03:34:07.283 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.283 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.283 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.283 INFO:teuthology.orchestra.run.smithi027.stdout: "jwt_token_ttl": { 2024-05-21T03:34:07.283 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "jwt_token_ttl", 2024-05-21T03:34:07.283 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.283 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.283 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.283 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "28800", 2024-05-21T03:34:07.283 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.283 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.284 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.284 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.284 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.284 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.284 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.284 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-05-21T03:34:07.284 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-05-21T03:34:07.284 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.284 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.284 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.284 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.284 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.284 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.285 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.285 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.285 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.285 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.285 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.285 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.285 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.285 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.285 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.285 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.285 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.285 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.285 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.285 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.286 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.286 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.286 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.286 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.286 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.286 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.286 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.286 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.286 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.286 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.286 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.286 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.287 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.287 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.287 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.287 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.287 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.287 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.287 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.287 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.287 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.287 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.287 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.287 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.287 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.288 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.288 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.288 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.288 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.288 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.288 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.288 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.288 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.288 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.288 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.288 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.288 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.288 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.288 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.289 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.289 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.289 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.289 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.289 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.289 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.289 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.289 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.289 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.289 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.289 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.289 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.289 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.289 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.290 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.290 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.290 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.290 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.290 INFO:teuthology.orchestra.run.smithi027.stdout: "motd": { 2024-05-21T03:34:07.290 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "motd", 2024-05-21T03:34:07.290 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.290 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.290 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.290 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.290 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.290 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.290 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.291 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "The message of the day", 2024-05-21T03:34:07.291 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.291 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.291 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.291 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect_resolve_ip_addr": { 2024-05-21T03:34:07.291 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "redirect_resolve_ip_addr", 2024-05-21T03:34:07.291 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.291 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.291 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.291 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.291 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.291 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.291 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.292 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.292 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.292 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.292 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.292 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-05-21T03:34:07.292 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-05-21T03:34:07.292 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.292 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.292 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.292 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-05-21T03:34:07.292 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.292 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.292 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.292 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.293 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.293 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.293 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.293 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-05-21T03:34:07.293 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-05-21T03:34:07.293 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.293 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.293 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.293 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8080", 2024-05-21T03:34:07.293 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.293 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.293 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.293 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.293 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.294 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.294 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.294 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.294 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.294 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.294 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.294 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.294 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.294 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.294 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.294 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.294 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.295 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.295 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.295 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.295 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-05-21T03:34:07.295 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-05-21T03:34:07.295 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.295 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.295 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.295 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.295 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.295 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.295 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.295 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.295 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.296 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.296 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.296 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl_server_port": { 2024-05-21T03:34:07.296 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl_server_port", 2024-05-21T03:34:07.296 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.296 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.296 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.296 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8443", 2024-05-21T03:34:07.296 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.296 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.296 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.296 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.296 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.297 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.297 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.297 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-05-21T03:34:07.297 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-05-21T03:34:07.297 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.297 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.297 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.297 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "redirect", 2024-05-21T03:34:07.297 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.297 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.297 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.297 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.298 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect" 2024-05-21T03:34:07.298 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.298 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.298 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.298 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.298 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.298 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-05-21T03:34:07.298 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-05-21T03:34:07.298 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.298 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.298 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.298 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-05-21T03:34:07.299 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-05-21T03:34:07.299 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-05-21T03:34:07.299 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.299 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.299 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.299 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.299 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.299 INFO:teuthology.orchestra.run.smithi027.stdout: "url_prefix": { 2024-05-21T03:34:07.299 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url_prefix", 2024-05-21T03:34:07.299 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.299 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.299 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.299 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.299 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.300 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.300 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.300 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.300 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.300 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.300 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.300 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.300 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.300 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-05-21T03:34:07.300 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.300 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.300 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.301 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_monitoring": { 2024-05-21T03:34:07.301 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_monitoring", 2024-05-21T03:34:07.301 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.301 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.301 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.301 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.301 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.301 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.301 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.301 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "monitor device health metrics", 2024-05-21T03:34:07.301 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.301 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.301 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.301 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.301 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.302 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.302 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.302 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.302 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.302 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.302 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.302 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.302 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.302 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.302 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.302 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.302 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.302 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.302 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.303 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.303 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.303 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.303 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.303 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.303 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.303 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.303 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.303 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.303 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.303 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.303 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.303 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.303 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.304 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.304 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.304 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.304 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.304 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.304 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.304 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.304 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.304 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.304 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.304 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.304 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.304 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.304 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.304 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.305 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.305 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.305 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.305 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.305 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.305 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.305 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.305 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.305 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.305 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.305 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.305 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.305 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.305 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.306 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.306 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.306 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.306 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.306 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.306 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.306 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.306 INFO:teuthology.orchestra.run.smithi027.stdout: "mark_out_threshold": { 2024-05-21T03:34:07.306 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mark_out_threshold", 2024-05-21T03:34:07.306 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:07.306 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.306 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.307 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2419200", 2024-05-21T03:34:07.307 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.307 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.307 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.307 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-21T03:34:07.307 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.307 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.307 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.307 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": { 2024-05-21T03:34:07.307 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_name", 2024-05-21T03:34:07.307 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.307 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.308 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.308 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "device_health_metrics", 2024-05-21T03:34:07.308 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.308 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.308 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.308 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-21T03:34:07.308 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.308 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.308 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.308 INFO:teuthology.orchestra.run.smithi027.stdout: "retention_period": { 2024-05-21T03:34:07.308 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retention_period", 2024-05-21T03:34:07.308 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:07.308 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.308 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.308 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15552000", 2024-05-21T03:34:07.309 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.309 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.309 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.309 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain device health metrics", 2024-05-21T03:34:07.309 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.309 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.309 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.309 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_frequency": { 2024-05-21T03:34:07.309 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_frequency", 2024-05-21T03:34:07.309 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:07.309 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.309 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.309 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-05-21T03:34:07.309 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.310 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.310 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.310 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-21T03:34:07.310 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.310 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.310 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.310 INFO:teuthology.orchestra.run.smithi027.stdout: "self_heal": { 2024-05-21T03:34:07.310 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "self_heal", 2024-05-21T03:34:07.310 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.310 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.310 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.310 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.311 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.311 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.311 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.311 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-21T03:34:07.311 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.311 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.311 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.311 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-05-21T03:34:07.311 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-05-21T03:34:07.311 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:07.311 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.311 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.311 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-05-21T03:34:07.312 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.312 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.312 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.312 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and check device health", 2024-05-21T03:34:07.312 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.312 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.312 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.312 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.312 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.312 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.312 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.312 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.312 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.313 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.313 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.313 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.313 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.313 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.313 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.313 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.313 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_threshold": { 2024-05-21T03:34:07.313 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_threshold", 2024-05-21T03:34:07.313 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:07.313 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.313 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.313 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7257600", 2024-05-21T03:34:07.313 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.314 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.314 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.314 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-21T03:34:07.314 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.314 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.314 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.314 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.314 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.314 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "diskprediction_local", 2024-05-21T03:34:07.314 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.314 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.314 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.315 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.315 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.315 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.315 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.315 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.315 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.315 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.315 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.315 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.315 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.315 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.315 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.315 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.315 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.316 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.316 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.316 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.316 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.316 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.316 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.316 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.316 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.316 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.316 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.316 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.316 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.316 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.316 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.317 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.317 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.317 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.317 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.317 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.317 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.317 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.317 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.317 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.317 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.317 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.317 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.317 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.317 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.318 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.318 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.318 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.318 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.318 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.318 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.318 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.318 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.318 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.318 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.318 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.318 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.319 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.319 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.319 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.319 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.319 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.319 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.319 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.319 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.319 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.319 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.319 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.319 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.319 INFO:teuthology.orchestra.run.smithi027.stdout: "predict_interval": { 2024-05-21T03:34:07.320 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predict_interval", 2024-05-21T03:34:07.320 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.320 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.320 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.320 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-05-21T03:34:07.320 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.320 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.320 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.320 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.320 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.320 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.320 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.320 INFO:teuthology.orchestra.run.smithi027.stdout: "predictor_model": { 2024-05-21T03:34:07.320 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predictor_model", 2024-05-21T03:34:07.321 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.321 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.321 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.321 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "prophetstor", 2024-05-21T03:34:07.321 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.321 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.321 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.321 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.321 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.321 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.321 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.321 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-05-21T03:34:07.321 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-05-21T03:34:07.321 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.322 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.322 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.322 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-05-21T03:34:07.322 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.322 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.322 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.322 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.322 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.322 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.322 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.322 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.322 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.322 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.322 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.323 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.323 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.323 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.323 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.323 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.323 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.323 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.323 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.323 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.323 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.323 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.324 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "influx", 2024-05-21T03:34:07.324 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": false, 2024-05-21T03:34:07.324 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "influxdb python module not found", 2024-05-21T03:34:07.324 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.324 INFO:teuthology.orchestra.run.smithi027.stdout: "batch_size": { 2024-05-21T03:34:07.324 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "batch_size", 2024-05-21T03:34:07.324 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.324 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.324 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.324 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5000", 2024-05-21T03:34:07.324 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.324 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.324 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.324 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-21T03:34:07.325 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.325 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.325 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.325 INFO:teuthology.orchestra.run.smithi027.stdout: "database": { 2024-05-21T03:34:07.325 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "database", 2024-05-21T03:34:07.325 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.325 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.325 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.325 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "ceph", 2024-05-21T03:34:07.325 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.325 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.325 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.325 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-21T03:34:07.325 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.326 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.326 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.326 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": { 2024-05-21T03:34:07.326 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hostname", 2024-05-21T03:34:07.326 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.326 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.326 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.326 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.326 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.326 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.326 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.326 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server hostname", 2024-05-21T03:34:07.327 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.327 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.327 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.327 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-05-21T03:34:07.327 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-05-21T03:34:07.327 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:07.327 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.327 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.327 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "30", 2024-05-21T03:34:07.327 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "5", 2024-05-21T03:34:07.327 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.327 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.327 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-21T03:34:07.327 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.328 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.328 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.328 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.328 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.328 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.328 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.328 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.328 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.328 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.328 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.328 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.328 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.328 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.329 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.329 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.329 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.329 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.329 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.329 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.329 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.329 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.329 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.329 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.329 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.329 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.329 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.329 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.330 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.330 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.330 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.330 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.330 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.330 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.330 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.330 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.330 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.330 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.330 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.330 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.330 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.330 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.331 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.331 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.331 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.331 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.331 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.331 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.331 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.331 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.331 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.331 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.331 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.331 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.331 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.331 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.332 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.332 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.332 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.332 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.332 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.332 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.332 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.332 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.332 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.332 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.332 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.332 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.332 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.332 INFO:teuthology.orchestra.run.smithi027.stdout: "password": { 2024-05-21T03:34:07.333 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "password", 2024-05-21T03:34:07.333 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.333 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.333 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.333 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.333 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.333 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.333 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.333 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "password of InfluxDB server user", 2024-05-21T03:34:07.333 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.333 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.333 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.333 INFO:teuthology.orchestra.run.smithi027.stdout: "port": { 2024-05-21T03:34:07.333 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "port", 2024-05-21T03:34:07.334 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.334 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.334 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.334 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8086", 2024-05-21T03:34:07.334 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.334 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.334 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.334 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server port", 2024-05-21T03:34:07.334 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.334 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.334 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.334 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.334 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.335 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.335 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.335 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.335 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.335 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.335 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.335 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.335 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.335 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.335 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.335 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.335 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-05-21T03:34:07.335 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-05-21T03:34:07.336 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.336 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.336 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.336 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "false", 2024-05-21T03:34:07.336 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.336 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.336 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.336 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-21T03:34:07.336 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.336 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.336 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.336 INFO:teuthology.orchestra.run.smithi027.stdout: "threads": { 2024-05-21T03:34:07.336 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threads", 2024-05-21T03:34:07.336 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.337 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.337 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.337 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-05-21T03:34:07.337 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-05-21T03:34:07.337 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "32", 2024-05-21T03:34:07.337 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.337 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-21T03:34:07.337 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.337 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.337 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.337 INFO:teuthology.orchestra.run.smithi027.stdout: "username": { 2024-05-21T03:34:07.337 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "username", 2024-05-21T03:34:07.337 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.337 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.338 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.338 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.338 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.338 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.338 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.338 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "username of InfluxDB server user", 2024-05-21T03:34:07.338 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.338 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.338 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.338 INFO:teuthology.orchestra.run.smithi027.stdout: "verify_ssl": { 2024-05-21T03:34:07.338 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "verify_ssl", 2024-05-21T03:34:07.338 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.339 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.339 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.339 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "true", 2024-05-21T03:34:07.339 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.339 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.339 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.339 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-21T03:34:07.339 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.339 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.339 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.339 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.339 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.339 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "insights", 2024-05-21T03:34:07.340 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.340 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.340 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.340 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.340 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.340 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.340 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.340 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.340 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.340 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.340 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.340 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.340 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.340 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.341 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.341 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.341 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.341 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.341 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.341 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.341 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.341 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.341 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.341 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.341 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.341 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.341 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.342 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.342 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.342 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.342 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.342 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.342 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.342 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.342 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.342 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.342 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.342 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.342 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.343 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.343 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.343 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.343 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.343 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.343 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.343 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.343 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.343 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.343 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.343 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.343 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.343 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.343 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.343 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.344 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.344 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.344 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.344 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.344 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.344 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.344 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.344 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.344 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.344 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.344 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.345 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.345 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.345 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.345 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.345 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.345 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.345 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.345 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.345 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.345 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.345 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.345 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.345 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.345 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.346 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.346 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.346 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.346 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.346 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.346 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.346 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "iostat", 2024-05-21T03:34:07.346 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.346 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.346 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.346 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.346 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.346 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.347 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.347 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.347 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.347 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.347 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.347 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.347 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.347 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.347 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.347 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.347 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.347 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.347 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.347 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.347 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.348 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.348 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.348 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.348 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.348 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.348 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.348 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.348 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.348 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.348 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.348 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.348 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.348 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.348 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.349 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.349 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.349 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.349 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.349 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.349 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.349 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.349 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.349 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.349 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.349 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.349 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.349 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.349 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.350 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.350 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.350 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.350 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.350 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.350 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.350 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.350 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.350 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.350 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.350 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.350 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.351 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.351 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.351 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.351 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.351 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.351 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.351 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.351 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.351 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.351 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.351 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.351 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.351 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.352 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.352 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.352 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.352 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.352 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.352 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.352 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.352 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.352 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.352 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.352 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.352 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.352 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "k8sevents", 2024-05-21T03:34:07.352 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.353 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.353 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.353 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_event_retention_days": { 2024-05-21T03:34:07.353 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ceph_event_retention_days", 2024-05-21T03:34:07.353 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.353 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.353 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.353 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7", 2024-05-21T03:34:07.353 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.353 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.353 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.353 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-21T03:34:07.353 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.353 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.354 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.354 INFO:teuthology.orchestra.run.smithi027.stdout: "config_check_secs": { 2024-05-21T03:34:07.354 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_check_secs", 2024-05-21T03:34:07.354 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.354 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.354 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.354 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-05-21T03:34:07.354 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "10", 2024-05-21T03:34:07.354 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.354 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.354 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-21T03:34:07.354 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.355 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.355 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.355 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.355 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.355 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.355 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.355 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.355 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.355 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.355 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.355 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.355 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.355 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.356 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.356 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.356 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.356 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.356 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.356 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.356 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.356 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.356 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.356 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.356 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.356 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.357 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.357 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.357 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.357 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.357 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.357 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.357 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.357 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.357 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.357 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.357 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.357 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.357 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.357 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.358 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.358 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.358 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.358 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.358 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.358 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.358 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.358 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.358 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.358 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.358 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.358 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.358 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.359 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.359 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.359 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.359 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.359 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.359 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.359 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.359 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.359 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.359 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.359 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.359 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.359 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.360 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.360 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.360 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.360 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.360 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.360 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.360 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.360 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.360 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.360 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.360 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.360 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.360 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.361 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.361 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.361 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.361 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.361 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.361 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "localpool", 2024-05-21T03:34:07.361 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.361 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.361 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.361 INFO:teuthology.orchestra.run.smithi027.stdout: "failure_domain": { 2024-05-21T03:34:07.361 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "failure_domain", 2024-05-21T03:34:07.361 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.361 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.362 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.362 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "host", 2024-05-21T03:34:07.362 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.362 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.362 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.362 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "failure domain for any created local pool", 2024-05-21T03:34:07.362 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-21T03:34:07.362 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.362 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.362 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.362 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.362 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.362 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.363 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.363 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.363 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.363 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.363 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.363 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.363 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.363 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.363 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.363 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.363 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.363 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.363 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.363 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.364 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.364 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.364 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.364 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.364 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.364 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.364 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.364 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.364 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.364 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.364 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.364 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.364 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.365 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.365 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.365 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.365 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.365 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.365 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.365 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.365 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.365 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.365 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.365 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.365 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.365 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.365 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.366 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.366 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.366 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.366 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.366 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.366 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.366 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.366 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.366 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.366 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.366 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.366 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.367 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.367 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.367 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.367 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.367 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.367 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.367 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.367 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.367 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.367 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": { 2024-05-21T03:34:07.367 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_size", 2024-05-21T03:34:07.367 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.367 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.367 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.368 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.368 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.368 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.368 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.368 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default min_size for any created local pool", 2024-05-21T03:34:07.368 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-21T03:34:07.368 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.368 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.368 INFO:teuthology.orchestra.run.smithi027.stdout: "num_rep": { 2024-05-21T03:34:07.368 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "num_rep", 2024-05-21T03:34:07.368 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.368 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.369 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.369 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-05-21T03:34:07.369 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.369 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.369 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.369 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default replica count for any created local pool", 2024-05-21T03:34:07.369 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.369 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.369 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.369 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": { 2024-05-21T03:34:07.369 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_num", 2024-05-21T03:34:07.369 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.369 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.369 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.370 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "128", 2024-05-21T03:34:07.370 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.370 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.370 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.370 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default pg_num for any created local pool", 2024-05-21T03:34:07.370 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.370 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.370 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.370 INFO:teuthology.orchestra.run.smithi027.stdout: "prefix": { 2024-05-21T03:34:07.370 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prefix", 2024-05-21T03:34:07.370 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.370 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.370 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.371 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.371 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.371 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.371 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.371 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name prefix for any created local pool", 2024-05-21T03:34:07.371 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.371 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.371 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.371 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.371 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.371 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.371 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.371 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.372 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.372 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.372 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.372 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.372 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.372 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.372 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.372 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.372 INFO:teuthology.orchestra.run.smithi027.stdout: "subtree": { 2024-05-21T03:34:07.372 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "subtree", 2024-05-21T03:34:07.372 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.372 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.372 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.372 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "rack", 2024-05-21T03:34:07.373 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.373 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.373 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.373 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-21T03:34:07.373 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-21T03:34:07.373 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.373 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.373 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.373 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.373 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.373 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mds_autoscaler", 2024-05-21T03:34:07.373 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.373 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.374 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.374 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.374 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.374 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.374 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.374 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.374 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.374 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.374 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.374 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.374 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.374 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.374 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.375 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.375 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.375 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.375 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.375 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.375 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.375 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.375 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.375 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.375 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.375 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.375 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.375 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.375 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.375 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.376 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.376 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.376 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.376 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.376 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.376 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.376 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.376 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.376 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.376 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.376 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.376 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.376 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.376 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.377 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.377 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.377 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.377 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.377 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.377 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.377 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.377 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.377 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.377 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.377 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.377 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.377 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.377 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.378 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.378 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.378 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.378 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.378 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.378 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.378 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.378 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.378 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.378 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.378 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.378 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.378 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.379 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.379 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.379 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.379 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.379 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.379 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.379 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.379 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.379 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.379 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.379 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.379 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.379 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.380 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.380 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirroring", 2024-05-21T03:34:07.380 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.380 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.380 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.380 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.380 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.380 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.380 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.380 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.380 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.380 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.380 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.380 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.380 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.381 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.381 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.381 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.381 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.381 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.381 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.381 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.381 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.381 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.381 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.381 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.381 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.381 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.381 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.382 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.382 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.382 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.382 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.382 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.382 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.382 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.382 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.382 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.382 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.382 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.382 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.382 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.382 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.383 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.383 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.383 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.383 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.383 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.383 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.383 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.383 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.383 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.383 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.383 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.383 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.383 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.383 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.383 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.384 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.384 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.384 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.384 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.384 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.384 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.384 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.384 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.384 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.384 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.384 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.384 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.384 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.385 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "nfs", 2024-05-21T03:34:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.386 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.387 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.388 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.389 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-05-21T03:34:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "fail_fs": { 2024-05-21T03:34:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "fail_fs", 2024-05-21T03:34:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.392 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-21T03:34:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.393 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.394 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.395 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.397 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator": { 2024-05-21T03:34:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-05-21T03:34:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm", 2024-05-21T03:34:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: "rook", 2024-05-21T03:34:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: "test_orchestrator" 2024-05-21T03:34:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.398 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Orchestrator backend", 2024-05-21T03:34:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.399 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_perf_query", 2024-05-21T03:34:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.400 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.401 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.402 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.403 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.404 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.405 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_support", 2024-05-21T03:34:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.406 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.407 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.408 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.409 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.410 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.411 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_autoscaler", 2024-05-21T03:34:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.413 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.414 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.416 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.417 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-05-21T03:34:07.418 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-05-21T03:34:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-05-21T03:34:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.419 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "threshold": { 2024-05-21T03:34:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threshold", 2024-05-21T03:34:07.420 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-05-21T03:34:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-05-21T03:34:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1.0", 2024-05-21T03:34:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "scaling threshold", 2024-05-21T03:34:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-21T03:34:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.421 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "progress", 2024-05-21T03:34:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_pg_recovery_event": { 2024-05-21T03:34:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_pg_recovery_event", 2024-05-21T03:34:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-21T03:34:07.422 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-05-21T03:34:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-05-21T03:34:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.423 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.424 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.425 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.426 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.427 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: "max_completed_events": { 2024-05-21T03:34:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_completed_events", 2024-05-21T03:34:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.428 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "50", 2024-05-21T03:34:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "number of past completed events to remember", 2024-05-21T03:34:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-05-21T03:34:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-05-21T03:34:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-05-21T03:34:07.429 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long the module is going to sleep", 2024-05-21T03:34:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.430 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus", 2024-05-21T03:34:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "cache": { 2024-05-21T03:34:07.431 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cache", 2024-05-21T03:34:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "exclude_perf_counters": { 2024-05-21T03:34:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "exclude_perf_counters", 2024-05-21T03:34:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-21T03:34:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-21T03:34:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.433 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.434 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.436 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.437 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools": { 2024-05-21T03:34:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools", 2024-05-21T03:34:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.438 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-21T03:34:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-21T03:34:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "300", 2024-05-21T03:34:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.439 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_interval": { 2024-05-21T03:34:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_interval", 2024-05-21T03:34:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-05-21T03:34:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15.0", 2024-05-21T03:34:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.440 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-05-21T03:34:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-05-21T03:34:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-05-21T03:34:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-21T03:34:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-05-21T03:34:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-05-21T03:34:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "9283", 2024-05-21T03:34:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-21T03:34:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.442 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "stale_cache_strategy": { 2024-05-21T03:34:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stale_cache_strategy", 2024-05-21T03:34:07.443 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "log", 2024-05-21T03:34:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-05-21T03:34:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-05-21T03:34:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.444 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "default", 2024-05-21T03:34:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "default", 2024-05-21T03:34:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "error" 2024-05-21T03:34:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-05-21T03:34:07.445 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-05-21T03:34:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-05-21T03:34:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-05-21T03:34:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-05-21T03:34:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-05-21T03:34:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.447 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.448 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.449 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.450 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_snap_create": { 2024-05-21T03:34:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_snap_create", 2024-05-21T03:34:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-05-21T03:34:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "mirror_snapshot_schedule": { 2024-05-21T03:34:07.452 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirror_snapshot_schedule", 2024-05-21T03:34:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.453 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "trash_purge_schedule": { 2024-05-21T03:34:07.454 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "trash_purge_schedule", 2024-05-21T03:34:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "restful", 2024-05-21T03:34:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_auth": { 2024-05-21T03:34:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_auth", 2024-05-21T03:34:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.456 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-05-21T03:34:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-05-21T03:34:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.457 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.460 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.461 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.462 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-05-21T03:34:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-05-21T03:34:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.463 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-05-21T03:34:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-05-21T03:34:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.464 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.465 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rgw", 2024-05-21T03:34:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.466 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.467 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.469 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.470 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.471 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rook", 2024-05-21T03:34:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.472 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.473 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.474 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.475 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.476 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.477 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "storage_class": { 2024-05-21T03:34:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "storage_class", 2024-05-21T03:34:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "local", 2024-05-21T03:34:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.478 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-21T03:34:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "selftest", 2024-05-21T03:34:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.479 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.480 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.481 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.482 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.483 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "roption1": { 2024-05-21T03:34:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption1", 2024-05-21T03:34:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.484 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "roption2": { 2024-05-21T03:34:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption2", 2024-05-21T03:34:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "xyz", 2024-05-21T03:34:07.485 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption1": { 2024-05-21T03:34:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption1", 2024-05-21T03:34:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.486 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption2": { 2024-05-21T03:34:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption2", 2024-05-21T03:34:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.487 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption3": { 2024-05-21T03:34:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption3", 2024-05-21T03:34:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-05-21T03:34:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.488 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption4": { 2024-05-21T03:34:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption4", 2024-05-21T03:34:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.489 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption5": { 2024-05-21T03:34:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption5", 2024-05-21T03:34:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.490 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption6": { 2024-05-21T03:34:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption6", 2024-05-21T03:34:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.491 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption7": { 2024-05-21T03:34:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption7", 2024-05-21T03:34:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-05-21T03:34:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "42", 2024-05-21T03:34:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.492 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.493 INFO:teuthology.orchestra.run.smithi027.stdout: "testkey": { 2024-05-21T03:34:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testkey", 2024-05-21T03:34:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "testlkey": { 2024-05-21T03:34:07.494 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testlkey", 2024-05-21T03:34:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "testnewline": { 2024-05-21T03:34:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testnewline", 2024-05-21T03:34:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.495 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smb", 2024-05-21T03:34:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.497 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.498 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.499 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.500 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.501 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: "update_orchestration": { 2024-05-21T03:34:07.502 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "update_orchestration", 2024-05-21T03:34:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-21T03:34:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-05-21T03:34:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_m_granularity": { 2024-05-21T03:34:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_m_granularity", 2024-05-21T03:34:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow minute scheduled snapshots", 2024-05-21T03:34:07.504 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "dump_on_update": { 2024-05-21T03:34:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dump_on_update", 2024-05-21T03:34:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "dump database to debug log on update", 2024-05-21T03:34:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.505 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.506 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.507 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.508 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.509 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.510 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.510 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.510 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.510 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.510 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.510 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.510 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.510 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.510 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.510 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.510 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.510 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.510 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.511 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.511 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.511 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.511 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.511 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.511 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.511 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.511 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.511 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.511 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.511 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.511 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.511 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.511 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.512 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.512 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.512 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.512 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.512 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.512 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stats", 2024-05-21T03:34:07.512 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.512 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.512 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.512 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.512 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.512 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.512 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.513 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.513 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.513 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.513 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.513 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.513 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.513 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.513 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.513 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.513 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.513 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.513 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.513 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.513 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.514 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.514 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.514 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.514 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.514 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.514 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.514 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.514 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.514 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.514 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.514 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.514 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.515 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.515 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.515 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.515 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.515 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.515 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.515 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.515 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.515 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.515 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.515 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.515 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.515 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.516 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.516 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.516 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.516 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.516 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.516 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.516 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.516 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.516 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.516 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.516 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.516 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.516 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.516 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.517 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.517 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.517 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.517 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.517 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.517 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.517 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.517 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.517 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.517 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.517 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.517 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.518 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.518 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.518 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.518 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.518 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.518 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.518 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.518 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.518 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.518 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.518 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.518 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.519 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.519 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.519 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.519 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "status", 2024-05-21T03:34:07.519 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.519 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.519 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.519 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.519 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.519 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.519 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.519 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.519 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.519 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.520 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.520 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.520 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.520 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.520 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.520 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.520 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.520 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.520 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.520 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.520 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.520 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.520 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.520 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.521 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.521 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.521 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.521 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.521 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.521 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.521 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.521 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.521 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.521 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.521 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.521 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.521 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.521 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.522 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.522 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.522 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.522 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.522 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.522 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.522 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.522 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.522 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.522 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.522 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.522 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.522 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.522 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.523 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.523 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.523 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.523 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.523 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.523 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.523 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.523 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.523 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.523 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.523 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.523 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.523 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.523 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.523 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.523 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.524 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.524 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.524 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.524 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.524 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.524 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.524 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.524 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.524 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.524 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.524 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.524 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.524 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.524 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.524 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.525 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.525 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.525 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.525 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.525 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.525 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telegraf", 2024-05-21T03:34:07.525 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.525 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.525 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.525 INFO:teuthology.orchestra.run.smithi027.stdout: "address": { 2024-05-21T03:34:07.525 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "address", 2024-05-21T03:34:07.525 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.525 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.525 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.525 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-21T03:34:07.525 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.526 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.526 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.526 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.526 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.526 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.526 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.526 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-05-21T03:34:07.526 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-05-21T03:34:07.526 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:07.526 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.526 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.526 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15", 2024-05-21T03:34:07.526 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.527 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.527 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.527 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.527 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.527 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.527 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.527 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.527 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.527 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.527 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.527 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.527 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.527 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.527 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.528 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.528 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.528 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.528 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.528 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.528 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.528 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.528 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.528 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.528 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.528 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.528 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.528 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.528 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.529 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.529 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.529 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.529 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.529 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.529 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.529 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.529 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.529 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.529 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.529 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.529 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.529 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.529 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.529 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.529 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.530 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.530 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.530 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.530 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.530 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.530 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.530 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.530 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.530 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.530 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.530 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.530 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.530 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.530 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.531 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.531 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.531 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.531 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.531 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.531 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.531 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.531 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.531 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.531 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.531 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.531 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.531 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.531 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.532 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.532 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.532 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.532 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.532 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.532 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.532 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.532 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.532 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.532 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.532 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.532 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.532 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.532 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.533 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.533 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.533 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.533 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telemetry", 2024-05-21T03:34:07.533 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.533 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.533 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.533 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_basic": { 2024-05-21T03:34:07.533 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_basic", 2024-05-21T03:34:07.533 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.533 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.533 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.533 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.533 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.533 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.534 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.534 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-21T03:34:07.534 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.534 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.534 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.534 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.534 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_crash": { 2024-05-21T03:34:07.534 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_crash", 2024-05-21T03:34:07.534 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.534 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.534 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.534 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.534 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.535 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.535 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.535 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-21T03:34:07.535 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.535 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.535 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.535 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_device": { 2024-05-21T03:34:07.535 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_device", 2024-05-21T03:34:07.535 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.535 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.535 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.535 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.535 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.535 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.536 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.536 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-21T03:34:07.536 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.536 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.536 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.536 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_ident": { 2024-05-21T03:34:07.536 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_ident", 2024-05-21T03:34:07.536 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.536 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.536 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.536 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.536 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.536 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.536 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.537 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-21T03:34:07.537 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.537 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.537 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.537 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_perf": { 2024-05-21T03:34:07.537 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_perf", 2024-05-21T03:34:07.537 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.537 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.537 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.537 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.537 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.537 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.537 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.537 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-21T03:34:07.537 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.538 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.538 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.538 INFO:teuthology.orchestra.run.smithi027.stdout: "contact": { 2024-05-21T03:34:07.538 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "contact", 2024-05-21T03:34:07.538 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.538 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.538 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.538 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.538 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.538 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.538 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.538 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.538 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.539 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.539 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.539 INFO:teuthology.orchestra.run.smithi027.stdout: "description": { 2024-05-21T03:34:07.539 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "description", 2024-05-21T03:34:07.539 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.539 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.539 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.539 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.539 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.539 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.539 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.539 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.539 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.539 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.539 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.540 INFO:teuthology.orchestra.run.smithi027.stdout: "device_url": { 2024-05-21T03:34:07.540 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_url", 2024-05-21T03:34:07.540 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.540 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.540 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.540 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-21T03:34:07.540 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.540 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.540 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.540 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.540 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.540 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.540 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.541 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-05-21T03:34:07.541 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-05-21T03:34:07.541 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.541 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.541 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.541 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.541 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.541 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.541 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.541 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.541 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.541 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.541 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.541 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-05-21T03:34:07.542 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-05-21T03:34:07.542 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.542 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.542 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.542 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "24", 2024-05-21T03:34:07.542 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "8", 2024-05-21T03:34:07.542 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.542 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.542 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.542 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.542 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.542 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.543 INFO:teuthology.orchestra.run.smithi027.stdout: "last_opt_revision": { 2024-05-21T03:34:07.543 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "last_opt_revision", 2024-05-21T03:34:07.543 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.543 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.543 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.543 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1", 2024-05-21T03:34:07.543 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.543 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.543 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.543 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.543 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.543 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.543 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.543 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.543 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard": { 2024-05-21T03:34:07.543 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard", 2024-05-21T03:34:07.544 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.544 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.544 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.544 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.544 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.544 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.544 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.544 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.544 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.544 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.544 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.544 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.544 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard_description": { 2024-05-21T03:34:07.544 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard_description", 2024-05-21T03:34:07.544 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.545 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.545 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.545 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.545 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.545 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.545 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.545 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.545 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.545 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.545 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.545 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.545 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.545 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.545 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.545 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.546 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.546 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.546 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.546 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.546 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.546 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.546 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.546 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.546 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.546 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.546 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.546 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.546 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.547 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.547 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.547 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.547 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.547 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.547 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.547 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.547 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.547 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.547 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.547 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.547 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.547 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.547 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.547 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.548 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.548 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.548 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.548 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.548 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.548 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.548 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.548 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.548 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.548 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.548 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.548 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.548 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.548 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.549 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.549 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.549 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.549 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.549 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.549 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.549 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.549 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.549 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.549 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.549 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.549 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.549 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.549 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.550 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.550 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.550 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.550 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.550 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.550 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.550 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.550 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.550 INFO:teuthology.orchestra.run.smithi027.stdout: "organization": { 2024-05-21T03:34:07.550 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "organization", 2024-05-21T03:34:07.550 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.550 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.550 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.551 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.551 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.551 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.551 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.551 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.551 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.551 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.551 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.551 INFO:teuthology.orchestra.run.smithi027.stdout: "proxy": { 2024-05-21T03:34:07.551 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "proxy", 2024-05-21T03:34:07.551 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.551 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.551 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.551 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.552 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.552 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.552 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.552 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.552 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.552 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.552 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.552 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.552 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.552 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.552 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.552 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.552 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.552 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.553 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.553 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.553 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.553 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.553 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.553 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.553 INFO:teuthology.orchestra.run.smithi027.stdout: "url": { 2024-05-21T03:34:07.553 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url", 2024-05-21T03:34:07.553 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.553 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.553 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.553 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-21T03:34:07.553 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.553 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.554 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.554 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.554 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.554 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.554 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.554 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.554 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.554 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.554 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "test_orchestrator", 2024-05-21T03:34:07.554 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.554 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.554 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.554 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.555 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.555 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.555 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.555 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.555 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.555 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.555 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.555 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.555 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.555 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.555 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.555 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.555 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.555 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.555 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.556 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.556 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.556 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.556 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.556 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.556 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.556 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.556 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.556 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.556 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.556 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.556 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.556 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.556 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.557 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.557 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.557 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.557 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.557 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.557 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.557 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.557 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.557 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.557 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.557 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.557 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.557 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.557 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.558 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.558 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.558 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.558 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.558 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.558 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.558 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.558 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.558 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.558 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.558 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.558 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.558 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.558 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.559 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.559 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.559 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.559 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.559 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.559 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.559 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.559 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.559 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.559 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.559 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.559 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.559 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.559 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.560 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.560 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.560 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.560 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.560 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.560 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.560 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.560 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.560 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.560 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.560 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-05-21T03:34:07.560 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.560 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.560 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.560 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.561 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.561 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.561 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.561 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.561 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.561 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.561 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.561 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.561 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.561 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.561 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.561 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.561 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.561 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.561 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.562 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.562 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.562 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.562 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.562 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.562 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.562 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.562 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.562 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.562 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.562 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.562 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.562 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.563 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.563 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.563 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.563 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.563 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.563 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.563 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.563 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.563 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.563 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.563 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.563 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.564 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.564 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.564 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.564 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.564 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.564 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.564 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.564 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.564 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.564 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.564 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.564 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.564 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.565 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.565 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.565 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.565 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.565 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.565 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.565 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.565 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.565 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.565 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.565 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.565 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.565 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_clones": { 2024-05-21T03:34:07.565 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_clones", 2024-05-21T03:34:07.566 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.566 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.566 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.566 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4", 2024-05-21T03:34:07.566 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.566 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.566 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.566 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-21T03:34:07.566 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.566 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.566 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.566 INFO:teuthology.orchestra.run.smithi027.stdout: "periodic_async_work": { 2024-05-21T03:34:07.567 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "periodic_async_work", 2024-05-21T03:34:07.567 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.567 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.567 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.567 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.567 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.567 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.567 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.567 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Periodically check for async work", 2024-05-21T03:34:07.567 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.567 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.567 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.567 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_delay": { 2024-05-21T03:34:07.567 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_delay", 2024-05-21T03:34:07.567 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.568 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.568 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.568 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.568 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.568 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.568 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.568 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-21T03:34:07.568 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.568 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.568 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.568 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_no_wait": { 2024-05-21T03:34:07.568 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_no_wait", 2024-05-21T03:34:07.568 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.568 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.569 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.569 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-05-21T03:34:07.569 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.569 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.569 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.569 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-21T03:34:07.569 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.569 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.569 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.569 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.569 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.569 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.569 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.569 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.569 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.569 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.570 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.570 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.570 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.570 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.570 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.570 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.570 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.570 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.570 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.570 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.570 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix", 2024-05-21T03:34:07.570 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-05-21T03:34:07.570 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-05-21T03:34:07.571 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-05-21T03:34:07.571 INFO:teuthology.orchestra.run.smithi027.stdout: "discovery_interval": { 2024-05-21T03:34:07.571 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "discovery_interval", 2024-05-21T03:34:07.571 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-05-21T03:34:07.571 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.571 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.571 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "100", 2024-05-21T03:34:07.571 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.571 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.571 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.571 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.571 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.571 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.571 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.571 INFO:teuthology.orchestra.run.smithi027.stdout: "identifier": { 2024-05-21T03:34:07.572 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "identifier", 2024-05-21T03:34:07.572 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.572 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.572 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.572 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.572 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.572 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.572 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.572 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.572 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.572 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.572 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.572 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.572 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-05-21T03:34:07.572 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-05-21T03:34:07.573 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-05-21T03:34:07.573 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.573 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.573 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-05-21T03:34:07.573 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.573 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.573 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.573 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.573 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.573 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.573 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.573 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-05-21T03:34:07.573 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-05-21T03:34:07.573 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.573 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.574 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.574 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.574 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.574 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.574 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.574 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.574 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.574 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.574 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.574 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.574 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.574 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.574 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.574 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.574 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.575 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.575 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.575 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-05-21T03:34:07.575 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-05-21T03:34:07.575 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.575 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.575 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.575 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.575 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.575 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.575 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.575 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.575 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.575 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.575 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.576 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.576 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-05-21T03:34:07.576 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-05-21T03:34:07.576 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.576 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.576 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.576 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-05-21T03:34:07.576 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.576 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.576 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-05-21T03:34:07.576 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-05-21T03:34:07.576 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-05-21T03:34:07.576 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-05-21T03:34:07.576 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-05-21T03:34:07.576 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-05-21T03:34:07.577 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-05-21T03:34:07.577 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.577 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.577 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.577 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.577 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.577 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.577 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-05-21T03:34:07.577 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-05-21T03:34:07.577 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-05-21T03:34:07.577 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.577 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.577 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-05-21T03:34:07.577 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.577 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.577 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.578 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.578 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.578 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.578 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.578 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-05-21T03:34:07.578 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-05-21T03:34:07.578 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.578 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-05-21T03:34:07.578 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-05-21T03:34:07.578 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-05-21T03:34:07.578 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.578 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.578 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.579 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.579 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.579 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.579 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.579 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_host": { 2024-05-21T03:34:07.579 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_host", 2024-05-21T03:34:07.579 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.579 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.579 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.579 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-05-21T03:34:07.579 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.579 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.579 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.580 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.580 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.580 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.580 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.580 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.580 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_port": { 2024-05-21T03:34:07.580 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_port", 2024-05-21T03:34:07.580 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-05-21T03:34:07.580 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.580 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.580 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10051", 2024-05-21T03:34:07.580 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.580 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.581 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.581 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.581 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.581 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.581 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.581 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.581 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_sender": { 2024-05-21T03:34:07.581 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_sender", 2024-05-21T03:34:07.581 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-05-21T03:34:07.581 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-05-21T03:34:07.581 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-05-21T03:34:07.581 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-21T03:34:07.581 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-05-21T03:34:07.581 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-05-21T03:34:07.581 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-05-21T03:34:07.582 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-05-21T03:34:07.582 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-05-21T03:34:07.582 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-05-21T03:34:07.582 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-05-21T03:34:07.582 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.582 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.582 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.582 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.582 INFO:teuthology.orchestra.run.smithi027.stdout: "services": { 2024-05-21T03:34:07.582 INFO:teuthology.orchestra.run.smithi027.stdout: "dashboard": "https://172.21.15.27:8443/" 2024-05-21T03:34:07.582 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.582 INFO:teuthology.orchestra.run.smithi027.stdout: "always_on_modules": { 2024-05-21T03:34:07.582 INFO:teuthology.orchestra.run.smithi027.stdout: "octopus": [ 2024-05-21T03:34:07.583 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-05-21T03:34:07.583 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-05-21T03:34:07.583 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-05-21T03:34:07.583 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-05-21T03:34:07.583 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-05-21T03:34:07.583 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-05-21T03:34:07.583 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-05-21T03:34:07.583 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-05-21T03:34:07.583 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-05-21T03:34:07.583 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-05-21T03:34:07.583 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.583 INFO:teuthology.orchestra.run.smithi027.stdout: "pacific": [ 2024-05-21T03:34:07.583 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-05-21T03:34:07.583 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-05-21T03:34:07.584 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-05-21T03:34:07.584 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-05-21T03:34:07.584 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-05-21T03:34:07.584 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-05-21T03:34:07.584 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-05-21T03:34:07.584 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-05-21T03:34:07.584 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-05-21T03:34:07.584 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-05-21T03:34:07.584 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.584 INFO:teuthology.orchestra.run.smithi027.stdout: "quincy": [ 2024-05-21T03:34:07.584 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-05-21T03:34:07.584 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-05-21T03:34:07.584 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-05-21T03:34:07.584 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-05-21T03:34:07.584 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-05-21T03:34:07.585 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-05-21T03:34:07.585 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-05-21T03:34:07.585 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-05-21T03:34:07.585 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-05-21T03:34:07.585 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-05-21T03:34:07.585 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.585 INFO:teuthology.orchestra.run.smithi027.stdout: "reef": [ 2024-05-21T03:34:07.585 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-05-21T03:34:07.585 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-05-21T03:34:07.585 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-05-21T03:34:07.585 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-05-21T03:34:07.585 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-05-21T03:34:07.585 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-05-21T03:34:07.585 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-05-21T03:34:07.586 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-05-21T03:34:07.586 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-05-21T03:34:07.586 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-05-21T03:34:07.586 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-05-21T03:34:07.586 INFO:teuthology.orchestra.run.smithi027.stdout: "squid": [ 2024-05-21T03:34:07.586 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-05-21T03:34:07.586 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-05-21T03:34:07.586 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-05-21T03:34:07.586 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-05-21T03:34:07.586 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-05-21T03:34:07.586 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-05-21T03:34:07.586 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-05-21T03:34:07.586 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-05-21T03:34:07.586 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-05-21T03:34:07.587 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-05-21T03:34:07.587 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-05-21T03:34:07.587 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.587 INFO:teuthology.orchestra.run.smithi027.stdout: "last_failure_osd_epoch": 83, 2024-05-21T03:34:07.587 INFO:teuthology.orchestra.run.smithi027.stdout: "active_clients": [ 2024-05-21T03:34:07.587 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.587 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-05-21T03:34:07.587 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-05-21T03:34:07.587 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.587 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-05-21T03:34:07.587 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-05-21T03:34:07.587 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2048934645 2024-05-21T03:34:07.587 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.587 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-05-21T03:34:07.588 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.588 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.588 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "libcephsqlite", 2024-05-21T03:34:07.588 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-05-21T03:34:07.588 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.588 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-05-21T03:34:07.588 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-05-21T03:34:07.588 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1856432253 2024-05-21T03:34:07.588 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.588 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-05-21T03:34:07.588 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.588 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.588 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-05-21T03:34:07.588 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-05-21T03:34:07.588 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.589 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-05-21T03:34:07.589 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-05-21T03:34:07.589 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1126730894 2024-05-21T03:34:07.589 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.589 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-05-21T03:34:07.589 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.589 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.589 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-05-21T03:34:07.589 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-05-21T03:34:07.589 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.589 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-05-21T03:34:07.589 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-05-21T03:34:07.589 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2285350718 2024-05-21T03:34:07.589 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.589 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-05-21T03:34:07.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.590 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.590 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-05-21T03:34:07.590 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-05-21T03:34:07.590 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.590 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-05-21T03:34:07.590 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-05-21T03:34:07.590 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3685007387 2024-05-21T03:34:07.590 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.590 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-05-21T03:34:07.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:07.590 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.590 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-05-21T03:34:07.591 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-05-21T03:34:07.591 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-05-21T03:34:07.591 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-05-21T03:34:07.591 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-05-21T03:34:07.591 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 332608189 2024-05-21T03:34:07.591 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.591 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-05-21T03:34:07.591 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:07.591 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-05-21T03:34:07.591 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:34:07.598 DEBUG:tasks.check_counter:Getting stats from a 2024-05-21T03:34:07.598 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-21T03:34:07.599 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:06 smithi027 ceph-mon[32179]: pgmap v311: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 18 KiB/s rd, 45 KiB/s wr, 31 op/s 2024-05-21T03:34:07.599 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:06 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/784942473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T03:34:07.599 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:06 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/824687682' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-21T03:34:07.600 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:06 smithi165 ceph-mon[36269]: pgmap v311: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 18 KiB/s rd, 45 KiB/s wr, 31 op/s 2024-05-21T03:34:07.600 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:06 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/784942473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T03:34:07.600 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:06 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/824687682' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-21T03:34:07.600 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:06 smithi039 ceph-mon[36429]: pgmap v311: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 18 KiB/s rd, 45 KiB/s wr, 31 op/s 2024-05-21T03:34:07.600 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:06 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/784942473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T03:34:07.600 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:06 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/824687682' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-21T03:34:07.864 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:34:07.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:07 smithi027 ceph-mon[32179]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:34:08.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:07 smithi039 ceph-mon[36429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:34:08.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:07 smithi165 ceph-mon[36269]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:34:09.003 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:08 smithi027 ceph-mon[32179]: pgmap v312: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 28 op/s 2024-05-21T03:34:09.003 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:08 smithi027 ceph-mon[32179]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:34:09.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:08 smithi039 ceph-mon[36429]: pgmap v312: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 28 op/s 2024-05-21T03:34:09.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:08 smithi039 ceph-mon[36429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:34:09.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:08 smithi165 ceph-mon[36269]: pgmap v312: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 28 op/s 2024-05-21T03:34:09.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:08 smithi165 ceph-mon[36269]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:34:09.711 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:34:09.711 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T03:34:09.711 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 710, 2024-05-21T03:34:09.711 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 712, 2024-05-21T03:34:09.711 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 237528, 2024-05-21T03:34:09.711 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 225781, 2024-05-21T03:34:09.712 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 6, 2024-05-21T03:34:09.712 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 2, 2024-05-21T03:34:09.712 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.076838906, 2024-05-21T03:34:09.712 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.024491340, 2024-05-21T03:34:09.712 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.020416239, 2024-05-21T03:34:09.712 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.023998427, 2024-05-21T03:34:09.712 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:09.712 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 710, 2024-05-21T03:34:09.712 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.030282955, 2024-05-21T03:34:09.712 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000042652 2024-05-21T03:34:09.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.713 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:09.713 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.713 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.713 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.713 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-21T03:34:09.713 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 0 2024-05-21T03:34:09.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.713 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T03:34:09.714 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 877, 2024-05-21T03:34:09.714 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 843, 2024-05-21T03:34:09.714 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 2315662, 2024-05-21T03:34:09.714 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 332350, 2024-05-21T03:34:09.714 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 4, 2024-05-21T03:34:09.714 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 3, 2024-05-21T03:34:09.714 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.103075683, 2024-05-21T03:34:09.714 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.028379732, 2024-05-21T03:34:09.714 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.057577401, 2024-05-21T03:34:09.714 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.026943620, 2024-05-21T03:34:09.714 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:09.714 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 843, 2024-05-21T03:34:09.714 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.030298379, 2024-05-21T03:34:09.715 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000035941 2024-05-21T03:34:09.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.715 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:09.715 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.715 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.715 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.715 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 2166672, 2024-05-21T03:34:09.715 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 140528 2024-05-21T03:34:09.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.715 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T03:34:09.715 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 752, 2024-05-21T03:34:09.715 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 920, 2024-05-21T03:34:09.715 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 2873304, 2024-05-21T03:34:09.716 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 701174, 2024-05-21T03:34:09.716 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 6, 2024-05-21T03:34:09.716 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 4, 2024-05-21T03:34:09.716 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.079312247, 2024-05-21T03:34:09.716 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.028992590, 2024-05-21T03:34:09.716 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.021816159, 2024-05-21T03:34:09.716 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.020473576, 2024-05-21T03:34:09.716 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:09.716 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 917, 2024-05-21T03:34:09.716 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.043578972, 2024-05-21T03:34:09.716 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000047523 2024-05-21T03:34:09.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.716 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:09.717 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.717 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.717 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.717 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-21T03:34:09.717 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 462992 2024-05-21T03:34:09.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.717 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-05-21T03:34:09.717 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 1, 2024-05-21T03:34:09.717 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-05-21T03:34:09.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.717 INFO:teuthology.orchestra.run.smithi027.stdout: "finisher-MDSRank": { 2024-05-21T03:34:09.717 INFO:teuthology.orchestra.run.smithi027.stdout: "queue_len": 0, 2024-05-21T03:34:09.717 INFO:teuthology.orchestra.run.smithi027.stdout: "complete_latency": { 2024-05-21T03:34:09.717 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 3452, 2024-05-21T03:34:09.718 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.093135014, 2024-05-21T03:34:09.718 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000026980 2024-05-21T03:34:09.718 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:09.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.718 INFO:teuthology.orchestra.run.smithi027.stdout: "finisher-PurgeQueue": { 2024-05-21T03:34:09.718 INFO:teuthology.orchestra.run.smithi027.stdout: "queue_len": 0, 2024-05-21T03:34:09.718 INFO:teuthology.orchestra.run.smithi027.stdout: "complete_latency": { 2024-05-21T03:34:09.718 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.718 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.718 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.718 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:09.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.718 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2024-05-21T03:34:09.718 INFO:teuthology.orchestra.run.smithi027.stdout: "request": 0, 2024-05-21T03:34:09.719 INFO:teuthology.orchestra.run.smithi027.stdout: "reply": 0, 2024-05-21T03:34:09.719 INFO:teuthology.orchestra.run.smithi027.stdout: "reply_latency": { 2024-05-21T03:34:09.719 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.719 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.719 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.719 INFO:teuthology.orchestra.run.smithi027.stdout: "slow_reply": 0, 2024-05-21T03:34:09.719 INFO:teuthology.orchestra.run.smithi027.stdout: "forward": 0, 2024-05-21T03:34:09.719 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_fetch_complete": 0, 2024-05-21T03:34:09.719 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_fetch_keys": 0, 2024-05-21T03:34:09.719 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_commit": 0, 2024-05-21T03:34:09.719 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_split": 0, 2024-05-21T03:34:09.719 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_merge": 0, 2024-05-21T03:34:09.719 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes": 32, 2024-05-21T03:34:09.719 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_top": 0, 2024-05-21T03:34:09.720 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_bottom": 32, 2024-05-21T03:34:09.720 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_pin_tail": 0, 2024-05-21T03:34:09.720 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_pinned": 32, 2024-05-21T03:34:09.720 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_expired": 0, 2024-05-21T03:34:09.720 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_with_caps": 0, 2024-05-21T03:34:09.720 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": 0, 2024-05-21T03:34:09.720 INFO:teuthology.orchestra.run.smithi027.stdout: "subtrees": 1, 2024-05-21T03:34:09.720 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse": 0, 2024-05-21T03:34:09.720 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_hit": 0, 2024-05-21T03:34:09.720 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_forward": 0, 2024-05-21T03:34:09.720 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_discover": 0, 2024-05-21T03:34:09.720 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_dir_fetch": 0, 2024-05-21T03:34:09.720 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_remote_ino": 0, 2024-05-21T03:34:09.720 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_lock": 0, 2024-05-21T03:34:09.720 INFO:teuthology.orchestra.run.smithi027.stdout: "load_cent": 0, 2024-05-21T03:34:09.721 INFO:teuthology.orchestra.run.smithi027.stdout: "q": 0, 2024-05-21T03:34:09.721 INFO:teuthology.orchestra.run.smithi027.stdout: "exported": 0, 2024-05-21T03:34:09.721 INFO:teuthology.orchestra.run.smithi027.stdout: "exported_inodes": 0, 2024-05-21T03:34:09.721 INFO:teuthology.orchestra.run.smithi027.stdout: "imported": 0, 2024-05-21T03:34:09.721 INFO:teuthology.orchestra.run.smithi027.stdout: "imported_inodes": 0, 2024-05-21T03:34:09.721 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_dir_fetch": 0, 2024-05-21T03:34:09.721 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_backtrace_fetch": 0, 2024-05-21T03:34:09.721 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_peer_discover": 0, 2024-05-21T03:34:09.721 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rfiles": 0, 2024-05-21T03:34:09.721 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rbytes": 0, 2024-05-21T03:34:09.721 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rsnaps": 0, 2024-05-21T03:34:09.721 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_backtrace_fetch": 0, 2024-05-21T03:34:09.721 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_set_tag": 0, 2024-05-21T03:34:09.721 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T03:34:09.722 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_inotable_repaired": 0, 2024-05-21T03:34:09.722 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dir_inodes": 0, 2024-05-21T03:34:09.722 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T03:34:09.722 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T03:34:09.722 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_file_inodes": 0, 2024-05-21T03:34:09.722 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_inode_file_caps": 0, 2024-05-21T03:34:09.722 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_revoke": 0, 2024-05-21T03:34:09.722 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_grant": 0, 2024-05-21T03:34:09.722 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T03:34:09.722 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T03:34:09.722 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T03:34:09.722 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_caps": 0, 2024-05-21T03:34:09.722 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_caps_dirty": 0, 2024-05-21T03:34:09.723 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_cap_release": 0, 2024-05-21T03:34:09.723 INFO:teuthology.orchestra.run.smithi027.stdout: "process_request_cap_release": 0 2024-05-21T03:34:09.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.723 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache": { 2024-05-21T03:34:09.723 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_update": 0, 2024-05-21T03:34:09.723 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_update_receipt": 0, 2024-05-21T03:34:09.723 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_try_discover": 0, 2024-05-21T03:34:09.723 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_send_discover": 0, 2024-05-21T03:34:09.723 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_handle_discover": 0, 2024-05-21T03:34:09.723 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays": 0, 2024-05-21T03:34:09.723 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays_delayed": 0, 2024-05-21T03:34:09.723 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays_enqueuing": 0, 2024-05-21T03:34:09.723 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_created": 0, 2024-05-21T03:34:09.723 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_enqueued": 0, 2024-05-21T03:34:09.723 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_reintegrated": 0, 2024-05-21T03:34:09.724 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_migrated": 0, 2024-05-21T03:34:09.724 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_processing": 0, 2024-05-21T03:34:09.724 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_enqueued": 0, 2024-05-21T03:34:09.724 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_prioritized": 0, 2024-05-21T03:34:09.724 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_started": 0, 2024-05-21T03:34:09.724 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_completed": 0, 2024-05-21T03:34:09.724 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_quiesce_path": 0, 2024-05-21T03:34:09.724 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_quiesce_inode": 0, 2024-05-21T03:34:09.724 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T03:34:09.724 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_exportdir": 0, 2024-05-21T03:34:09.724 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_flush": 0, 2024-05-21T03:34:09.724 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_fragmentdir": 0, 2024-05-21T03:34:09.724 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_fragstats": 0, 2024-05-21T03:34:09.724 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_inodestats": 0 2024-05-21T03:34:09.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.725 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log": { 2024-05-21T03:34:09.725 INFO:teuthology.orchestra.run.smithi027.stdout: "evlrg": 0, 2024-05-21T03:34:09.725 INFO:teuthology.orchestra.run.smithi027.stdout: "evadd": 0, 2024-05-21T03:34:09.725 INFO:teuthology.orchestra.run.smithi027.stdout: "evex": 0, 2024-05-21T03:34:09.725 INFO:teuthology.orchestra.run.smithi027.stdout: "evtrm": 0, 2024-05-21T03:34:09.725 INFO:teuthology.orchestra.run.smithi027.stdout: "ev": 2071, 2024-05-21T03:34:09.725 INFO:teuthology.orchestra.run.smithi027.stdout: "evexg": 0, 2024-05-21T03:34:09.725 INFO:teuthology.orchestra.run.smithi027.stdout: "evexd": 0, 2024-05-21T03:34:09.725 INFO:teuthology.orchestra.run.smithi027.stdout: "segadd": 0, 2024-05-21T03:34:09.725 INFO:teuthology.orchestra.run.smithi027.stdout: "segex": 0, 2024-05-21T03:34:09.726 INFO:teuthology.orchestra.run.smithi027.stdout: "segtrm": 0, 2024-05-21T03:34:09.726 INFO:teuthology.orchestra.run.smithi027.stdout: "seg": 4, 2024-05-21T03:34:09.726 INFO:teuthology.orchestra.run.smithi027.stdout: "segmjr": 2, 2024-05-21T03:34:09.726 INFO:teuthology.orchestra.run.smithi027.stdout: "segexg": 0, 2024-05-21T03:34:09.726 INFO:teuthology.orchestra.run.smithi027.stdout: "segexd": 0, 2024-05-21T03:34:09.726 INFO:teuthology.orchestra.run.smithi027.stdout: "expos": 4194304, 2024-05-21T03:34:09.726 INFO:teuthology.orchestra.run.smithi027.stdout: "wrpos": 6873021, 2024-05-21T03:34:09.726 INFO:teuthology.orchestra.run.smithi027.stdout: "rdpos": 6870416, 2024-05-21T03:34:09.726 INFO:teuthology.orchestra.run.smithi027.stdout: "jlat": { 2024-05-21T03:34:09.726 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.727 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.727 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.727 INFO:teuthology.orchestra.run.smithi027.stdout: "replayed": 2071 2024-05-21T03:34:09.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.727 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_mem": { 2024-05-21T03:34:09.727 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 4, 2024-05-21T03:34:09.727 INFO:teuthology.orchestra.run.smithi027.stdout: "ino+": 186, 2024-05-21T03:34:09.727 INFO:teuthology.orchestra.run.smithi027.stdout: "ino-": 182, 2024-05-21T03:34:09.727 INFO:teuthology.orchestra.run.smithi027.stdout: "dir": 2, 2024-05-21T03:34:09.727 INFO:teuthology.orchestra.run.smithi027.stdout: "dir+": 14, 2024-05-21T03:34:09.727 INFO:teuthology.orchestra.run.smithi027.stdout: "dir-": 12, 2024-05-21T03:34:09.727 INFO:teuthology.orchestra.run.smithi027.stdout: "dn": 32, 2024-05-21T03:34:09.728 INFO:teuthology.orchestra.run.smithi027.stdout: "dn+": 214, 2024-05-21T03:34:09.728 INFO:teuthology.orchestra.run.smithi027.stdout: "dn-": 182, 2024-05-21T03:34:09.728 INFO:teuthology.orchestra.run.smithi027.stdout: "cap": 0, 2024-05-21T03:34:09.728 INFO:teuthology.orchestra.run.smithi027.stdout: "cap+": 0, 2024-05-21T03:34:09.728 INFO:teuthology.orchestra.run.smithi027.stdout: "cap-": 0, 2024-05-21T03:34:09.728 INFO:teuthology.orchestra.run.smithi027.stdout: "rss": 52480, 2024-05-21T03:34:09.728 INFO:teuthology.orchestra.run.smithi027.stdout: "heap": 215416 2024-05-21T03:34:09.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.728 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_server": { 2024-05-21T03:34:09.728 INFO:teuthology.orchestra.run.smithi027.stdout: "dispatch_client_request": 0, 2024-05-21T03:34:09.728 INFO:teuthology.orchestra.run.smithi027.stdout: "dispatch_server_request": 0, 2024-05-21T03:34:09.728 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_request": 0, 2024-05-21T03:34:09.728 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_session": 0, 2024-05-21T03:34:09.728 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_peer_request": 0, 2024-05-21T03:34:09.729 INFO:teuthology.orchestra.run.smithi027.stdout: "req_create_latency": { 2024-05-21T03:34:09.729 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.729 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.729 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.729 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getattr_latency": { 2024-05-21T03:34:09.729 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.729 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.729 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.729 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getfilelock_latency": { 2024-05-21T03:34:09.729 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.729 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.729 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.730 INFO:teuthology.orchestra.run.smithi027.stdout: "req_link_latency": { 2024-05-21T03:34:09.730 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.730 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.730 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.730 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookup_latency": { 2024-05-21T03:34:09.730 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.730 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.730 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.730 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookuphash_latency": { 2024-05-21T03:34:09.730 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.730 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.731 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.731 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupino_latency": { 2024-05-21T03:34:09.731 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.731 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.731 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.731 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupname_latency": { 2024-05-21T03:34:09.731 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.731 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.731 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.732 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupparent_latency": { 2024-05-21T03:34:09.732 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.732 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.732 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.732 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupsnap_latency": { 2024-05-21T03:34:09.732 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.732 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.732 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.732 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lssnap_latency": { 2024-05-21T03:34:09.732 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.732 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.732 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.733 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mkdir_latency": { 2024-05-21T03:34:09.733 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.733 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.733 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.733 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mknod_latency": { 2024-05-21T03:34:09.733 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.733 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.733 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.733 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mksnap_latency": { 2024-05-21T03:34:09.733 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.733 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.734 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.734 INFO:teuthology.orchestra.run.smithi027.stdout: "req_open_latency": { 2024-05-21T03:34:09.734 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.734 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.734 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.734 INFO:teuthology.orchestra.run.smithi027.stdout: "req_readdir_latency": { 2024-05-21T03:34:09.734 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.734 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.734 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.734 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rename_latency": { 2024-05-21T03:34:09.734 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.735 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.735 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.735 INFO:teuthology.orchestra.run.smithi027.stdout: "req_renamesnap_latency": { 2024-05-21T03:34:09.735 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.735 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.735 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.735 INFO:teuthology.orchestra.run.smithi027.stdout: "req_snapdiff_latency": { 2024-05-21T03:34:09.735 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.735 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.735 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.735 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmdir_latency": { 2024-05-21T03:34:09.736 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.736 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.736 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.736 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmsnap_latency": { 2024-05-21T03:34:09.736 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.736 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.736 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.736 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmxattr_latency": { 2024-05-21T03:34:09.736 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.736 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.736 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.737 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setattr_latency": { 2024-05-21T03:34:09.737 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.737 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.737 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.737 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setdirlayout_latency": { 2024-05-21T03:34:09.737 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.737 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.737 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.737 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setfilelock_latency": { 2024-05-21T03:34:09.737 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.738 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.738 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.738 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setlayout_latency": { 2024-05-21T03:34:09.738 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.738 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.738 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.738 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setxattr_latency": { 2024-05-21T03:34:09.738 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.738 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.738 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.739 INFO:teuthology.orchestra.run.smithi027.stdout: "req_symlink_latency": { 2024-05-21T03:34:09.739 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.739 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.739 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.739 INFO:teuthology.orchestra.run.smithi027.stdout: "req_unlink_latency": { 2024-05-21T03:34:09.739 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.739 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.739 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.739 INFO:teuthology.orchestra.run.smithi027.stdout: "cap_revoke_eviction": 0, 2024-05-21T03:34:09.739 INFO:teuthology.orchestra.run.smithi027.stdout: "cap_acquisition_throttle": 0, 2024-05-21T03:34:09.739 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getvxattr_latency": { 2024-05-21T03:34:09.740 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.740 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.740 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.740 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:09.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.740 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_sessions": { 2024-05-21T03:34:09.740 INFO:teuthology.orchestra.run.smithi027.stdout: "session_count": 2, 2024-05-21T03:34:09.740 INFO:teuthology.orchestra.run.smithi027.stdout: "session_add": 5, 2024-05-21T03:34:09.740 INFO:teuthology.orchestra.run.smithi027.stdout: "session_remove": 3, 2024-05-21T03:34:09.740 INFO:teuthology.orchestra.run.smithi027.stdout: "sessions_open": 2, 2024-05-21T03:34:09.740 INFO:teuthology.orchestra.run.smithi027.stdout: "sessions_stale": 0, 2024-05-21T03:34:09.740 INFO:teuthology.orchestra.run.smithi027.stdout: "total_load": 0, 2024-05-21T03:34:09.740 INFO:teuthology.orchestra.run.smithi027.stdout: "average_load": 0, 2024-05-21T03:34:09.741 INFO:teuthology.orchestra.run.smithi027.stdout: "avg_session_uptime": 4421, 2024-05-21T03:34:09.741 INFO:teuthology.orchestra.run.smithi027.stdout: "mdthresh_evicted": 0 2024-05-21T03:34:09.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.741 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-05-21T03:34:09.741 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-05-21T03:34:09.741 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-05-21T03:34:09.741 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T03:34:09.741 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-05-21T03:34:09.741 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T03:34:09.741 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-05-21T03:34:09.741 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T03:34:09.741 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T03:34:09.742 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T03:34:09.742 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T03:34:09.742 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T03:34:09.742 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-05-21T03:34:09.742 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T03:34:09.742 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T03:34:09.742 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-05-21T03:34:09.742 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-05-21T03:34:09.742 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-05-21T03:34:09.742 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-05-21T03:34:09.742 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T03:34:09.742 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T03:34:09.742 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T03:34:09.743 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T03:34:09.743 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T03:34:09.743 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-05-21T03:34:09.743 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-05-21T03:34:09.743 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-05-21T03:34:09.743 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T03:34:09.743 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T03:34:09.743 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-05-21T03:34:09.743 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-05-21T03:34:09.743 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-05-21T03:34:09.743 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-05-21T03:34:09.743 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T03:34:09.743 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-05-21T03:34:09.744 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T03:34:09.744 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-05-21T03:34:09.744 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 190201, 2024-05-21T03:34:09.744 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 53, 2024-05-21T03:34:09.744 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-05-21T03:34:09.744 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-05-21T03:34:09.744 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-05-21T03:34:09.744 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-05-21T03:34:09.744 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-05-21T03:34:09.744 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-05-21T03:34:09.744 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-05-21T03:34:09.744 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-05-21T03:34:09.744 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 8224, 2024-05-21T03:34:09.745 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 171, 2024-05-21T03:34:09.745 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T03:34:09.745 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-05-21T03:34:09.745 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-05-21T03:34:09.745 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-05-21T03:34:09.745 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 36256, 2024-05-21T03:34:09.745 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 160, 2024-05-21T03:34:09.745 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-05-21T03:34:09.745 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-05-21T03:34:09.745 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-05-21T03:34:09.745 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-05-21T03:34:09.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.745 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter": { 2024-05-21T03:34:09.746 INFO:teuthology.orchestra.run.smithi027.stdout: "op_active": 0, 2024-05-21T03:34:09.746 INFO:teuthology.orchestra.run.smithi027.stdout: "op_laggy": 0, 2024-05-21T03:34:09.746 INFO:teuthology.orchestra.run.smithi027.stdout: "op_send": 2062, 2024-05-21T03:34:09.746 INFO:teuthology.orchestra.run.smithi027.stdout: "op_send_bytes": 21, 2024-05-21T03:34:09.746 INFO:teuthology.orchestra.run.smithi027.stdout: "op_resend": 3, 2024-05-21T03:34:09.746 INFO:teuthology.orchestra.run.smithi027.stdout: "op_reply": 2062, 2024-05-21T03:34:09.746 INFO:teuthology.orchestra.run.smithi027.stdout: "op_latency": { 2024-05-21T03:34:09.746 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 2062, 2024-05-21T03:34:09.746 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 116.635965333, 2024-05-21T03:34:09.746 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.056564483 2024-05-21T03:34:09.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.746 INFO:teuthology.orchestra.run.smithi027.stdout: "op_inflight": 0, 2024-05-21T03:34:09.746 INFO:teuthology.orchestra.run.smithi027.stdout: "oplen_avg": { 2024-05-21T03:34:09.746 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 2062, 2024-05-21T03:34:09.747 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 2063 2024-05-21T03:34:09.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.747 INFO:teuthology.orchestra.run.smithi027.stdout: "op": 2062, 2024-05-21T03:34:09.747 INFO:teuthology.orchestra.run.smithi027.stdout: "op_r": 2062, 2024-05-21T03:34:09.747 INFO:teuthology.orchestra.run.smithi027.stdout: "op_w": 0, 2024-05-21T03:34:09.747 INFO:teuthology.orchestra.run.smithi027.stdout: "op_rmw": 0, 2024-05-21T03:34:09.747 INFO:teuthology.orchestra.run.smithi027.stdout: "op_pg": 0, 2024-05-21T03:34:09.747 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_stat": 1283, 2024-05-21T03:34:09.747 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_create": 0, 2024-05-21T03:34:09.747 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_read": 777, 2024-05-21T03:34:09.747 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_write": 0, 2024-05-21T03:34:09.747 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_writefull": 0, 2024-05-21T03:34:09.747 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_writesame": 0, 2024-05-21T03:34:09.747 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_append": 0, 2024-05-21T03:34:09.747 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_zero": 0, 2024-05-21T03:34:09.748 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_truncate": 0, 2024-05-21T03:34:09.748 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_delete": 0, 2024-05-21T03:34:09.748 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_mapext": 0, 2024-05-21T03:34:09.748 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_sparse_read": 0, 2024-05-21T03:34:09.748 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_clonerange": 0, 2024-05-21T03:34:09.748 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_getxattr": 1, 2024-05-21T03:34:09.748 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_setxattr": 0, 2024-05-21T03:34:09.748 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_cmpxattr": 0, 2024-05-21T03:34:09.748 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_rmxattr": 0, 2024-05-21T03:34:09.748 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_resetxattrs": 0, 2024-05-21T03:34:09.748 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_call": 0, 2024-05-21T03:34:09.748 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_watch": 0, 2024-05-21T03:34:09.748 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_notify": 0, 2024-05-21T03:34:09.749 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T03:34:09.749 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_pgls": 0, 2024-05-21T03:34:09.749 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_pgls_filter": 0, 2024-05-21T03:34:09.749 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_other": 0, 2024-05-21T03:34:09.749 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_active": 0, 2024-05-21T03:34:09.749 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_send": 0, 2024-05-21T03:34:09.749 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_resend": 0, 2024-05-21T03:34:09.749 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_ping": 0, 2024-05-21T03:34:09.749 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_active": 0, 2024-05-21T03:34:09.749 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_send": 0, 2024-05-21T03:34:09.749 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_resend": 0, 2024-05-21T03:34:09.749 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_active": 0, 2024-05-21T03:34:09.749 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_send": 0, 2024-05-21T03:34:09.749 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_resend": 0, 2024-05-21T03:34:09.749 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_active": 0, 2024-05-21T03:34:09.750 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_send": 0, 2024-05-21T03:34:09.750 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_resend": 0, 2024-05-21T03:34:09.750 INFO:teuthology.orchestra.run.smithi027.stdout: "command_active": 0, 2024-05-21T03:34:09.750 INFO:teuthology.orchestra.run.smithi027.stdout: "command_send": 0, 2024-05-21T03:34:09.750 INFO:teuthology.orchestra.run.smithi027.stdout: "command_resend": 0, 2024-05-21T03:34:09.750 INFO:teuthology.orchestra.run.smithi027.stdout: "map_epoch": 89, 2024-05-21T03:34:09.750 INFO:teuthology.orchestra.run.smithi027.stdout: "map_full": 0, 2024-05-21T03:34:09.750 INFO:teuthology.orchestra.run.smithi027.stdout: "map_inc": 9, 2024-05-21T03:34:09.750 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_sessions": 5, 2024-05-21T03:34:09.750 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_session_open": 5, 2024-05-21T03:34:09.750 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_session_close": 0, 2024-05-21T03:34:09.750 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_laggy": 0, 2024-05-21T03:34:09.750 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_wr": 0, 2024-05-21T03:34:09.750 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_rd": 2, 2024-05-21T03:34:09.751 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_del": 0 2024-05-21T03:34:09.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.751 INFO:teuthology.orchestra.run.smithi027.stdout: "oft": { 2024-05-21T03:34:09.751 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_objs": 0, 2024-05-21T03:34:09.751 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_kv_pairs": 0, 2024-05-21T03:34:09.751 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_updates": 0, 2024-05-21T03:34:09.751 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_removes": 0 2024-05-21T03:34:09.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.751 INFO:teuthology.orchestra.run.smithi027.stdout: "osd-slow-ops": { 2024-05-21T03:34:09.751 INFO:teuthology.orchestra.run.smithi027.stdout: "slow_ops_count": 0 2024-05-21T03:34:09.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.751 INFO:teuthology.orchestra.run.smithi027.stdout: "purge_queue": { 2024-05-21T03:34:09.751 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_ops": 0, 2024-05-21T03:34:09.752 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_ops_high_water": 0, 2024-05-21T03:34:09.752 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing": 0, 2024-05-21T03:34:09.752 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_high_water": 0, 2024-05-21T03:34:09.752 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executed_ops": 0, 2024-05-21T03:34:09.752 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executed": 0, 2024-05-21T03:34:09.752 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_item_in_journal": 0 2024-05-21T03:34:09.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.752 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T03:34:09.752 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-21T03:34:09.752 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-05-21T03:34:09.752 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-21T03:34:09.752 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 2335, 2024-05-21T03:34:09.752 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 5199596, 2024-05-21T03:34:09.752 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:09.753 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 2335, 2024-05-21T03:34:09.753 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-05-21T03:34:09.753 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-05-21T03:34:09.753 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 2335, 2024-05-21T03:34:09.753 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 5199596, 2024-05-21T03:34:09.753 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-21T03:34:09.753 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.753 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.753 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.753 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:09.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.753 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-objecter_bytes": { 2024-05-21T03:34:09.753 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-21T03:34:09.753 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-05-21T03:34:09.753 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-21T03:34:09.754 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-05-21T03:34:09.754 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-05-21T03:34:09.754 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:09.754 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-05-21T03:34:09.754 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 2062, 2024-05-21T03:34:09.754 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 2678722, 2024-05-21T03:34:09.754 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 2062, 2024-05-21T03:34:09.754 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 2678722, 2024-05-21T03:34:09.754 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-21T03:34:09.754 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.755 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.755 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.755 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:09.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.755 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-objecter_ops": { 2024-05-21T03:34:09.755 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-21T03:34:09.755 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 1024, 2024-05-21T03:34:09.755 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-21T03:34:09.755 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-05-21T03:34:09.755 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-05-21T03:34:09.755 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:09.755 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-05-21T03:34:09.756 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 2062, 2024-05-21T03:34:09.756 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 2062, 2024-05-21T03:34:09.756 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 2062, 2024-05-21T03:34:09.756 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 2062, 2024-05-21T03:34:09.756 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-21T03:34:09.756 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.756 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.756 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.756 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:09.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.756 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-write_buf_throttle": { 2024-05-21T03:34:09.756 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-21T03:34:09.756 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 3758096384, 2024-05-21T03:34:09.756 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-21T03:34:09.757 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-05-21T03:34:09.757 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-05-21T03:34:09.757 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:09.757 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-05-21T03:34:09.757 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-05-21T03:34:09.757 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-05-21T03:34:09.757 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 0, 2024-05-21T03:34:09.757 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 0, 2024-05-21T03:34:09.757 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-21T03:34:09.757 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.757 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.757 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.757 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:09.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:09.758 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-write_buf_throttle-0x55eb2d6b46e0": { 2024-05-21T03:34:09.758 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-21T03:34:09.758 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 3758096384, 2024-05-21T03:34:09.758 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-21T03:34:09.758 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-05-21T03:34:09.758 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-05-21T03:34:09.758 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:09.758 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-05-21T03:34:09.758 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-05-21T03:34:09.758 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-05-21T03:34:09.758 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 0, 2024-05-21T03:34:09.758 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 0, 2024-05-21T03:34:09.758 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-21T03:34:09.759 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:09.759 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:09.759 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:09.759 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:09.759 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:09.759 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:34:10.420 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-05-21T03:34:10.420 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-05-21T03:34:10.420 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-05-21T03:34:10.421 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-05-21T03:34:10.421 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-05-21T03:34:10.421 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-05-21T03:34:10.421 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-05-21T03:34:10.421 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-05-21T03:34:10.421 DEBUG:tasks.check_counter:Getting stats from d 2024-05-21T03:34:10.421 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-21T03:34:10.681 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:34:11.006 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:10 smithi027 ceph-mon[32179]: pgmap v313: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 18 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-05-21T03:34:11.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:10 smithi039 ceph-mon[36429]: pgmap v313: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 18 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-05-21T03:34:11.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:10 smithi165 ceph-mon[36269]: pgmap v313: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 18 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-05-21T03:34:12.000 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:11 smithi027 ceph-mon[32179]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:34:12.000 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:11 smithi027 ceph-mon[32179]: osdmap e90: 12 total, 12 up, 12 in 2024-05-21T03:34:12.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:11 smithi039 ceph-mon[36429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:34:12.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:11 smithi039 ceph-mon[36429]: osdmap e90: 12 total, 12 up, 12 in 2024-05-21T03:34:12.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:11 smithi165 ceph-mon[36269]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:34:12.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:11 smithi165 ceph-mon[36269]: osdmap e90: 12 total, 12 up, 12 in 2024-05-21T03:34:12.669 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:34:12.669 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T03:34:12.669 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 0, 2024-05-21T03:34:12.669 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 2, 2024-05-21T03:34:12.669 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 0, 2024-05-21T03:34:12.669 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 0, 2024-05-21T03:34:12.670 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 4, 2024-05-21T03:34:12.670 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 0, 2024-05-21T03:34:12.670 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.001398302, 2024-05-21T03:34:12.670 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.000000000, 2024-05-21T03:34:12.670 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.000328568, 2024-05-21T03:34:12.670 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-21T03:34:12.670 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:12.670 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:12.670 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:12.670 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:12.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:12.671 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:12.671 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:12.671 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:12.671 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:12.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:12.671 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-21T03:34:12.671 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 0 2024-05-21T03:34:12.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:12.671 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T03:34:12.671 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 247, 2024-05-21T03:34:12.671 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 232, 2024-05-21T03:34:12.671 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 2018784, 2024-05-21T03:34:12.671 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 141952, 2024-05-21T03:34:12.672 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 2, 2024-05-21T03:34:12.672 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 1, 2024-05-21T03:34:12.672 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.049634693, 2024-05-21T03:34:12.672 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.013073508, 2024-05-21T03:34:12.672 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.045878005, 2024-05-21T03:34:12.672 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.004692168, 2024-05-21T03:34:12.672 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:12.672 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 232, 2024-05-21T03:34:12.672 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.013303950, 2024-05-21T03:34:12.672 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000057344 2024-05-21T03:34:12.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:12.672 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:12.672 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:12.673 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:12.673 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:12.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:12.673 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 2018784, 2024-05-21T03:34:12.673 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 141952 2024-05-21T03:34:12.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:12.673 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T03:34:12.673 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 2, 2024-05-21T03:34:12.673 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 176, 2024-05-21T03:34:12.673 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 320, 2024-05-21T03:34:12.673 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 188368, 2024-05-21T03:34:12.673 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 5, 2024-05-21T03:34:12.673 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 3, 2024-05-21T03:34:12.674 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.013856693, 2024-05-21T03:34:12.674 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.009611453, 2024-05-21T03:34:12.674 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.001882096, 2024-05-21T03:34:12.674 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-21T03:34:12.674 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:12.674 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 173, 2024-05-21T03:34:12.674 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.013754834, 2024-05-21T03:34:12.674 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000079507 2024-05-21T03:34:12.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:12.674 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:12.674 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:12.674 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:12.674 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:12.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:12.675 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-21T03:34:12.675 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 188368 2024-05-21T03:34:12.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:12.675 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-05-21T03:34:12.675 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 0, 2024-05-21T03:34:12.675 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-05-21T03:34:12.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:12.675 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-05-21T03:34:12.675 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-05-21T03:34:12.675 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-05-21T03:34:12.675 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T03:34:12.675 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-05-21T03:34:12.676 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T03:34:12.676 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-05-21T03:34:12.676 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T03:34:12.676 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T03:34:12.676 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T03:34:12.676 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T03:34:12.676 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T03:34:12.676 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-05-21T03:34:12.676 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T03:34:12.676 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T03:34:12.676 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-05-21T03:34:12.676 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-05-21T03:34:12.676 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-05-21T03:34:12.677 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-05-21T03:34:12.677 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T03:34:12.677 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T03:34:12.677 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T03:34:12.677 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T03:34:12.677 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T03:34:12.677 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-05-21T03:34:12.677 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-05-21T03:34:12.677 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-05-21T03:34:12.677 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T03:34:12.677 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T03:34:12.677 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-05-21T03:34:12.677 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-05-21T03:34:12.677 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-05-21T03:34:12.678 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-05-21T03:34:12.678 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T03:34:12.678 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-05-21T03:34:12.678 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T03:34:12.678 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-05-21T03:34:12.678 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 148748, 2024-05-21T03:34:12.678 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 32, 2024-05-21T03:34:12.678 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-05-21T03:34:12.678 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-05-21T03:34:12.678 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-05-21T03:34:12.678 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-05-21T03:34:12.678 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-05-21T03:34:12.678 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-05-21T03:34:12.679 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-05-21T03:34:12.679 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-05-21T03:34:12.679 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 0, 2024-05-21T03:34:12.679 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 0, 2024-05-21T03:34:12.679 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T03:34:12.679 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-05-21T03:34:12.679 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-05-21T03:34:12.679 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-05-21T03:34:12.679 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 928, 2024-05-21T03:34:12.679 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 2, 2024-05-21T03:34:12.679 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-05-21T03:34:12.679 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-05-21T03:34:12.679 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-05-21T03:34:12.680 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-05-21T03:34:12.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:12.680 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T03:34:12.680 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-21T03:34:12.680 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-05-21T03:34:12.680 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-21T03:34:12.680 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 245, 2024-05-21T03:34:12.680 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 1981512, 2024-05-21T03:34:12.680 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:12.680 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 245, 2024-05-21T03:34:12.680 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-05-21T03:34:12.680 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-05-21T03:34:12.680 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 245, 2024-05-21T03:34:12.681 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 1981512, 2024-05-21T03:34:12.681 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-21T03:34:12.681 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:12.681 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:12.681 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:12.681 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:12.681 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:12.681 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:34:12.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:12 smithi027 ceph-mon[32179]: pgmap v315: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 33 op/s 2024-05-21T03:34:13.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:12 smithi039 ceph-mon[36429]: pgmap v315: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 33 op/s 2024-05-21T03:34:13.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:12 smithi165 ceph-mon[36269]: pgmap v315: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 33 op/s 2024-05-21T03:34:13.625 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-21T03:34:13.625 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-21T03:34:13.625 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-05-21T03:34:13.625 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-05-21T03:34:13.625 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-05-21T03:34:13.625 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-05-21T03:34:13.625 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-05-21T03:34:13.625 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-05-21T03:34:13.625 DEBUG:tasks.check_counter:Getting stats from g 2024-05-21T03:34:13.626 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-21T03:34:13.883 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:34:15.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:14 smithi165 ceph-mon[36269]: pgmap v316: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 33 op/s 2024-05-21T03:34:15.023 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:14 smithi039 ceph-mon[36429]: pgmap v316: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 33 op/s 2024-05-21T03:34:15.033 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:14 smithi027 ceph-mon[32179]: pgmap v316: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 33 op/s 2024-05-21T03:34:15.842 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:15 smithi027 ceph-mon[32179]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:34:16.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:15 smithi039 ceph-mon[36429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:34:16.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:15 smithi165 ceph-mon[36269]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:34:16.074 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:34:16.074 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T03:34:16.074 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 737, 2024-05-21T03:34:16.074 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 739, 2024-05-21T03:34:16.074 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 247577, 2024-05-21T03:34:16.075 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 234367, 2024-05-21T03:34:16.075 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 6, 2024-05-21T03:34:16.075 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 2, 2024-05-21T03:34:16.075 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.079904621, 2024-05-21T03:34:16.075 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.031589823, 2024-05-21T03:34:16.075 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.018422329, 2024-05-21T03:34:16.075 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.021888705, 2024-05-21T03:34:16.075 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:16.075 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 737, 2024-05-21T03:34:16.075 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.034153788, 2024-05-21T03:34:16.075 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000046341 2024-05-21T03:34:16.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.075 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:16.076 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.076 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.076 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.076 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-21T03:34:16.076 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 0 2024-05-21T03:34:16.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.076 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T03:34:16.076 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 1484, 2024-05-21T03:34:16.076 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 1654, 2024-05-21T03:34:16.076 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 2909488, 2024-05-21T03:34:16.076 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 935975, 2024-05-21T03:34:16.076 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 7, 2024-05-21T03:34:16.076 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 5, 2024-05-21T03:34:16.076 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.112105928, 2024-05-21T03:34:16.077 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.032955608, 2024-05-21T03:34:16.077 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.031049204, 2024-05-21T03:34:16.077 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.039033274, 2024-05-21T03:34:16.077 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:16.077 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 1651, 2024-05-21T03:34:16.077 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.061834457, 2024-05-21T03:34:16.077 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000037452 2024-05-21T03:34:16.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.077 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:16.077 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.077 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.077 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.078 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-21T03:34:16.078 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 464960 2024-05-21T03:34:16.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.078 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T03:34:16.078 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 283, 2024-05-21T03:34:16.078 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 246, 2024-05-21T03:34:16.078 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 2189700, 2024-05-21T03:34:16.078 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 142949, 2024-05-21T03:34:16.078 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 4, 2024-05-21T03:34:16.078 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 3, 2024-05-21T03:34:16.078 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.048941730, 2024-05-21T03:34:16.079 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.014013547, 2024-05-21T03:34:16.079 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.044725921, 2024-05-21T03:34:16.079 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.004538893, 2024-05-21T03:34:16.079 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:16.079 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 246, 2024-05-21T03:34:16.079 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.013136103, 2024-05-21T03:34:16.079 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000053398 2024-05-21T03:34:16.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.079 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:16.079 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.079 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.079 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.079 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 2188352, 2024-05-21T03:34:16.080 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 141984 2024-05-21T03:34:16.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.080 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-05-21T03:34:16.080 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 1, 2024-05-21T03:34:16.080 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-05-21T03:34:16.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.080 INFO:teuthology.orchestra.run.smithi027.stdout: "finisher-MDSRank": { 2024-05-21T03:34:16.080 INFO:teuthology.orchestra.run.smithi027.stdout: "queue_len": 0, 2024-05-21T03:34:16.080 INFO:teuthology.orchestra.run.smithi027.stdout: "complete_latency": { 2024-05-21T03:34:16.080 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 3752, 2024-05-21T03:34:16.080 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.090886798, 2024-05-21T03:34:16.080 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000024223 2024-05-21T03:34:16.080 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:16.081 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.081 INFO:teuthology.orchestra.run.smithi027.stdout: "finisher-PurgeQueue": { 2024-05-21T03:34:16.081 INFO:teuthology.orchestra.run.smithi027.stdout: "queue_len": 0, 2024-05-21T03:34:16.081 INFO:teuthology.orchestra.run.smithi027.stdout: "complete_latency": { 2024-05-21T03:34:16.081 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.081 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.081 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.081 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:16.081 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.081 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2024-05-21T03:34:16.081 INFO:teuthology.orchestra.run.smithi027.stdout: "request": 0, 2024-05-21T03:34:16.081 INFO:teuthology.orchestra.run.smithi027.stdout: "reply": 0, 2024-05-21T03:34:16.081 INFO:teuthology.orchestra.run.smithi027.stdout: "reply_latency": { 2024-05-21T03:34:16.081 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.082 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.082 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.082 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.082 INFO:teuthology.orchestra.run.smithi027.stdout: "slow_reply": 0, 2024-05-21T03:34:16.082 INFO:teuthology.orchestra.run.smithi027.stdout: "forward": 0, 2024-05-21T03:34:16.082 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_fetch_complete": 0, 2024-05-21T03:34:16.082 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_fetch_keys": 0, 2024-05-21T03:34:16.082 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_commit": 0, 2024-05-21T03:34:16.082 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_split": 0, 2024-05-21T03:34:16.082 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_merge": 0, 2024-05-21T03:34:16.082 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes": 31, 2024-05-21T03:34:16.082 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_top": 0, 2024-05-21T03:34:16.082 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_bottom": 31, 2024-05-21T03:34:16.083 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_pin_tail": 0, 2024-05-21T03:34:16.083 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_pinned": 31, 2024-05-21T03:34:16.083 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_expired": 0, 2024-05-21T03:34:16.083 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_with_caps": 0, 2024-05-21T03:34:16.083 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": 0, 2024-05-21T03:34:16.083 INFO:teuthology.orchestra.run.smithi027.stdout: "subtrees": 1, 2024-05-21T03:34:16.083 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse": 0, 2024-05-21T03:34:16.083 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_hit": 0, 2024-05-21T03:34:16.083 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_forward": 0, 2024-05-21T03:34:16.083 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_discover": 0, 2024-05-21T03:34:16.083 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_dir_fetch": 0, 2024-05-21T03:34:16.083 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_remote_ino": 0, 2024-05-21T03:34:16.083 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_lock": 0, 2024-05-21T03:34:16.083 INFO:teuthology.orchestra.run.smithi027.stdout: "load_cent": 0, 2024-05-21T03:34:16.083 INFO:teuthology.orchestra.run.smithi027.stdout: "q": 0, 2024-05-21T03:34:16.083 INFO:teuthology.orchestra.run.smithi027.stdout: "exported": 0, 2024-05-21T03:34:16.084 INFO:teuthology.orchestra.run.smithi027.stdout: "exported_inodes": 0, 2024-05-21T03:34:16.084 INFO:teuthology.orchestra.run.smithi027.stdout: "imported": 0, 2024-05-21T03:34:16.084 INFO:teuthology.orchestra.run.smithi027.stdout: "imported_inodes": 0, 2024-05-21T03:34:16.084 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_dir_fetch": 0, 2024-05-21T03:34:16.084 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_backtrace_fetch": 0, 2024-05-21T03:34:16.084 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_peer_discover": 0, 2024-05-21T03:34:16.084 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rfiles": 0, 2024-05-21T03:34:16.084 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rbytes": 0, 2024-05-21T03:34:16.084 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rsnaps": 0, 2024-05-21T03:34:16.084 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_backtrace_fetch": 0, 2024-05-21T03:34:16.084 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_set_tag": 0, 2024-05-21T03:34:16.084 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T03:34:16.084 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_inotable_repaired": 0, 2024-05-21T03:34:16.084 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dir_inodes": 0, 2024-05-21T03:34:16.085 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T03:34:16.085 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T03:34:16.085 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_file_inodes": 0, 2024-05-21T03:34:16.085 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_inode_file_caps": 0, 2024-05-21T03:34:16.085 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_revoke": 0, 2024-05-21T03:34:16.085 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_grant": 0, 2024-05-21T03:34:16.085 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T03:34:16.085 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T03:34:16.085 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T03:34:16.085 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_caps": 0, 2024-05-21T03:34:16.085 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_caps_dirty": 0, 2024-05-21T03:34:16.085 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_cap_release": 0, 2024-05-21T03:34:16.085 INFO:teuthology.orchestra.run.smithi027.stdout: "process_request_cap_release": 0 2024-05-21T03:34:16.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.085 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache": { 2024-05-21T03:34:16.085 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_update": 0, 2024-05-21T03:34:16.086 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_update_receipt": 0, 2024-05-21T03:34:16.086 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_try_discover": 0, 2024-05-21T03:34:16.086 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_send_discover": 0, 2024-05-21T03:34:16.086 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_handle_discover": 0, 2024-05-21T03:34:16.086 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays": 0, 2024-05-21T03:34:16.086 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays_delayed": 0, 2024-05-21T03:34:16.086 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays_enqueuing": 0, 2024-05-21T03:34:16.086 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_created": 0, 2024-05-21T03:34:16.086 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_enqueued": 0, 2024-05-21T03:34:16.086 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_reintegrated": 0, 2024-05-21T03:34:16.086 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_migrated": 0, 2024-05-21T03:34:16.086 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_processing": 0, 2024-05-21T03:34:16.086 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_enqueued": 0, 2024-05-21T03:34:16.086 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_prioritized": 0, 2024-05-21T03:34:16.087 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_started": 0, 2024-05-21T03:34:16.087 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_completed": 0, 2024-05-21T03:34:16.087 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_quiesce_path": 0, 2024-05-21T03:34:16.087 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_quiesce_inode": 0, 2024-05-21T03:34:16.087 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T03:34:16.087 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_exportdir": 0, 2024-05-21T03:34:16.087 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_flush": 0, 2024-05-21T03:34:16.087 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_fragmentdir": 0, 2024-05-21T03:34:16.087 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_fragstats": 0, 2024-05-21T03:34:16.087 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_inodestats": 0 2024-05-21T03:34:16.087 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.087 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log": { 2024-05-21T03:34:16.087 INFO:teuthology.orchestra.run.smithi027.stdout: "evlrg": 0, 2024-05-21T03:34:16.087 INFO:teuthology.orchestra.run.smithi027.stdout: "evadd": 0, 2024-05-21T03:34:16.087 INFO:teuthology.orchestra.run.smithi027.stdout: "evex": 0, 2024-05-21T03:34:16.087 INFO:teuthology.orchestra.run.smithi027.stdout: "evtrm": 0, 2024-05-21T03:34:16.088 INFO:teuthology.orchestra.run.smithi027.stdout: "ev": 1942, 2024-05-21T03:34:16.088 INFO:teuthology.orchestra.run.smithi027.stdout: "evexg": 0, 2024-05-21T03:34:16.088 INFO:teuthology.orchestra.run.smithi027.stdout: "evexd": 0, 2024-05-21T03:34:16.088 INFO:teuthology.orchestra.run.smithi027.stdout: "segadd": 0, 2024-05-21T03:34:16.088 INFO:teuthology.orchestra.run.smithi027.stdout: "segex": 0, 2024-05-21T03:34:16.088 INFO:teuthology.orchestra.run.smithi027.stdout: "segtrm": 0, 2024-05-21T03:34:16.088 INFO:teuthology.orchestra.run.smithi027.stdout: "seg": 3, 2024-05-21T03:34:16.088 INFO:teuthology.orchestra.run.smithi027.stdout: "segmjr": 2, 2024-05-21T03:34:16.088 INFO:teuthology.orchestra.run.smithi027.stdout: "segexg": 0, 2024-05-21T03:34:16.088 INFO:teuthology.orchestra.run.smithi027.stdout: "segexd": 0, 2024-05-21T03:34:16.088 INFO:teuthology.orchestra.run.smithi027.stdout: "expos": 4194304, 2024-05-21T03:34:16.088 INFO:teuthology.orchestra.run.smithi027.stdout: "wrpos": 6726884, 2024-05-21T03:34:16.088 INFO:teuthology.orchestra.run.smithi027.stdout: "rdpos": 6724279, 2024-05-21T03:34:16.088 INFO:teuthology.orchestra.run.smithi027.stdout: "jlat": { 2024-05-21T03:34:16.088 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.089 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.089 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.089 INFO:teuthology.orchestra.run.smithi027.stdout: "replayed": 1942 2024-05-21T03:34:16.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.089 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_mem": { 2024-05-21T03:34:16.089 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 4, 2024-05-21T03:34:16.089 INFO:teuthology.orchestra.run.smithi027.stdout: "ino+": 180, 2024-05-21T03:34:16.089 INFO:teuthology.orchestra.run.smithi027.stdout: "ino-": 176, 2024-05-21T03:34:16.089 INFO:teuthology.orchestra.run.smithi027.stdout: "dir": 2, 2024-05-21T03:34:16.089 INFO:teuthology.orchestra.run.smithi027.stdout: "dir+": 14, 2024-05-21T03:34:16.089 INFO:teuthology.orchestra.run.smithi027.stdout: "dir-": 12, 2024-05-21T03:34:16.089 INFO:teuthology.orchestra.run.smithi027.stdout: "dn": 31, 2024-05-21T03:34:16.089 INFO:teuthology.orchestra.run.smithi027.stdout: "dn+": 207, 2024-05-21T03:34:16.089 INFO:teuthology.orchestra.run.smithi027.stdout: "dn-": 176, 2024-05-21T03:34:16.089 INFO:teuthology.orchestra.run.smithi027.stdout: "cap": 0, 2024-05-21T03:34:16.090 INFO:teuthology.orchestra.run.smithi027.stdout: "cap+": 0, 2024-05-21T03:34:16.090 INFO:teuthology.orchestra.run.smithi027.stdout: "cap-": 0, 2024-05-21T03:34:16.090 INFO:teuthology.orchestra.run.smithi027.stdout: "rss": 52508, 2024-05-21T03:34:16.090 INFO:teuthology.orchestra.run.smithi027.stdout: "heap": 215416 2024-05-21T03:34:16.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.090 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_server": { 2024-05-21T03:34:16.090 INFO:teuthology.orchestra.run.smithi027.stdout: "dispatch_client_request": 0, 2024-05-21T03:34:16.090 INFO:teuthology.orchestra.run.smithi027.stdout: "dispatch_server_request": 0, 2024-05-21T03:34:16.090 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_request": 0, 2024-05-21T03:34:16.090 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_session": 0, 2024-05-21T03:34:16.090 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_peer_request": 0, 2024-05-21T03:34:16.090 INFO:teuthology.orchestra.run.smithi027.stdout: "req_create_latency": { 2024-05-21T03:34:16.090 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.090 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.091 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.091 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getattr_latency": { 2024-05-21T03:34:16.091 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.091 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.091 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.091 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getfilelock_latency": { 2024-05-21T03:34:16.091 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.091 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.091 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.091 INFO:teuthology.orchestra.run.smithi027.stdout: "req_link_latency": { 2024-05-21T03:34:16.091 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.091 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.092 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.092 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookup_latency": { 2024-05-21T03:34:16.092 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.092 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.092 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.092 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookuphash_latency": { 2024-05-21T03:34:16.092 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.092 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.092 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.092 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupino_latency": { 2024-05-21T03:34:16.092 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.092 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.093 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.093 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupname_latency": { 2024-05-21T03:34:16.093 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.093 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.093 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.093 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupparent_latency": { 2024-05-21T03:34:16.093 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.093 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.093 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.093 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupsnap_latency": { 2024-05-21T03:34:16.093 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.093 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.094 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.094 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lssnap_latency": { 2024-05-21T03:34:16.094 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.094 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.094 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.094 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mkdir_latency": { 2024-05-21T03:34:16.095 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.095 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.095 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.095 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mknod_latency": { 2024-05-21T03:34:16.095 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.095 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.095 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.095 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mksnap_latency": { 2024-05-21T03:34:16.095 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.095 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.095 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.096 INFO:teuthology.orchestra.run.smithi027.stdout: "req_open_latency": { 2024-05-21T03:34:16.096 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.096 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.096 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.096 INFO:teuthology.orchestra.run.smithi027.stdout: "req_readdir_latency": { 2024-05-21T03:34:16.096 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.096 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.096 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.096 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rename_latency": { 2024-05-21T03:34:16.096 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.096 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.096 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.097 INFO:teuthology.orchestra.run.smithi027.stdout: "req_renamesnap_latency": { 2024-05-21T03:34:16.097 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.097 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.097 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.097 INFO:teuthology.orchestra.run.smithi027.stdout: "req_snapdiff_latency": { 2024-05-21T03:34:16.097 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.097 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.097 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.097 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmdir_latency": { 2024-05-21T03:34:16.097 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.097 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.098 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.098 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmsnap_latency": { 2024-05-21T03:34:16.098 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.098 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.098 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.098 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmxattr_latency": { 2024-05-21T03:34:16.098 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.098 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.098 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.098 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setattr_latency": { 2024-05-21T03:34:16.098 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.098 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.099 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.099 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setdirlayout_latency": { 2024-05-21T03:34:16.099 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.099 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.099 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.099 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setfilelock_latency": { 2024-05-21T03:34:16.099 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.099 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.099 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.099 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setlayout_latency": { 2024-05-21T03:34:16.099 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.099 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.100 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.100 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setxattr_latency": { 2024-05-21T03:34:16.100 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.100 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.100 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.100 INFO:teuthology.orchestra.run.smithi027.stdout: "req_symlink_latency": { 2024-05-21T03:34:16.100 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.100 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.100 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.100 INFO:teuthology.orchestra.run.smithi027.stdout: "req_unlink_latency": { 2024-05-21T03:34:16.100 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.100 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.101 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.101 INFO:teuthology.orchestra.run.smithi027.stdout: "cap_revoke_eviction": 0, 2024-05-21T03:34:16.101 INFO:teuthology.orchestra.run.smithi027.stdout: "cap_acquisition_throttle": 0, 2024-05-21T03:34:16.101 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getvxattr_latency": { 2024-05-21T03:34:16.101 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.101 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.101 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.101 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:16.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.101 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_sessions": { 2024-05-21T03:34:16.101 INFO:teuthology.orchestra.run.smithi027.stdout: "session_count": 2, 2024-05-21T03:34:16.101 INFO:teuthology.orchestra.run.smithi027.stdout: "session_add": 4, 2024-05-21T03:34:16.101 INFO:teuthology.orchestra.run.smithi027.stdout: "session_remove": 2, 2024-05-21T03:34:16.102 INFO:teuthology.orchestra.run.smithi027.stdout: "sessions_open": 2, 2024-05-21T03:34:16.102 INFO:teuthology.orchestra.run.smithi027.stdout: "sessions_stale": 0, 2024-05-21T03:34:16.102 INFO:teuthology.orchestra.run.smithi027.stdout: "total_load": 0, 2024-05-21T03:34:16.102 INFO:teuthology.orchestra.run.smithi027.stdout: "average_load": 0, 2024-05-21T03:34:16.102 INFO:teuthology.orchestra.run.smithi027.stdout: "avg_session_uptime": 1883, 2024-05-21T03:34:16.102 INFO:teuthology.orchestra.run.smithi027.stdout: "mdthresh_evicted": 0 2024-05-21T03:34:16.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.102 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-05-21T03:34:16.102 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-05-21T03:34:16.102 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-05-21T03:34:16.102 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T03:34:16.102 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-05-21T03:34:16.102 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T03:34:16.102 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-05-21T03:34:16.103 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T03:34:16.103 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T03:34:16.103 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T03:34:16.103 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T03:34:16.103 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T03:34:16.103 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-05-21T03:34:16.103 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T03:34:16.103 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T03:34:16.103 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-05-21T03:34:16.103 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-05-21T03:34:16.103 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-05-21T03:34:16.103 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-05-21T03:34:16.103 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T03:34:16.103 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T03:34:16.103 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T03:34:16.103 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T03:34:16.104 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T03:34:16.104 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-05-21T03:34:16.104 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-05-21T03:34:16.104 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-05-21T03:34:16.104 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T03:34:16.104 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T03:34:16.104 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-05-21T03:34:16.104 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-05-21T03:34:16.104 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-05-21T03:34:16.104 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-05-21T03:34:16.104 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T03:34:16.104 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-05-21T03:34:16.104 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T03:34:16.104 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-05-21T03:34:16.105 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 198321, 2024-05-21T03:34:16.105 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 58, 2024-05-21T03:34:16.105 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-05-21T03:34:16.105 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-05-21T03:34:16.105 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-05-21T03:34:16.105 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-05-21T03:34:16.105 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-05-21T03:34:16.105 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-05-21T03:34:16.105 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-05-21T03:34:16.105 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-05-21T03:34:16.105 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 8344, 2024-05-21T03:34:16.105 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 174, 2024-05-21T03:34:16.105 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T03:34:16.105 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-05-21T03:34:16.106 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-05-21T03:34:16.106 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-05-21T03:34:16.106 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 35552, 2024-05-21T03:34:16.106 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 156, 2024-05-21T03:34:16.106 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-05-21T03:34:16.106 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-05-21T03:34:16.106 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-05-21T03:34:16.106 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-05-21T03:34:16.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.106 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter": { 2024-05-21T03:34:16.106 INFO:teuthology.orchestra.run.smithi027.stdout: "op_active": 0, 2024-05-21T03:34:16.106 INFO:teuthology.orchestra.run.smithi027.stdout: "op_laggy": 0, 2024-05-21T03:34:16.106 INFO:teuthology.orchestra.run.smithi027.stdout: "op_send": 2221, 2024-05-21T03:34:16.106 INFO:teuthology.orchestra.run.smithi027.stdout: "op_send_bytes": 21, 2024-05-21T03:34:16.106 INFO:teuthology.orchestra.run.smithi027.stdout: "op_resend": 3, 2024-05-21T03:34:16.107 INFO:teuthology.orchestra.run.smithi027.stdout: "op_reply": 2221, 2024-05-21T03:34:16.107 INFO:teuthology.orchestra.run.smithi027.stdout: "op_latency": { 2024-05-21T03:34:16.107 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 2221, 2024-05-21T03:34:16.107 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 79.019947638, 2024-05-21T03:34:16.107 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.035578544 2024-05-21T03:34:16.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.107 INFO:teuthology.orchestra.run.smithi027.stdout: "op_inflight": 0, 2024-05-21T03:34:16.107 INFO:teuthology.orchestra.run.smithi027.stdout: "oplen_avg": { 2024-05-21T03:34:16.107 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 2221, 2024-05-21T03:34:16.107 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 2222 2024-05-21T03:34:16.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.107 INFO:teuthology.orchestra.run.smithi027.stdout: "op": 2221, 2024-05-21T03:34:16.107 INFO:teuthology.orchestra.run.smithi027.stdout: "op_r": 2221, 2024-05-21T03:34:16.107 INFO:teuthology.orchestra.run.smithi027.stdout: "op_w": 0, 2024-05-21T03:34:16.108 INFO:teuthology.orchestra.run.smithi027.stdout: "op_rmw": 0, 2024-05-21T03:34:16.108 INFO:teuthology.orchestra.run.smithi027.stdout: "op_pg": 0, 2024-05-21T03:34:16.108 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_stat": 1389, 2024-05-21T03:34:16.108 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_create": 0, 2024-05-21T03:34:16.108 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_read": 830, 2024-05-21T03:34:16.108 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_write": 0, 2024-05-21T03:34:16.108 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_writefull": 0, 2024-05-21T03:34:16.108 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_writesame": 0, 2024-05-21T03:34:16.108 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_append": 0, 2024-05-21T03:34:16.108 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_zero": 0, 2024-05-21T03:34:16.108 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_truncate": 0, 2024-05-21T03:34:16.108 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_delete": 0, 2024-05-21T03:34:16.108 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_mapext": 0, 2024-05-21T03:34:16.108 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_sparse_read": 0, 2024-05-21T03:34:16.108 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_clonerange": 0, 2024-05-21T03:34:16.109 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_getxattr": 1, 2024-05-21T03:34:16.109 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_setxattr": 0, 2024-05-21T03:34:16.109 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_cmpxattr": 0, 2024-05-21T03:34:16.109 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_rmxattr": 0, 2024-05-21T03:34:16.109 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_resetxattrs": 0, 2024-05-21T03:34:16.109 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_call": 0, 2024-05-21T03:34:16.109 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_watch": 0, 2024-05-21T03:34:16.109 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_notify": 0, 2024-05-21T03:34:16.109 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T03:34:16.109 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_pgls": 0, 2024-05-21T03:34:16.109 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_pgls_filter": 0, 2024-05-21T03:34:16.109 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_other": 0, 2024-05-21T03:34:16.109 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_active": 0, 2024-05-21T03:34:16.109 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_send": 0, 2024-05-21T03:34:16.110 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_resend": 0, 2024-05-21T03:34:16.110 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_ping": 0, 2024-05-21T03:34:16.110 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_active": 0, 2024-05-21T03:34:16.110 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_send": 0, 2024-05-21T03:34:16.110 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_resend": 0, 2024-05-21T03:34:16.110 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_active": 0, 2024-05-21T03:34:16.110 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_send": 0, 2024-05-21T03:34:16.110 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_resend": 0, 2024-05-21T03:34:16.110 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_active": 0, 2024-05-21T03:34:16.110 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_send": 0, 2024-05-21T03:34:16.110 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_resend": 0, 2024-05-21T03:34:16.110 INFO:teuthology.orchestra.run.smithi027.stdout: "command_active": 0, 2024-05-21T03:34:16.110 INFO:teuthology.orchestra.run.smithi027.stdout: "command_send": 0, 2024-05-21T03:34:16.110 INFO:teuthology.orchestra.run.smithi027.stdout: "command_resend": 0, 2024-05-21T03:34:16.111 INFO:teuthology.orchestra.run.smithi027.stdout: "map_epoch": 90, 2024-05-21T03:34:16.111 INFO:teuthology.orchestra.run.smithi027.stdout: "map_full": 0, 2024-05-21T03:34:16.111 INFO:teuthology.orchestra.run.smithi027.stdout: "map_inc": 10, 2024-05-21T03:34:16.111 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_sessions": 6, 2024-05-21T03:34:16.111 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_session_open": 6, 2024-05-21T03:34:16.111 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_session_close": 0, 2024-05-21T03:34:16.111 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_laggy": 0, 2024-05-21T03:34:16.111 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_wr": 0, 2024-05-21T03:34:16.111 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_rd": 2, 2024-05-21T03:34:16.111 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_del": 0 2024-05-21T03:34:16.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.111 INFO:teuthology.orchestra.run.smithi027.stdout: "oft": { 2024-05-21T03:34:16.111 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_objs": 0, 2024-05-21T03:34:16.111 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_kv_pairs": 0, 2024-05-21T03:34:16.112 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_updates": 0, 2024-05-21T03:34:16.112 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_removes": 0 2024-05-21T03:34:16.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.112 INFO:teuthology.orchestra.run.smithi027.stdout: "osd-slow-ops": { 2024-05-21T03:34:16.112 INFO:teuthology.orchestra.run.smithi027.stdout: "slow_ops_count": 0 2024-05-21T03:34:16.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.112 INFO:teuthology.orchestra.run.smithi027.stdout: "purge_queue": { 2024-05-21T03:34:16.112 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_ops": 0, 2024-05-21T03:34:16.112 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_ops_high_water": 0, 2024-05-21T03:34:16.112 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing": 0, 2024-05-21T03:34:16.112 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_high_water": 0, 2024-05-21T03:34:16.112 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executed_ops": 0, 2024-05-21T03:34:16.112 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executed": 0, 2024-05-21T03:34:16.112 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_item_in_journal": 0 2024-05-21T03:34:16.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.112 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T03:34:16.113 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-21T03:34:16.113 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-05-21T03:34:16.113 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-21T03:34:16.113 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 2500, 2024-05-21T03:34:16.113 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 5104727, 2024-05-21T03:34:16.113 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:16.113 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 2500, 2024-05-21T03:34:16.113 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-05-21T03:34:16.113 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-05-21T03:34:16.113 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 2500, 2024-05-21T03:34:16.113 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 5104727, 2024-05-21T03:34:16.113 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-21T03:34:16.113 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.113 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.113 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.114 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:16.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.114 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-objecter_bytes": { 2024-05-21T03:34:16.114 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-21T03:34:16.114 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-05-21T03:34:16.114 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-21T03:34:16.114 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-05-21T03:34:16.114 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-05-21T03:34:16.114 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:16.114 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-05-21T03:34:16.114 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 2221, 2024-05-21T03:34:16.114 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 2532585, 2024-05-21T03:34:16.114 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 2221, 2024-05-21T03:34:16.114 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 2532585, 2024-05-21T03:34:16.114 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-21T03:34:16.114 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.115 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.115 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.115 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:16.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.115 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-objecter_ops": { 2024-05-21T03:34:16.115 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-21T03:34:16.115 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 1024, 2024-05-21T03:34:16.115 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-21T03:34:16.115 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-05-21T03:34:16.115 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-05-21T03:34:16.115 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:16.115 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-05-21T03:34:16.115 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 2221, 2024-05-21T03:34:16.115 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 2221, 2024-05-21T03:34:16.115 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 2221, 2024-05-21T03:34:16.116 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 2221, 2024-05-21T03:34:16.116 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-21T03:34:16.116 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.116 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.116 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.116 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:16.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.116 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-write_buf_throttle": { 2024-05-21T03:34:16.116 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-21T03:34:16.116 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 3758096384, 2024-05-21T03:34:16.116 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-21T03:34:16.116 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-05-21T03:34:16.116 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-05-21T03:34:16.116 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:16.116 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-05-21T03:34:16.117 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-05-21T03:34:16.117 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-05-21T03:34:16.117 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 0, 2024-05-21T03:34:16.117 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 0, 2024-05-21T03:34:16.117 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-21T03:34:16.117 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.117 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.117 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.117 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:16.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:16.117 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-write_buf_throttle-0x5617b76066e0": { 2024-05-21T03:34:16.117 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-21T03:34:16.117 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 3758096384, 2024-05-21T03:34:16.117 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-21T03:34:16.117 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-05-21T03:34:16.117 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-05-21T03:34:16.118 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:16.118 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-05-21T03:34:16.118 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-05-21T03:34:16.118 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-05-21T03:34:16.118 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 0, 2024-05-21T03:34:16.118 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 0, 2024-05-21T03:34:16.118 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-21T03:34:16.118 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:16.118 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:16.118 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:16.118 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:16.118 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:16.118 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:34:16.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:16 smithi027 ceph-mon[32179]: pgmap v317: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T03:34:16.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:16 smithi027 ceph-mon[32179]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:34:17.018 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-05-21T03:34:17.018 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-05-21T03:34:17.018 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-05-21T03:34:17.018 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-05-21T03:34:17.018 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-05-21T03:34:17.018 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-05-21T03:34:17.018 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-05-21T03:34:17.018 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-05-21T03:34:17.019 DEBUG:tasks.check_counter:Getting stats from j 2024-05-21T03:34:17.019 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-21T03:34:17.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:16 smithi039 ceph-mon[36429]: pgmap v317: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T03:34:17.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:16 smithi039 ceph-mon[36429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:34:17.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:16 smithi165 ceph-mon[36269]: pgmap v317: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T03:34:17.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:16 smithi165 ceph-mon[36269]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:34:17.280 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:34:17.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:17 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:34:17.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:17 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:17.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:17 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:17.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:17 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:17.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:17 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:17.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:17 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:17.928 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:17 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:18.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:17 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:34:18.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:17 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:18.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:17 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:18.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:17 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:18.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:17 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:18.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:17 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:18.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:17 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:18.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:17 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T03:34:18.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:17 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:18.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:17 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:18.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:17 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:18.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:17 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:18.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:17 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:18.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:17 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:18.829 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:18 smithi027 ceph-mon[32179]: pgmap v318: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T03:34:18.829 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:18 smithi027 ceph-mon[32179]: osdmap e91: 12 total, 12 up, 12 in 2024-05-21T03:34:19.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:18 smithi039 ceph-mon[36429]: pgmap v318: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T03:34:19.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:18 smithi039 ceph-mon[36429]: osdmap e91: 12 total, 12 up, 12 in 2024-05-21T03:34:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:18 smithi165 ceph-mon[36269]: pgmap v318: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T03:34:19.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:18 smithi165 ceph-mon[36269]: osdmap e91: 12 total, 12 up, 12 in 2024-05-21T03:34:19.201 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-21T03:34:19.202 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T03:34:19.202 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 0, 2024-05-21T03:34:19.202 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 2, 2024-05-21T03:34:19.202 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 0, 2024-05-21T03:34:19.202 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 0, 2024-05-21T03:34:19.202 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 4, 2024-05-21T03:34:19.202 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 0, 2024-05-21T03:34:19.202 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.001242442, 2024-05-21T03:34:19.202 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.000000000, 2024-05-21T03:34:19.202 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.000251021, 2024-05-21T03:34:19.202 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-21T03:34:19.202 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:19.203 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:19.203 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:19.203 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:19.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:19.203 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:19.203 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:19.203 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:19.203 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:19.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:19.203 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-21T03:34:19.203 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 0 2024-05-21T03:34:19.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:19.203 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T03:34:19.204 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 2, 2024-05-21T03:34:19.204 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 178, 2024-05-21T03:34:19.204 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 320, 2024-05-21T03:34:19.204 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 188720, 2024-05-21T03:34:19.204 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 5, 2024-05-21T03:34:19.204 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 3, 2024-05-21T03:34:19.204 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.013144934, 2024-05-21T03:34:19.204 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.009170684, 2024-05-21T03:34:19.204 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.001625752, 2024-05-21T03:34:19.204 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-21T03:34:19.204 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:19.204 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 175, 2024-05-21T03:34:19.204 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.012470397, 2024-05-21T03:34:19.204 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000071259 2024-05-21T03:34:19.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:19.205 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:19.205 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:19.205 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:19.205 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:19.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:19.205 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-21T03:34:19.205 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 188720 2024-05-21T03:34:19.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:19.205 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T03:34:19.205 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 249, 2024-05-21T03:34:19.205 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 234, 2024-05-21T03:34:19.205 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 2019328, 2024-05-21T03:34:19.206 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 143232, 2024-05-21T03:34:19.206 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 2, 2024-05-21T03:34:19.206 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 1, 2024-05-21T03:34:19.206 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.048470058, 2024-05-21T03:34:19.206 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.014581948, 2024-05-21T03:34:19.206 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.043085360, 2024-05-21T03:34:19.206 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.004714284, 2024-05-21T03:34:19.206 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:19.206 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 234, 2024-05-21T03:34:19.206 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.012233263, 2024-05-21T03:34:19.206 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000052278 2024-05-21T03:34:19.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:19.206 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:19.207 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:19.207 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:19.207 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:19.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:19.207 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 2019328, 2024-05-21T03:34:19.207 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 143232 2024-05-21T03:34:19.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:19.207 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-05-21T03:34:19.207 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 0, 2024-05-21T03:34:19.207 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-05-21T03:34:19.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:19.207 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-05-21T03:34:19.207 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-05-21T03:34:19.207 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-05-21T03:34:19.208 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T03:34:19.208 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-05-21T03:34:19.208 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T03:34:19.208 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-05-21T03:34:19.208 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T03:34:19.208 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T03:34:19.208 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T03:34:19.208 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T03:34:19.208 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T03:34:19.208 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-05-21T03:34:19.208 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T03:34:19.208 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T03:34:19.208 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-05-21T03:34:19.209 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-05-21T03:34:19.209 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-05-21T03:34:19.209 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-05-21T03:34:19.209 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T03:34:19.209 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T03:34:19.209 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T03:34:19.209 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T03:34:19.209 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T03:34:19.209 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-05-21T03:34:19.209 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-05-21T03:34:19.209 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-05-21T03:34:19.209 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T03:34:19.209 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T03:34:19.209 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-05-21T03:34:19.210 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-05-21T03:34:19.210 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-05-21T03:34:19.210 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-05-21T03:34:19.210 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T03:34:19.210 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-05-21T03:34:19.210 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T03:34:19.210 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-05-21T03:34:19.210 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 148754, 2024-05-21T03:34:19.210 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 31, 2024-05-21T03:34:19.210 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-05-21T03:34:19.210 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-05-21T03:34:19.210 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-05-21T03:34:19.210 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-05-21T03:34:19.211 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-05-21T03:34:19.211 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-05-21T03:34:19.211 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-05-21T03:34:19.211 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-05-21T03:34:19.211 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 0, 2024-05-21T03:34:19.211 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 0, 2024-05-21T03:34:19.211 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T03:34:19.211 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-05-21T03:34:19.211 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-05-21T03:34:19.211 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-05-21T03:34:19.211 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 928, 2024-05-21T03:34:19.211 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 2, 2024-05-21T03:34:19.211 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-05-21T03:34:19.212 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-05-21T03:34:19.212 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-05-21T03:34:19.212 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-05-21T03:34:19.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-21T03:34:19.212 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T03:34:19.212 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-21T03:34:19.212 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-05-21T03:34:19.212 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-21T03:34:19.212 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 247, 2024-05-21T03:34:19.212 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 1981772, 2024-05-21T03:34:19.212 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:19.212 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 247, 2024-05-21T03:34:19.212 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-05-21T03:34:19.213 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-05-21T03:34:19.213 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 247, 2024-05-21T03:34:19.213 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 1981772, 2024-05-21T03:34:19.213 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-21T03:34:19.213 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-21T03:34:19.213 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-21T03:34:19.213 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-21T03:34:19.213 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:19.213 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-21T03:34:19.213 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-21T03:34:19.998 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-21T03:34:19.999 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-21T03:34:19.999 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-05-21T03:34:19.999 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-05-21T03:34:19.999 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-05-21T03:34:19.999 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-05-21T03:34:19.999 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-05-21T03:34:19.999 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-05-21T03:34:19.999 DEBUG:tasks.check_counter:Getting stats from b 2024-05-21T03:34:19.999 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-21T03:34:20.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:19 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:20.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:19 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:20.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:19 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:20.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:19 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:20.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:19 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:20.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:19 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:20.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:19 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:20.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:19 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:20.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:19 smithi027 ceph-mon[32179]: Adjusting osd_memory_target on smithi165 to 176.2M 2024-05-21T03:34:20.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:19 smithi027 ceph-mon[32179]: Unable to set osd_memory_target on smithi165 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-05-21T03:34:20.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:19 smithi027 ceph-mon[32179]: pgmap v320: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.4 KiB/s rd, 115 B/s wr, 7 op/s 2024-05-21T03:34:20.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:19 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:20.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:19 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:20.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:19 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:20.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:19 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:20.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:19 smithi027 ceph-mon[32179]: Adjusting osd_memory_target on smithi039 to 176.2M 2024-05-21T03:34:20.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:19 smithi027 ceph-mon[32179]: Unable to set osd_memory_target on smithi039 to 184789606: error parsing value: Value '184789606' is below minimum 939524096 2024-05-21T03:34:20.246 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T03:34:20.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:19 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:20.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:19 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:20.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:19 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:20.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:19 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:20.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:19 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:20.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:19 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:20.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:19 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:20.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:19 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:20.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:19 smithi039 ceph-mon[36429]: Adjusting osd_memory_target on smithi165 to 176.2M 2024-05-21T03:34:20.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:19 smithi039 ceph-mon[36429]: Unable to set osd_memory_target on smithi165 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-05-21T03:34:20.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:19 smithi039 ceph-mon[36429]: pgmap v320: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.4 KiB/s rd, 115 B/s wr, 7 op/s 2024-05-21T03:34:20.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:19 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:20.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:19 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:20.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:19 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:20.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:19 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:20.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:19 smithi039 ceph-mon[36429]: Adjusting osd_memory_target on smithi039 to 176.2M 2024-05-21T03:34:20.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:19 smithi039 ceph-mon[36429]: Unable to set osd_memory_target on smithi039 to 184789606: error parsing value: Value '184789606' is below minimum 939524096 2024-05-21T03:34:20.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:19 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:20.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:19 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:20.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:19 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:20.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:19 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:20.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:19 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:20.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:19 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:20.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:19 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:20.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:19 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:20.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:19 smithi165 ceph-mon[36269]: Adjusting osd_memory_target on smithi165 to 176.2M 2024-05-21T03:34:20.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:19 smithi165 ceph-mon[36269]: Unable to set osd_memory_target on smithi165 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-05-21T03:34:20.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:19 smithi165 ceph-mon[36269]: pgmap v320: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.4 KiB/s rd, 115 B/s wr, 7 op/s 2024-05-21T03:34:20.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:19 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:20.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:19 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:20.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:19 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:20.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:19 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:20.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:19 smithi165 ceph-mon[36269]: Adjusting osd_memory_target on smithi039 to 176.2M 2024-05-21T03:34:20.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:19 smithi165 ceph-mon[36269]: Unable to set osd_memory_target on smithi039 to 184789606: error parsing value: Value '184789606' is below minimum 939524096 2024-05-21T03:34:22.189 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:22 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:22.189 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:22 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:22.189 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:22 smithi039 ceph-mon[36429]: pgmap v321: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2024-05-21T03:34:22.189 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:22 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:22.189 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:22 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:22.189 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:22 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:22.189 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:22 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:22.190 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:22 smithi039 ceph-mon[36429]: Adjusting osd_memory_target on smithi027 to 176.1M 2024-05-21T03:34:22.190 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:22 smithi039 ceph-mon[36429]: Unable to set osd_memory_target on smithi027 to 184745164: error parsing value: Value '184745164' is below minimum 939524096 2024-05-21T03:34:22.190 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:22 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:34:22.190 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:22 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:34:22.190 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:22 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:22.190 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-21T03:34:22.190 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T03:34:22.190 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 15426, 2024-05-21T03:34:22.191 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 19172, 2024-05-21T03:34:22.191 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 5767276, 2024-05-21T03:34:22.191 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 22941734, 2024-05-21T03:34:22.191 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 17, 2024-05-21T03:34:22.191 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 5, 2024-05-21T03:34:22.191 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 1.154497832, 2024-05-21T03:34:22.191 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.446764279, 2024-05-21T03:34:22.191 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 1.087825627, 2024-05-21T03:34:22.191 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.128104256, 2024-05-21T03:34:22.191 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:22.191 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 19171, 2024-05-21T03:34:22.191 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.940251671, 2024-05-21T03:34:22.192 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000049045 2024-05-21T03:34:22.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.192 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:22.192 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 28559, 2024-05-21T03:34:22.192 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.004691614, 2024-05-21T03:34:22.192 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000164 2024-05-21T03:34:22.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.192 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 4227312, 2024-05-21T03:34:22.192 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 8682240 2024-05-21T03:34:22.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.192 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T03:34:22.192 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 4701, 2024-05-21T03:34:22.192 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 4369, 2024-05-21T03:34:22.192 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 3818826, 2024-05-21T03:34:22.193 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 4837277, 2024-05-21T03:34:22.193 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 14, 2024-05-21T03:34:22.193 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 6, 2024-05-21T03:34:22.193 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.445235644, 2024-05-21T03:34:22.193 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.172185869, 2024-05-21T03:34:22.193 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.218095617, 2024-05-21T03:34:22.193 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.078888660, 2024-05-21T03:34:22.193 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:22.193 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 4368, 2024-05-21T03:34:22.193 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 1.073166674, 2024-05-21T03:34:22.193 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000245688 2024-05-21T03:34:22.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.193 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:22.193 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 5557, 2024-05-21T03:34:22.193 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.001391268, 2024-05-21T03:34:22.194 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000250 2024-05-21T03:34:22.194 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.194 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 2281728, 2024-05-21T03:34:22.194 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 159696 2024-05-21T03:34:22.194 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.194 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T03:34:22.194 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 7143, 2024-05-21T03:34:22.194 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 6008, 2024-05-21T03:34:22.194 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 3191001, 2024-05-21T03:34:22.194 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 4322327, 2024-05-21T03:34:22.194 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 37, 2024-05-21T03:34:22.194 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 7, 2024-05-21T03:34:22.194 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.512814695, 2024-05-21T03:34:22.194 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.167497600, 2024-05-21T03:34:22.195 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.286850090, 2024-05-21T03:34:22.195 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.088984842, 2024-05-21T03:34:22.195 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:22.195 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 6005, 2024-05-21T03:34:22.195 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.424620653, 2024-05-21T03:34:22.195 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000070711 2024-05-21T03:34:22.195 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.195 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:22.195 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 11791, 2024-05-21T03:34:22.195 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.002151841, 2024-05-21T03:34:22.195 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000182 2024-05-21T03:34:22.195 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.196 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 315040, 2024-05-21T03:34:22.196 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 240592 2024-05-21T03:34:22.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.196 INFO:teuthology.orchestra.run.smithi039.stdout: "cct": { 2024-05-21T03:34:22.196 INFO:teuthology.orchestra.run.smithi039.stdout: "total_workers": 1, 2024-05-21T03:34:22.196 INFO:teuthology.orchestra.run.smithi039.stdout: "unhealthy_workers": 0 2024-05-21T03:34:22.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.196 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-MDSRank": { 2024-05-21T03:34:22.196 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-05-21T03:34:22.196 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-05-21T03:34:22.196 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 3462, 2024-05-21T03:34:22.196 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 3.489880472, 2024-05-21T03:34:22.196 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.001008053 2024-05-21T03:34:22.197 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:22.197 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.197 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-PurgeQueue": { 2024-05-21T03:34:22.197 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-05-21T03:34:22.197 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-05-21T03:34:22.197 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 1573, 2024-05-21T03:34:22.197 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.083980645, 2024-05-21T03:34:22.197 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000053388 2024-05-21T03:34:22.197 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:22.197 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.197 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": { 2024-05-21T03:34:22.197 INFO:teuthology.orchestra.run.smithi039.stdout: "request": 2392, 2024-05-21T03:34:22.197 INFO:teuthology.orchestra.run.smithi039.stdout: "reply": 2350, 2024-05-21T03:34:22.197 INFO:teuthology.orchestra.run.smithi039.stdout: "reply_latency": { 2024-05-21T03:34:22.198 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 2350, 2024-05-21T03:34:22.198 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 304.604559849, 2024-05-21T03:34:22.198 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.129618961 2024-05-21T03:34:22.198 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.198 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_reply": 0, 2024-05-21T03:34:22.198 INFO:teuthology.orchestra.run.smithi039.stdout: "forward": 42, 2024-05-21T03:34:22.198 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_complete": 0, 2024-05-21T03:34:22.198 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_keys": 0, 2024-05-21T03:34:22.198 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_commit": 21, 2024-05-21T03:34:22.198 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_split": 1, 2024-05-21T03:34:22.198 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_merge": 0, 2024-05-21T03:34:22.198 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes": 1046, 2024-05-21T03:34:22.198 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_top": 46, 2024-05-21T03:34:22.198 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_bottom": 921, 2024-05-21T03:34:22.199 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pin_tail": 79, 2024-05-21T03:34:22.199 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pinned": 979, 2024-05-21T03:34:22.199 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_expired": 767, 2024-05-21T03:34:22.199 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_with_caps": 2, 2024-05-21T03:34:22.199 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": 2, 2024-05-21T03:34:22.199 INFO:teuthology.orchestra.run.smithi039.stdout: "subtrees": 6, 2024-05-21T03:34:22.199 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse": 3395, 2024-05-21T03:34:22.199 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_hit": 3037, 2024-05-21T03:34:22.199 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_forward": 5, 2024-05-21T03:34:22.199 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_discover": 78, 2024-05-21T03:34:22.199 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_dir_fetch": 0, 2024-05-21T03:34:22.199 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_remote_ino": 0, 2024-05-21T03:34:22.199 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_lock": 49, 2024-05-21T03:34:22.199 INFO:teuthology.orchestra.run.smithi039.stdout: "load_cent": 139, 2024-05-21T03:34:22.200 INFO:teuthology.orchestra.run.smithi039.stdout: "q": 0, 2024-05-21T03:34:22.200 INFO:teuthology.orchestra.run.smithi039.stdout: "exported": 11, 2024-05-21T03:34:22.200 INFO:teuthology.orchestra.run.smithi039.stdout: "exported_inodes": 517, 2024-05-21T03:34:22.200 INFO:teuthology.orchestra.run.smithi039.stdout: "imported": 14, 2024-05-21T03:34:22.200 INFO:teuthology.orchestra.run.smithi039.stdout: "imported_inodes": 823, 2024-05-21T03:34:22.200 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_dir_fetch": 0, 2024-05-21T03:34:22.200 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_backtrace_fetch": 0, 2024-05-21T03:34:22.200 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_peer_discover": 0, 2024-05-21T03:34:22.200 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rfiles": 4, 2024-05-21T03:34:22.200 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rbytes": 234, 2024-05-21T03:34:22.200 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rsnaps": 6, 2024-05-21T03:34:22.200 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_fetch": 0, 2024-05-21T03:34:22.200 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_set_tag": 0, 2024-05-21T03:34:22.200 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T03:34:22.201 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_inotable_repaired": 0, 2024-05-21T03:34:22.201 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_inodes": 0, 2024-05-21T03:34:22.201 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T03:34:22.201 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T03:34:22.201 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_file_inodes": 0, 2024-05-21T03:34:22.201 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_inode_file_caps": 12, 2024-05-21T03:34:22.201 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_revoke": 378, 2024-05-21T03:34:22.201 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_grant": 3178, 2024-05-21T03:34:22.201 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_trunc": 6, 2024-05-21T03:34:22.201 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T03:34:22.201 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T03:34:22.201 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps": 7713, 2024-05-21T03:34:22.201 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps_dirty": 6939, 2024-05-21T03:34:22.202 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_cap_release": 161, 2024-05-21T03:34:22.202 INFO:teuthology.orchestra.run.smithi039.stdout: "process_request_cap_release": 1003 2024-05-21T03:34:22.202 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.202 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache": { 2024-05-21T03:34:22.202 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update": 347, 2024-05-21T03:34:22.202 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update_receipt": 220, 2024-05-21T03:34:22.202 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_try_discover": 150, 2024-05-21T03:34:22.202 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_send_discover": 142, 2024-05-21T03:34:22.202 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_handle_discover": 718, 2024-05-21T03:34:22.202 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays": 6, 2024-05-21T03:34:22.202 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_delayed": 0, 2024-05-21T03:34:22.202 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_enqueuing": 0, 2024-05-21T03:34:22.202 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_created": 562, 2024-05-21T03:34:22.202 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_enqueued": 495, 2024-05-21T03:34:22.202 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_reintegrated": 61, 2024-05-21T03:34:22.203 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_migrated": 0, 2024-05-21T03:34:22.203 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_processing": 0, 2024-05-21T03:34:22.203 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_enqueued": 0, 2024-05-21T03:34:22.203 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_prioritized": 0, 2024-05-21T03:34:22.203 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_started": 0, 2024-05-21T03:34:22.203 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_completed": 0, 2024-05-21T03:34:22.203 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_quiesce_path": 5, 2024-05-21T03:34:22.203 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_quiesce_inode": 398, 2024-05-21T03:34:22.203 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T03:34:22.203 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_exportdir": 11, 2024-05-21T03:34:22.203 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_flush": 0, 2024-05-21T03:34:22.203 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragmentdir": 1, 2024-05-21T03:34:22.203 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragstats": 0, 2024-05-21T03:34:22.203 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_inodestats": 0 2024-05-21T03:34:22.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.204 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log": { 2024-05-21T03:34:22.204 INFO:teuthology.orchestra.run.smithi039.stdout: "evlrg": 0, 2024-05-21T03:34:22.204 INFO:teuthology.orchestra.run.smithi039.stdout: "evadd": 9086, 2024-05-21T03:34:22.204 INFO:teuthology.orchestra.run.smithi039.stdout: "evex": 0, 2024-05-21T03:34:22.204 INFO:teuthology.orchestra.run.smithi039.stdout: "evtrm": 0, 2024-05-21T03:34:22.204 INFO:teuthology.orchestra.run.smithi039.stdout: "ev": 9086, 2024-05-21T03:34:22.204 INFO:teuthology.orchestra.run.smithi039.stdout: "evexg": 0, 2024-05-21T03:34:22.204 INFO:teuthology.orchestra.run.smithi039.stdout: "evexd": 0, 2024-05-21T03:34:22.204 INFO:teuthology.orchestra.run.smithi039.stdout: "segadd": 12, 2024-05-21T03:34:22.204 INFO:teuthology.orchestra.run.smithi039.stdout: "segex": 0, 2024-05-21T03:34:22.204 INFO:teuthology.orchestra.run.smithi039.stdout: "segtrm": 0, 2024-05-21T03:34:22.204 INFO:teuthology.orchestra.run.smithi039.stdout: "seg": 12, 2024-05-21T03:34:22.204 INFO:teuthology.orchestra.run.smithi039.stdout: "segmjr": 2, 2024-05-21T03:34:22.204 INFO:teuthology.orchestra.run.smithi039.stdout: "segexg": 0, 2024-05-21T03:34:22.204 INFO:teuthology.orchestra.run.smithi039.stdout: "segexd": 0, 2024-05-21T03:34:22.205 INFO:teuthology.orchestra.run.smithi039.stdout: "expos": 4194304, 2024-05-21T03:34:22.205 INFO:teuthology.orchestra.run.smithi039.stdout: "wrpos": 27025580, 2024-05-21T03:34:22.205 INFO:teuthology.orchestra.run.smithi039.stdout: "rdpos": 0, 2024-05-21T03:34:22.205 INFO:teuthology.orchestra.run.smithi039.stdout: "jlat": { 2024-05-21T03:34:22.205 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 1932, 2024-05-21T03:34:22.205 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 381.408665221, 2024-05-21T03:34:22.205 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.197416493 2024-05-21T03:34:22.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.205 INFO:teuthology.orchestra.run.smithi039.stdout: "replayed": 0 2024-05-21T03:34:22.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.205 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_mem": { 2024-05-21T03:34:22.205 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 713, 2024-05-21T03:34:22.205 INFO:teuthology.orchestra.run.smithi039.stdout: "ino+": 1653, 2024-05-21T03:34:22.205 INFO:teuthology.orchestra.run.smithi039.stdout: "ino-": 940, 2024-05-21T03:34:22.205 INFO:teuthology.orchestra.run.smithi039.stdout: "dir": 46, 2024-05-21T03:34:22.206 INFO:teuthology.orchestra.run.smithi039.stdout: "dir+": 210, 2024-05-21T03:34:22.206 INFO:teuthology.orchestra.run.smithi039.stdout: "dir-": 164, 2024-05-21T03:34:22.206 INFO:teuthology.orchestra.run.smithi039.stdout: "dn": 1046, 2024-05-21T03:34:22.206 INFO:teuthology.orchestra.run.smithi039.stdout: "dn+": 2394, 2024-05-21T03:34:22.206 INFO:teuthology.orchestra.run.smithi039.stdout: "dn-": 1348, 2024-05-21T03:34:22.206 INFO:teuthology.orchestra.run.smithi039.stdout: "cap": 2, 2024-05-21T03:34:22.206 INFO:teuthology.orchestra.run.smithi039.stdout: "cap+": 979, 2024-05-21T03:34:22.206 INFO:teuthology.orchestra.run.smithi039.stdout: "cap-": 977, 2024-05-21T03:34:22.206 INFO:teuthology.orchestra.run.smithi039.stdout: "rss": 76076, 2024-05-21T03:34:22.206 INFO:teuthology.orchestra.run.smithi039.stdout: "heap": 231800 2024-05-21T03:34:22.206 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.206 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_server": { 2024-05-21T03:34:22.206 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_client_request": 2993, 2024-05-21T03:34:22.206 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_server_request": 391, 2024-05-21T03:34:22.207 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_request": 2392, 2024-05-21T03:34:22.207 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_session": 45, 2024-05-21T03:34:22.207 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_peer_request": 1085, 2024-05-21T03:34:22.207 INFO:teuthology.orchestra.run.smithi039.stdout: "req_create_latency": { 2024-05-21T03:34:22.207 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 765, 2024-05-21T03:34:22.207 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.887700647, 2024-05-21T03:34:22.207 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.001160393 2024-05-21T03:34:22.207 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.207 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getattr_latency": { 2024-05-21T03:34:22.207 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 160, 2024-05-21T03:34:22.207 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.029343293, 2024-05-21T03:34:22.207 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000183395 2024-05-21T03:34:22.207 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.207 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getfilelock_latency": { 2024-05-21T03:34:22.207 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:22.208 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:22.208 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:22.208 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.208 INFO:teuthology.orchestra.run.smithi039.stdout: "req_link_latency": { 2024-05-21T03:34:22.208 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 61, 2024-05-21T03:34:22.208 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.050246858, 2024-05-21T03:34:22.208 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000823718 2024-05-21T03:34:22.208 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.208 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookup_latency": { 2024-05-21T03:34:22.208 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 296, 2024-05-21T03:34:22.208 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 260.299954266, 2024-05-21T03:34:22.208 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.879391737 2024-05-21T03:34:22.208 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.208 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookuphash_latency": { 2024-05-21T03:34:22.209 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:22.209 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:22.209 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:22.209 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.209 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupino_latency": { 2024-05-21T03:34:22.209 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:22.209 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:22.209 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:22.209 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.209 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupname_latency": { 2024-05-21T03:34:22.209 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:22.209 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:22.209 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:22.209 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.209 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupparent_latency": { 2024-05-21T03:34:22.210 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:22.210 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:22.210 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:22.210 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.210 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupsnap_latency": { 2024-05-21T03:34:22.210 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:22.210 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:22.210 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:22.210 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.210 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lssnap_latency": { 2024-05-21T03:34:22.211 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 10, 2024-05-21T03:34:22.211 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.001903616, 2024-05-21T03:34:22.211 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000190361 2024-05-21T03:34:22.211 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.211 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mkdir_latency": { 2024-05-21T03:34:22.211 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 105, 2024-05-21T03:34:22.211 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 14.791690169, 2024-05-21T03:34:22.211 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.140873239 2024-05-21T03:34:22.211 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.211 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mknod_latency": { 2024-05-21T03:34:22.211 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:22.212 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:22.212 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:22.212 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.212 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mksnap_latency": { 2024-05-21T03:34:22.212 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 10, 2024-05-21T03:34:22.212 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 10.906039350, 2024-05-21T03:34:22.212 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 1.090603935 2024-05-21T03:34:22.212 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.212 INFO:teuthology.orchestra.run.smithi039.stdout: "req_open_latency": { 2024-05-21T03:34:22.212 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 91, 2024-05-21T03:34:22.212 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.111590902, 2024-05-21T03:34:22.212 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.001226273 2024-05-21T03:34:22.212 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.212 INFO:teuthology.orchestra.run.smithi039.stdout: "req_readdir_latency": { 2024-05-21T03:34:22.212 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 175, 2024-05-21T03:34:22.213 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 1.113431071, 2024-05-21T03:34:22.213 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.006362463 2024-05-21T03:34:22.213 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.213 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rename_latency": { 2024-05-21T03:34:22.213 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 113, 2024-05-21T03:34:22.213 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 12.449621712, 2024-05-21T03:34:22.213 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.110173643 2024-05-21T03:34:22.213 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.213 INFO:teuthology.orchestra.run.smithi039.stdout: "req_renamesnap_latency": { 2024-05-21T03:34:22.213 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:22.213 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:22.213 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:22.213 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.213 INFO:teuthology.orchestra.run.smithi039.stdout: "req_snapdiff_latency": { 2024-05-21T03:34:22.213 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:22.214 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:22.214 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:22.214 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.214 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmdir_latency": { 2024-05-21T03:34:22.214 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 22, 2024-05-21T03:34:22.214 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.293165700, 2024-05-21T03:34:22.214 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.013325713 2024-05-21T03:34:22.214 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.214 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmsnap_latency": { 2024-05-21T03:34:22.214 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 4, 2024-05-21T03:34:22.214 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 2.695963651, 2024-05-21T03:34:22.214 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.673990912 2024-05-21T03:34:22.214 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.214 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmxattr_latency": { 2024-05-21T03:34:22.214 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:22.215 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:22.215 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:22.215 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.215 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setattr_latency": { 2024-05-21T03:34:22.215 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 7, 2024-05-21T03:34:22.215 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.049855371, 2024-05-21T03:34:22.215 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.007122195 2024-05-21T03:34:22.215 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.215 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setdirlayout_latency": { 2024-05-21T03:34:22.215 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:22.215 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:22.215 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:22.215 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.215 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setfilelock_latency": { 2024-05-21T03:34:22.216 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 4, 2024-05-21T03:34:22.216 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.001165161, 2024-05-21T03:34:22.216 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000291290 2024-05-21T03:34:22.216 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.216 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setlayout_latency": { 2024-05-21T03:34:22.216 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:22.216 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:22.216 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:22.216 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.216 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setxattr_latency": { 2024-05-21T03:34:22.216 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 7, 2024-05-21T03:34:22.216 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.020335489, 2024-05-21T03:34:22.216 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.002905069 2024-05-21T03:34:22.216 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.216 INFO:teuthology.orchestra.run.smithi039.stdout: "req_symlink_latency": { 2024-05-21T03:34:22.217 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 13, 2024-05-21T03:34:22.217 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.008049478, 2024-05-21T03:34:22.217 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000619190 2024-05-21T03:34:22.217 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.217 INFO:teuthology.orchestra.run.smithi039.stdout: "req_unlink_latency": { 2024-05-21T03:34:22.217 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 507, 2024-05-21T03:34:22.217 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.894503115, 2024-05-21T03:34:22.217 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.001764305 2024-05-21T03:34:22.217 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.217 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_revoke_eviction": 0, 2024-05-21T03:34:22.217 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_acquisition_throttle": 0, 2024-05-21T03:34:22.217 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getvxattr_latency": { 2024-05-21T03:34:22.217 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:22.217 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:22.217 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:22.218 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:22.218 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.218 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_sessions": { 2024-05-21T03:34:22.218 INFO:teuthology.orchestra.run.smithi039.stdout: "session_count": 2, 2024-05-21T03:34:22.218 INFO:teuthology.orchestra.run.smithi039.stdout: "session_add": 6, 2024-05-21T03:34:22.218 INFO:teuthology.orchestra.run.smithi039.stdout: "session_remove": 4, 2024-05-21T03:34:22.218 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_open": 2, 2024-05-21T03:34:22.218 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_stale": 0, 2024-05-21T03:34:22.218 INFO:teuthology.orchestra.run.smithi039.stdout: "total_load": 61, 2024-05-21T03:34:22.218 INFO:teuthology.orchestra.run.smithi039.stdout: "average_load": 30, 2024-05-21T03:34:22.218 INFO:teuthology.orchestra.run.smithi039.stdout: "avg_session_uptime": 523, 2024-05-21T03:34:22.218 INFO:teuthology.orchestra.run.smithi039.stdout: "mdthresh_evicted": 0 2024-05-21T03:34:22.218 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.218 INFO:teuthology.orchestra.run.smithi039.stdout: "mempool": { 2024-05-21T03:34:22.218 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_bytes": 0, 2024-05-21T03:34:22.219 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_items": 0, 2024-05-21T03:34:22.219 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T03:34:22.219 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_items": 0, 2024-05-21T03:34:22.219 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T03:34:22.219 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_items": 0, 2024-05-21T03:34:22.219 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T03:34:22.219 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T03:34:22.219 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T03:34:22.219 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T03:34:22.219 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T03:34:22.219 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_items": 0, 2024-05-21T03:34:22.219 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T03:34:22.219 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T03:34:22.219 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_bytes": 0, 2024-05-21T03:34:22.220 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_items": 0, 2024-05-21T03:34:22.220 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_bytes": 0, 2024-05-21T03:34:22.220 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_items": 0, 2024-05-21T03:34:22.220 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T03:34:22.220 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T03:34:22.220 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T03:34:22.220 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T03:34:22.220 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T03:34:22.220 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_items": 0, 2024-05-21T03:34:22.220 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_bytes": 0, 2024-05-21T03:34:22.220 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_items": 0, 2024-05-21T03:34:22.220 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T03:34:22.220 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T03:34:22.220 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_bytes": 0, 2024-05-21T03:34:22.221 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_items": 0, 2024-05-21T03:34:22.221 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_bytes": 0, 2024-05-21T03:34:22.221 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_items": 0, 2024-05-21T03:34:22.221 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T03:34:22.221 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_items": 0, 2024-05-21T03:34:22.221 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T03:34:22.221 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_items": 0, 2024-05-21T03:34:22.221 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_bytes": 406556, 2024-05-21T03:34:22.221 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_items": 740, 2024-05-21T03:34:22.221 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_bytes": 0, 2024-05-21T03:34:22.221 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_items": 0, 2024-05-21T03:34:22.221 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bytes": 0, 2024-05-21T03:34:22.221 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_items": 0, 2024-05-21T03:34:22.221 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_bytes": 0, 2024-05-21T03:34:22.222 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_items": 0, 2024-05-21T03:34:22.222 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_bytes": 0, 2024-05-21T03:34:22.222 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_items": 0, 2024-05-21T03:34:22.222 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_bytes": 8224, 2024-05-21T03:34:22.222 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_items": 171, 2024-05-21T03:34:22.222 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T03:34:22.222 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_items": 0, 2024-05-21T03:34:22.222 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_bytes": 0, 2024-05-21T03:34:22.222 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_items": 0, 2024-05-21T03:34:22.222 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_bytes": 2497572, 2024-05-21T03:34:22.222 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_items": 22170, 2024-05-21T03:34:22.222 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_bytes": 0, 2024-05-21T03:34:22.222 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_items": 0, 2024-05-21T03:34:22.222 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_bytes": 0, 2024-05-21T03:34:22.222 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_items": 0 2024-05-21T03:34:22.222 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.223 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter": { 2024-05-21T03:34:22.223 INFO:teuthology.orchestra.run.smithi039.stdout: "op_active": 0, 2024-05-21T03:34:22.223 INFO:teuthology.orchestra.run.smithi039.stdout: "op_laggy": 0, 2024-05-21T03:34:22.223 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send": 3819, 2024-05-21T03:34:22.223 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send_bytes": 23152753, 2024-05-21T03:34:22.223 INFO:teuthology.orchestra.run.smithi039.stdout: "op_resend": 33, 2024-05-21T03:34:22.223 INFO:teuthology.orchestra.run.smithi039.stdout: "op_reply": 3819, 2024-05-21T03:34:22.223 INFO:teuthology.orchestra.run.smithi039.stdout: "op_latency": { 2024-05-21T03:34:22.223 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 3819, 2024-05-21T03:34:22.223 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 578.295371953, 2024-05-21T03:34:22.223 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.151425863 2024-05-21T03:34:22.223 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.223 INFO:teuthology.orchestra.run.smithi039.stdout: "op_inflight": 0, 2024-05-21T03:34:22.224 INFO:teuthology.orchestra.run.smithi039.stdout: "oplen_avg": { 2024-05-21T03:34:22.224 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 3819, 2024-05-21T03:34:22.224 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 4018 2024-05-21T03:34:22.224 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.224 INFO:teuthology.orchestra.run.smithi039.stdout: "op": 3819, 2024-05-21T03:34:22.224 INFO:teuthology.orchestra.run.smithi039.stdout: "op_r": 14, 2024-05-21T03:34:22.224 INFO:teuthology.orchestra.run.smithi039.stdout: "op_w": 3805, 2024-05-21T03:34:22.224 INFO:teuthology.orchestra.run.smithi039.stdout: "op_rmw": 0, 2024-05-21T03:34:22.224 INFO:teuthology.orchestra.run.smithi039.stdout: "op_pg": 0, 2024-05-21T03:34:22.224 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_stat": 0, 2024-05-21T03:34:22.224 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_create": 11, 2024-05-21T03:34:22.224 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_read": 14, 2024-05-21T03:34:22.224 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_write": 1946, 2024-05-21T03:34:22.224 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writefull": 50, 2024-05-21T03:34:22.224 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writesame": 0, 2024-05-21T03:34:22.224 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_append": 0, 2024-05-21T03:34:22.225 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_zero": 0, 2024-05-21T03:34:22.225 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_truncate": 1, 2024-05-21T03:34:22.225 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_delete": 1636, 2024-05-21T03:34:22.225 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_mapext": 0, 2024-05-21T03:34:22.225 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_sparse_read": 0, 2024-05-21T03:34:22.225 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_clonerange": 0, 2024-05-21T03:34:22.225 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_getxattr": 0, 2024-05-21T03:34:22.225 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_setxattr": 22, 2024-05-21T03:34:22.225 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_cmpxattr": 0, 2024-05-21T03:34:22.225 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_rmxattr": 0, 2024-05-21T03:34:22.225 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_resetxattrs": 0, 2024-05-21T03:34:22.225 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_call": 0, 2024-05-21T03:34:22.225 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_watch": 0, 2024-05-21T03:34:22.225 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_notify": 0, 2024-05-21T03:34:22.226 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T03:34:22.226 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls": 0, 2024-05-21T03:34:22.226 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls_filter": 0, 2024-05-21T03:34:22.226 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_other": 6, 2024-05-21T03:34:22.226 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_active": 0, 2024-05-21T03:34:22.226 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_send": 0, 2024-05-21T03:34:22.226 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_resend": 0, 2024-05-21T03:34:22.226 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_ping": 0, 2024-05-21T03:34:22.226 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_active": 0, 2024-05-21T03:34:22.226 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_send": 0, 2024-05-21T03:34:22.226 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_resend": 0, 2024-05-21T03:34:22.226 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_active": 0, 2024-05-21T03:34:22.226 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_send": 0, 2024-05-21T03:34:22.227 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_resend": 0, 2024-05-21T03:34:22.227 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_active": 0, 2024-05-21T03:34:22.227 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_send": 0, 2024-05-21T03:34:22.227 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_resend": 0, 2024-05-21T03:34:22.227 INFO:teuthology.orchestra.run.smithi039.stdout: "command_active": 0, 2024-05-21T03:34:22.227 INFO:teuthology.orchestra.run.smithi039.stdout: "command_send": 0, 2024-05-21T03:34:22.227 INFO:teuthology.orchestra.run.smithi039.stdout: "command_resend": 0, 2024-05-21T03:34:22.227 INFO:teuthology.orchestra.run.smithi039.stdout: "map_epoch": 91, 2024-05-21T03:34:22.227 INFO:teuthology.orchestra.run.smithi039.stdout: "map_full": 0, 2024-05-21T03:34:22.227 INFO:teuthology.orchestra.run.smithi039.stdout: "map_inc": 11, 2024-05-21T03:34:22.227 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_sessions": 12, 2024-05-21T03:34:22.227 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_open": 12, 2024-05-21T03:34:22.227 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_close": 0, 2024-05-21T03:34:22.227 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_laggy": 0, 2024-05-21T03:34:22.228 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_wr": 242, 2024-05-21T03:34:22.228 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_rd": 0, 2024-05-21T03:34:22.228 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_del": 90 2024-05-21T03:34:22.228 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.228 INFO:teuthology.orchestra.run.smithi039.stdout: "oft": { 2024-05-21T03:34:22.228 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_objs": 1, 2024-05-21T03:34:22.228 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_kv_pairs": 0, 2024-05-21T03:34:22.228 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_updates": 0, 2024-05-21T03:34:22.228 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_removes": 0 2024-05-21T03:34:22.228 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.228 INFO:teuthology.orchestra.run.smithi039.stdout: "osd-slow-ops": { 2024-05-21T03:34:22.228 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_ops_count": 585 2024-05-21T03:34:22.228 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.228 INFO:teuthology.orchestra.run.smithi039.stdout: "purge_queue": { 2024-05-21T03:34:22.229 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops": 0, 2024-05-21T03:34:22.229 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops_high_water": 25, 2024-05-21T03:34:22.229 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing": 0, 2024-05-21T03:34:22.229 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_high_water": 6, 2024-05-21T03:34:22.229 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed_ops": 1119, 2024-05-21T03:34:22.229 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed": 495, 2024-05-21T03:34:22.229 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_item_in_journal": 0 2024-05-21T03:34:22.229 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.229 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T03:34:22.229 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-21T03:34:22.229 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-05-21T03:34:22.229 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-21T03:34:22.229 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 27266, 2024-05-21T03:34:22.229 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 9741120, 2024-05-21T03:34:22.230 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:22.230 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 27266, 2024-05-21T03:34:22.230 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-05-21T03:34:22.230 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-05-21T03:34:22.230 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 27266, 2024-05-21T03:34:22.230 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 9741120, 2024-05-21T03:34:22.230 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-21T03:34:22.230 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:22.230 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:22.230 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:22.230 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:22.230 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.230 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_bytes": { 2024-05-21T03:34:22.230 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-21T03:34:22.230 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-05-21T03:34:22.230 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-21T03:34:22.231 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-21T03:34:22.231 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-21T03:34:22.231 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:22.231 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-21T03:34:22.231 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 3819, 2024-05-21T03:34:22.231 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 23203788, 2024-05-21T03:34:22.231 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 3819, 2024-05-21T03:34:22.231 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 23203788, 2024-05-21T03:34:22.231 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-21T03:34:22.231 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:22.231 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:22.231 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:22.231 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:22.231 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.231 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_ops": { 2024-05-21T03:34:22.232 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-21T03:34:22.232 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 1024, 2024-05-21T03:34:22.232 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-21T03:34:22.232 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-21T03:34:22.232 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-21T03:34:22.232 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:22.232 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-21T03:34:22.232 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 3819, 2024-05-21T03:34:22.232 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 3819, 2024-05-21T03:34:22.232 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 3819, 2024-05-21T03:34:22.232 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 3819, 2024-05-21T03:34:22.232 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-21T03:34:22.232 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:22.232 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:22.232 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:22.232 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:22.233 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.233 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle": { 2024-05-21T03:34:22.233 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-21T03:34:22.233 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-05-21T03:34:22.233 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-21T03:34:22.233 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 495, 2024-05-21T03:34:22.233 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 51035, 2024-05-21T03:34:22.233 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:22.233 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 495, 2024-05-21T03:34:22.233 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-05-21T03:34:22.233 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-05-21T03:34:22.233 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 14, 2024-05-21T03:34:22.233 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 51035, 2024-05-21T03:34:22.233 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-21T03:34:22.233 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:22.234 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:22.234 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:22.234 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:22.234 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:22.234 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle-0x55a214f36500": { 2024-05-21T03:34:22.234 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-21T03:34:22.234 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-05-21T03:34:22.234 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-21T03:34:22.234 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 9086, 2024-05-21T03:34:22.234 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 22831276, 2024-05-21T03:34:22.234 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:22.234 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 9086, 2024-05-21T03:34:22.234 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-05-21T03:34:22.234 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-05-21T03:34:22.234 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 1932, 2024-05-21T03:34:22.234 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 22831276, 2024-05-21T03:34:22.235 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-21T03:34:22.235 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:22.235 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:22.235 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:22.235 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:22.235 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:22.235 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-21T03:34:22.415 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:22 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:22.416 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:22 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:22.416 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:22 smithi027 ceph-mon[32179]: pgmap v321: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2024-05-21T03:34:22.416 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:22 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:22.416 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:22 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:22.416 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:22 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:22.416 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:22 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:22.416 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:22 smithi027 ceph-mon[32179]: Adjusting osd_memory_target on smithi027 to 176.1M 2024-05-21T03:34:22.416 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:22 smithi027 ceph-mon[32179]: Unable to set osd_memory_target on smithi027 to 184745164: error parsing value: Value '184745164' is below minimum 939524096 2024-05-21T03:34:22.416 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:22 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:34:22.416 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:22 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:34:22.416 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:22 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:22.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:22 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:22.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:22 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:22.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:22 smithi165 ceph-mon[36269]: pgmap v321: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2024-05-21T03:34:22.521 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:22 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:22.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:22 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:22.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:22 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:22.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:22 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T03:34:22.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:22 smithi165 ceph-mon[36269]: Adjusting osd_memory_target on smithi027 to 176.1M 2024-05-21T03:34:22.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:22 smithi165 ceph-mon[36269]: Unable to set osd_memory_target on smithi027 to 184745164: error parsing value: Value '184745164' is below minimum 939524096 2024-05-21T03:34:22.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:22 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T03:34:22.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:22 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T03:34:22.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:22 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' 2024-05-21T03:34:22.982 INFO:tasks.check_counter:Daemon mds.b mds.exported=11 2024-05-21T03:34:22.982 INFO:tasks.check_counter:Daemon mds.b mds.imported=14 2024-05-21T03:34:22.982 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=347 2024-05-21T03:34:22.982 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=220 2024-05-21T03:34:22.982 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-05-21T03:34:22.982 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-05-21T03:34:22.983 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-05-21T03:34:22.983 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=1 2024-05-21T03:34:22.983 DEBUG:tasks.check_counter:Getting stats from e 2024-05-21T03:34:22.983 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-21T03:34:23.234 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T03:34:24.713 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:24 smithi039 ceph-mon[36429]: pgmap v322: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2024-05-21T03:34:24.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:24 smithi027 ceph-mon[32179]: pgmap v322: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2024-05-21T03:34:25.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:24 smithi165 ceph-mon[36269]: pgmap v322: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2024-05-21T03:34:25.100 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-21T03:34:25.100 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T03:34:25.101 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 1520, 2024-05-21T03:34:25.101 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 1523, 2024-05-21T03:34:25.101 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 8835785, 2024-05-21T03:34:25.101 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 483447, 2024-05-21T03:34:25.101 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 7, 2024-05-21T03:34:25.101 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 3, 2024-05-21T03:34:25.101 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.147051919, 2024-05-21T03:34:25.101 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.049563074, 2024-05-21T03:34:25.101 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.038756814, 2024-05-21T03:34:25.101 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.045596843, 2024-05-21T03:34:25.101 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:25.101 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 1521, 2024-05-21T03:34:25.101 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.061485639, 2024-05-21T03:34:25.101 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000040424 2024-05-21T03:34:25.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.102 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:25.102 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.102 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.102 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.102 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-21T03:34:25.102 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 0 2024-05-21T03:34:25.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.102 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T03:34:25.102 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 69, 2024-05-21T03:34:25.102 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 242, 2024-05-21T03:34:25.102 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 8408442, 2024-05-21T03:34:25.103 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 487828, 2024-05-21T03:34:25.103 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 7, 2024-05-21T03:34:25.103 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 5, 2024-05-21T03:34:25.103 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.026386600, 2024-05-21T03:34:25.103 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.007946282, 2024-05-21T03:34:25.103 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.014155154, 2024-05-21T03:34:25.103 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.002044235, 2024-05-21T03:34:25.103 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:25.103 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 240, 2024-05-21T03:34:25.103 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.025677724, 2024-05-21T03:34:25.103 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000106990 2024-05-21T03:34:25.103 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.103 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:25.103 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.103 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.104 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.104 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-21T03:34:25.104 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 467728 2024-05-21T03:34:25.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.104 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T03:34:25.104 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 1034, 2024-05-21T03:34:25.104 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 999, 2024-05-21T03:34:25.104 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 8429767, 2024-05-21T03:34:25.104 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 384213, 2024-05-21T03:34:25.104 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 4, 2024-05-21T03:34:25.104 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 3, 2024-05-21T03:34:25.104 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.123667229, 2024-05-21T03:34:25.105 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.031777231, 2024-05-21T03:34:25.105 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.070705588, 2024-05-21T03:34:25.105 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.030382338, 2024-05-21T03:34:25.105 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:25.105 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 998, 2024-05-21T03:34:25.105 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.035745913, 2024-05-21T03:34:25.105 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000035817 2024-05-21T03:34:25.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.105 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:25.105 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.105 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.105 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.105 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 2187616, 2024-05-21T03:34:25.106 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 145712 2024-05-21T03:34:25.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.106 INFO:teuthology.orchestra.run.smithi039.stdout: "cct": { 2024-05-21T03:34:25.106 INFO:teuthology.orchestra.run.smithi039.stdout: "total_workers": 1, 2024-05-21T03:34:25.106 INFO:teuthology.orchestra.run.smithi039.stdout: "unhealthy_workers": 0 2024-05-21T03:34:25.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.106 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-MDSRank": { 2024-05-21T03:34:25.106 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-05-21T03:34:25.106 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-05-21T03:34:25.106 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 3895, 2024-05-21T03:34:25.106 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.099898421, 2024-05-21T03:34:25.106 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000025647 2024-05-21T03:34:25.106 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:25.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.107 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-PurgeQueue": { 2024-05-21T03:34:25.107 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-05-21T03:34:25.107 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-05-21T03:34:25.107 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.107 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.107 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.107 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:25.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.107 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": { 2024-05-21T03:34:25.107 INFO:teuthology.orchestra.run.smithi039.stdout: "request": 0, 2024-05-21T03:34:25.107 INFO:teuthology.orchestra.run.smithi039.stdout: "reply": 0, 2024-05-21T03:34:25.107 INFO:teuthology.orchestra.run.smithi039.stdout: "reply_latency": { 2024-05-21T03:34:25.107 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.107 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.107 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.108 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.108 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_reply": 0, 2024-05-21T03:34:25.108 INFO:teuthology.orchestra.run.smithi039.stdout: "forward": 0, 2024-05-21T03:34:25.108 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_complete": 0, 2024-05-21T03:34:25.108 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_keys": 0, 2024-05-21T03:34:25.108 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_commit": 0, 2024-05-21T03:34:25.108 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_split": 0, 2024-05-21T03:34:25.108 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_merge": 0, 2024-05-21T03:34:25.108 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes": 1530, 2024-05-21T03:34:25.108 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_top": 44, 2024-05-21T03:34:25.108 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_bottom": 1486, 2024-05-21T03:34:25.108 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pin_tail": 0, 2024-05-21T03:34:25.108 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pinned": 1467, 2024-05-21T03:34:25.108 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_expired": 0, 2024-05-21T03:34:25.109 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_with_caps": 0, 2024-05-21T03:34:25.109 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": 0, 2024-05-21T03:34:25.109 INFO:teuthology.orchestra.run.smithi039.stdout: "subtrees": 2, 2024-05-21T03:34:25.109 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse": 0, 2024-05-21T03:34:25.109 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_hit": 0, 2024-05-21T03:34:25.109 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_forward": 0, 2024-05-21T03:34:25.109 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_discover": 0, 2024-05-21T03:34:25.109 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_dir_fetch": 0, 2024-05-21T03:34:25.109 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_remote_ino": 0, 2024-05-21T03:34:25.109 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_lock": 0, 2024-05-21T03:34:25.109 INFO:teuthology.orchestra.run.smithi039.stdout: "load_cent": 0, 2024-05-21T03:34:25.109 INFO:teuthology.orchestra.run.smithi039.stdout: "q": 0, 2024-05-21T03:34:25.109 INFO:teuthology.orchestra.run.smithi039.stdout: "exported": 0, 2024-05-21T03:34:25.109 INFO:teuthology.orchestra.run.smithi039.stdout: "exported_inodes": 0, 2024-05-21T03:34:25.110 INFO:teuthology.orchestra.run.smithi039.stdout: "imported": 0, 2024-05-21T03:34:25.110 INFO:teuthology.orchestra.run.smithi039.stdout: "imported_inodes": 0, 2024-05-21T03:34:25.110 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_dir_fetch": 0, 2024-05-21T03:34:25.110 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_backtrace_fetch": 0, 2024-05-21T03:34:25.110 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_peer_discover": 0, 2024-05-21T03:34:25.110 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rfiles": 4, 2024-05-21T03:34:25.110 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rbytes": 234, 2024-05-21T03:34:25.110 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rsnaps": 6, 2024-05-21T03:34:25.110 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_fetch": 0, 2024-05-21T03:34:25.110 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_set_tag": 0, 2024-05-21T03:34:25.110 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T03:34:25.110 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_inotable_repaired": 0, 2024-05-21T03:34:25.110 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_inodes": 0, 2024-05-21T03:34:25.110 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T03:34:25.110 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T03:34:25.111 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_file_inodes": 0, 2024-05-21T03:34:25.111 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_inode_file_caps": 0, 2024-05-21T03:34:25.111 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_revoke": 0, 2024-05-21T03:34:25.111 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_grant": 0, 2024-05-21T03:34:25.111 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T03:34:25.111 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T03:34:25.111 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T03:34:25.111 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps": 0, 2024-05-21T03:34:25.111 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps_dirty": 0, 2024-05-21T03:34:25.111 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_cap_release": 0, 2024-05-21T03:34:25.111 INFO:teuthology.orchestra.run.smithi039.stdout: "process_request_cap_release": 0 2024-05-21T03:34:25.111 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.112 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache": { 2024-05-21T03:34:25.112 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update": 0, 2024-05-21T03:34:25.112 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update_receipt": 0, 2024-05-21T03:34:25.112 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_try_discover": 0, 2024-05-21T03:34:25.112 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_send_discover": 0, 2024-05-21T03:34:25.112 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_handle_discover": 0, 2024-05-21T03:34:25.112 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays": 0, 2024-05-21T03:34:25.112 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_delayed": 0, 2024-05-21T03:34:25.112 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_enqueuing": 0, 2024-05-21T03:34:25.112 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_created": 0, 2024-05-21T03:34:25.112 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_enqueued": 0, 2024-05-21T03:34:25.112 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_reintegrated": 0, 2024-05-21T03:34:25.112 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_migrated": 0, 2024-05-21T03:34:25.112 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_processing": 0, 2024-05-21T03:34:25.112 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_enqueued": 0, 2024-05-21T03:34:25.113 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_prioritized": 0, 2024-05-21T03:34:25.113 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_started": 0, 2024-05-21T03:34:25.113 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_completed": 0, 2024-05-21T03:34:25.113 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_quiesce_path": 0, 2024-05-21T03:34:25.113 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_quiesce_inode": 0, 2024-05-21T03:34:25.113 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T03:34:25.113 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_exportdir": 0, 2024-05-21T03:34:25.113 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_flush": 0, 2024-05-21T03:34:25.113 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragmentdir": 0, 2024-05-21T03:34:25.113 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragstats": 0, 2024-05-21T03:34:25.113 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_inodestats": 0 2024-05-21T03:34:25.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.113 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log": { 2024-05-21T03:34:25.113 INFO:teuthology.orchestra.run.smithi039.stdout: "evlrg": 0, 2024-05-21T03:34:25.113 INFO:teuthology.orchestra.run.smithi039.stdout: "evadd": 0, 2024-05-21T03:34:25.114 INFO:teuthology.orchestra.run.smithi039.stdout: "evex": 0, 2024-05-21T03:34:25.114 INFO:teuthology.orchestra.run.smithi039.stdout: "evtrm": 0, 2024-05-21T03:34:25.114 INFO:teuthology.orchestra.run.smithi039.stdout: "ev": 9086, 2024-05-21T03:34:25.114 INFO:teuthology.orchestra.run.smithi039.stdout: "evexg": 0, 2024-05-21T03:34:25.114 INFO:teuthology.orchestra.run.smithi039.stdout: "evexd": 0, 2024-05-21T03:34:25.114 INFO:teuthology.orchestra.run.smithi039.stdout: "segadd": 0, 2024-05-21T03:34:25.114 INFO:teuthology.orchestra.run.smithi039.stdout: "segex": 0, 2024-05-21T03:34:25.114 INFO:teuthology.orchestra.run.smithi039.stdout: "segtrm": 0, 2024-05-21T03:34:25.114 INFO:teuthology.orchestra.run.smithi039.stdout: "seg": 12, 2024-05-21T03:34:25.114 INFO:teuthology.orchestra.run.smithi039.stdout: "segmjr": 2, 2024-05-21T03:34:25.114 INFO:teuthology.orchestra.run.smithi039.stdout: "segexg": 0, 2024-05-21T03:34:25.114 INFO:teuthology.orchestra.run.smithi039.stdout: "segexd": 0, 2024-05-21T03:34:25.114 INFO:teuthology.orchestra.run.smithi039.stdout: "expos": 4194304, 2024-05-21T03:34:25.114 INFO:teuthology.orchestra.run.smithi039.stdout: "wrpos": 27025580, 2024-05-21T03:34:25.114 INFO:teuthology.orchestra.run.smithi039.stdout: "rdpos": 27025468, 2024-05-21T03:34:25.115 INFO:teuthology.orchestra.run.smithi039.stdout: "jlat": { 2024-05-21T03:34:25.115 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.115 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.115 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.115 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.115 INFO:teuthology.orchestra.run.smithi039.stdout: "replayed": 9086 2024-05-21T03:34:25.115 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.115 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_mem": { 2024-05-21T03:34:25.115 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 698, 2024-05-21T03:34:25.115 INFO:teuthology.orchestra.run.smithi039.stdout: "ino+": 1572, 2024-05-21T03:34:25.115 INFO:teuthology.orchestra.run.smithi039.stdout: "ino-": 874, 2024-05-21T03:34:25.115 INFO:teuthology.orchestra.run.smithi039.stdout: "dir": 31, 2024-05-21T03:34:25.115 INFO:teuthology.orchestra.run.smithi039.stdout: "dir+": 126, 2024-05-21T03:34:25.116 INFO:teuthology.orchestra.run.smithi039.stdout: "dir-": 95, 2024-05-21T03:34:25.116 INFO:teuthology.orchestra.run.smithi039.stdout: "dn": 1530, 2024-05-21T03:34:25.116 INFO:teuthology.orchestra.run.smithi039.stdout: "dn+": 2092, 2024-05-21T03:34:25.116 INFO:teuthology.orchestra.run.smithi039.stdout: "dn-": 562, 2024-05-21T03:34:25.116 INFO:teuthology.orchestra.run.smithi039.stdout: "cap": 0, 2024-05-21T03:34:25.116 INFO:teuthology.orchestra.run.smithi039.stdout: "cap+": 0, 2024-05-21T03:34:25.116 INFO:teuthology.orchestra.run.smithi039.stdout: "cap-": 0, 2024-05-21T03:34:25.116 INFO:teuthology.orchestra.run.smithi039.stdout: "rss": 55536, 2024-05-21T03:34:25.116 INFO:teuthology.orchestra.run.smithi039.stdout: "heap": 215416 2024-05-21T03:34:25.116 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.116 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_server": { 2024-05-21T03:34:25.116 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_client_request": 0, 2024-05-21T03:34:25.116 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_server_request": 0, 2024-05-21T03:34:25.116 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_request": 0, 2024-05-21T03:34:25.116 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_session": 0, 2024-05-21T03:34:25.117 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_peer_request": 0, 2024-05-21T03:34:25.117 INFO:teuthology.orchestra.run.smithi039.stdout: "req_create_latency": { 2024-05-21T03:34:25.117 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.117 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.117 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.117 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getattr_latency": { 2024-05-21T03:34:25.117 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.117 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.117 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.117 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getfilelock_latency": { 2024-05-21T03:34:25.117 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.117 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.117 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.118 INFO:teuthology.orchestra.run.smithi039.stdout: "req_link_latency": { 2024-05-21T03:34:25.118 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.118 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.118 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.118 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookup_latency": { 2024-05-21T03:34:25.118 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.118 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.118 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.118 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookuphash_latency": { 2024-05-21T03:34:25.118 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.118 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.118 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.119 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupino_latency": { 2024-05-21T03:34:25.119 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.119 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.119 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.119 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupname_latency": { 2024-05-21T03:34:25.119 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.119 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.119 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.119 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupparent_latency": { 2024-05-21T03:34:25.119 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.119 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.119 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.120 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupsnap_latency": { 2024-05-21T03:34:25.120 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.120 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.120 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.120 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lssnap_latency": { 2024-05-21T03:34:25.120 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.120 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.120 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.120 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mkdir_latency": { 2024-05-21T03:34:25.121 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.121 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.121 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.121 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mknod_latency": { 2024-05-21T03:34:25.121 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.121 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.121 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.121 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mksnap_latency": { 2024-05-21T03:34:25.121 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.122 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.122 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.122 INFO:teuthology.orchestra.run.smithi039.stdout: "req_open_latency": { 2024-05-21T03:34:25.122 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.122 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.122 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.122 INFO:teuthology.orchestra.run.smithi039.stdout: "req_readdir_latency": { 2024-05-21T03:34:25.122 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.122 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.122 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.122 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rename_latency": { 2024-05-21T03:34:25.123 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.123 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.123 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.123 INFO:teuthology.orchestra.run.smithi039.stdout: "req_renamesnap_latency": { 2024-05-21T03:34:25.123 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.123 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.123 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.123 INFO:teuthology.orchestra.run.smithi039.stdout: "req_snapdiff_latency": { 2024-05-21T03:34:25.123 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.123 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.123 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.124 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmdir_latency": { 2024-05-21T03:34:25.124 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.124 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.124 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.124 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmsnap_latency": { 2024-05-21T03:34:25.124 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.124 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.124 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.124 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmxattr_latency": { 2024-05-21T03:34:25.125 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.125 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.125 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.125 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.125 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setattr_latency": { 2024-05-21T03:34:25.125 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.125 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.125 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.125 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.125 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setdirlayout_latency": { 2024-05-21T03:34:25.125 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.125 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.125 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.125 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.126 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setfilelock_latency": { 2024-05-21T03:34:25.126 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.126 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.126 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.126 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.126 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setlayout_latency": { 2024-05-21T03:34:25.126 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.126 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.126 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.126 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.126 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setxattr_latency": { 2024-05-21T03:34:25.126 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.126 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.126 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.126 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.127 INFO:teuthology.orchestra.run.smithi039.stdout: "req_symlink_latency": { 2024-05-21T03:34:25.127 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.127 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.127 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.127 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.127 INFO:teuthology.orchestra.run.smithi039.stdout: "req_unlink_latency": { 2024-05-21T03:34:25.127 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.127 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.127 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.127 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.127 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_revoke_eviction": 0, 2024-05-21T03:34:25.127 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_acquisition_throttle": 0, 2024-05-21T03:34:25.127 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getvxattr_latency": { 2024-05-21T03:34:25.128 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.128 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.128 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.128 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:25.128 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.128 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_sessions": { 2024-05-21T03:34:25.128 INFO:teuthology.orchestra.run.smithi039.stdout: "session_count": 2, 2024-05-21T03:34:25.128 INFO:teuthology.orchestra.run.smithi039.stdout: "session_add": 6, 2024-05-21T03:34:25.128 INFO:teuthology.orchestra.run.smithi039.stdout: "session_remove": 4, 2024-05-21T03:34:25.128 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_open": 2, 2024-05-21T03:34:25.128 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_stale": 0, 2024-05-21T03:34:25.128 INFO:teuthology.orchestra.run.smithi039.stdout: "total_load": 0, 2024-05-21T03:34:25.128 INFO:teuthology.orchestra.run.smithi039.stdout: "average_load": 0, 2024-05-21T03:34:25.128 INFO:teuthology.orchestra.run.smithi039.stdout: "avg_session_uptime": 7161, 2024-05-21T03:34:25.128 INFO:teuthology.orchestra.run.smithi039.stdout: "mdthresh_evicted": 0 2024-05-21T03:34:25.129 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.129 INFO:teuthology.orchestra.run.smithi039.stdout: "mempool": { 2024-05-21T03:34:25.129 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_bytes": 0, 2024-05-21T03:34:25.129 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_items": 0, 2024-05-21T03:34:25.129 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T03:34:25.129 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_items": 0, 2024-05-21T03:34:25.129 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T03:34:25.129 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_items": 0, 2024-05-21T03:34:25.129 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T03:34:25.129 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T03:34:25.129 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T03:34:25.129 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T03:34:25.130 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T03:34:25.130 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_items": 0, 2024-05-21T03:34:25.130 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T03:34:25.130 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T03:34:25.130 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_bytes": 0, 2024-05-21T03:34:25.130 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_items": 0, 2024-05-21T03:34:25.130 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_bytes": 0, 2024-05-21T03:34:25.130 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_items": 0, 2024-05-21T03:34:25.130 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T03:34:25.130 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T03:34:25.130 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T03:34:25.130 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T03:34:25.130 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T03:34:25.131 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_items": 0, 2024-05-21T03:34:25.131 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_bytes": 0, 2024-05-21T03:34:25.131 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_items": 0, 2024-05-21T03:34:25.131 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T03:34:25.131 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T03:34:25.131 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_bytes": 0, 2024-05-21T03:34:25.131 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_items": 0, 2024-05-21T03:34:25.131 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_bytes": 0, 2024-05-21T03:34:25.131 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_items": 0, 2024-05-21T03:34:25.131 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T03:34:25.131 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_items": 0, 2024-05-21T03:34:25.131 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T03:34:25.131 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_items": 0, 2024-05-21T03:34:25.131 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_bytes": 234075, 2024-05-21T03:34:25.132 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_items": 771, 2024-05-21T03:34:25.132 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_bytes": 0, 2024-05-21T03:34:25.132 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_items": 0, 2024-05-21T03:34:25.132 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bytes": 0, 2024-05-21T03:34:25.132 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_items": 0, 2024-05-21T03:34:25.132 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_bytes": 0, 2024-05-21T03:34:25.132 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_items": 0, 2024-05-21T03:34:25.132 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_bytes": 0, 2024-05-21T03:34:25.132 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_items": 0, 2024-05-21T03:34:25.132 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_bytes": 8224, 2024-05-21T03:34:25.132 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_items": 171, 2024-05-21T03:34:25.132 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T03:34:25.132 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_items": 0, 2024-05-21T03:34:25.133 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_bytes": 0, 2024-05-21T03:34:25.133 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_items": 0, 2024-05-21T03:34:25.133 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_bytes": 2767749, 2024-05-21T03:34:25.133 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_items": 22289, 2024-05-21T03:34:25.133 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_bytes": 0, 2024-05-21T03:34:25.133 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_items": 0, 2024-05-21T03:34:25.133 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_bytes": 0, 2024-05-21T03:34:25.133 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_items": 0 2024-05-21T03:34:25.133 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.133 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter": { 2024-05-21T03:34:25.133 INFO:teuthology.orchestra.run.smithi039.stdout: "op_active": 0, 2024-05-21T03:34:25.133 INFO:teuthology.orchestra.run.smithi039.stdout: "op_laggy": 0, 2024-05-21T03:34:25.133 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send": 2333, 2024-05-21T03:34:25.134 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send_bytes": 26, 2024-05-21T03:34:25.134 INFO:teuthology.orchestra.run.smithi039.stdout: "op_resend": 4, 2024-05-21T03:34:25.134 INFO:teuthology.orchestra.run.smithi039.stdout: "op_reply": 2333, 2024-05-21T03:34:25.134 INFO:teuthology.orchestra.run.smithi039.stdout: "op_latency": { 2024-05-21T03:34:25.134 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 2333, 2024-05-21T03:34:25.134 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 101.012142297, 2024-05-21T03:34:25.134 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.043297103 2024-05-21T03:34:25.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.134 INFO:teuthology.orchestra.run.smithi039.stdout: "op_inflight": 0, 2024-05-21T03:34:25.134 INFO:teuthology.orchestra.run.smithi039.stdout: "oplen_avg": { 2024-05-21T03:34:25.134 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 2333, 2024-05-21T03:34:25.134 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 2334 2024-05-21T03:34:25.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.135 INFO:teuthology.orchestra.run.smithi039.stdout: "op": 2333, 2024-05-21T03:34:25.135 INFO:teuthology.orchestra.run.smithi039.stdout: "op_r": 2333, 2024-05-21T03:34:25.135 INFO:teuthology.orchestra.run.smithi039.stdout: "op_w": 0, 2024-05-21T03:34:25.135 INFO:teuthology.orchestra.run.smithi039.stdout: "op_rmw": 0, 2024-05-21T03:34:25.135 INFO:teuthology.orchestra.run.smithi039.stdout: "op_pg": 0, 2024-05-21T03:34:25.135 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_stat": 1431, 2024-05-21T03:34:25.135 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_create": 0, 2024-05-21T03:34:25.135 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_read": 899, 2024-05-21T03:34:25.135 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_write": 0, 2024-05-21T03:34:25.135 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writefull": 0, 2024-05-21T03:34:25.136 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writesame": 0, 2024-05-21T03:34:25.136 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_append": 0, 2024-05-21T03:34:25.136 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_zero": 0, 2024-05-21T03:34:25.136 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_truncate": 0, 2024-05-21T03:34:25.136 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_delete": 0, 2024-05-21T03:34:25.136 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_mapext": 0, 2024-05-21T03:34:25.136 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_sparse_read": 0, 2024-05-21T03:34:25.136 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_clonerange": 0, 2024-05-21T03:34:25.136 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_getxattr": 2, 2024-05-21T03:34:25.136 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_setxattr": 0, 2024-05-21T03:34:25.136 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_cmpxattr": 0, 2024-05-21T03:34:25.136 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_rmxattr": 0, 2024-05-21T03:34:25.136 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_resetxattrs": 0, 2024-05-21T03:34:25.137 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_call": 0, 2024-05-21T03:34:25.137 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_watch": 0, 2024-05-21T03:34:25.137 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_notify": 0, 2024-05-21T03:34:25.137 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T03:34:25.137 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls": 0, 2024-05-21T03:34:25.137 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls_filter": 0, 2024-05-21T03:34:25.137 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_other": 0, 2024-05-21T03:34:25.137 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_active": 0, 2024-05-21T03:34:25.137 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_send": 0, 2024-05-21T03:34:25.137 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_resend": 0, 2024-05-21T03:34:25.137 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_ping": 0, 2024-05-21T03:34:25.137 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_active": 0, 2024-05-21T03:34:25.137 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_send": 0, 2024-05-21T03:34:25.137 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_resend": 0, 2024-05-21T03:34:25.137 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_active": 0, 2024-05-21T03:34:25.138 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_send": 0, 2024-05-21T03:34:25.138 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_resend": 0, 2024-05-21T03:34:25.138 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_active": 0, 2024-05-21T03:34:25.138 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_send": 0, 2024-05-21T03:34:25.138 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_resend": 0, 2024-05-21T03:34:25.138 INFO:teuthology.orchestra.run.smithi039.stdout: "command_active": 0, 2024-05-21T03:34:25.138 INFO:teuthology.orchestra.run.smithi039.stdout: "command_send": 0, 2024-05-21T03:34:25.138 INFO:teuthology.orchestra.run.smithi039.stdout: "command_resend": 0, 2024-05-21T03:34:25.139 INFO:teuthology.orchestra.run.smithi039.stdout: "map_epoch": 91, 2024-05-21T03:34:25.139 INFO:teuthology.orchestra.run.smithi039.stdout: "map_full": 0, 2024-05-21T03:34:25.139 INFO:teuthology.orchestra.run.smithi039.stdout: "map_inc": 11, 2024-05-21T03:34:25.139 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_sessions": 7, 2024-05-21T03:34:25.139 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_open": 7, 2024-05-21T03:34:25.139 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_close": 0, 2024-05-21T03:34:25.140 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_laggy": 0, 2024-05-21T03:34:25.140 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_wr": 0, 2024-05-21T03:34:25.140 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_rd": 2, 2024-05-21T03:34:25.140 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_del": 0 2024-05-21T03:34:25.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.140 INFO:teuthology.orchestra.run.smithi039.stdout: "oft": { 2024-05-21T03:34:25.140 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_objs": 0, 2024-05-21T03:34:25.140 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_kv_pairs": 0, 2024-05-21T03:34:25.140 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_updates": 0, 2024-05-21T03:34:25.141 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_removes": 0 2024-05-21T03:34:25.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.141 INFO:teuthology.orchestra.run.smithi039.stdout: "osd-slow-ops": { 2024-05-21T03:34:25.141 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_ops_count": 0 2024-05-21T03:34:25.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.141 INFO:teuthology.orchestra.run.smithi039.stdout: "purge_queue": { 2024-05-21T03:34:25.141 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops": 0, 2024-05-21T03:34:25.141 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops_high_water": 0, 2024-05-21T03:34:25.141 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing": 0, 2024-05-21T03:34:25.141 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_high_water": 0, 2024-05-21T03:34:25.141 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed_ops": 0, 2024-05-21T03:34:25.141 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed": 0, 2024-05-21T03:34:25.141 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_item_in_journal": 0 2024-05-21T03:34:25.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.141 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T03:34:25.142 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-21T03:34:25.142 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-05-21T03:34:25.142 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-21T03:34:25.142 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 2619, 2024-05-21T03:34:25.142 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 25420846, 2024-05-21T03:34:25.142 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:25.142 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 2619, 2024-05-21T03:34:25.142 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-05-21T03:34:25.142 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-05-21T03:34:25.142 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 2619, 2024-05-21T03:34:25.142 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 25420846, 2024-05-21T03:34:25.142 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-21T03:34:25.142 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.142 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.143 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.143 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:25.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.143 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_bytes": { 2024-05-21T03:34:25.143 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-21T03:34:25.143 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-05-21T03:34:25.144 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-21T03:34:25.144 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-21T03:34:25.144 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-21T03:34:25.144 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:25.144 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-21T03:34:25.145 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 2333, 2024-05-21T03:34:25.145 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 22831286, 2024-05-21T03:34:25.145 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 2333, 2024-05-21T03:34:25.145 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 22831286, 2024-05-21T03:34:25.145 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-21T03:34:25.145 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.146 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.146 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.146 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:25.146 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.146 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_ops": { 2024-05-21T03:34:25.146 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-21T03:34:25.146 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 1024, 2024-05-21T03:34:25.146 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-21T03:34:25.146 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-21T03:34:25.146 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-21T03:34:25.146 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:25.147 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-21T03:34:25.147 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 2333, 2024-05-21T03:34:25.147 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 2333, 2024-05-21T03:34:25.147 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 2333, 2024-05-21T03:34:25.147 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 2333, 2024-05-21T03:34:25.147 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-21T03:34:25.147 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.147 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.148 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.148 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:25.148 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.148 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle": { 2024-05-21T03:34:25.148 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-21T03:34:25.148 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-05-21T03:34:25.148 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-21T03:34:25.148 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-21T03:34:25.148 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-21T03:34:25.148 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:25.148 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-21T03:34:25.148 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-05-21T03:34:25.148 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-05-21T03:34:25.148 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 0, 2024-05-21T03:34:25.148 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 0, 2024-05-21T03:34:25.149 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-21T03:34:25.149 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.149 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.149 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.149 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:25.149 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:25.149 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle-0x55f7f491c6e0": { 2024-05-21T03:34:25.149 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-21T03:34:25.149 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-05-21T03:34:25.149 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-21T03:34:25.149 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-21T03:34:25.149 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-21T03:34:25.149 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:25.149 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-21T03:34:25.149 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-05-21T03:34:25.150 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-05-21T03:34:25.150 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 0, 2024-05-21T03:34:25.150 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 0, 2024-05-21T03:34:25.150 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-21T03:34:25.150 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:25.150 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:25.150 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:25.150 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:25.150 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:25.150 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-21T03:34:25.748 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-21T03:34:25.748 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-21T03:34:25.748 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-05-21T03:34:25.749 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-05-21T03:34:25.749 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-05-21T03:34:25.749 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-05-21T03:34:25.749 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-05-21T03:34:25.749 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-05-21T03:34:25.749 DEBUG:tasks.check_counter:Getting stats from h 2024-05-21T03:34:25.749 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-21T03:34:25.997 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T03:34:26.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:26 smithi039 ceph-mon[36429]: pgmap v323: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T03:34:26.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:26 smithi027 ceph-mon[32179]: pgmap v323: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T03:34:27.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:26 smithi165 ceph-mon[36269]: pgmap v323: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T03:34:27.893 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-21T03:34:27.893 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T03:34:27.894 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 3099, 2024-05-21T03:34:27.894 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 3275, 2024-05-21T03:34:27.894 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 1144485, 2024-05-21T03:34:27.894 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 1412082, 2024-05-21T03:34:27.894 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 13, 2024-05-21T03:34:27.894 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 3, 2024-05-21T03:34:27.894 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.283783209, 2024-05-21T03:34:27.894 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.126975681, 2024-05-21T03:34:27.894 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.144109243, 2024-05-21T03:34:27.894 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.018192605, 2024-05-21T03:34:27.894 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:27.894 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 3276, 2024-05-21T03:34:27.895 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.212031346, 2024-05-21T03:34:27.895 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000064722 2024-05-21T03:34:27.895 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.895 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:27.895 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 5368, 2024-05-21T03:34:27.895 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.001526742, 2024-05-21T03:34:27.895 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000284 2024-05-21T03:34:27.895 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.895 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-21T03:34:27.895 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 448 2024-05-21T03:34:27.895 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.895 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T03:34:27.895 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 3006, 2024-05-21T03:34:27.896 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 2185, 2024-05-21T03:34:27.896 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 3477927, 2024-05-21T03:34:27.896 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 3403988, 2024-05-21T03:34:27.896 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 10, 2024-05-21T03:34:27.896 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 7, 2024-05-21T03:34:27.896 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.208642439, 2024-05-21T03:34:27.896 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.070008870, 2024-05-21T03:34:27.896 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.152071750, 2024-05-21T03:34:27.896 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.027810513, 2024-05-21T03:34:27.896 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:27.896 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 2184, 2024-05-21T03:34:27.896 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.233574383, 2024-05-21T03:34:27.896 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000106947 2024-05-21T03:34:27.896 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.897 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:27.897 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 3234, 2024-05-21T03:34:27.897 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000575445, 2024-05-21T03:34:27.897 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000177 2024-05-21T03:34:27.897 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.897 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 2212080, 2024-05-21T03:34:27.897 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 164320 2024-05-21T03:34:27.897 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.897 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T03:34:27.897 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 2775, 2024-05-21T03:34:27.897 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 3411, 2024-05-21T03:34:27.897 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 1118166, 2024-05-21T03:34:27.897 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 2009531, 2024-05-21T03:34:27.897 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 21, 2024-05-21T03:34:27.898 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 9, 2024-05-21T03:34:27.898 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.220384778, 2024-05-21T03:34:27.898 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.084313072, 2024-05-21T03:34:27.898 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.193046878, 2024-05-21T03:34:27.898 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.010943631, 2024-05-21T03:34:27.898 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:27.898 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 3406, 2024-05-21T03:34:27.898 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.315046912, 2024-05-21T03:34:27.898 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000092497 2024-05-21T03:34:27.898 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.898 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:27.898 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 4868, 2024-05-21T03:34:27.898 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000876494, 2024-05-21T03:34:27.899 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000180 2024-05-21T03:34:27.899 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.899 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 928544, 2024-05-21T03:34:27.899 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 1708320 2024-05-21T03:34:27.899 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.899 INFO:teuthology.orchestra.run.smithi039.stdout: "cct": { 2024-05-21T03:34:27.899 INFO:teuthology.orchestra.run.smithi039.stdout: "total_workers": 1, 2024-05-21T03:34:27.899 INFO:teuthology.orchestra.run.smithi039.stdout: "unhealthy_workers": 0 2024-05-21T03:34:27.899 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.899 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-MDSRank": { 2024-05-21T03:34:27.899 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-05-21T03:34:27.899 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-05-21T03:34:27.899 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 557, 2024-05-21T03:34:27.899 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.439644072, 2024-05-21T03:34:27.900 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000789307 2024-05-21T03:34:27.900 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:27.900 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.900 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-PurgeQueue": { 2024-05-21T03:34:27.900 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-05-21T03:34:27.900 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-05-21T03:34:27.900 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 840, 2024-05-21T03:34:27.900 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.028916380, 2024-05-21T03:34:27.900 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000034424 2024-05-21T03:34:27.900 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:27.901 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.901 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": { 2024-05-21T03:34:27.901 INFO:teuthology.orchestra.run.smithi039.stdout: "request": 211, 2024-05-21T03:34:27.901 INFO:teuthology.orchestra.run.smithi039.stdout: "reply": 187, 2024-05-21T03:34:27.901 INFO:teuthology.orchestra.run.smithi039.stdout: "reply_latency": { 2024-05-21T03:34:27.901 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 187, 2024-05-21T03:34:27.901 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 49.539920681, 2024-05-21T03:34:27.901 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.264919361 2024-05-21T03:34:27.901 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.901 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_reply": 0, 2024-05-21T03:34:27.901 INFO:teuthology.orchestra.run.smithi039.stdout: "forward": 24, 2024-05-21T03:34:27.901 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_complete": 0, 2024-05-21T03:34:27.902 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_keys": 0, 2024-05-21T03:34:27.902 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_commit": 11, 2024-05-21T03:34:27.902 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_split": 0, 2024-05-21T03:34:27.902 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_merge": 0, 2024-05-21T03:34:27.902 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes": 16, 2024-05-21T03:34:27.902 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_top": 2, 2024-05-21T03:34:27.902 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_bottom": 4, 2024-05-21T03:34:27.902 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pin_tail": 10, 2024-05-21T03:34:27.902 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pinned": 13, 2024-05-21T03:34:27.902 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_expired": 374, 2024-05-21T03:34:27.902 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_with_caps": 1, 2024-05-21T03:34:27.902 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": 1, 2024-05-21T03:34:27.903 INFO:teuthology.orchestra.run.smithi039.stdout: "subtrees": 4, 2024-05-21T03:34:27.903 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse": 629, 2024-05-21T03:34:27.903 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_hit": 406, 2024-05-21T03:34:27.903 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_forward": 22, 2024-05-21T03:34:27.903 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_discover": 128, 2024-05-21T03:34:27.903 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_dir_fetch": 0, 2024-05-21T03:34:27.903 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_remote_ino": 0, 2024-05-21T03:34:27.903 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_lock": 3, 2024-05-21T03:34:27.903 INFO:teuthology.orchestra.run.smithi039.stdout: "load_cent": 0, 2024-05-21T03:34:27.903 INFO:teuthology.orchestra.run.smithi039.stdout: "q": 0, 2024-05-21T03:34:27.903 INFO:teuthology.orchestra.run.smithi039.stdout: "exported": 2, 2024-05-21T03:34:27.903 INFO:teuthology.orchestra.run.smithi039.stdout: "exported_inodes": 172, 2024-05-21T03:34:27.903 INFO:teuthology.orchestra.run.smithi039.stdout: "imported": 2, 2024-05-21T03:34:27.904 INFO:teuthology.orchestra.run.smithi039.stdout: "imported_inodes": 31, 2024-05-21T03:34:27.904 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_dir_fetch": 0, 2024-05-21T03:34:27.904 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_backtrace_fetch": 0, 2024-05-21T03:34:27.904 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_peer_discover": 0, 2024-05-21T03:34:27.904 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rfiles": 325, 2024-05-21T03:34:27.904 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rbytes": 13424230508, 2024-05-21T03:34:27.904 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rsnaps": 6, 2024-05-21T03:34:27.904 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_fetch": 0, 2024-05-21T03:34:27.904 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_set_tag": 0, 2024-05-21T03:34:27.904 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T03:34:27.904 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_inotable_repaired": 0, 2024-05-21T03:34:27.904 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_inodes": 0, 2024-05-21T03:34:27.904 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T03:34:27.904 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T03:34:27.905 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_file_inodes": 0, 2024-05-21T03:34:27.905 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_inode_file_caps": 14, 2024-05-21T03:34:27.905 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_revoke": 253, 2024-05-21T03:34:27.905 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_grant": 484, 2024-05-21T03:34:27.905 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T03:34:27.905 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T03:34:27.905 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T03:34:27.905 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps": 2155, 2024-05-21T03:34:27.905 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps_dirty": 1808, 2024-05-21T03:34:27.905 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_cap_release": 14, 2024-05-21T03:34:27.905 INFO:teuthology.orchestra.run.smithi039.stdout: "process_request_cap_release": 65 2024-05-21T03:34:27.905 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.905 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache": { 2024-05-21T03:34:27.906 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update": 8, 2024-05-21T03:34:27.906 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update_receipt": 366, 2024-05-21T03:34:27.906 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_try_discover": 248, 2024-05-21T03:34:27.906 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_send_discover": 227, 2024-05-21T03:34:27.906 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_handle_discover": 8, 2024-05-21T03:34:27.906 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays": 0, 2024-05-21T03:34:27.906 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_delayed": 0, 2024-05-21T03:34:27.906 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_enqueuing": 0, 2024-05-21T03:34:27.906 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_created": 31, 2024-05-21T03:34:27.906 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_enqueued": 31, 2024-05-21T03:34:27.906 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_reintegrated": 0, 2024-05-21T03:34:27.906 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_migrated": 0, 2024-05-21T03:34:27.906 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_processing": 0, 2024-05-21T03:34:27.907 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_enqueued": 0, 2024-05-21T03:34:27.907 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_prioritized": 0, 2024-05-21T03:34:27.907 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_started": 0, 2024-05-21T03:34:27.907 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_completed": 0, 2024-05-21T03:34:27.907 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_quiesce_path": 5, 2024-05-21T03:34:27.907 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_quiesce_inode": 353, 2024-05-21T03:34:27.907 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T03:34:27.907 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_exportdir": 3, 2024-05-21T03:34:27.907 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_flush": 0, 2024-05-21T03:34:27.907 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragmentdir": 0, 2024-05-21T03:34:27.907 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragstats": 0, 2024-05-21T03:34:27.907 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_inodestats": 0 2024-05-21T03:34:27.907 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.907 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log": { 2024-05-21T03:34:27.908 INFO:teuthology.orchestra.run.smithi039.stdout: "evlrg": 0, 2024-05-21T03:34:27.908 INFO:teuthology.orchestra.run.smithi039.stdout: "evadd": 2071, 2024-05-21T03:34:27.908 INFO:teuthology.orchestra.run.smithi039.stdout: "evex": 0, 2024-05-21T03:34:27.908 INFO:teuthology.orchestra.run.smithi039.stdout: "evtrm": 0, 2024-05-21T03:34:27.908 INFO:teuthology.orchestra.run.smithi039.stdout: "ev": 2071, 2024-05-21T03:34:27.908 INFO:teuthology.orchestra.run.smithi039.stdout: "evexg": 0, 2024-05-21T03:34:27.908 INFO:teuthology.orchestra.run.smithi039.stdout: "evexd": 0, 2024-05-21T03:34:27.908 INFO:teuthology.orchestra.run.smithi039.stdout: "segadd": 4, 2024-05-21T03:34:27.908 INFO:teuthology.orchestra.run.smithi039.stdout: "segex": 0, 2024-05-21T03:34:27.908 INFO:teuthology.orchestra.run.smithi039.stdout: "segtrm": 0, 2024-05-21T03:34:27.908 INFO:teuthology.orchestra.run.smithi039.stdout: "seg": 4, 2024-05-21T03:34:27.908 INFO:teuthology.orchestra.run.smithi039.stdout: "segmjr": 2, 2024-05-21T03:34:27.908 INFO:teuthology.orchestra.run.smithi039.stdout: "segexg": 0, 2024-05-21T03:34:27.908 INFO:teuthology.orchestra.run.smithi039.stdout: "segexd": 0, 2024-05-21T03:34:27.909 INFO:teuthology.orchestra.run.smithi039.stdout: "expos": 4194304, 2024-05-21T03:34:27.909 INFO:teuthology.orchestra.run.smithi039.stdout: "wrpos": 6873021, 2024-05-21T03:34:27.909 INFO:teuthology.orchestra.run.smithi039.stdout: "rdpos": 0, 2024-05-21T03:34:27.909 INFO:teuthology.orchestra.run.smithi039.stdout: "jlat": { 2024-05-21T03:34:27.909 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 251, 2024-05-21T03:34:27.909 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 457.522347296, 2024-05-21T03:34:27.909 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 1.822798196 2024-05-21T03:34:27.909 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.909 INFO:teuthology.orchestra.run.smithi039.stdout: "replayed": 0 2024-05-21T03:34:27.909 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.909 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_mem": { 2024-05-21T03:34:27.909 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 21, 2024-05-21T03:34:27.909 INFO:teuthology.orchestra.run.smithi039.stdout: "ino+": 310, 2024-05-21T03:34:27.909 INFO:teuthology.orchestra.run.smithi039.stdout: "ino-": 289, 2024-05-21T03:34:27.910 INFO:teuthology.orchestra.run.smithi039.stdout: "dir": 20, 2024-05-21T03:34:27.910 INFO:teuthology.orchestra.run.smithi039.stdout: "dir+": 153, 2024-05-21T03:34:27.910 INFO:teuthology.orchestra.run.smithi039.stdout: "dir-": 133, 2024-05-21T03:34:27.910 INFO:teuthology.orchestra.run.smithi039.stdout: "dn": 16, 2024-05-21T03:34:27.910 INFO:teuthology.orchestra.run.smithi039.stdout: "dn+": 421, 2024-05-21T03:34:27.910 INFO:teuthology.orchestra.run.smithi039.stdout: "dn-": 405, 2024-05-21T03:34:27.910 INFO:teuthology.orchestra.run.smithi039.stdout: "cap": 1, 2024-05-21T03:34:27.910 INFO:teuthology.orchestra.run.smithi039.stdout: "cap+": 108, 2024-05-21T03:34:27.910 INFO:teuthology.orchestra.run.smithi039.stdout: "cap-": 107, 2024-05-21T03:34:27.910 INFO:teuthology.orchestra.run.smithi039.stdout: "rss": 64120, 2024-05-21T03:34:27.910 INFO:teuthology.orchestra.run.smithi039.stdout: "heap": 223608 2024-05-21T03:34:27.910 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.910 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_server": { 2024-05-21T03:34:27.910 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_client_request": 360, 2024-05-21T03:34:27.911 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_server_request": 102, 2024-05-21T03:34:27.911 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_request": 211, 2024-05-21T03:34:27.911 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_session": 28, 2024-05-21T03:34:27.911 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_peer_request": 498, 2024-05-21T03:34:27.911 INFO:teuthology.orchestra.run.smithi039.stdout: "req_create_latency": { 2024-05-21T03:34:27.911 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:27.911 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:27.911 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:27.911 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.911 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getattr_latency": { 2024-05-21T03:34:27.911 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 23, 2024-05-21T03:34:27.911 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 20.157140256, 2024-05-21T03:34:27.911 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.876397402 2024-05-21T03:34:27.912 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.912 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getfilelock_latency": { 2024-05-21T03:34:27.912 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:27.912 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:27.912 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:27.912 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.912 INFO:teuthology.orchestra.run.smithi039.stdout: "req_link_latency": { 2024-05-21T03:34:27.912 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:27.912 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:27.912 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:27.912 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.912 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookup_latency": { 2024-05-21T03:34:27.912 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 69, 2024-05-21T03:34:27.913 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 25.794759655, 2024-05-21T03:34:27.913 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.373837096 2024-05-21T03:34:27.913 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.913 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookuphash_latency": { 2024-05-21T03:34:27.913 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:27.913 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:27.913 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:27.913 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.913 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupino_latency": { 2024-05-21T03:34:27.913 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:27.913 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:27.913 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:27.913 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.914 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupname_latency": { 2024-05-21T03:34:27.914 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:27.914 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:27.914 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:27.914 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.914 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupparent_latency": { 2024-05-21T03:34:27.914 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:27.914 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:27.914 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:27.914 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.914 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupsnap_latency": { 2024-05-21T03:34:27.914 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:27.914 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:27.914 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:27.915 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.915 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lssnap_latency": { 2024-05-21T03:34:27.915 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:27.915 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:27.915 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:27.916 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.916 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mkdir_latency": { 2024-05-21T03:34:27.916 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:27.916 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:27.916 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:27.916 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.916 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mknod_latency": { 2024-05-21T03:34:27.916 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:27.916 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:27.916 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:27.916 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.917 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mksnap_latency": { 2024-05-21T03:34:27.917 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:27.917 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:27.917 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:27.917 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.917 INFO:teuthology.orchestra.run.smithi039.stdout: "req_open_latency": { 2024-05-21T03:34:27.917 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 60, 2024-05-21T03:34:27.917 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 3.506618586, 2024-05-21T03:34:27.917 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.058443643 2024-05-21T03:34:27.917 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.917 INFO:teuthology.orchestra.run.smithi039.stdout: "req_readdir_latency": { 2024-05-21T03:34:27.917 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 4, 2024-05-21T03:34:27.917 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.002777415, 2024-05-21T03:34:27.918 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000694353 2024-05-21T03:34:27.918 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.918 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rename_latency": { 2024-05-21T03:34:27.918 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:27.918 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:27.918 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:27.918 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.918 INFO:teuthology.orchestra.run.smithi039.stdout: "req_renamesnap_latency": { 2024-05-21T03:34:27.918 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:27.918 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:27.918 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:27.918 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.918 INFO:teuthology.orchestra.run.smithi039.stdout: "req_snapdiff_latency": { 2024-05-21T03:34:27.918 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:27.918 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:27.919 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:27.919 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.919 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmdir_latency": { 2024-05-21T03:34:27.919 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:27.919 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:27.919 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:27.919 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.919 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmsnap_latency": { 2024-05-21T03:34:27.919 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:27.919 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:27.919 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:27.919 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.919 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmxattr_latency": { 2024-05-21T03:34:27.919 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:27.920 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:27.920 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:27.920 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.920 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setattr_latency": { 2024-05-21T03:34:27.920 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:27.920 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:27.920 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:27.920 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.920 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setdirlayout_latency": { 2024-05-21T03:34:27.920 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:27.920 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:27.920 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:27.920 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.920 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setfilelock_latency": { 2024-05-21T03:34:27.920 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:27.921 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:27.921 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:27.921 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.921 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setlayout_latency": { 2024-05-21T03:34:27.921 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:27.921 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:27.921 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:27.921 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.921 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setxattr_latency": { 2024-05-21T03:34:27.921 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:27.921 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:27.921 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:27.921 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.921 INFO:teuthology.orchestra.run.smithi039.stdout: "req_symlink_latency": { 2024-05-21T03:34:27.922 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:27.922 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:27.922 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:27.922 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.922 INFO:teuthology.orchestra.run.smithi039.stdout: "req_unlink_latency": { 2024-05-21T03:34:27.922 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 31, 2024-05-21T03:34:27.922 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.078624769, 2024-05-21T03:34:27.922 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.002536282 2024-05-21T03:34:27.922 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.922 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_revoke_eviction": 0, 2024-05-21T03:34:27.922 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_acquisition_throttle": 0, 2024-05-21T03:34:27.922 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getvxattr_latency": { 2024-05-21T03:34:27.922 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:27.922 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:27.922 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:27.923 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:27.923 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.923 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_sessions": { 2024-05-21T03:34:27.923 INFO:teuthology.orchestra.run.smithi039.stdout: "session_count": 2, 2024-05-21T03:34:27.923 INFO:teuthology.orchestra.run.smithi039.stdout: "session_add": 5, 2024-05-21T03:34:27.923 INFO:teuthology.orchestra.run.smithi039.stdout: "session_remove": 3, 2024-05-21T03:34:27.923 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_open": 2, 2024-05-21T03:34:27.923 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_stale": 0, 2024-05-21T03:34:27.923 INFO:teuthology.orchestra.run.smithi039.stdout: "total_load": 93, 2024-05-21T03:34:27.923 INFO:teuthology.orchestra.run.smithi039.stdout: "average_load": 46, 2024-05-21T03:34:27.923 INFO:teuthology.orchestra.run.smithi039.stdout: "avg_session_uptime": 535, 2024-05-21T03:34:27.923 INFO:teuthology.orchestra.run.smithi039.stdout: "mdthresh_evicted": 0 2024-05-21T03:34:27.923 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.923 INFO:teuthology.orchestra.run.smithi039.stdout: "mempool": { 2024-05-21T03:34:27.923 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_bytes": 0, 2024-05-21T03:34:27.924 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_items": 0, 2024-05-21T03:34:27.924 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T03:34:27.924 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_items": 0, 2024-05-21T03:34:27.924 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T03:34:27.924 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_items": 0, 2024-05-21T03:34:27.924 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T03:34:27.924 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T03:34:27.924 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T03:34:27.924 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T03:34:27.924 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T03:34:27.924 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_items": 0, 2024-05-21T03:34:27.924 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T03:34:27.924 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T03:34:27.924 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_bytes": 0, 2024-05-21T03:34:27.924 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_items": 0, 2024-05-21T03:34:27.925 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_bytes": 0, 2024-05-21T03:34:27.925 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_items": 0, 2024-05-21T03:34:27.925 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T03:34:27.925 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T03:34:27.925 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T03:34:27.925 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T03:34:27.925 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T03:34:27.925 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_items": 0, 2024-05-21T03:34:27.925 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_bytes": 0, 2024-05-21T03:34:27.925 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_items": 0, 2024-05-21T03:34:27.925 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T03:34:27.925 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T03:34:27.925 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_bytes": 0, 2024-05-21T03:34:27.925 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_items": 0, 2024-05-21T03:34:27.925 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_bytes": 0, 2024-05-21T03:34:27.926 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_items": 0, 2024-05-21T03:34:27.926 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T03:34:27.926 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_items": 0, 2024-05-21T03:34:27.926 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T03:34:27.926 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_items": 0, 2024-05-21T03:34:27.926 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_bytes": 312743, 2024-05-21T03:34:27.926 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_items": 115, 2024-05-21T03:34:27.926 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_bytes": 0, 2024-05-21T03:34:27.926 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_items": 0, 2024-05-21T03:34:27.926 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bytes": 0, 2024-05-21T03:34:27.926 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_items": 0, 2024-05-21T03:34:27.926 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_bytes": 0, 2024-05-21T03:34:27.926 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_items": 0, 2024-05-21T03:34:27.927 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_bytes": 0, 2024-05-21T03:34:27.927 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_items": 0, 2024-05-21T03:34:27.927 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_bytes": 8224, 2024-05-21T03:34:27.927 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_items": 171, 2024-05-21T03:34:27.927 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T03:34:27.927 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_items": 0, 2024-05-21T03:34:27.927 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_bytes": 0, 2024-05-21T03:34:27.927 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_items": 0, 2024-05-21T03:34:27.927 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_bytes": 95541, 2024-05-21T03:34:27.927 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_items": 392, 2024-05-21T03:34:27.927 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_bytes": 0, 2024-05-21T03:34:27.927 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_items": 0, 2024-05-21T03:34:27.927 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_bytes": 0, 2024-05-21T03:34:27.927 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_items": 0 2024-05-21T03:34:27.928 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.928 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter": { 2024-05-21T03:34:27.928 INFO:teuthology.orchestra.run.smithi039.stdout: "op_active": 0, 2024-05-21T03:34:27.928 INFO:teuthology.orchestra.run.smithi039.stdout: "op_laggy": 0, 2024-05-21T03:34:27.928 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send": 1160, 2024-05-21T03:34:27.928 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send_bytes": 2707610, 2024-05-21T03:34:27.928 INFO:teuthology.orchestra.run.smithi039.stdout: "op_resend": 33, 2024-05-21T03:34:27.928 INFO:teuthology.orchestra.run.smithi039.stdout: "op_reply": 1160, 2024-05-21T03:34:27.928 INFO:teuthology.orchestra.run.smithi039.stdout: "op_latency": { 2024-05-21T03:34:27.928 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 1160, 2024-05-21T03:34:27.928 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 650.511047863, 2024-05-21T03:34:27.928 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.560785386 2024-05-21T03:34:27.928 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.928 INFO:teuthology.orchestra.run.smithi039.stdout: "op_inflight": 0, 2024-05-21T03:34:27.929 INFO:teuthology.orchestra.run.smithi039.stdout: "oplen_avg": { 2024-05-21T03:34:27.929 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 1160, 2024-05-21T03:34:27.929 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 1253 2024-05-21T03:34:27.929 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.929 INFO:teuthology.orchestra.run.smithi039.stdout: "op": 1160, 2024-05-21T03:34:27.929 INFO:teuthology.orchestra.run.smithi039.stdout: "op_r": 1, 2024-05-21T03:34:27.929 INFO:teuthology.orchestra.run.smithi039.stdout: "op_w": 1159, 2024-05-21T03:34:27.929 INFO:teuthology.orchestra.run.smithi039.stdout: "op_rmw": 0, 2024-05-21T03:34:27.929 INFO:teuthology.orchestra.run.smithi039.stdout: "op_pg": 0, 2024-05-21T03:34:27.929 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_stat": 0, 2024-05-21T03:34:27.929 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_create": 10, 2024-05-21T03:34:27.929 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_read": 1, 2024-05-21T03:34:27.929 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_write": 252, 2024-05-21T03:34:27.929 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writefull": 28, 2024-05-21T03:34:27.930 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writesame": 0, 2024-05-21T03:34:27.930 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_append": 0, 2024-05-21T03:34:27.930 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_zero": 0, 2024-05-21T03:34:27.930 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_truncate": 0, 2024-05-21T03:34:27.930 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_delete": 787, 2024-05-21T03:34:27.930 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_mapext": 0, 2024-05-21T03:34:27.930 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_sparse_read": 0, 2024-05-21T03:34:27.930 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_clonerange": 0, 2024-05-21T03:34:27.930 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_getxattr": 0, 2024-05-21T03:34:27.930 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_setxattr": 20, 2024-05-21T03:34:27.930 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_cmpxattr": 0, 2024-05-21T03:34:27.930 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_rmxattr": 0, 2024-05-21T03:34:27.930 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_resetxattrs": 0, 2024-05-21T03:34:27.931 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_call": 0, 2024-05-21T03:34:27.931 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_watch": 0, 2024-05-21T03:34:27.931 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_notify": 0, 2024-05-21T03:34:27.931 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T03:34:27.931 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls": 0, 2024-05-21T03:34:27.931 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls_filter": 0, 2024-05-21T03:34:27.931 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_other": 0, 2024-05-21T03:34:27.931 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_active": 0, 2024-05-21T03:34:27.931 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_send": 0, 2024-05-21T03:34:27.931 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_resend": 0, 2024-05-21T03:34:27.931 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_ping": 0, 2024-05-21T03:34:27.931 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_active": 0, 2024-05-21T03:34:27.931 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_send": 0, 2024-05-21T03:34:27.931 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_resend": 0, 2024-05-21T03:34:27.932 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_active": 0, 2024-05-21T03:34:27.932 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_send": 0, 2024-05-21T03:34:27.932 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_resend": 0, 2024-05-21T03:34:27.932 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_active": 0, 2024-05-21T03:34:27.932 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_send": 0, 2024-05-21T03:34:27.932 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_resend": 0, 2024-05-21T03:34:27.932 INFO:teuthology.orchestra.run.smithi039.stdout: "command_active": 0, 2024-05-21T03:34:27.932 INFO:teuthology.orchestra.run.smithi039.stdout: "command_send": 0, 2024-05-21T03:34:27.932 INFO:teuthology.orchestra.run.smithi039.stdout: "command_resend": 0, 2024-05-21T03:34:27.932 INFO:teuthology.orchestra.run.smithi039.stdout: "map_epoch": 91, 2024-05-21T03:34:27.932 INFO:teuthology.orchestra.run.smithi039.stdout: "map_full": 0, 2024-05-21T03:34:27.932 INFO:teuthology.orchestra.run.smithi039.stdout: "map_inc": 11, 2024-05-21T03:34:27.932 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_sessions": 12, 2024-05-21T03:34:27.933 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_open": 12, 2024-05-21T03:34:27.933 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_close": 0, 2024-05-21T03:34:27.933 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_laggy": 0, 2024-05-21T03:34:27.933 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_wr": 115, 2024-05-21T03:34:27.933 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_rd": 0, 2024-05-21T03:34:27.933 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_del": 40 2024-05-21T03:34:27.933 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.933 INFO:teuthology.orchestra.run.smithi039.stdout: "oft": { 2024-05-21T03:34:27.933 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_objs": 1, 2024-05-21T03:34:27.933 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_kv_pairs": 0, 2024-05-21T03:34:27.933 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_updates": 0, 2024-05-21T03:34:27.933 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_removes": 0 2024-05-21T03:34:27.933 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.933 INFO:teuthology.orchestra.run.smithi039.stdout: "osd-slow-ops": { 2024-05-21T03:34:27.934 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_ops_count": 451 2024-05-21T03:34:27.934 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.934 INFO:teuthology.orchestra.run.smithi039.stdout: "purge_queue": { 2024-05-21T03:34:27.934 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops": 0, 2024-05-21T03:34:27.934 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops_high_water": 25, 2024-05-21T03:34:27.934 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing": 0, 2024-05-21T03:34:27.934 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_high_water": 1, 2024-05-21T03:34:27.934 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed_ops": 775, 2024-05-21T03:34:27.934 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed": 31, 2024-05-21T03:34:27.934 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_item_in_journal": 0 2024-05-21T03:34:27.934 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.934 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T03:34:27.935 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-21T03:34:27.935 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-05-21T03:34:27.935 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-21T03:34:27.935 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 8876, 2024-05-21T03:34:27.935 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 4805917, 2024-05-21T03:34:27.935 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:27.935 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 8876, 2024-05-21T03:34:27.935 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-05-21T03:34:27.935 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-05-21T03:34:27.935 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 8876, 2024-05-21T03:34:27.935 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 4805917, 2024-05-21T03:34:27.935 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-21T03:34:27.936 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:27.936 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:27.936 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:27.936 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:27.936 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.936 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_bytes": { 2024-05-21T03:34:27.936 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-21T03:34:27.936 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-05-21T03:34:27.936 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-21T03:34:27.936 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-21T03:34:27.936 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-21T03:34:27.936 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:27.936 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-21T03:34:27.936 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 1160, 2024-05-21T03:34:27.937 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 2711981, 2024-05-21T03:34:27.937 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 1160, 2024-05-21T03:34:27.937 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 2711981, 2024-05-21T03:34:27.937 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-21T03:34:27.937 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:27.937 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:27.937 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:27.937 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:27.937 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.937 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_ops": { 2024-05-21T03:34:27.937 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-21T03:34:27.937 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 1024, 2024-05-21T03:34:27.937 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-21T03:34:27.937 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-21T03:34:27.937 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-21T03:34:27.938 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:27.938 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-21T03:34:27.938 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 1160, 2024-05-21T03:34:27.938 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 1160, 2024-05-21T03:34:27.938 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 1160, 2024-05-21T03:34:27.938 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 1160, 2024-05-21T03:34:27.938 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-21T03:34:27.938 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:27.938 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:27.938 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:27.938 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:27.939 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.939 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle": { 2024-05-21T03:34:27.939 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-21T03:34:27.939 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-05-21T03:34:27.939 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-21T03:34:27.939 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 31, 2024-05-21T03:34:27.939 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 4371, 2024-05-21T03:34:27.939 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:27.939 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 31, 2024-05-21T03:34:27.939 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-05-21T03:34:27.939 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-05-21T03:34:27.939 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 1, 2024-05-21T03:34:27.939 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 4371, 2024-05-21T03:34:27.939 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-21T03:34:27.940 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:27.940 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:27.940 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:27.940 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:27.940 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:27.940 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle-0x55fd30960500": { 2024-05-21T03:34:27.940 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-21T03:34:27.940 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-05-21T03:34:27.940 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-21T03:34:27.940 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 2071, 2024-05-21T03:34:27.940 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 2678717, 2024-05-21T03:34:27.940 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:27.940 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 2071, 2024-05-21T03:34:27.940 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-05-21T03:34:27.940 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-05-21T03:34:27.940 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 251, 2024-05-21T03:34:27.941 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 2678717, 2024-05-21T03:34:27.941 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-21T03:34:27.941 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:27.941 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:27.941 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:27.941 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:27.941 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:27.941 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-21T03:34:28.664 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:28 smithi039 ceph-mon[36429]: pgmap v324: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-05-21T03:34:28.693 INFO:tasks.check_counter:Daemon mds.h mds.exported=2 2024-05-21T03:34:28.693 INFO:tasks.check_counter:Daemon mds.h mds.imported=2 2024-05-21T03:34:28.693 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=8 2024-05-21T03:34:28.693 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=366 2024-05-21T03:34:28.693 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-05-21T03:34:28.693 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-05-21T03:34:28.694 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-05-21T03:34:28.694 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-05-21T03:34:28.694 DEBUG:tasks.check_counter:Getting stats from k 2024-05-21T03:34:28.694 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-21T03:34:28.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:28 smithi027 ceph-mon[32179]: pgmap v324: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-05-21T03:34:28.945 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T03:34:29.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:28 smithi165 ceph-mon[36269]: pgmap v324: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-05-21T03:34:30.712 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-21T03:34:30.713 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T03:34:30.713 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 853, 2024-05-21T03:34:30.713 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 855, 2024-05-21T03:34:30.713 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 4067597, 2024-05-21T03:34:30.713 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 271318, 2024-05-21T03:34:30.713 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 6, 2024-05-21T03:34:30.713 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 2, 2024-05-21T03:34:30.713 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.078680429, 2024-05-21T03:34:30.714 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.028097653, 2024-05-21T03:34:30.714 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.021216503, 2024-05-21T03:34:30.714 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.021641920, 2024-05-21T03:34:30.714 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:30.714 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 853, 2024-05-21T03:34:30.714 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.036835135, 2024-05-21T03:34:30.714 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000043183 2024-05-21T03:34:30.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.714 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:30.714 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.714 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.714 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.714 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-21T03:34:30.715 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 0 2024-05-21T03:34:30.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.715 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T03:34:30.715 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 683, 2024-05-21T03:34:30.715 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 860, 2024-05-21T03:34:30.715 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 169876, 2024-05-21T03:34:30.715 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 686955, 2024-05-21T03:34:30.715 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 7, 2024-05-21T03:34:30.715 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 5, 2024-05-21T03:34:30.715 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.072227555, 2024-05-21T03:34:30.715 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.023258080, 2024-05-21T03:34:30.715 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.016704488, 2024-05-21T03:34:30.715 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.025033435, 2024-05-21T03:34:30.715 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:30.716 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 858, 2024-05-21T03:34:30.716 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.039877845, 2024-05-21T03:34:30.716 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000046477 2024-05-21T03:34:30.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.716 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:30.716 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.716 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.716 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.716 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-21T03:34:30.716 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 471664 2024-05-21T03:34:30.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.716 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T03:34:30.716 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 1047, 2024-05-21T03:34:30.716 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 1010, 2024-05-21T03:34:30.717 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 2450406, 2024-05-21T03:34:30.717 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 389156, 2024-05-21T03:34:30.717 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 3, 2024-05-21T03:34:30.717 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 2, 2024-05-21T03:34:30.717 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.141353739, 2024-05-21T03:34:30.717 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.041517589, 2024-05-21T03:34:30.717 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.071855301, 2024-05-21T03:34:30.717 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.034555387, 2024-05-21T03:34:30.717 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:30.717 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 1009, 2024-05-21T03:34:30.717 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.050396556, 2024-05-21T03:34:30.717 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000049947 2024-05-21T03:34:30.717 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.717 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:30.718 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.718 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.718 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.718 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 2195296, 2024-05-21T03:34:30.718 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 148112 2024-05-21T03:34:30.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.718 INFO:teuthology.orchestra.run.smithi039.stdout: "cct": { 2024-05-21T03:34:30.718 INFO:teuthology.orchestra.run.smithi039.stdout: "total_workers": 1, 2024-05-21T03:34:30.718 INFO:teuthology.orchestra.run.smithi039.stdout: "unhealthy_workers": 0 2024-05-21T03:34:30.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.718 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-MDSRank": { 2024-05-21T03:34:30.718 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-05-21T03:34:30.718 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-05-21T03:34:30.719 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 3832, 2024-05-21T03:34:30.719 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.108019909, 2024-05-21T03:34:30.719 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000028188 2024-05-21T03:34:30.719 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:30.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.719 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-PurgeQueue": { 2024-05-21T03:34:30.719 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-05-21T03:34:30.719 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-05-21T03:34:30.719 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.719 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.719 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.719 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:30.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.719 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": { 2024-05-21T03:34:30.719 INFO:teuthology.orchestra.run.smithi039.stdout: "request": 0, 2024-05-21T03:34:30.720 INFO:teuthology.orchestra.run.smithi039.stdout: "reply": 0, 2024-05-21T03:34:30.720 INFO:teuthology.orchestra.run.smithi039.stdout: "reply_latency": { 2024-05-21T03:34:30.720 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.720 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.720 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.720 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_reply": 0, 2024-05-21T03:34:30.720 INFO:teuthology.orchestra.run.smithi039.stdout: "forward": 0, 2024-05-21T03:34:30.720 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_complete": 0, 2024-05-21T03:34:30.720 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_keys": 0, 2024-05-21T03:34:30.720 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_commit": 0, 2024-05-21T03:34:30.720 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_split": 0, 2024-05-21T03:34:30.720 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_merge": 0, 2024-05-21T03:34:30.720 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes": 21, 2024-05-21T03:34:30.721 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_top": 0, 2024-05-21T03:34:30.721 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_bottom": 21, 2024-05-21T03:34:30.721 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pin_tail": 0, 2024-05-21T03:34:30.721 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pinned": 21, 2024-05-21T03:34:30.721 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_expired": 0, 2024-05-21T03:34:30.721 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_with_caps": 0, 2024-05-21T03:34:30.721 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": 0, 2024-05-21T03:34:30.721 INFO:teuthology.orchestra.run.smithi039.stdout: "subtrees": 1, 2024-05-21T03:34:30.721 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse": 0, 2024-05-21T03:34:30.721 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_hit": 0, 2024-05-21T03:34:30.721 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_forward": 0, 2024-05-21T03:34:30.721 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_discover": 0, 2024-05-21T03:34:30.721 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_dir_fetch": 0, 2024-05-21T03:34:30.721 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_remote_ino": 0, 2024-05-21T03:34:30.721 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_lock": 0, 2024-05-21T03:34:30.722 INFO:teuthology.orchestra.run.smithi039.stdout: "load_cent": 0, 2024-05-21T03:34:30.722 INFO:teuthology.orchestra.run.smithi039.stdout: "q": 0, 2024-05-21T03:34:30.722 INFO:teuthology.orchestra.run.smithi039.stdout: "exported": 0, 2024-05-21T03:34:30.722 INFO:teuthology.orchestra.run.smithi039.stdout: "exported_inodes": 0, 2024-05-21T03:34:30.722 INFO:teuthology.orchestra.run.smithi039.stdout: "imported": 0, 2024-05-21T03:34:30.722 INFO:teuthology.orchestra.run.smithi039.stdout: "imported_inodes": 0, 2024-05-21T03:34:30.722 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_dir_fetch": 0, 2024-05-21T03:34:30.722 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_backtrace_fetch": 0, 2024-05-21T03:34:30.722 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_peer_discover": 0, 2024-05-21T03:34:30.722 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rfiles": 0, 2024-05-21T03:34:30.722 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rbytes": 0, 2024-05-21T03:34:30.722 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rsnaps": 0, 2024-05-21T03:34:30.722 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_fetch": 0, 2024-05-21T03:34:30.722 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_set_tag": 0, 2024-05-21T03:34:30.723 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T03:34:30.723 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_inotable_repaired": 0, 2024-05-21T03:34:30.723 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_inodes": 0, 2024-05-21T03:34:30.723 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T03:34:30.723 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T03:34:30.723 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_file_inodes": 0, 2024-05-21T03:34:30.723 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_inode_file_caps": 0, 2024-05-21T03:34:30.723 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_revoke": 0, 2024-05-21T03:34:30.723 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_grant": 0, 2024-05-21T03:34:30.723 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T03:34:30.723 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T03:34:30.723 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T03:34:30.724 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps": 0, 2024-05-21T03:34:30.724 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps_dirty": 0, 2024-05-21T03:34:30.724 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_cap_release": 0, 2024-05-21T03:34:30.724 INFO:teuthology.orchestra.run.smithi039.stdout: "process_request_cap_release": 0 2024-05-21T03:34:30.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.724 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache": { 2024-05-21T03:34:30.724 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update": 0, 2024-05-21T03:34:30.724 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update_receipt": 0, 2024-05-21T03:34:30.724 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_try_discover": 0, 2024-05-21T03:34:30.724 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_send_discover": 0, 2024-05-21T03:34:30.724 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_handle_discover": 0, 2024-05-21T03:34:30.724 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays": 0, 2024-05-21T03:34:30.725 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_delayed": 0, 2024-05-21T03:34:30.725 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_enqueuing": 0, 2024-05-21T03:34:30.725 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_created": 0, 2024-05-21T03:34:30.725 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_enqueued": 0, 2024-05-21T03:34:30.725 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_reintegrated": 0, 2024-05-21T03:34:30.725 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_migrated": 0, 2024-05-21T03:34:30.725 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_processing": 0, 2024-05-21T03:34:30.725 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_enqueued": 0, 2024-05-21T03:34:30.725 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_prioritized": 0, 2024-05-21T03:34:30.725 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_started": 0, 2024-05-21T03:34:30.725 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_completed": 0, 2024-05-21T03:34:30.725 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_quiesce_path": 0, 2024-05-21T03:34:30.725 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_quiesce_inode": 0, 2024-05-21T03:34:30.725 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T03:34:30.725 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_exportdir": 0, 2024-05-21T03:34:30.726 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_flush": 0, 2024-05-21T03:34:30.726 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragmentdir": 0, 2024-05-21T03:34:30.726 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragstats": 0, 2024-05-21T03:34:30.726 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_inodestats": 0 2024-05-21T03:34:30.726 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.726 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log": { 2024-05-21T03:34:30.726 INFO:teuthology.orchestra.run.smithi039.stdout: "evlrg": 0, 2024-05-21T03:34:30.726 INFO:teuthology.orchestra.run.smithi039.stdout: "evadd": 0, 2024-05-21T03:34:30.726 INFO:teuthology.orchestra.run.smithi039.stdout: "evex": 0, 2024-05-21T03:34:30.726 INFO:teuthology.orchestra.run.smithi039.stdout: "evtrm": 0, 2024-05-21T03:34:30.726 INFO:teuthology.orchestra.run.smithi039.stdout: "ev": 2986, 2024-05-21T03:34:30.726 INFO:teuthology.orchestra.run.smithi039.stdout: "evexg": 0, 2024-05-21T03:34:30.726 INFO:teuthology.orchestra.run.smithi039.stdout: "evexd": 0, 2024-05-21T03:34:30.726 INFO:teuthology.orchestra.run.smithi039.stdout: "segadd": 0, 2024-05-21T03:34:30.726 INFO:teuthology.orchestra.run.smithi039.stdout: "segex": 0, 2024-05-21T03:34:30.727 INFO:teuthology.orchestra.run.smithi039.stdout: "segtrm": 0, 2024-05-21T03:34:30.727 INFO:teuthology.orchestra.run.smithi039.stdout: "seg": 4, 2024-05-21T03:34:30.727 INFO:teuthology.orchestra.run.smithi039.stdout: "segmjr": 2, 2024-05-21T03:34:30.727 INFO:teuthology.orchestra.run.smithi039.stdout: "segexg": 0, 2024-05-21T03:34:30.727 INFO:teuthology.orchestra.run.smithi039.stdout: "segexd": 0, 2024-05-21T03:34:30.727 INFO:teuthology.orchestra.run.smithi039.stdout: "expos": 4194304, 2024-05-21T03:34:30.727 INFO:teuthology.orchestra.run.smithi039.stdout: "wrpos": 8039248, 2024-05-21T03:34:30.727 INFO:teuthology.orchestra.run.smithi039.stdout: "rdpos": 8036643, 2024-05-21T03:34:30.727 INFO:teuthology.orchestra.run.smithi039.stdout: "jlat": { 2024-05-21T03:34:30.727 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.727 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.728 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.728 INFO:teuthology.orchestra.run.smithi039.stdout: "replayed": 2986 2024-05-21T03:34:30.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.728 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_mem": { 2024-05-21T03:34:30.728 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 4, 2024-05-21T03:34:30.728 INFO:teuthology.orchestra.run.smithi039.stdout: "ino+": 237, 2024-05-21T03:34:30.728 INFO:teuthology.orchestra.run.smithi039.stdout: "ino-": 233, 2024-05-21T03:34:30.728 INFO:teuthology.orchestra.run.smithi039.stdout: "dir": 2, 2024-05-21T03:34:30.728 INFO:teuthology.orchestra.run.smithi039.stdout: "dir+": 22, 2024-05-21T03:34:30.728 INFO:teuthology.orchestra.run.smithi039.stdout: "dir-": 20, 2024-05-21T03:34:30.728 INFO:teuthology.orchestra.run.smithi039.stdout: "dn": 21, 2024-05-21T03:34:30.728 INFO:teuthology.orchestra.run.smithi039.stdout: "dn+": 254, 2024-05-21T03:34:30.728 INFO:teuthology.orchestra.run.smithi039.stdout: "dn-": 233, 2024-05-21T03:34:30.728 INFO:teuthology.orchestra.run.smithi039.stdout: "cap": 0, 2024-05-21T03:34:30.729 INFO:teuthology.orchestra.run.smithi039.stdout: "cap+": 0, 2024-05-21T03:34:30.729 INFO:teuthology.orchestra.run.smithi039.stdout: "cap-": 0, 2024-05-21T03:34:30.729 INFO:teuthology.orchestra.run.smithi039.stdout: "rss": 52160, 2024-05-21T03:34:30.729 INFO:teuthology.orchestra.run.smithi039.stdout: "heap": 215416 2024-05-21T03:34:30.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.729 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_server": { 2024-05-21T03:34:30.729 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_client_request": 0, 2024-05-21T03:34:30.729 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_server_request": 0, 2024-05-21T03:34:30.729 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_request": 0, 2024-05-21T03:34:30.729 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_session": 0, 2024-05-21T03:34:30.729 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_peer_request": 0, 2024-05-21T03:34:30.729 INFO:teuthology.orchestra.run.smithi039.stdout: "req_create_latency": { 2024-05-21T03:34:30.729 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.729 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.729 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.730 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getattr_latency": { 2024-05-21T03:34:30.730 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.730 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.730 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.730 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getfilelock_latency": { 2024-05-21T03:34:30.730 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.730 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.730 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.730 INFO:teuthology.orchestra.run.smithi039.stdout: "req_link_latency": { 2024-05-21T03:34:30.730 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.730 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.730 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.731 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookup_latency": { 2024-05-21T03:34:30.731 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.731 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.731 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.731 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookuphash_latency": { 2024-05-21T03:34:30.731 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.731 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.731 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.731 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupino_latency": { 2024-05-21T03:34:30.731 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.731 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.731 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.732 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupname_latency": { 2024-05-21T03:34:30.732 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.732 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.732 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.732 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupparent_latency": { 2024-05-21T03:34:30.732 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.732 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.732 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.732 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupsnap_latency": { 2024-05-21T03:34:30.732 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.732 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.732 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.733 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lssnap_latency": { 2024-05-21T03:34:30.733 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.733 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.733 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.733 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mkdir_latency": { 2024-05-21T03:34:30.733 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.734 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.734 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.734 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mknod_latency": { 2024-05-21T03:34:30.734 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.734 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.734 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.734 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mksnap_latency": { 2024-05-21T03:34:30.735 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.735 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.735 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.735 INFO:teuthology.orchestra.run.smithi039.stdout: "req_open_latency": { 2024-05-21T03:34:30.735 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.735 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.735 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.735 INFO:teuthology.orchestra.run.smithi039.stdout: "req_readdir_latency": { 2024-05-21T03:34:30.735 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.735 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.735 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.735 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rename_latency": { 2024-05-21T03:34:30.736 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.736 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.736 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.736 INFO:teuthology.orchestra.run.smithi039.stdout: "req_renamesnap_latency": { 2024-05-21T03:34:30.736 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.736 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.736 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.736 INFO:teuthology.orchestra.run.smithi039.stdout: "req_snapdiff_latency": { 2024-05-21T03:34:30.736 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.736 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.736 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.737 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmdir_latency": { 2024-05-21T03:34:30.737 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.737 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.737 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.737 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmsnap_latency": { 2024-05-21T03:34:30.737 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.737 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.737 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.737 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmxattr_latency": { 2024-05-21T03:34:30.737 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.737 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.737 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.737 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setattr_latency": { 2024-05-21T03:34:30.738 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.738 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.738 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.738 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setdirlayout_latency": { 2024-05-21T03:34:30.738 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.738 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.738 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.738 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setfilelock_latency": { 2024-05-21T03:34:30.738 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.738 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.738 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.738 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setlayout_latency": { 2024-05-21T03:34:30.739 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.739 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.739 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.739 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setxattr_latency": { 2024-05-21T03:34:30.739 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.739 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.739 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.739 INFO:teuthology.orchestra.run.smithi039.stdout: "req_symlink_latency": { 2024-05-21T03:34:30.739 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.739 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.739 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.739 INFO:teuthology.orchestra.run.smithi039.stdout: "req_unlink_latency": { 2024-05-21T03:34:30.740 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.740 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.740 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.740 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_revoke_eviction": 0, 2024-05-21T03:34:30.740 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_acquisition_throttle": 0, 2024-05-21T03:34:30.740 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getvxattr_latency": { 2024-05-21T03:34:30.740 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.740 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.740 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.740 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:30.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.740 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_sessions": { 2024-05-21T03:34:30.740 INFO:teuthology.orchestra.run.smithi039.stdout: "session_count": 2, 2024-05-21T03:34:30.741 INFO:teuthology.orchestra.run.smithi039.stdout: "session_add": 4, 2024-05-21T03:34:30.741 INFO:teuthology.orchestra.run.smithi039.stdout: "session_remove": 2, 2024-05-21T03:34:30.741 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_open": 2, 2024-05-21T03:34:30.741 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_stale": 0, 2024-05-21T03:34:30.741 INFO:teuthology.orchestra.run.smithi039.stdout: "total_load": 0, 2024-05-21T03:34:30.741 INFO:teuthology.orchestra.run.smithi039.stdout: "average_load": 0, 2024-05-21T03:34:30.741 INFO:teuthology.orchestra.run.smithi039.stdout: "avg_session_uptime": 3865, 2024-05-21T03:34:30.741 INFO:teuthology.orchestra.run.smithi039.stdout: "mdthresh_evicted": 0 2024-05-21T03:34:30.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.741 INFO:teuthology.orchestra.run.smithi039.stdout: "mempool": { 2024-05-21T03:34:30.741 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_bytes": 0, 2024-05-21T03:34:30.741 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_items": 0, 2024-05-21T03:34:30.741 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T03:34:30.742 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_items": 0, 2024-05-21T03:34:30.742 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T03:34:30.742 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_items": 0, 2024-05-21T03:34:30.742 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T03:34:30.742 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T03:34:30.742 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T03:34:30.742 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T03:34:30.742 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T03:34:30.742 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_items": 0, 2024-05-21T03:34:30.742 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T03:34:30.742 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T03:34:30.742 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_bytes": 0, 2024-05-21T03:34:30.743 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_items": 0, 2024-05-21T03:34:30.743 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_bytes": 0, 2024-05-21T03:34:30.743 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_items": 0, 2024-05-21T03:34:30.743 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T03:34:30.743 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T03:34:30.743 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T03:34:30.743 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T03:34:30.743 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T03:34:30.743 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_items": 0, 2024-05-21T03:34:30.743 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_bytes": 0, 2024-05-21T03:34:30.743 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_items": 0, 2024-05-21T03:34:30.743 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T03:34:30.743 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T03:34:30.743 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_bytes": 0, 2024-05-21T03:34:30.743 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_items": 0, 2024-05-21T03:34:30.745 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_bytes": 0, 2024-05-21T03:34:30.745 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_items": 0, 2024-05-21T03:34:30.745 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T03:34:30.745 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_items": 0, 2024-05-21T03:34:30.745 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T03:34:30.745 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_items": 0, 2024-05-21T03:34:30.745 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_bytes": 190151, 2024-05-21T03:34:30.745 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_items": 51, 2024-05-21T03:34:30.746 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_bytes": 0, 2024-05-21T03:34:30.746 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_items": 0, 2024-05-21T03:34:30.746 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bytes": 0, 2024-05-21T03:34:30.746 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_items": 0, 2024-05-21T03:34:30.746 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_bytes": 0, 2024-05-21T03:34:30.746 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_items": 0, 2024-05-21T03:34:30.746 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_bytes": 0, 2024-05-21T03:34:30.746 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_items": 0, 2024-05-21T03:34:30.746 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_bytes": 8224, 2024-05-21T03:34:30.746 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_items": 171, 2024-05-21T03:34:30.746 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T03:34:30.746 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_items": 0, 2024-05-21T03:34:30.746 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_bytes": 0, 2024-05-21T03:34:30.746 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_items": 0, 2024-05-21T03:34:30.747 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_bytes": 28512, 2024-05-21T03:34:30.747 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_items": 116, 2024-05-21T03:34:30.747 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_bytes": 0, 2024-05-21T03:34:30.747 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_items": 0, 2024-05-21T03:34:30.747 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_bytes": 0, 2024-05-21T03:34:30.747 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_items": 0 2024-05-21T03:34:30.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.747 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter": { 2024-05-21T03:34:30.747 INFO:teuthology.orchestra.run.smithi039.stdout: "op_active": 0, 2024-05-21T03:34:30.747 INFO:teuthology.orchestra.run.smithi039.stdout: "op_laggy": 0, 2024-05-21T03:34:30.747 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send": 2288, 2024-05-21T03:34:30.747 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send_bytes": 21, 2024-05-21T03:34:30.747 INFO:teuthology.orchestra.run.smithi039.stdout: "op_resend": 3, 2024-05-21T03:34:30.747 INFO:teuthology.orchestra.run.smithi039.stdout: "op_reply": 2288, 2024-05-21T03:34:30.747 INFO:teuthology.orchestra.run.smithi039.stdout: "op_latency": { 2024-05-21T03:34:30.747 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 2288, 2024-05-21T03:34:30.748 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 62.903843765, 2024-05-21T03:34:30.748 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.027492938 2024-05-21T03:34:30.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.748 INFO:teuthology.orchestra.run.smithi039.stdout: "op_inflight": 0, 2024-05-21T03:34:30.748 INFO:teuthology.orchestra.run.smithi039.stdout: "oplen_avg": { 2024-05-21T03:34:30.748 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 2288, 2024-05-21T03:34:30.748 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 2289 2024-05-21T03:34:30.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.748 INFO:teuthology.orchestra.run.smithi039.stdout: "op": 2288, 2024-05-21T03:34:30.748 INFO:teuthology.orchestra.run.smithi039.stdout: "op_r": 2288, 2024-05-21T03:34:30.748 INFO:teuthology.orchestra.run.smithi039.stdout: "op_w": 0, 2024-05-21T03:34:30.748 INFO:teuthology.orchestra.run.smithi039.stdout: "op_rmw": 0, 2024-05-21T03:34:30.748 INFO:teuthology.orchestra.run.smithi039.stdout: "op_pg": 0, 2024-05-21T03:34:30.748 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_stat": 1433, 2024-05-21T03:34:30.748 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_create": 0, 2024-05-21T03:34:30.749 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_read": 853, 2024-05-21T03:34:30.749 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_write": 0, 2024-05-21T03:34:30.749 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writefull": 0, 2024-05-21T03:34:30.749 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writesame": 0, 2024-05-21T03:34:30.749 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_append": 0, 2024-05-21T03:34:30.749 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_zero": 0, 2024-05-21T03:34:30.749 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_truncate": 0, 2024-05-21T03:34:30.749 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_delete": 0, 2024-05-21T03:34:30.749 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_mapext": 0, 2024-05-21T03:34:30.749 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_sparse_read": 0, 2024-05-21T03:34:30.749 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_clonerange": 0, 2024-05-21T03:34:30.749 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_getxattr": 1, 2024-05-21T03:34:30.749 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_setxattr": 0, 2024-05-21T03:34:30.749 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_cmpxattr": 0, 2024-05-21T03:34:30.749 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_rmxattr": 0, 2024-05-21T03:34:30.749 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_resetxattrs": 0, 2024-05-21T03:34:30.750 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_call": 0, 2024-05-21T03:34:30.750 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_watch": 0, 2024-05-21T03:34:30.750 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_notify": 0, 2024-05-21T03:34:30.750 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T03:34:30.750 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls": 0, 2024-05-21T03:34:30.750 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls_filter": 0, 2024-05-21T03:34:30.750 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_other": 0, 2024-05-21T03:34:30.750 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_active": 0, 2024-05-21T03:34:30.750 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_send": 0, 2024-05-21T03:34:30.750 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_resend": 0, 2024-05-21T03:34:30.750 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_ping": 0, 2024-05-21T03:34:30.750 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_active": 0, 2024-05-21T03:34:30.750 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_send": 0, 2024-05-21T03:34:30.750 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_resend": 0, 2024-05-21T03:34:30.751 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_active": 0, 2024-05-21T03:34:30.751 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_send": 0, 2024-05-21T03:34:30.751 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_resend": 0, 2024-05-21T03:34:30.751 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_active": 0, 2024-05-21T03:34:30.751 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_send": 0, 2024-05-21T03:34:30.751 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_resend": 0, 2024-05-21T03:34:30.751 INFO:teuthology.orchestra.run.smithi039.stdout: "command_active": 0, 2024-05-21T03:34:30.751 INFO:teuthology.orchestra.run.smithi039.stdout: "command_send": 0, 2024-05-21T03:34:30.751 INFO:teuthology.orchestra.run.smithi039.stdout: "command_resend": 0, 2024-05-21T03:34:30.751 INFO:teuthology.orchestra.run.smithi039.stdout: "map_epoch": 91, 2024-05-21T03:34:30.751 INFO:teuthology.orchestra.run.smithi039.stdout: "map_full": 0, 2024-05-21T03:34:30.751 INFO:teuthology.orchestra.run.smithi039.stdout: "map_inc": 11, 2024-05-21T03:34:30.751 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_sessions": 5, 2024-05-21T03:34:30.751 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_open": 5, 2024-05-21T03:34:30.751 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_close": 0, 2024-05-21T03:34:30.752 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_laggy": 0, 2024-05-21T03:34:30.752 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_wr": 0, 2024-05-21T03:34:30.752 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_rd": 2, 2024-05-21T03:34:30.752 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_del": 0 2024-05-21T03:34:30.752 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.752 INFO:teuthology.orchestra.run.smithi039.stdout: "oft": { 2024-05-21T03:34:30.752 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_objs": 0, 2024-05-21T03:34:30.752 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_kv_pairs": 0, 2024-05-21T03:34:30.752 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_updates": 0, 2024-05-21T03:34:30.752 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_removes": 0 2024-05-21T03:34:30.752 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.752 INFO:teuthology.orchestra.run.smithi039.stdout: "osd-slow-ops": { 2024-05-21T03:34:30.752 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_ops_count": 0 2024-05-21T03:34:30.752 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.752 INFO:teuthology.orchestra.run.smithi039.stdout: "purge_queue": { 2024-05-21T03:34:30.752 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops": 0, 2024-05-21T03:34:30.753 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops_high_water": 0, 2024-05-21T03:34:30.753 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing": 0, 2024-05-21T03:34:30.753 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_high_water": 0, 2024-05-21T03:34:30.753 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed_ops": 0, 2024-05-21T03:34:30.753 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed": 0, 2024-05-21T03:34:30.753 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_item_in_journal": 0 2024-05-21T03:34:30.753 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.753 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T03:34:30.753 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-21T03:34:30.753 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-05-21T03:34:30.753 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-21T03:34:30.753 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 2579, 2024-05-21T03:34:30.753 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 6438116, 2024-05-21T03:34:30.753 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:30.753 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 2579, 2024-05-21T03:34:30.754 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-05-21T03:34:30.754 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-05-21T03:34:30.754 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 2579, 2024-05-21T03:34:30.754 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 6438116, 2024-05-21T03:34:30.754 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-21T03:34:30.754 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.754 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.754 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.754 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:30.754 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.754 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_bytes": { 2024-05-21T03:34:30.754 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-21T03:34:30.754 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-05-21T03:34:30.754 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-21T03:34:30.755 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-21T03:34:30.755 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-21T03:34:30.755 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:30.755 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-21T03:34:30.755 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 2288, 2024-05-21T03:34:30.755 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 3844949, 2024-05-21T03:34:30.755 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 2288, 2024-05-21T03:34:30.755 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 3844949, 2024-05-21T03:34:30.755 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-21T03:34:30.755 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.755 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.755 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.756 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:30.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.756 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_ops": { 2024-05-21T03:34:30.756 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-21T03:34:30.756 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 1024, 2024-05-21T03:34:30.756 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-21T03:34:30.756 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-21T03:34:30.756 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-21T03:34:30.756 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:30.756 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-21T03:34:30.756 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 2288, 2024-05-21T03:34:30.756 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 2288, 2024-05-21T03:34:30.756 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 2288, 2024-05-21T03:34:30.756 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 2288, 2024-05-21T03:34:30.756 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-21T03:34:30.757 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.757 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.757 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.757 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:30.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.757 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle": { 2024-05-21T03:34:30.757 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-21T03:34:30.757 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-05-21T03:34:30.757 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-21T03:34:30.757 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-21T03:34:30.757 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-21T03:34:30.757 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:30.757 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-21T03:34:30.757 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-05-21T03:34:30.757 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-05-21T03:34:30.758 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 0, 2024-05-21T03:34:30.758 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 0, 2024-05-21T03:34:30.758 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-21T03:34:30.758 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.758 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.758 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.758 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:30.758 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-21T03:34:30.758 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle-0x55ca4722e6e0": { 2024-05-21T03:34:30.758 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-21T03:34:30.758 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-05-21T03:34:30.758 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-21T03:34:30.758 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-21T03:34:30.758 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-21T03:34:30.758 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:30.759 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-21T03:34:30.759 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-05-21T03:34:30.759 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-05-21T03:34:30.759 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 0, 2024-05-21T03:34:30.759 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 0, 2024-05-21T03:34:30.759 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-21T03:34:30.759 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-21T03:34:30.759 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-21T03:34:30.759 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-21T03:34:30.759 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:30.759 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-21T03:34:30.759 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-21T03:34:30.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:30 smithi027 ceph-mon[32179]: pgmap v325: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-05-21T03:34:30.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:30 smithi027 ceph-mon[32179]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:34:31.006 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:30 smithi039 ceph-mon[36429]: pgmap v325: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-05-21T03:34:31.007 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:30 smithi039 ceph-mon[36429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:34:31.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:30 smithi165 ceph-mon[36269]: pgmap v325: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-05-21T03:34:31.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:30 smithi165 ceph-mon[36269]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:34:31.529 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-05-21T03:34:31.529 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-05-21T03:34:31.529 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-05-21T03:34:31.529 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-05-21T03:34:31.529 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-05-21T03:34:31.529 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-05-21T03:34:31.530 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-05-21T03:34:31.530 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-05-21T03:34:31.530 DEBUG:tasks.check_counter:Getting stats from c 2024-05-21T03:34:31.530 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-21T03:34:31.781 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T03:34:32.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:32 smithi027 ceph-mon[32179]: pgmap v326: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-05-21T03:34:32.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:32 smithi027 ceph-mon[32179]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:34:32.933 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:32 smithi165 ceph-mon[36269]: pgmap v326: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-05-21T03:34:32.933 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:32 smithi165 ceph-mon[36269]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:34:33.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:32 smithi039 ceph-mon[36429]: pgmap v326: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-05-21T03:34:33.020 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:32 smithi039 ceph-mon[36429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:34:33.633 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-05-21T03:34:33.633 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T03:34:33.633 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 5518, 2024-05-21T03:34:33.633 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 4694, 2024-05-21T03:34:33.633 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 2282026, 2024-05-21T03:34:33.633 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 4891109, 2024-05-21T03:34:33.634 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 10, 2024-05-21T03:34:33.634 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 3, 2024-05-21T03:34:33.634 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.356561516, 2024-05-21T03:34:33.634 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.144215848, 2024-05-21T03:34:33.634 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.226322150, 2024-05-21T03:34:33.634 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.028407981, 2024-05-21T03:34:33.634 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:33.634 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 4696, 2024-05-21T03:34:33.634 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.356024395, 2024-05-21T03:34:33.634 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000075814 2024-05-21T03:34:33.634 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.634 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:33.634 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 8673, 2024-05-21T03:34:33.635 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.002278196, 2024-05-21T03:34:33.635 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000262 2024-05-21T03:34:33.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.635 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-21T03:34:33.635 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 0 2024-05-21T03:34:33.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.635 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T03:34:33.635 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 4044, 2024-05-21T03:34:33.635 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 4530, 2024-05-21T03:34:33.635 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 3733093, 2024-05-21T03:34:33.635 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 2077588, 2024-05-21T03:34:33.635 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 9, 2024-05-21T03:34:33.635 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 7, 2024-05-21T03:34:33.636 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.306726744, 2024-05-21T03:34:33.636 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.115751352, 2024-05-21T03:34:33.636 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.286834138, 2024-05-21T03:34:33.636 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.015115793, 2024-05-21T03:34:33.636 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:33.636 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 4529, 2024-05-21T03:34:33.636 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.295833367, 2024-05-21T03:34:33.636 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000065319 2024-05-21T03:34:33.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.636 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:33.636 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 6739, 2024-05-21T03:34:33.636 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.001167159, 2024-05-21T03:34:33.636 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000173 2024-05-21T03:34:33.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.637 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 3558496, 2024-05-21T03:34:33.637 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 1832064 2024-05-21T03:34:33.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.637 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T03:34:33.637 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 324, 2024-05-21T03:34:33.637 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 517, 2024-05-21T03:34:33.637 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 168657, 2024-05-21T03:34:33.637 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 778345, 2024-05-21T03:34:33.637 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 24, 2024-05-21T03:34:33.637 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 11, 2024-05-21T03:34:33.637 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.044063835, 2024-05-21T03:34:33.637 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.016547489, 2024-05-21T03:34:33.637 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.017829886, 2024-05-21T03:34:33.637 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.006615639, 2024-05-21T03:34:33.637 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:33.638 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 511, 2024-05-21T03:34:33.638 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.053197399, 2024-05-21T03:34:33.638 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000104104 2024-05-21T03:34:33.638 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.638 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:33.638 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 208, 2024-05-21T03:34:33.638 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000042882, 2024-05-21T03:34:33.638 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000206 2024-05-21T03:34:33.638 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.638 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 8784, 2024-05-21T03:34:33.638 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 482208 2024-05-21T03:34:33.638 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.638 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-05-21T03:34:33.639 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-05-21T03:34:33.639 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-05-21T03:34:33.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.639 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-05-21T03:34:33.639 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-05-21T03:34:33.639 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-05-21T03:34:33.639 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 754, 2024-05-21T03:34:33.639 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.599903790, 2024-05-21T03:34:33.639 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000795628 2024-05-21T03:34:33.639 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:33.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.639 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-05-21T03:34:33.639 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-05-21T03:34:33.639 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-05-21T03:34:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 543, 2024-05-21T03:34:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.018228976, 2024-05-21T03:34:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000033570 2024-05-21T03:34:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-05-21T03:34:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 248, 2024-05-21T03:34:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 223, 2024-05-21T03:34:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-05-21T03:34:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 223, 2024-05-21T03:34:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 20.968723902, 2024-05-21T03:34:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.094030152 2024-05-21T03:34:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-05-21T03:34:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 25, 2024-05-21T03:34:33.641 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 0, 2024-05-21T03:34:33.641 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-05-21T03:34:33.641 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 19, 2024-05-21T03:34:33.641 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 1, 2024-05-21T03:34:33.641 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-05-21T03:34:33.641 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 17, 2024-05-21T03:34:33.641 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 2, 2024-05-21T03:34:33.641 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 5, 2024-05-21T03:34:33.641 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 10, 2024-05-21T03:34:33.641 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 13, 2024-05-21T03:34:33.641 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 428, 2024-05-21T03:34:33.641 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 2, 2024-05-21T03:34:33.641 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 2, 2024-05-21T03:34:33.641 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 4, 2024-05-21T03:34:33.642 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 649, 2024-05-21T03:34:33.642 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 460, 2024-05-21T03:34:33.642 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 22, 2024-05-21T03:34:33.642 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 127, 2024-05-21T03:34:33.642 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-05-21T03:34:33.642 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-05-21T03:34:33.642 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 0, 2024-05-21T03:34:33.642 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 2000, 2024-05-21T03:34:33.642 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 2, 2024-05-21T03:34:33.642 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 9, 2024-05-21T03:34:33.642 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 223, 2024-05-21T03:34:33.642 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 2, 2024-05-21T03:34:33.642 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 48, 2024-05-21T03:34:33.642 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-05-21T03:34:33.643 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 0, 2024-05-21T03:34:33.643 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-05-21T03:34:33.643 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 325, 2024-05-21T03:34:33.643 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 13424230508, 2024-05-21T03:34:33.643 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 6, 2024-05-21T03:34:33.643 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 0, 2024-05-21T03:34:33.643 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-05-21T03:34:33.643 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T03:34:33.643 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-05-21T03:34:33.643 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 0, 2024-05-21T03:34:33.643 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T03:34:33.643 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T03:34:33.643 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 0, 2024-05-21T03:34:33.643 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 40, 2024-05-21T03:34:33.644 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 372, 2024-05-21T03:34:33.644 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 498, 2024-05-21T03:34:33.644 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T03:34:33.644 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T03:34:33.644 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T03:34:33.644 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 3092, 2024-05-21T03:34:33.644 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 2651, 2024-05-21T03:34:33.644 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 22, 2024-05-21T03:34:33.644 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 51 2024-05-21T03:34:33.644 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.644 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-05-21T03:34:33.644 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 24, 2024-05-21T03:34:33.644 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 353, 2024-05-21T03:34:33.644 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 238, 2024-05-21T03:34:33.644 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 221, 2024-05-21T03:34:33.645 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 20, 2024-05-21T03:34:33.645 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 0, 2024-05-21T03:34:33.645 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-05-21T03:34:33.645 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-05-21T03:34:33.645 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 20, 2024-05-21T03:34:33.645 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 20, 2024-05-21T03:34:33.645 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 0, 2024-05-21T03:34:33.645 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-05-21T03:34:33.645 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-05-21T03:34:33.645 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-05-21T03:34:33.645 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-05-21T03:34:33.645 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-05-21T03:34:33.645 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-05-21T03:34:33.645 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_path": 5, 2024-05-21T03:34:33.645 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_inode": 373, 2024-05-21T03:34:33.645 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T03:34:33.646 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 10, 2024-05-21T03:34:33.646 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-05-21T03:34:33.646 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 1, 2024-05-21T03:34:33.646 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-05-21T03:34:33.646 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0 2024-05-21T03:34:33.646 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.646 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-05-21T03:34:33.646 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-05-21T03:34:33.646 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 2986, 2024-05-21T03:34:33.646 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 0, 2024-05-21T03:34:33.646 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 0, 2024-05-21T03:34:33.646 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 2986, 2024-05-21T03:34:33.646 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-05-21T03:34:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 0, 2024-05-21T03:34:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 4, 2024-05-21T03:34:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 0, 2024-05-21T03:34:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 0, 2024-05-21T03:34:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 4, 2024-05-21T03:34:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 2, 2024-05-21T03:34:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-05-21T03:34:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 0, 2024-05-21T03:34:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 4194304, 2024-05-21T03:34:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 8039248, 2024-05-21T03:34:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 0, 2024-05-21T03:34:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-05-21T03:34:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 330, 2024-05-21T03:34:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 185.298039537, 2024-05-21T03:34:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.561509210 2024-05-21T03:34:33.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.648 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 0 2024-05-21T03:34:33.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.648 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-05-21T03:34:33.648 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 22, 2024-05-21T03:34:33.648 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 358, 2024-05-21T03:34:33.648 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 336, 2024-05-21T03:34:33.648 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 21, 2024-05-21T03:34:33.648 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 151, 2024-05-21T03:34:33.648 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 130, 2024-05-21T03:34:33.648 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 17, 2024-05-21T03:34:33.648 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 465, 2024-05-21T03:34:33.648 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 448, 2024-05-21T03:34:33.648 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 2, 2024-05-21T03:34:33.648 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 131, 2024-05-21T03:34:33.648 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 129, 2024-05-21T03:34:33.649 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 61512, 2024-05-21T03:34:33.649 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 223608 2024-05-21T03:34:33.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.649 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-05-21T03:34:33.649 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 363, 2024-05-21T03:34:33.649 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 102, 2024-05-21T03:34:33.649 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 248, 2024-05-21T03:34:33.649 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 28, 2024-05-21T03:34:33.649 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 493, 2024-05-21T03:34:33.649 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-05-21T03:34:33.649 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:33.649 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:33.649 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:33.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.650 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-05-21T03:34:33.650 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 48, 2024-05-21T03:34:33.650 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 5.737380853, 2024-05-21T03:34:33.650 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.119528767 2024-05-21T03:34:33.650 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.650 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-05-21T03:34:33.650 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:33.650 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:33.650 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:33.650 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.650 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-05-21T03:34:33.650 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:33.650 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:33.651 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:33.651 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.651 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-05-21T03:34:33.651 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 67, 2024-05-21T03:34:33.651 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 15.091401201, 2024-05-21T03:34:33.651 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.225244794 2024-05-21T03:34:33.651 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.651 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-05-21T03:34:33.651 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:33.651 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:33.651 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:33.651 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.651 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-05-21T03:34:33.651 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2, 2024-05-21T03:34:33.651 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000193426, 2024-05-21T03:34:33.652 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000096713 2024-05-21T03:34:33.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.652 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-05-21T03:34:33.652 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:33.652 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:33.652 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:33.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.652 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-05-21T03:34:33.652 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:33.652 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:33.652 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:33.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.652 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-05-21T03:34:33.652 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:33.653 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:33.653 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:33.653 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.653 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-05-21T03:34:33.653 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:33.653 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:33.653 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:33.653 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.654 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-05-21T03:34:33.654 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:33.654 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:33.654 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:33.654 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.654 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-05-21T03:34:33.654 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:33.654 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:33.654 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:33.654 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.654 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-05-21T03:34:33.654 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:33.654 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:33.655 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:33.655 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.655 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-05-21T03:34:33.655 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 78, 2024-05-21T03:34:33.655 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.088718650, 2024-05-21T03:34:33.655 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001137418 2024-05-21T03:34:33.655 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.655 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-05-21T03:34:33.655 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 8, 2024-05-21T03:34:33.655 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.003496502, 2024-05-21T03:34:33.655 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000437062 2024-05-21T03:34:33.655 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.655 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-05-21T03:34:33.655 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:33.655 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:33.656 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:33.656 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.656 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-05-21T03:34:33.656 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:33.656 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:33.656 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:33.656 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.656 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-05-21T03:34:33.656 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:33.656 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:33.656 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:33.656 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.656 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-05-21T03:34:33.657 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:33.657 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:33.657 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:33.657 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.657 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-05-21T03:34:33.657 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:33.657 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:33.657 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:33.657 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.657 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-05-21T03:34:33.657 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:33.657 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:33.657 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:33.657 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.658 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-05-21T03:34:33.658 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:33.658 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:33.658 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:33.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.658 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-05-21T03:34:33.658 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:33.658 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:33.658 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:33.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.658 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-05-21T03:34:33.658 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:33.658 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:33.659 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:33.659 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.659 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-05-21T03:34:33.659 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:33.659 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:33.659 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:33.659 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.659 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-05-21T03:34:33.659 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:33.659 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:33.659 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:33.659 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.659 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-05-21T03:34:33.660 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:33.660 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:33.660 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:33.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.660 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-05-21T03:34:33.660 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 20, 2024-05-21T03:34:33.660 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.047533270, 2024-05-21T03:34:33.660 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.002376663 2024-05-21T03:34:33.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.660 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-05-21T03:34:33.660 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-05-21T03:34:33.660 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-05-21T03:34:33.660 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:33.660 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:33.660 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:33.661 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:33.661 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.661 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-05-21T03:34:33.661 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 2, 2024-05-21T03:34:33.661 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 4, 2024-05-21T03:34:33.661 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 2, 2024-05-21T03:34:33.661 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 2, 2024-05-21T03:34:33.661 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-05-21T03:34:33.661 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 82, 2024-05-21T03:34:33.661 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 41, 2024-05-21T03:34:33.661 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 519, 2024-05-21T03:34:33.661 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-05-21T03:34:33.661 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.662 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-05-21T03:34:33.662 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-05-21T03:34:33.662 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-05-21T03:34:33.662 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T03:34:33.662 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-05-21T03:34:33.662 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T03:34:33.662 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-05-21T03:34:33.662 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T03:34:33.662 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T03:34:33.662 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T03:34:33.662 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T03:34:33.662 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T03:34:33.662 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-05-21T03:34:33.663 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T03:34:33.663 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T03:34:33.663 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-05-21T03:34:33.663 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-05-21T03:34:33.663 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-05-21T03:34:33.663 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-05-21T03:34:33.663 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T03:34:33.663 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T03:34:33.663 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T03:34:33.663 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T03:34:33.663 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T03:34:33.663 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-05-21T03:34:33.663 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-05-21T03:34:33.663 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-05-21T03:34:33.664 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T03:34:33.664 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T03:34:33.664 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-05-21T03:34:33.664 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-05-21T03:34:33.664 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-05-21T03:34:33.664 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-05-21T03:34:33.664 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T03:34:33.664 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-05-21T03:34:33.664 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T03:34:33.664 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-05-21T03:34:33.664 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 288407, 2024-05-21T03:34:33.664 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 105, 2024-05-21T03:34:33.664 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 0, 2024-05-21T03:34:33.664 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 0, 2024-05-21T03:34:33.665 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-05-21T03:34:33.665 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-05-21T03:34:33.665 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-05-21T03:34:33.665 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-05-21T03:34:33.665 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-05-21T03:34:33.665 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-05-21T03:34:33.665 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 8224, 2024-05-21T03:34:33.665 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 171, 2024-05-21T03:34:33.665 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T03:34:33.665 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-05-21T03:34:33.665 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-05-21T03:34:33.665 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-05-21T03:34:33.665 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 100917, 2024-05-21T03:34:33.665 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 406, 2024-05-21T03:34:33.666 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-05-21T03:34:33.666 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-05-21T03:34:33.666 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-05-21T03:34:33.666 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-05-21T03:34:33.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.666 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-05-21T03:34:33.666 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-05-21T03:34:33.666 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-05-21T03:34:33.666 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 981, 2024-05-21T03:34:33.666 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 3936844, 2024-05-21T03:34:33.666 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 33, 2024-05-21T03:34:33.666 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 981, 2024-05-21T03:34:33.666 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-05-21T03:34:33.667 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 981, 2024-05-21T03:34:33.667 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 331.337837990, 2024-05-21T03:34:33.667 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.337755186 2024-05-21T03:34:33.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.667 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-05-21T03:34:33.667 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-05-21T03:34:33.667 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 981, 2024-05-21T03:34:33.667 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 1092 2024-05-21T03:34:33.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.667 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 981, 2024-05-21T03:34:33.667 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 1, 2024-05-21T03:34:33.667 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 980, 2024-05-21T03:34:33.667 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-05-21T03:34:33.667 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-05-21T03:34:33.668 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 0, 2024-05-21T03:34:33.668 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 10, 2024-05-21T03:34:33.668 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 1, 2024-05-21T03:34:33.668 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 331, 2024-05-21T03:34:33.668 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 29, 2024-05-21T03:34:33.668 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-05-21T03:34:33.668 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-05-21T03:34:33.668 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 0, 2024-05-21T03:34:33.668 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 0, 2024-05-21T03:34:33.668 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 513, 2024-05-21T03:34:33.668 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-05-21T03:34:33.668 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-05-21T03:34:33.668 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-05-21T03:34:33.668 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 0, 2024-05-21T03:34:33.668 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 20, 2024-05-21T03:34:33.669 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-05-21T03:34:33.669 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-05-21T03:34:33.669 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-05-21T03:34:33.669 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-05-21T03:34:33.669 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-05-21T03:34:33.669 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-05-21T03:34:33.669 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T03:34:33.669 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-05-21T03:34:33.669 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-05-21T03:34:33.669 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-05-21T03:34:33.669 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-05-21T03:34:33.669 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-05-21T03:34:33.669 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-05-21T03:34:33.669 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-05-21T03:34:33.669 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-05-21T03:34:33.670 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-05-21T03:34:33.670 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-05-21T03:34:33.670 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-05-21T03:34:33.670 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-05-21T03:34:33.670 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-05-21T03:34:33.670 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-05-21T03:34:33.670 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-05-21T03:34:33.670 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-05-21T03:34:33.670 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-05-21T03:34:33.670 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-05-21T03:34:33.670 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-05-21T03:34:33.670 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 91, 2024-05-21T03:34:33.670 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-05-21T03:34:33.671 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 11, 2024-05-21T03:34:33.671 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 12, 2024-05-21T03:34:33.671 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 12, 2024-05-21T03:34:33.671 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 0, 2024-05-21T03:34:33.671 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-05-21T03:34:33.671 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 135, 2024-05-21T03:34:33.671 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 0, 2024-05-21T03:34:33.671 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 53 2024-05-21T03:34:33.671 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.671 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-05-21T03:34:33.671 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 1, 2024-05-21T03:34:33.671 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-05-21T03:34:33.671 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-05-21T03:34:33.671 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-05-21T03:34:33.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.672 INFO:teuthology.orchestra.run.smithi165.stdout: "osd-slow-ops": { 2024-05-21T03:34:33.672 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 458 2024-05-21T03:34:33.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.672 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-05-21T03:34:33.672 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-05-21T03:34:33.672 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 25, 2024-05-21T03:34:33.672 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-05-21T03:34:33.672 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 1, 2024-05-21T03:34:33.672 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 500, 2024-05-21T03:34:33.672 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 20, 2024-05-21T03:34:33.672 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-05-21T03:34:33.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.672 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T03:34:33.672 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-21T03:34:33.673 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-05-21T03:34:33.673 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-21T03:34:33.673 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 9882, 2024-05-21T03:34:33.673 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 5107973, 2024-05-21T03:34:33.673 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:33.673 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 9882, 2024-05-21T03:34:33.673 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-05-21T03:34:33.673 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-05-21T03:34:33.673 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 9882, 2024-05-21T03:34:33.673 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 5107973, 2024-05-21T03:34:33.673 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-21T03:34:33.673 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:33.673 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:33.673 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:33.673 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:33.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.674 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-05-21T03:34:33.674 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-21T03:34:33.674 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-05-21T03:34:33.674 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-21T03:34:33.674 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-05-21T03:34:33.674 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-05-21T03:34:33.674 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:33.674 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-05-21T03:34:33.674 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 981, 2024-05-21T03:34:33.674 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 3939664, 2024-05-21T03:34:33.674 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 981, 2024-05-21T03:34:33.674 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 3939664, 2024-05-21T03:34:33.675 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-21T03:34:33.675 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:33.675 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:33.675 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:33.675 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:33.675 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.675 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-05-21T03:34:33.675 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-21T03:34:33.675 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-05-21T03:34:33.675 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-21T03:34:33.675 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-05-21T03:34:33.675 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-05-21T03:34:33.676 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:33.676 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-05-21T03:34:33.676 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 981, 2024-05-21T03:34:33.676 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 981, 2024-05-21T03:34:33.676 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 981, 2024-05-21T03:34:33.676 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 981, 2024-05-21T03:34:33.676 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-21T03:34:33.676 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:33.676 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:33.676 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:33.676 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:33.676 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.676 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-05-21T03:34:33.677 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-21T03:34:33.677 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-05-21T03:34:33.677 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-21T03:34:33.677 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 20, 2024-05-21T03:34:33.677 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 2820, 2024-05-21T03:34:33.677 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:33.677 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 20, 2024-05-21T03:34:33.677 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-05-21T03:34:33.677 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-05-21T03:34:33.677 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 1, 2024-05-21T03:34:33.677 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 2820, 2024-05-21T03:34:33.677 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-21T03:34:33.677 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:33.677 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:33.678 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:33.678 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:33.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:33.678 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x560520cb6500": { 2024-05-21T03:34:33.678 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-21T03:34:33.678 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-05-21T03:34:33.678 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-21T03:34:33.678 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 2986, 2024-05-21T03:34:33.678 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 3844944, 2024-05-21T03:34:33.678 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:33.678 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 2986, 2024-05-21T03:34:33.678 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-05-21T03:34:33.678 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-05-21T03:34:33.679 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 330, 2024-05-21T03:34:33.679 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 3844944, 2024-05-21T03:34:33.679 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-21T03:34:33.679 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:33.679 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:33.679 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:33.679 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:33.679 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:33.679 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-05-21T03:34:33.771 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:33 smithi165 ceph-mon[36269]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:34:33.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:33 smithi027 ceph-mon[32179]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:34:34.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:33 smithi039 ceph-mon[36429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:34:34.391 INFO:tasks.check_counter:Daemon mds.c mds.exported=9 2024-05-21T03:34:34.391 INFO:tasks.check_counter:Daemon mds.c mds.imported=2 2024-05-21T03:34:34.391 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=24 2024-05-21T03:34:34.391 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=353 2024-05-21T03:34:34.391 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-05-21T03:34:34.391 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-05-21T03:34:34.391 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-05-21T03:34:34.391 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=1 2024-05-21T03:34:34.391 DEBUG:tasks.check_counter:Getting stats from f 2024-05-21T03:34:34.391 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-21T03:34:34.636 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T03:34:34.925 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:34 smithi165 ceph-mon[36269]: pgmap v327: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:34:34.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:34 smithi027 ceph-mon[32179]: pgmap v327: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:34:35.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:34 smithi039 ceph-mon[36429]: pgmap v327: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:34:36.431 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-05-21T03:34:36.431 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T03:34:36.431 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 2600, 2024-05-21T03:34:36.432 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 3131, 2024-05-21T03:34:36.432 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 967306, 2024-05-21T03:34:36.432 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 3810740, 2024-05-21T03:34:36.432 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 20, 2024-05-21T03:34:36.432 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 6, 2024-05-21T03:34:36.432 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.200296769, 2024-05-21T03:34:36.432 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.078063944, 2024-05-21T03:34:36.432 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.172365028, 2024-05-21T03:34:36.432 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.013605140, 2024-05-21T03:34:36.432 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:36.432 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 3124, 2024-05-21T03:34:36.432 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.135976228, 2024-05-21T03:34:36.433 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000043526 2024-05-21T03:34:36.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.433 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:36.433 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 4413, 2024-05-21T03:34:36.433 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000719752, 2024-05-21T03:34:36.433 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000163 2024-05-21T03:34:36.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.433 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 859040, 2024-05-21T03:34:36.433 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 1447968 2024-05-21T03:34:36.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.433 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T03:34:36.433 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 4980, 2024-05-21T03:34:36.433 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 4286, 2024-05-21T03:34:36.433 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 2210594, 2024-05-21T03:34:36.434 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 1779633, 2024-05-21T03:34:36.434 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 12, 2024-05-21T03:34:36.434 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 7, 2024-05-21T03:34:36.434 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.323042780, 2024-05-21T03:34:36.434 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.129840170, 2024-05-21T03:34:36.434 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.212819418, 2024-05-21T03:34:36.434 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.018770257, 2024-05-21T03:34:36.434 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:36.434 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 4288, 2024-05-21T03:34:36.434 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.229127934, 2024-05-21T03:34:36.434 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000053434 2024-05-21T03:34:36.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.435 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:36.435 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 8182, 2024-05-21T03:34:36.435 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.001877246, 2024-05-21T03:34:36.435 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000229 2024-05-21T03:34:36.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.435 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-21T03:34:36.435 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 156480 2024-05-21T03:34:36.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.435 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T03:34:36.435 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 825, 2024-05-21T03:34:36.435 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 786, 2024-05-21T03:34:36.435 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 2439307, 2024-05-21T03:34:36.435 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 446637, 2024-05-21T03:34:36.435 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 11, 2024-05-21T03:34:36.435 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 6, 2024-05-21T03:34:36.436 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.101153352, 2024-05-21T03:34:36.436 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.028326775, 2024-05-21T03:34:36.436 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.067906853, 2024-05-21T03:34:36.436 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.019713012, 2024-05-21T03:34:36.436 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:36.436 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 786, 2024-05-21T03:34:36.436 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.067169921, 2024-05-21T03:34:36.436 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000085457 2024-05-21T03:34:36.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.436 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:36.436 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 144, 2024-05-21T03:34:36.436 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000044229, 2024-05-21T03:34:36.436 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000307 2024-05-21T03:34:36.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.437 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 2223664, 2024-05-21T03:34:36.437 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 152768 2024-05-21T03:34:36.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.437 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-05-21T03:34:36.437 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-05-21T03:34:36.437 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-05-21T03:34:36.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.437 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-05-21T03:34:36.437 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-05-21T03:34:36.437 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-05-21T03:34:36.437 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 466, 2024-05-21T03:34:36.438 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.343728050, 2024-05-21T03:34:36.438 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000737613 2024-05-21T03:34:36.438 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:36.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.438 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-05-21T03:34:36.438 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-05-21T03:34:36.438 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-05-21T03:34:36.438 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 813, 2024-05-21T03:34:36.438 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.028498625, 2024-05-21T03:34:36.438 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000035053 2024-05-21T03:34:36.438 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:36.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.439 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-05-21T03:34:36.439 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 156, 2024-05-21T03:34:36.439 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 156, 2024-05-21T03:34:36.439 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-05-21T03:34:36.439 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 156, 2024-05-21T03:34:36.439 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.273395135, 2024-05-21T03:34:36.439 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001752532 2024-05-21T03:34:36.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.439 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-05-21T03:34:36.439 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 0, 2024-05-21T03:34:36.439 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 0, 2024-05-21T03:34:36.439 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-05-21T03:34:36.439 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 11, 2024-05-21T03:34:36.439 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 0, 2024-05-21T03:34:36.440 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-05-21T03:34:36.440 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 17, 2024-05-21T03:34:36.440 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 2, 2024-05-21T03:34:36.440 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 5, 2024-05-21T03:34:36.440 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 10, 2024-05-21T03:34:36.440 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 13, 2024-05-21T03:34:36.440 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 367, 2024-05-21T03:34:36.440 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 0, 2024-05-21T03:34:36.440 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 0, 2024-05-21T03:34:36.440 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 4, 2024-05-21T03:34:36.440 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 530, 2024-05-21T03:34:36.440 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 349, 2024-05-21T03:34:36.440 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 0, 2024-05-21T03:34:36.440 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 125, 2024-05-21T03:34:36.441 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-05-21T03:34:36.441 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-05-21T03:34:36.441 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 0, 2024-05-21T03:34:36.441 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 0, 2024-05-21T03:34:36.441 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-05-21T03:34:36.441 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 2, 2024-05-21T03:34:36.441 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 166, 2024-05-21T03:34:36.441 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 2, 2024-05-21T03:34:36.441 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 30, 2024-05-21T03:34:36.441 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-05-21T03:34:36.441 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 0, 2024-05-21T03:34:36.441 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-05-21T03:34:36.441 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 325, 2024-05-21T03:34:36.441 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 13424230508, 2024-05-21T03:34:36.442 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 6, 2024-05-21T03:34:36.442 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 0, 2024-05-21T03:34:36.442 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-05-21T03:34:36.442 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T03:34:36.442 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-05-21T03:34:36.442 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 0, 2024-05-21T03:34:36.442 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T03:34:36.442 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T03:34:36.442 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 0, 2024-05-21T03:34:36.442 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 2, 2024-05-21T03:34:36.442 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 185, 2024-05-21T03:34:36.442 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 439, 2024-05-21T03:34:36.442 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T03:34:36.442 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T03:34:36.443 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T03:34:36.443 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 1939, 2024-05-21T03:34:36.443 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 1637, 2024-05-21T03:34:36.443 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 17, 2024-05-21T03:34:36.443 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 63 2024-05-21T03:34:36.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.443 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-05-21T03:34:36.443 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 5, 2024-05-21T03:34:36.443 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 362, 2024-05-21T03:34:36.443 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 243, 2024-05-21T03:34:36.443 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 229, 2024-05-21T03:34:36.443 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 5, 2024-05-21T03:34:36.443 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 0, 2024-05-21T03:34:36.443 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-05-21T03:34:36.443 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-05-21T03:34:36.443 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 30, 2024-05-21T03:34:36.444 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 30, 2024-05-21T03:34:36.444 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 0, 2024-05-21T03:34:36.444 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-05-21T03:34:36.444 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-05-21T03:34:36.444 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-05-21T03:34:36.444 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-05-21T03:34:36.444 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-05-21T03:34:36.444 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-05-21T03:34:36.444 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_path": 5, 2024-05-21T03:34:36.444 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_inode": 350, 2024-05-21T03:34:36.444 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T03:34:36.444 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 2, 2024-05-21T03:34:36.444 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-05-21T03:34:36.444 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 0, 2024-05-21T03:34:36.445 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-05-21T03:34:36.445 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0 2024-05-21T03:34:36.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.445 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-05-21T03:34:36.445 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-05-21T03:34:36.445 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 1942, 2024-05-21T03:34:36.445 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 0, 2024-05-21T03:34:36.445 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 0, 2024-05-21T03:34:36.445 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 1942, 2024-05-21T03:34:36.445 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-05-21T03:34:36.445 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 0, 2024-05-21T03:34:36.445 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 3, 2024-05-21T03:34:36.445 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 0, 2024-05-21T03:34:36.445 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 0, 2024-05-21T03:34:36.446 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 3, 2024-05-21T03:34:36.446 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 2, 2024-05-21T03:34:36.446 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-05-21T03:34:36.446 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 0, 2024-05-21T03:34:36.446 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 4194304, 2024-05-21T03:34:36.446 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 6726884, 2024-05-21T03:34:36.446 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 0, 2024-05-21T03:34:36.446 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-05-21T03:34:36.446 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 207, 2024-05-21T03:34:36.446 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 206.296928856, 2024-05-21T03:34:36.446 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.996603521 2024-05-21T03:34:36.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.446 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 0 2024-05-21T03:34:36.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.447 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-05-21T03:34:36.447 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 22, 2024-05-21T03:34:36.447 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 304, 2024-05-21T03:34:36.447 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 282, 2024-05-21T03:34:36.447 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 21, 2024-05-21T03:34:36.447 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 155, 2024-05-21T03:34:36.447 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 134, 2024-05-21T03:34:36.447 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 17, 2024-05-21T03:34:36.447 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 414, 2024-05-21T03:34:36.447 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 397, 2024-05-21T03:34:36.447 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 0, 2024-05-21T03:34:36.447 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 105, 2024-05-21T03:34:36.447 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 105, 2024-05-21T03:34:36.448 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 58584, 2024-05-21T03:34:36.449 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 223608 2024-05-21T03:34:36.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.449 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-05-21T03:34:36.449 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 260, 2024-05-21T03:34:36.449 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 75, 2024-05-21T03:34:36.449 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 156, 2024-05-21T03:34:36.449 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 21, 2024-05-21T03:34:36.450 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 442, 2024-05-21T03:34:36.450 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-05-21T03:34:36.450 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.450 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.450 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.450 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-05-21T03:34:36.450 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.450 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.450 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.450 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-05-21T03:34:36.450 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.451 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.451 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.451 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-05-21T03:34:36.451 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.451 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.451 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.451 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-05-21T03:34:36.451 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 63, 2024-05-21T03:34:36.451 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.137440731, 2024-05-21T03:34:36.451 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.002181598 2024-05-21T03:34:36.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.451 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-05-21T03:34:36.452 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.452 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.452 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.452 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-05-21T03:34:36.452 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.452 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.452 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.452 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-05-21T03:34:36.452 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.452 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.452 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.452 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-05-21T03:34:36.453 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.453 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.453 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.453 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-05-21T03:34:36.453 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.453 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.453 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.453 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-05-21T03:34:36.453 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.453 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.454 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.454 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-05-21T03:34:36.454 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.454 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.454 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.454 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-05-21T03:34:36.454 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.454 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.455 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.455 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-05-21T03:34:36.455 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.455 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.455 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.455 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-05-21T03:34:36.455 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 59, 2024-05-21T03:34:36.455 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.048334809, 2024-05-21T03:34:36.455 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000819234 2024-05-21T03:34:36.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.455 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-05-21T03:34:36.455 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 4, 2024-05-21T03:34:36.456 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.002612856, 2024-05-21T03:34:36.456 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000653214 2024-05-21T03:34:36.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.456 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-05-21T03:34:36.456 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.456 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.456 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.456 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-05-21T03:34:36.456 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.456 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.456 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.457 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-05-21T03:34:36.457 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.457 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.457 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.457 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-05-21T03:34:36.457 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.457 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.457 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.457 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-05-21T03:34:36.457 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.457 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.457 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.458 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-05-21T03:34:36.458 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.458 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.458 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.458 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-05-21T03:34:36.458 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.458 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.458 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.458 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-05-21T03:34:36.458 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.459 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.459 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.459 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-05-21T03:34:36.459 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.459 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.459 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.459 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-05-21T03:34:36.459 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.459 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.459 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.459 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-05-21T03:34:36.459 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.460 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.460 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.460 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-05-21T03:34:36.460 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.460 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.460 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.460 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-05-21T03:34:36.460 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 30, 2024-05-21T03:34:36.460 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.085006739, 2024-05-21T03:34:36.460 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.002833557 2024-05-21T03:34:36.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.461 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-05-21T03:34:36.461 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-05-21T03:34:36.461 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-05-21T03:34:36.461 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.461 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.461 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.461 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:36.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.461 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-05-21T03:34:36.461 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 2, 2024-05-21T03:34:36.461 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 4, 2024-05-21T03:34:36.461 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 2, 2024-05-21T03:34:36.462 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 2, 2024-05-21T03:34:36.462 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-05-21T03:34:36.462 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 82, 2024-05-21T03:34:36.462 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 41, 2024-05-21T03:34:36.462 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 1362, 2024-05-21T03:34:36.462 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-05-21T03:34:36.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.462 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-05-21T03:34:36.462 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-05-21T03:34:36.462 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-05-21T03:34:36.462 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T03:34:36.462 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-05-21T03:34:36.462 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T03:34:36.463 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-05-21T03:34:36.463 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T03:34:36.463 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T03:34:36.463 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T03:34:36.463 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T03:34:36.463 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T03:34:36.463 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-05-21T03:34:36.463 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T03:34:36.463 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T03:34:36.463 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-05-21T03:34:36.463 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-05-21T03:34:36.463 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-05-21T03:34:36.463 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-05-21T03:34:36.464 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T03:34:36.464 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T03:34:36.464 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T03:34:36.464 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T03:34:36.464 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T03:34:36.464 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-05-21T03:34:36.464 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-05-21T03:34:36.464 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-05-21T03:34:36.464 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T03:34:36.464 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T03:34:36.464 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-05-21T03:34:36.464 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-05-21T03:34:36.464 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-05-21T03:34:36.465 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-05-21T03:34:36.465 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T03:34:36.465 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-05-21T03:34:36.465 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T03:34:36.465 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-05-21T03:34:36.465 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 297255, 2024-05-21T03:34:36.465 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 110, 2024-05-21T03:34:36.465 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 0, 2024-05-21T03:34:36.465 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 0, 2024-05-21T03:34:36.465 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-05-21T03:34:36.465 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-05-21T03:34:36.465 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-05-21T03:34:36.465 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-05-21T03:34:36.466 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-05-21T03:34:36.466 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-05-21T03:34:36.466 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 8224, 2024-05-21T03:34:36.466 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 171, 2024-05-21T03:34:36.466 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T03:34:36.466 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-05-21T03:34:36.466 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-05-21T03:34:36.466 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-05-21T03:34:36.466 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 100133, 2024-05-21T03:34:36.466 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 396, 2024-05-21T03:34:36.466 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-05-21T03:34:36.466 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-05-21T03:34:36.466 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-05-21T03:34:36.466 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-05-21T03:34:36.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.467 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-05-21T03:34:36.467 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-05-21T03:34:36.467 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-05-21T03:34:36.467 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 1097, 2024-05-21T03:34:36.467 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 2561584, 2024-05-21T03:34:36.467 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 33, 2024-05-21T03:34:36.467 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 1097, 2024-05-21T03:34:36.467 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-05-21T03:34:36.467 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1097, 2024-05-21T03:34:36.467 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 366.348295540, 2024-05-21T03:34:36.467 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.333954690 2024-05-21T03:34:36.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.468 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-05-21T03:34:36.468 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-05-21T03:34:36.468 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1097, 2024-05-21T03:34:36.468 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 1200 2024-05-21T03:34:36.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.468 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 1097, 2024-05-21T03:34:36.468 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 1, 2024-05-21T03:34:36.468 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 1096, 2024-05-21T03:34:36.468 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-05-21T03:34:36.468 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-05-21T03:34:36.468 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 0, 2024-05-21T03:34:36.468 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 10, 2024-05-21T03:34:36.468 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 1, 2024-05-21T03:34:36.469 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 208, 2024-05-21T03:34:36.469 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 29, 2024-05-21T03:34:36.469 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-05-21T03:34:36.469 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-05-21T03:34:36.469 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 0, 2024-05-21T03:34:36.469 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 0, 2024-05-21T03:34:36.469 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 762, 2024-05-21T03:34:36.469 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-05-21T03:34:36.469 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-05-21T03:34:36.469 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-05-21T03:34:36.469 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 0, 2024-05-21T03:34:36.469 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 20, 2024-05-21T03:34:36.469 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-05-21T03:34:36.469 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-05-21T03:34:36.470 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-05-21T03:34:36.470 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-05-21T03:34:36.470 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-05-21T03:34:36.470 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-05-21T03:34:36.470 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T03:34:36.470 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-05-21T03:34:36.470 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-05-21T03:34:36.470 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-05-21T03:34:36.470 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-05-21T03:34:36.470 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-05-21T03:34:36.470 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-05-21T03:34:36.470 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-05-21T03:34:36.470 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-05-21T03:34:36.470 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-05-21T03:34:36.471 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-05-21T03:34:36.471 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-05-21T03:34:36.471 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-05-21T03:34:36.471 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-05-21T03:34:36.471 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-05-21T03:34:36.471 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-05-21T03:34:36.471 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-05-21T03:34:36.471 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-05-21T03:34:36.471 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-05-21T03:34:36.471 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-05-21T03:34:36.471 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 91, 2024-05-21T03:34:36.471 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-05-21T03:34:36.471 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 11, 2024-05-21T03:34:36.472 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 12, 2024-05-21T03:34:36.472 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 12, 2024-05-21T03:34:36.472 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 0, 2024-05-21T03:34:36.472 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-05-21T03:34:36.472 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 114, 2024-05-21T03:34:36.472 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 0, 2024-05-21T03:34:36.472 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 56 2024-05-21T03:34:36.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.472 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-05-21T03:34:36.472 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 1, 2024-05-21T03:34:36.472 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-05-21T03:34:36.472 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-05-21T03:34:36.472 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-05-21T03:34:36.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.472 INFO:teuthology.orchestra.run.smithi165.stdout: "osd-slow-ops": { 2024-05-21T03:34:36.473 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 427 2024-05-21T03:34:36.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.473 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-05-21T03:34:36.473 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-05-21T03:34:36.473 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 25, 2024-05-21T03:34:36.473 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-05-21T03:34:36.473 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 1, 2024-05-21T03:34:36.473 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 750, 2024-05-21T03:34:36.473 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 30, 2024-05-21T03:34:36.473 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-05-21T03:34:36.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.473 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T03:34:36.473 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-21T03:34:36.473 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-05-21T03:34:36.474 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-21T03:34:36.474 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 8401, 2024-05-21T03:34:36.474 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 4734863, 2024-05-21T03:34:36.474 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:36.474 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 8401, 2024-05-21T03:34:36.474 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-05-21T03:34:36.474 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-05-21T03:34:36.474 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 8401, 2024-05-21T03:34:36.474 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 4734863, 2024-05-21T03:34:36.474 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-21T03:34:36.474 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.474 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.474 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.474 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:36.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.475 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-05-21T03:34:36.475 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-21T03:34:36.475 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-05-21T03:34:36.475 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-21T03:34:36.475 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-05-21T03:34:36.475 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-05-21T03:34:36.475 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:36.475 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-05-21T03:34:36.475 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 1097, 2024-05-21T03:34:36.475 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 2565814, 2024-05-21T03:34:36.475 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 1097, 2024-05-21T03:34:36.475 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 2565814, 2024-05-21T03:34:36.475 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-21T03:34:36.476 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.476 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.476 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.476 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:36.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.476 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-05-21T03:34:36.476 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-21T03:34:36.476 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-05-21T03:34:36.476 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-21T03:34:36.476 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-05-21T03:34:36.476 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-05-21T03:34:36.476 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:36.476 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-05-21T03:34:36.476 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 1097, 2024-05-21T03:34:36.476 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 1097, 2024-05-21T03:34:36.477 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 1097, 2024-05-21T03:34:36.477 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 1097, 2024-05-21T03:34:36.477 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-21T03:34:36.477 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.477 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.477 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.477 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:36.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.477 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-05-21T03:34:36.477 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-21T03:34:36.477 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-05-21T03:34:36.477 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-21T03:34:36.477 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 30, 2024-05-21T03:34:36.477 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 4230, 2024-05-21T03:34:36.477 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:36.477 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 30, 2024-05-21T03:34:36.478 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-05-21T03:34:36.478 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-05-21T03:34:36.478 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 1, 2024-05-21T03:34:36.478 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 4230, 2024-05-21T03:34:36.478 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-21T03:34:36.478 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.478 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.478 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.478 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:36.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:36.478 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x556c57e92500": { 2024-05-21T03:34:36.478 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-21T03:34:36.478 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-05-21T03:34:36.478 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-21T03:34:36.479 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 1942, 2024-05-21T03:34:36.479 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 2532580, 2024-05-21T03:34:36.479 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:36.479 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 1942, 2024-05-21T03:34:36.479 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-05-21T03:34:36.479 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-05-21T03:34:36.479 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 207, 2024-05-21T03:34:36.479 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 2532580, 2024-05-21T03:34:36.479 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-21T03:34:36.479 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:36.479 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:36.479 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:36.479 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:36.479 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:36.479 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-05-21T03:34:36.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:36 smithi027 ceph-mon[32179]: pgmap v328: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T03:34:37.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:36 smithi039 ceph-mon[36429]: pgmap v328: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T03:34:37.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:36 smithi165 ceph-mon[36269]: pgmap v328: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T03:34:37.266 INFO:tasks.check_counter:Daemon mds.f mds.exported=2 2024-05-21T03:34:37.266 INFO:tasks.check_counter:Daemon mds.f mds.imported=2 2024-05-21T03:34:37.266 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=5 2024-05-21T03:34:37.266 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=362 2024-05-21T03:34:37.266 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-05-21T03:34:37.266 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-05-21T03:34:37.267 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-05-21T03:34:37.267 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-05-21T03:34:37.267 DEBUG:tasks.check_counter:Getting stats from i 2024-05-21T03:34:37.267 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-21T03:34:37.505 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T03:34:38.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:38 smithi027 ceph-mon[32179]: pgmap v329: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:34:39.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:38 smithi039 ceph-mon[36429]: pgmap v329: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:34:39.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:38 smithi165 ceph-mon[36269]: pgmap v329: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:34:39.287 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-05-21T03:34:39.287 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T03:34:39.287 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 5038, 2024-05-21T03:34:39.287 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 6348, 2024-05-21T03:34:39.287 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 4023053, 2024-05-21T03:34:39.287 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 6152442, 2024-05-21T03:34:39.287 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 12, 2024-05-21T03:34:39.287 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 3, 2024-05-21T03:34:39.287 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.399549187, 2024-05-21T03:34:39.287 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.143952561, 2024-05-21T03:34:39.287 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.265040026, 2024-05-21T03:34:39.288 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.043518063, 2024-05-21T03:34:39.288 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:39.288 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 6350, 2024-05-21T03:34:39.288 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.839277766, 2024-05-21T03:34:39.288 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000132169 2024-05-21T03:34:39.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.288 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:39.288 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 9270, 2024-05-21T03:34:39.288 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.003562211, 2024-05-21T03:34:39.288 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000384 2024-05-21T03:34:39.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.288 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 2238448, 2024-05-21T03:34:39.288 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 151984 2024-05-21T03:34:39.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.289 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T03:34:39.289 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 1665, 2024-05-21T03:34:39.289 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 2664, 2024-05-21T03:34:39.289 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 532196, 2024-05-21T03:34:39.289 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 1788529, 2024-05-21T03:34:39.289 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 19, 2024-05-21T03:34:39.289 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 8, 2024-05-21T03:34:39.289 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.245751780, 2024-05-21T03:34:39.289 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.152248861, 2024-05-21T03:34:39.289 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.137702915, 2024-05-21T03:34:39.289 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.007656112, 2024-05-21T03:34:39.289 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:39.290 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2659, 2024-05-21T03:34:39.290 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.319859767, 2024-05-21T03:34:39.290 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000120293 2024-05-21T03:34:39.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.290 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:39.290 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 3402, 2024-05-21T03:34:39.290 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000755101, 2024-05-21T03:34:39.290 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000221 2024-05-21T03:34:39.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.290 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 2656, 2024-05-21T03:34:39.290 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 479648 2024-05-21T03:34:39.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.291 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T03:34:39.291 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 2908, 2024-05-21T03:34:39.291 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 4771, 2024-05-21T03:34:39.291 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 984455, 2024-05-21T03:34:39.291 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 2625597, 2024-05-21T03:34:39.291 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 10, 2024-05-21T03:34:39.291 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 7, 2024-05-21T03:34:39.291 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.252098348, 2024-05-21T03:34:39.291 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.119658915, 2024-05-21T03:34:39.291 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.140551923, 2024-05-21T03:34:39.291 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.010781757, 2024-05-21T03:34:39.291 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:39.291 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 4769, 2024-05-21T03:34:39.292 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.608996286, 2024-05-21T03:34:39.292 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000127698 2024-05-21T03:34:39.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.292 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:39.292 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 6001, 2024-05-21T03:34:39.292 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.001204238, 2024-05-21T03:34:39.292 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000200 2024-05-21T03:34:39.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.292 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 482400, 2024-05-21T03:34:39.292 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 1301872 2024-05-21T03:34:39.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.292 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-05-21T03:34:39.292 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-05-21T03:34:39.293 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-05-21T03:34:39.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.293 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-05-21T03:34:39.293 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-05-21T03:34:39.293 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-05-21T03:34:39.293 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 483, 2024-05-21T03:34:39.293 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.590222285, 2024-05-21T03:34:39.293 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001221992 2024-05-21T03:34:39.293 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:39.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.293 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-05-21T03:34:39.293 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-05-21T03:34:39.293 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-05-21T03:34:39.293 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 942, 2024-05-21T03:34:39.294 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.055818705, 2024-05-21T03:34:39.294 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000059255 2024-05-21T03:34:39.294 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:39.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.294 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-05-21T03:34:39.294 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 625, 2024-05-21T03:34:39.294 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 610, 2024-05-21T03:34:39.294 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-05-21T03:34:39.294 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 610, 2024-05-21T03:34:39.294 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 16.660361723, 2024-05-21T03:34:39.294 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.027312068 2024-05-21T03:34:39.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.294 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-05-21T03:34:39.295 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 15, 2024-05-21T03:34:39.295 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 0, 2024-05-21T03:34:39.295 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-05-21T03:34:39.295 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 11, 2024-05-21T03:34:39.295 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 0, 2024-05-21T03:34:39.295 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-05-21T03:34:39.295 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 542, 2024-05-21T03:34:39.295 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 0, 2024-05-21T03:34:39.295 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 1, 2024-05-21T03:34:39.295 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 541, 2024-05-21T03:34:39.295 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 541, 2024-05-21T03:34:39.295 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 377, 2024-05-21T03:34:39.295 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 0, 2024-05-21T03:34:39.296 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 0, 2024-05-21T03:34:39.296 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 2, 2024-05-21T03:34:39.296 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 1357, 2024-05-21T03:34:39.296 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 990, 2024-05-21T03:34:39.296 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 15, 2024-05-21T03:34:39.296 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 96, 2024-05-21T03:34:39.296 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-05-21T03:34:39.296 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-05-21T03:34:39.296 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 0, 2024-05-21T03:34:39.296 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 0, 2024-05-21T03:34:39.296 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-05-21T03:34:39.296 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 3, 2024-05-21T03:34:39.296 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 342, 2024-05-21T03:34:39.296 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 7, 2024-05-21T03:34:39.297 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 488, 2024-05-21T03:34:39.297 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-05-21T03:34:39.297 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 0, 2024-05-21T03:34:39.297 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-05-21T03:34:39.297 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 325, 2024-05-21T03:34:39.297 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 13424230508, 2024-05-21T03:34:39.297 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 6, 2024-05-21T03:34:39.297 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 0, 2024-05-21T03:34:39.297 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-05-21T03:34:39.297 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T03:34:39.297 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-05-21T03:34:39.297 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 0, 2024-05-21T03:34:39.297 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T03:34:39.297 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T03:34:39.298 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 0, 2024-05-21T03:34:39.298 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 0, 2024-05-21T03:34:39.298 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 376, 2024-05-21T03:34:39.298 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 521, 2024-05-21T03:34:39.298 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T03:34:39.298 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T03:34:39.298 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T03:34:39.298 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 562, 2024-05-21T03:34:39.298 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 127, 2024-05-21T03:34:39.298 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 11, 2024-05-21T03:34:39.298 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 547 2024-05-21T03:34:39.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.298 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-05-21T03:34:39.299 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 164, 2024-05-21T03:34:39.299 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 353, 2024-05-21T03:34:39.299 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 231, 2024-05-21T03:34:39.299 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 248, 2024-05-21T03:34:39.299 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 318, 2024-05-21T03:34:39.299 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 77, 2024-05-21T03:34:39.299 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-05-21T03:34:39.299 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-05-21T03:34:39.299 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 291, 2024-05-21T03:34:39.299 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 214, 2024-05-21T03:34:39.299 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 0, 2024-05-21T03:34:39.299 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-05-21T03:34:39.299 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-05-21T03:34:39.299 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-05-21T03:34:39.300 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-05-21T03:34:39.300 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-05-21T03:34:39.300 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-05-21T03:34:39.300 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_path": 5, 2024-05-21T03:34:39.300 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_inode": 875, 2024-05-21T03:34:39.300 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T03:34:39.300 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 3, 2024-05-21T03:34:39.300 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-05-21T03:34:39.300 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 2, 2024-05-21T03:34:39.300 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-05-21T03:34:39.300 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0 2024-05-21T03:34:39.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.300 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-05-21T03:34:39.300 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-05-21T03:34:39.301 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 947, 2024-05-21T03:34:39.301 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 0, 2024-05-21T03:34:39.301 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 0, 2024-05-21T03:34:39.301 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 947, 2024-05-21T03:34:39.301 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-05-21T03:34:39.301 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 0, 2024-05-21T03:34:39.301 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 2, 2024-05-21T03:34:39.301 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 0, 2024-05-21T03:34:39.301 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 0, 2024-05-21T03:34:39.301 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 2, 2024-05-21T03:34:39.301 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 2, 2024-05-21T03:34:39.301 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-05-21T03:34:39.301 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 0, 2024-05-21T03:34:39.301 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 4194304, 2024-05-21T03:34:39.302 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 7099339, 2024-05-21T03:34:39.302 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 0, 2024-05-21T03:34:39.302 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-05-21T03:34:39.302 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 263, 2024-05-21T03:34:39.302 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 1238.305083265, 2024-05-21T03:34:39.302 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 4.708384347 2024-05-21T03:34:39.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.302 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 0 2024-05-21T03:34:39.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.302 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-05-21T03:34:39.302 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 201, 2024-05-21T03:34:39.302 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 619, 2024-05-21T03:34:39.302 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 418, 2024-05-21T03:34:39.302 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 90, 2024-05-21T03:34:39.302 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 152, 2024-05-21T03:34:39.303 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 62, 2024-05-21T03:34:39.303 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 542, 2024-05-21T03:34:39.303 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 1133, 2024-05-21T03:34:39.303 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 591, 2024-05-21T03:34:39.303 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 0, 2024-05-21T03:34:39.303 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 573, 2024-05-21T03:34:39.303 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 573, 2024-05-21T03:34:39.303 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 65612, 2024-05-21T03:34:39.303 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 223608 2024-05-21T03:34:39.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.303 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-05-21T03:34:39.303 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 1340, 2024-05-21T03:34:39.303 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 656, 2024-05-21T03:34:39.303 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 625, 2024-05-21T03:34:39.303 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 19, 2024-05-21T03:34:39.304 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 1468, 2024-05-21T03:34:39.304 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-05-21T03:34:39.304 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:39.304 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:39.304 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:39.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.304 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-05-21T03:34:39.304 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-05-21T03:34:39.304 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.239115150, 2024-05-21T03:34:39.304 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.239115150 2024-05-21T03:34:39.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.304 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-05-21T03:34:39.304 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:39.304 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:39.305 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:39.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.305 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-05-21T03:34:39.305 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:39.305 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:39.305 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:39.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.305 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-05-21T03:34:39.305 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 231, 2024-05-21T03:34:39.305 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 14.591168922, 2024-05-21T03:34:39.305 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.063165233 2024-05-21T03:34:39.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.305 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-05-21T03:34:39.305 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:39.306 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:39.306 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:39.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.306 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-05-21T03:34:39.306 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:39.306 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:39.306 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:39.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.306 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-05-21T03:34:39.306 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:39.306 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:39.306 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:39.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.306 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-05-21T03:34:39.306 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:39.307 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:39.307 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:39.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.307 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-05-21T03:34:39.307 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:39.307 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:39.307 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:39.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.307 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-05-21T03:34:39.307 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:39.308 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:39.308 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:39.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.308 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-05-21T03:34:39.308 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:39.308 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:39.308 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:39.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.308 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-05-21T03:34:39.308 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:39.308 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:39.308 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:39.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.309 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-05-21T03:34:39.309 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:39.309 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:39.309 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:39.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.309 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-05-21T03:34:39.309 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 9, 2024-05-21T03:34:39.309 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.007767586, 2024-05-21T03:34:39.309 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000863065 2024-05-21T03:34:39.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.309 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-05-21T03:34:39.309 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 78, 2024-05-21T03:34:39.309 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.075774452, 2024-05-21T03:34:39.309 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000971467 2024-05-21T03:34:39.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.310 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-05-21T03:34:39.310 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:39.310 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:39.310 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:39.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.310 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-05-21T03:34:39.310 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:39.310 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:39.310 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:39.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.310 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-05-21T03:34:39.310 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:39.311 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:39.311 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:39.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.311 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-05-21T03:34:39.311 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 77, 2024-05-21T03:34:39.311 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 1.273689957, 2024-05-21T03:34:39.311 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.016541428 2024-05-21T03:34:39.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.311 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-05-21T03:34:39.311 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:39.311 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:39.311 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:39.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.311 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-05-21T03:34:39.312 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:39.312 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:39.312 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:39.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.312 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-05-21T03:34:39.312 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:39.312 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:39.312 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:39.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.312 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-05-21T03:34:39.312 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:39.312 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:39.312 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:39.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.312 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-05-21T03:34:39.313 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:39.313 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:39.313 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:39.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.313 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-05-21T03:34:39.313 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:39.313 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:39.313 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:39.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.313 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-05-21T03:34:39.313 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:39.313 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:39.313 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:39.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.313 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-05-21T03:34:39.314 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:39.314 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:39.314 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:39.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.314 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-05-21T03:34:39.314 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 214, 2024-05-21T03:34:39.314 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.472845656, 2024-05-21T03:34:39.314 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.002209559 2024-05-21T03:34:39.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.314 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-05-21T03:34:39.314 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-05-21T03:34:39.314 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-05-21T03:34:39.314 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:39.314 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:39.315 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:39.315 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:39.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.315 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-05-21T03:34:39.315 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 2, 2024-05-21T03:34:39.315 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 3, 2024-05-21T03:34:39.315 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 1, 2024-05-21T03:34:39.315 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 2, 2024-05-21T03:34:39.315 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-05-21T03:34:39.315 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 575, 2024-05-21T03:34:39.315 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 287, 2024-05-21T03:34:39.315 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 472, 2024-05-21T03:34:39.315 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-05-21T03:34:39.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.316 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-05-21T03:34:39.316 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-05-21T03:34:39.316 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-05-21T03:34:39.316 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T03:34:39.316 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-05-21T03:34:39.316 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T03:34:39.316 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-05-21T03:34:39.316 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T03:34:39.316 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T03:34:39.316 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T03:34:39.316 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T03:34:39.316 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T03:34:39.316 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-05-21T03:34:39.316 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T03:34:39.317 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T03:34:39.317 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-05-21T03:34:39.317 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-05-21T03:34:39.317 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-05-21T03:34:39.317 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-05-21T03:34:39.317 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T03:34:39.317 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T03:34:39.317 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T03:34:39.317 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T03:34:39.317 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T03:34:39.317 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-05-21T03:34:39.317 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-05-21T03:34:39.317 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-05-21T03:34:39.317 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T03:34:39.318 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T03:34:39.318 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-05-21T03:34:39.318 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-05-21T03:34:39.318 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-05-21T03:34:39.318 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-05-21T03:34:39.318 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T03:34:39.318 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-05-21T03:34:39.318 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T03:34:39.318 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-05-21T03:34:39.318 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 920313, 2024-05-21T03:34:39.318 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 445, 2024-05-21T03:34:39.318 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 0, 2024-05-21T03:34:39.318 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 0, 2024-05-21T03:34:39.318 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-05-21T03:34:39.319 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-05-21T03:34:39.319 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-05-21T03:34:39.319 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-05-21T03:34:39.319 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-05-21T03:34:39.319 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-05-21T03:34:39.319 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 8224, 2024-05-21T03:34:39.319 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 171, 2024-05-21T03:34:39.319 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T03:34:39.319 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-05-21T03:34:39.319 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-05-21T03:34:39.319 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-05-21T03:34:39.319 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 1146062, 2024-05-21T03:34:39.319 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 30779, 2024-05-21T03:34:39.319 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-05-21T03:34:39.319 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-05-21T03:34:39.320 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-05-21T03:34:39.320 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-05-21T03:34:39.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.320 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-05-21T03:34:39.320 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-05-21T03:34:39.320 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-05-21T03:34:39.320 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 1107, 2024-05-21T03:34:39.320 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 2980973, 2024-05-21T03:34:39.320 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 28, 2024-05-21T03:34:39.320 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 1107, 2024-05-21T03:34:39.320 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-05-21T03:34:39.320 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1107, 2024-05-21T03:34:39.320 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 1363.767704834, 2024-05-21T03:34:39.320 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 1.231949146 2024-05-21T03:34:39.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.321 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-05-21T03:34:39.321 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-05-21T03:34:39.321 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1107, 2024-05-21T03:34:39.321 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 1183 2024-05-21T03:34:39.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.321 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 1107, 2024-05-21T03:34:39.321 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 10, 2024-05-21T03:34:39.321 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 1097, 2024-05-21T03:34:39.321 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-05-21T03:34:39.321 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-05-21T03:34:39.321 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 0, 2024-05-21T03:34:39.321 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 10, 2024-05-21T03:34:39.321 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 10, 2024-05-21T03:34:39.321 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 273, 2024-05-21T03:34:39.322 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 26, 2024-05-21T03:34:39.322 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-05-21T03:34:39.322 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-05-21T03:34:39.322 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 0, 2024-05-21T03:34:39.322 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 0, 2024-05-21T03:34:39.322 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 730, 2024-05-21T03:34:39.322 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-05-21T03:34:39.322 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-05-21T03:34:39.322 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-05-21T03:34:39.322 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 0, 2024-05-21T03:34:39.322 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 20, 2024-05-21T03:34:39.322 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-05-21T03:34:39.322 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-05-21T03:34:39.322 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-05-21T03:34:39.323 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-05-21T03:34:39.323 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-05-21T03:34:39.323 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-05-21T03:34:39.323 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T03:34:39.323 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-05-21T03:34:39.323 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-05-21T03:34:39.323 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-05-21T03:34:39.323 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-05-21T03:34:39.323 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-05-21T03:34:39.323 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-05-21T03:34:39.323 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-05-21T03:34:39.323 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-05-21T03:34:39.323 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-05-21T03:34:39.323 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-05-21T03:34:39.323 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-05-21T03:34:39.324 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-05-21T03:34:39.324 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-05-21T03:34:39.324 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-05-21T03:34:39.324 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-05-21T03:34:39.324 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-05-21T03:34:39.324 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-05-21T03:34:39.324 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-05-21T03:34:39.324 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-05-21T03:34:39.324 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 91, 2024-05-21T03:34:39.324 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-05-21T03:34:39.324 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 11, 2024-05-21T03:34:39.324 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 12, 2024-05-21T03:34:39.324 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 12, 2024-05-21T03:34:39.324 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 0, 2024-05-21T03:34:39.324 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-05-21T03:34:39.325 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 90, 2024-05-21T03:34:39.325 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 0, 2024-05-21T03:34:39.325 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 24 2024-05-21T03:34:39.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.325 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-05-21T03:34:39.325 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 1, 2024-05-21T03:34:39.325 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-05-21T03:34:39.325 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-05-21T03:34:39.325 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-05-21T03:34:39.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.325 INFO:teuthology.orchestra.run.smithi165.stdout: "osd-slow-ops": { 2024-05-21T03:34:39.325 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 1534 2024-05-21T03:34:39.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.325 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-05-21T03:34:39.325 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-05-21T03:34:39.326 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 25, 2024-05-21T03:34:39.326 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-05-21T03:34:39.326 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 6, 2024-05-21T03:34:39.326 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 718, 2024-05-21T03:34:39.326 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 214, 2024-05-21T03:34:39.326 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-05-21T03:34:39.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.326 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T03:34:39.326 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-21T03:34:39.326 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-05-21T03:34:39.326 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-21T03:34:39.326 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 9607, 2024-05-21T03:34:39.326 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 4597155, 2024-05-21T03:34:39.326 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:39.326 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 9607, 2024-05-21T03:34:39.327 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-05-21T03:34:39.327 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-05-21T03:34:39.327 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 9607, 2024-05-21T03:34:39.327 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 4597155, 2024-05-21T03:34:39.327 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-21T03:34:39.327 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:39.327 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:39.327 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:39.327 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:39.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.327 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-05-21T03:34:39.327 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-21T03:34:39.327 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-05-21T03:34:39.327 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-21T03:34:39.328 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-05-21T03:34:39.328 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-05-21T03:34:39.328 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:39.328 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-05-21T03:34:39.328 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 1107, 2024-05-21T03:34:39.328 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 3012531, 2024-05-21T03:34:39.328 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 1107, 2024-05-21T03:34:39.328 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 3012531, 2024-05-21T03:34:39.328 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-21T03:34:39.328 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:39.328 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:39.328 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:39.328 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:39.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.328 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-05-21T03:34:39.329 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-21T03:34:39.329 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-05-21T03:34:39.329 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-21T03:34:39.329 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-05-21T03:34:39.329 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-05-21T03:34:39.329 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:39.329 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-05-21T03:34:39.329 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 1107, 2024-05-21T03:34:39.329 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 1107, 2024-05-21T03:34:39.329 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 1107, 2024-05-21T03:34:39.329 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 1107, 2024-05-21T03:34:39.329 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-21T03:34:39.329 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:39.330 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:39.330 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:39.330 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:39.330 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.330 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-05-21T03:34:39.330 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-21T03:34:39.330 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-05-21T03:34:39.330 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-21T03:34:39.330 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 214, 2024-05-21T03:34:39.330 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 31558, 2024-05-21T03:34:39.330 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:39.330 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 214, 2024-05-21T03:34:39.330 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-05-21T03:34:39.330 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-05-21T03:34:39.331 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 10, 2024-05-21T03:34:39.331 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 31558, 2024-05-21T03:34:39.331 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-21T03:34:39.331 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:39.331 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:39.331 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:39.331 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:39.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:39.331 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x563c11eb0500": { 2024-05-21T03:34:39.331 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-21T03:34:39.331 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-05-21T03:34:39.331 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-21T03:34:39.331 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 947, 2024-05-21T03:34:39.331 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 2905035, 2024-05-21T03:34:39.331 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:39.332 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 947, 2024-05-21T03:34:39.332 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-05-21T03:34:39.332 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-05-21T03:34:39.332 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 263, 2024-05-21T03:34:39.332 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 2905035, 2024-05-21T03:34:39.332 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-21T03:34:39.332 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:39.332 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:39.332 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:39.332 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:39.332 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:39.332 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-05-21T03:34:40.067 INFO:tasks.check_counter:Daemon mds.i mds.exported=3 2024-05-21T03:34:40.067 INFO:tasks.check_counter:Daemon mds.i mds.imported=7 2024-05-21T03:34:40.067 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=164 2024-05-21T03:34:40.067 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=353 2024-05-21T03:34:40.067 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-05-21T03:34:40.067 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-05-21T03:34:40.068 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-05-21T03:34:40.068 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-05-21T03:34:40.068 DEBUG:tasks.check_counter:Getting stats from l 2024-05-21T03:34:40.068 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-21T03:34:40.308 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T03:34:40.681 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:40 smithi165 ceph-mon[36269]: pgmap v330: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:34:40.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:40 smithi027 ceph-mon[32179]: pgmap v330: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:34:41.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:40 smithi039 ceph-mon[36429]: pgmap v330: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-05-21T03:34:42.010 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-05-21T03:34:42.010 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T03:34:42.010 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 2134, 2024-05-21T03:34:42.010 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 2136, 2024-05-21T03:34:42.010 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 3512772, 2024-05-21T03:34:42.010 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 678619, 2024-05-21T03:34:42.011 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 6, 2024-05-21T03:34:42.011 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 2, 2024-05-21T03:34:42.011 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.184650640, 2024-05-21T03:34:42.011 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.057156778, 2024-05-21T03:34:42.011 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.047457950, 2024-05-21T03:34:42.011 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.066032083, 2024-05-21T03:34:42.011 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:42.011 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2134, 2024-05-21T03:34:42.011 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.085669085, 2024-05-21T03:34:42.011 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000040144 2024-05-21T03:34:42.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.011 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:42.011 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.011 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.012 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.012 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-21T03:34:42.012 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 0 2024-05-21T03:34:42.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.012 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T03:34:42.012 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 8, 2024-05-21T03:34:42.012 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 187, 2024-05-21T03:34:42.012 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 3868, 2024-05-21T03:34:42.012 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 474330, 2024-05-21T03:34:42.012 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 6, 2024-05-21T03:34:42.012 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 4, 2024-05-21T03:34:42.012 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.011022221, 2024-05-21T03:34:42.012 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.006853113, 2024-05-21T03:34:42.012 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.002167423, 2024-05-21T03:34:42.013 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.000074090, 2024-05-21T03:34:42.013 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:42.013 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 185, 2024-05-21T03:34:42.013 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.012320856, 2024-05-21T03:34:42.013 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000066599 2024-05-21T03:34:42.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.013 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:42.013 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.013 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.013 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.013 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-21T03:34:42.013 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 473632 2024-05-21T03:34:42.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.014 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T03:34:42.014 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 295, 2024-05-21T03:34:42.014 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 257, 2024-05-21T03:34:42.014 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 2210124, 2024-05-21T03:34:42.014 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 150990, 2024-05-21T03:34:42.014 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 3, 2024-05-21T03:34:42.014 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 2, 2024-05-21T03:34:42.014 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.054292818, 2024-05-21T03:34:42.014 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.014739235, 2024-05-21T03:34:42.014 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.049013070, 2024-05-21T03:34:42.014 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.005905436, 2024-05-21T03:34:42.014 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T03:34:42.014 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 256, 2024-05-21T03:34:42.014 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.013221149, 2024-05-21T03:34:42.015 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000051645 2024-05-21T03:34:42.015 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.015 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-05-21T03:34:42.015 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.015 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.015 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.015 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.015 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 2209856, 2024-05-21T03:34:42.015 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 150672 2024-05-21T03:34:42.015 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.015 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-05-21T03:34:42.015 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-05-21T03:34:42.015 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-05-21T03:34:42.015 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.016 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-05-21T03:34:42.016 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-05-21T03:34:42.016 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-05-21T03:34:42.016 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 3720, 2024-05-21T03:34:42.016 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.099957865, 2024-05-21T03:34:42.016 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000026870 2024-05-21T03:34:42.016 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:42.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.016 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-05-21T03:34:42.016 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-05-21T03:34:42.016 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-05-21T03:34:42.016 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.016 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.017 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.017 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:42.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.017 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-05-21T03:34:42.017 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 0, 2024-05-21T03:34:42.017 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 0, 2024-05-21T03:34:42.017 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-05-21T03:34:42.017 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.017 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.017 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.017 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-05-21T03:34:42.017 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 0, 2024-05-21T03:34:42.017 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 0, 2024-05-21T03:34:42.018 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-05-21T03:34:42.018 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 0, 2024-05-21T03:34:42.018 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 0, 2024-05-21T03:34:42.018 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-05-21T03:34:42.018 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 749, 2024-05-21T03:34:42.018 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 2, 2024-05-21T03:34:42.018 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 747, 2024-05-21T03:34:42.018 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 0, 2024-05-21T03:34:42.018 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 746, 2024-05-21T03:34:42.018 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 0, 2024-05-21T03:34:42.018 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 0, 2024-05-21T03:34:42.018 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 0, 2024-05-21T03:34:42.018 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 1, 2024-05-21T03:34:42.019 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 0, 2024-05-21T03:34:42.019 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 0, 2024-05-21T03:34:42.019 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 0, 2024-05-21T03:34:42.019 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 0, 2024-05-21T03:34:42.019 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-05-21T03:34:42.019 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-05-21T03:34:42.019 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 0, 2024-05-21T03:34:42.019 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 0, 2024-05-21T03:34:42.019 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-05-21T03:34:42.019 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 0, 2024-05-21T03:34:42.019 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 0, 2024-05-21T03:34:42.019 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 0, 2024-05-21T03:34:42.019 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 0, 2024-05-21T03:34:42.019 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-05-21T03:34:42.020 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 0, 2024-05-21T03:34:42.020 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-05-21T03:34:42.020 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 0, 2024-05-21T03:34:42.020 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 0, 2024-05-21T03:34:42.020 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 0, 2024-05-21T03:34:42.020 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 0, 2024-05-21T03:34:42.020 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-05-21T03:34:42.020 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T03:34:42.020 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-05-21T03:34:42.020 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 0, 2024-05-21T03:34:42.020 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T03:34:42.020 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T03:34:42.020 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 0, 2024-05-21T03:34:42.020 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 0, 2024-05-21T03:34:42.021 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 0, 2024-05-21T03:34:42.021 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 0, 2024-05-21T03:34:42.021 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T03:34:42.021 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T03:34:42.021 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T03:34:42.021 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 0, 2024-05-21T03:34:42.021 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 0, 2024-05-21T03:34:42.021 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 0, 2024-05-21T03:34:42.021 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 0 2024-05-21T03:34:42.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.021 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-05-21T03:34:42.021 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 0, 2024-05-21T03:34:42.021 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 0, 2024-05-21T03:34:42.021 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 0, 2024-05-21T03:34:42.021 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 0, 2024-05-21T03:34:42.022 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 0, 2024-05-21T03:34:42.022 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 0, 2024-05-21T03:34:42.022 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-05-21T03:34:42.022 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-05-21T03:34:42.022 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 0, 2024-05-21T03:34:42.022 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 0, 2024-05-21T03:34:42.022 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 0, 2024-05-21T03:34:42.022 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-05-21T03:34:42.022 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-05-21T03:34:42.022 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-05-21T03:34:42.022 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-05-21T03:34:42.022 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-05-21T03:34:42.022 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-05-21T03:34:42.022 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_path": 0, 2024-05-21T03:34:42.023 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_inode": 0, 2024-05-21T03:34:42.023 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T03:34:42.023 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 0, 2024-05-21T03:34:42.023 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-05-21T03:34:42.023 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 0, 2024-05-21T03:34:42.023 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-05-21T03:34:42.023 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0 2024-05-21T03:34:42.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.023 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-05-21T03:34:42.023 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-05-21T03:34:42.023 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 0, 2024-05-21T03:34:42.023 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 0, 2024-05-21T03:34:42.023 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 0, 2024-05-21T03:34:42.023 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 947, 2024-05-21T03:34:42.023 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-05-21T03:34:42.024 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 0, 2024-05-21T03:34:42.024 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 0, 2024-05-21T03:34:42.024 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 0, 2024-05-21T03:34:42.024 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 0, 2024-05-21T03:34:42.024 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 2, 2024-05-21T03:34:42.024 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 2, 2024-05-21T03:34:42.024 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-05-21T03:34:42.024 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 0, 2024-05-21T03:34:42.024 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 4194304, 2024-05-21T03:34:42.024 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 7099339, 2024-05-21T03:34:42.024 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 7095315, 2024-05-21T03:34:42.024 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-05-21T03:34:42.024 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.024 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.025 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.025 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 947 2024-05-21T03:34:42.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.025 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-05-21T03:34:42.025 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 190, 2024-05-21T03:34:42.025 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 601, 2024-05-21T03:34:42.025 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 411, 2024-05-21T03:34:42.025 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 79, 2024-05-21T03:34:42.025 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 94, 2024-05-21T03:34:42.025 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 15, 2024-05-21T03:34:42.025 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 749, 2024-05-21T03:34:42.025 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 1104, 2024-05-21T03:34:42.025 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 355, 2024-05-21T03:34:42.025 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 0, 2024-05-21T03:34:42.025 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 0, 2024-05-21T03:34:42.026 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 0, 2024-05-21T03:34:42.026 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 55080, 2024-05-21T03:34:42.026 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 215416 2024-05-21T03:34:42.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.026 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-05-21T03:34:42.026 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 0, 2024-05-21T03:34:42.026 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 0, 2024-05-21T03:34:42.026 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 0, 2024-05-21T03:34:42.026 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 0, 2024-05-21T03:34:42.026 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 0, 2024-05-21T03:34:42.026 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-05-21T03:34:42.026 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.026 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.026 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.027 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-05-21T03:34:42.027 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.027 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.027 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.027 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-05-21T03:34:42.027 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.027 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.027 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.027 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-05-21T03:34:42.027 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.027 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.028 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.028 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-05-21T03:34:42.028 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.028 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.028 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.028 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-05-21T03:34:42.028 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.028 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.028 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.028 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-05-21T03:34:42.028 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.028 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.029 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.029 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-05-21T03:34:42.029 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.029 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.029 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.029 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-05-21T03:34:42.029 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.029 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.029 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.029 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-05-21T03:34:42.029 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.029 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.030 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.030 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-05-21T03:34:42.030 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.030 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.030 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.030 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-05-21T03:34:42.031 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.031 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.031 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.031 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-05-21T03:34:42.031 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.031 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.031 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.031 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-05-21T03:34:42.031 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.031 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.032 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.032 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-05-21T03:34:42.032 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.032 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.032 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.032 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-05-21T03:34:42.032 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.032 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.032 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.032 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-05-21T03:34:42.033 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.033 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.033 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.033 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-05-21T03:34:42.033 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.033 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.033 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.033 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-05-21T03:34:42.033 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.033 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.033 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.033 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-05-21T03:34:42.034 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.034 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.034 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.034 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-05-21T03:34:42.034 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.034 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.034 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.034 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-05-21T03:34:42.034 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.034 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.034 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.035 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-05-21T03:34:42.035 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.035 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.035 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.035 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-05-21T03:34:42.035 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.035 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.035 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.035 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-05-21T03:34:42.035 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.035 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.035 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.036 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-05-21T03:34:42.036 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.036 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.036 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.036 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-05-21T03:34:42.036 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.036 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.036 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.036 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-05-21T03:34:42.036 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.036 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.036 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.037 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-05-21T03:34:42.037 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.037 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.037 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.037 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-05-21T03:34:42.037 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-05-21T03:34:42.037 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-05-21T03:34:42.037 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.037 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.037 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.037 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:42.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.038 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-05-21T03:34:42.038 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 2, 2024-05-21T03:34:42.038 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 3, 2024-05-21T03:34:42.038 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 1, 2024-05-21T03:34:42.038 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 2, 2024-05-21T03:34:42.038 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-05-21T03:34:42.038 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 0, 2024-05-21T03:34:42.038 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 0, 2024-05-21T03:34:42.038 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 4018, 2024-05-21T03:34:42.038 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-05-21T03:34:42.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.038 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-05-21T03:34:42.038 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-05-21T03:34:42.038 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-05-21T03:34:42.038 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T03:34:42.039 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-05-21T03:34:42.039 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T03:34:42.039 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-05-21T03:34:42.039 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T03:34:42.039 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T03:34:42.039 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T03:34:42.039 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T03:34:42.039 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T03:34:42.039 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-05-21T03:34:42.039 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T03:34:42.039 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T03:34:42.039 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-05-21T03:34:42.039 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-05-21T03:34:42.039 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-05-21T03:34:42.040 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-05-21T03:34:42.040 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T03:34:42.040 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T03:34:42.040 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T03:34:42.040 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T03:34:42.040 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T03:34:42.040 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-05-21T03:34:42.040 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-05-21T03:34:42.040 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-05-21T03:34:42.040 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T03:34:42.040 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T03:34:42.040 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-05-21T03:34:42.040 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-05-21T03:34:42.040 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-05-21T03:34:42.041 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-05-21T03:34:42.041 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T03:34:42.041 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-05-21T03:34:42.041 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T03:34:42.041 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-05-21T03:34:42.041 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 194207, 2024-05-21T03:34:42.041 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 375, 2024-05-21T03:34:42.041 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 0, 2024-05-21T03:34:42.041 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 0, 2024-05-21T03:34:42.041 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-05-21T03:34:42.041 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-05-21T03:34:42.041 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-05-21T03:34:42.041 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-05-21T03:34:42.041 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-05-21T03:34:42.042 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-05-21T03:34:42.042 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 8224, 2024-05-21T03:34:42.042 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 171, 2024-05-21T03:34:42.042 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T03:34:42.042 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-05-21T03:34:42.042 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-05-21T03:34:42.042 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-05-21T03:34:42.042 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 1222107, 2024-05-21T03:34:42.042 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 28667, 2024-05-21T03:34:42.042 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-05-21T03:34:42.042 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-05-21T03:34:42.042 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-05-21T03:34:42.042 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-05-21T03:34:42.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.043 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-05-21T03:34:42.043 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-05-21T03:34:42.043 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-05-21T03:34:42.043 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 2137, 2024-05-21T03:34:42.043 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 21, 2024-05-21T03:34:42.043 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 3, 2024-05-21T03:34:42.043 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 2137, 2024-05-21T03:34:42.043 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-05-21T03:34:42.043 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2137, 2024-05-21T03:34:42.043 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 60.347057186, 2024-05-21T03:34:42.043 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.028239147 2024-05-21T03:34:42.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.043 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-05-21T03:34:42.043 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-05-21T03:34:42.044 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2137, 2024-05-21T03:34:42.044 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 2138 2024-05-21T03:34:42.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.044 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 2137, 2024-05-21T03:34:42.044 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 2137, 2024-05-21T03:34:42.044 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 0, 2024-05-21T03:34:42.044 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-05-21T03:34:42.044 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-05-21T03:34:42.044 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 1299, 2024-05-21T03:34:42.044 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 0, 2024-05-21T03:34:42.044 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 836, 2024-05-21T03:34:42.044 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 0, 2024-05-21T03:34:42.044 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 0, 2024-05-21T03:34:42.044 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-05-21T03:34:42.044 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-05-21T03:34:42.045 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 0, 2024-05-21T03:34:42.045 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 0, 2024-05-21T03:34:42.045 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 0, 2024-05-21T03:34:42.045 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-05-21T03:34:42.045 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-05-21T03:34:42.045 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-05-21T03:34:42.045 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 1, 2024-05-21T03:34:42.045 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 0, 2024-05-21T03:34:42.045 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-05-21T03:34:42.045 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-05-21T03:34:42.045 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-05-21T03:34:42.045 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-05-21T03:34:42.045 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-05-21T03:34:42.045 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-05-21T03:34:42.045 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T03:34:42.046 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-05-21T03:34:42.046 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-05-21T03:34:42.046 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-05-21T03:34:42.046 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-05-21T03:34:42.046 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-05-21T03:34:42.046 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-05-21T03:34:42.046 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-05-21T03:34:42.046 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-05-21T03:34:42.046 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-05-21T03:34:42.046 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-05-21T03:34:42.046 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-05-21T03:34:42.046 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-05-21T03:34:42.046 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-05-21T03:34:42.046 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-05-21T03:34:42.047 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-05-21T03:34:42.047 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-05-21T03:34:42.047 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-05-21T03:34:42.047 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-05-21T03:34:42.047 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-05-21T03:34:42.047 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 91, 2024-05-21T03:34:42.047 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-05-21T03:34:42.047 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 11, 2024-05-21T03:34:42.047 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 4, 2024-05-21T03:34:42.047 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 4, 2024-05-21T03:34:42.047 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 0, 2024-05-21T03:34:42.047 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-05-21T03:34:42.047 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 0, 2024-05-21T03:34:42.047 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 2, 2024-05-21T03:34:42.047 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 0 2024-05-21T03:34:42.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.048 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-05-21T03:34:42.048 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 0, 2024-05-21T03:34:42.048 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-05-21T03:34:42.048 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-05-21T03:34:42.048 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-05-21T03:34:42.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.048 INFO:teuthology.orchestra.run.smithi165.stdout: "osd-slow-ops": { 2024-05-21T03:34:42.048 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 0 2024-05-21T03:34:42.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.048 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-05-21T03:34:42.048 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-05-21T03:34:42.048 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 0, 2024-05-21T03:34:42.049 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-05-21T03:34:42.049 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 0, 2024-05-21T03:34:42.049 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 0, 2024-05-21T03:34:42.049 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 0, 2024-05-21T03:34:42.049 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-05-21T03:34:42.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.049 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T03:34:42.049 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-21T03:34:42.049 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-05-21T03:34:42.049 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-21T03:34:42.049 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 2433, 2024-05-21T03:34:42.049 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 5489847, 2024-05-21T03:34:42.049 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:42.049 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 2433, 2024-05-21T03:34:42.050 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-05-21T03:34:42.050 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-05-21T03:34:42.050 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 2433, 2024-05-21T03:34:42.050 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 5489847, 2024-05-21T03:34:42.050 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-21T03:34:42.050 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.050 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.050 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.050 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:42.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.050 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-05-21T03:34:42.050 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-21T03:34:42.050 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-05-21T03:34:42.050 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-21T03:34:42.051 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-05-21T03:34:42.051 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-05-21T03:34:42.051 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:42.051 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-05-21T03:34:42.051 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 2137, 2024-05-21T03:34:42.051 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 2905040, 2024-05-21T03:34:42.051 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 2137, 2024-05-21T03:34:42.051 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 2905040, 2024-05-21T03:34:42.051 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-21T03:34:42.051 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.051 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.051 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.051 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:42.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.051 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-05-21T03:34:42.052 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-21T03:34:42.052 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-05-21T03:34:42.052 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-21T03:34:42.052 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-05-21T03:34:42.052 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-05-21T03:34:42.052 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:42.052 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-05-21T03:34:42.052 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 2137, 2024-05-21T03:34:42.052 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 2137, 2024-05-21T03:34:42.052 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 2137, 2024-05-21T03:34:42.052 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 2137, 2024-05-21T03:34:42.052 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-21T03:34:42.052 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.052 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.052 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.053 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:42.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.053 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-05-21T03:34:42.053 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-21T03:34:42.053 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-05-21T03:34:42.053 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-21T03:34:42.053 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-05-21T03:34:42.053 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-05-21T03:34:42.053 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:42.053 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-05-21T03:34:42.053 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-05-21T03:34:42.053 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-05-21T03:34:42.053 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 0, 2024-05-21T03:34:42.053 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 0, 2024-05-21T03:34:42.054 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-21T03:34:42.054 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.054 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.054 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.054 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:42.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-05-21T03:34:42.054 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x559be523e6e0": { 2024-05-21T03:34:42.054 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-05-21T03:34:42.054 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-05-21T03:34:42.054 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-05-21T03:34:42.054 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-05-21T03:34:42.054 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-05-21T03:34:42.054 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-05-21T03:34:42.054 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-05-21T03:34:42.055 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-05-21T03:34:42.055 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-05-21T03:34:42.055 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 0, 2024-05-21T03:34:42.055 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 0, 2024-05-21T03:34:42.055 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-05-21T03:34:42.055 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-05-21T03:34:42.055 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-05-21T03:34:42.055 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-05-21T03:34:42.055 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:42.055 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-05-21T03:34:42.055 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-05-21T03:34:42.928 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-05-21T03:34:42.928 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-05-21T03:34:42.928 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-05-21T03:34:42.928 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-05-21T03:34:42.928 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-05-21T03:34:42.928 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-05-21T03:34:42.928 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-05-21T03:34:42.928 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-05-21T03:34:42.929 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-21T03:34:42.942 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-21T03:34:42.942 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-05-21T03:34:42.942 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-21T03:34:42.964 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-05-21T03:34:42.964 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-21T03:34:43.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:42 smithi039 ceph-mon[36429]: pgmap v331: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-21T03:34:43.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:42 smithi165 ceph-mon[36269]: pgmap v331: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-21T03:34:43.025 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd blocklist ls 2024-05-21T03:34:43.093 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:42 smithi027 ceph-mon[32179]: pgmap v331: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-21T03:34:43.330 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:34:44.846 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:44 smithi027 ceph-mon[32179]: pgmap v332: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:34:45.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:44 smithi039 ceph-mon[36429]: pgmap v332: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:34:45.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:44 smithi165 ceph-mon[36269]: pgmap v332: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T03:34:45.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.211+0000 7fed5811d700 1 -- 172.21.15.27:0/1019979908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed50103850 msgr2=0x7fed50103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.211+0000 7fed5811d700 1 --2- 172.21.15.27:0/1019979908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed50103850 0x7fed50103c30 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fed4000b0d0 tx=0x7fed4000b3e0 comp rx=0 tx=0).stop 2024-05-21T03:34:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.211+0000 7fed5811d700 1 -- 172.21.15.27:0/1019979908 shutdown_connections 2024-05-21T03:34:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.211+0000 7fed5811d700 1 --2- 172.21.15.27:0/1019979908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed50108920 0x7fed501104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.211+0000 7fed5811d700 1 --2- 172.21.15.27:0/1019979908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed50103850 0x7fed50103c30 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.211+0000 7fed5811d700 1 --2- 172.21.15.27:0/1019979908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed50104200 0x7fed501081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.211+0000 7fed5811d700 1 -- 172.21.15.27:0/1019979908 >> 172.21.15.27:0/1019979908 conn(0x7fed5007a5d0 msgr2=0x7fed5007a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:34:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.211+0000 7fed5811d700 1 -- 172.21.15.27:0/1019979908 shutdown_connections 2024-05-21T03:34:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.211+0000 7fed5811d700 1 -- 172.21.15.27:0/1019979908 wait complete. 2024-05-21T03:34:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.212+0000 7fed5811d700 1 Processor -- start 2024-05-21T03:34:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.212+0000 7fed5811d700 1 -- start start 2024-05-21T03:34:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.212+0000 7fed5811d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed50103850 0x7fed5019f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.212+0000 7fed5811d700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed50104200 0x7fed5019fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.212+0000 7fed5811d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed50108920 0x7fed501999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:45.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.212+0000 7fed5811d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fed50113750 con 0x7fed50103850 2024-05-21T03:34:45.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.212+0000 7fed5811d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fed501135d0 con 0x7fed50108920 2024-05-21T03:34:45.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.212+0000 7fed5811d700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fed501138d0 con 0x7fed50104200 2024-05-21T03:34:45.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.213+0000 7fed55eb9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed50103850 0x7fed5019f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:45.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.213+0000 7fed55eb9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed50103850 0x7fed5019f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52138/0 (socket says 172.21.15.27:52138) 2024-05-21T03:34:45.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.213+0000 7fed55eb9700 1 -- 172.21.15.27:0/2660246348 learned_addr learned my addr 172.21.15.27:0/2660246348 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:34:45.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.213+0000 7fed556b8700 1 --2- 172.21.15.27:0/2660246348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed50104200 0x7fed5019fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:45.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.213+0000 7fed566ba700 1 --2- 172.21.15.27:0/2660246348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed50108920 0x7fed501999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:45.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.213+0000 7fed55eb9700 1 -- 172.21.15.27:0/2660246348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed50104200 msgr2=0x7fed5019fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:45.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.213+0000 7fed55eb9700 1 --2- 172.21.15.27:0/2660246348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed50104200 0x7fed5019fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:45.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.213+0000 7fed55eb9700 1 -- 172.21.15.27:0/2660246348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed50108920 msgr2=0x7fed501999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:45.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.213+0000 7fed55eb9700 1 --2- 172.21.15.27:0/2660246348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed50108920 0x7fed501999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:45.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.213+0000 7fed55eb9700 1 -- 172.21.15.27:0/2660246348 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed5019a150 con 0x7fed50103850 2024-05-21T03:34:45.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.213+0000 7fed55eb9700 1 --2- 172.21.15.27:0/2660246348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed50103850 0x7fed5019f830 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fed40009fd0 tx=0x7fed40007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:34:45.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.213+0000 7fed4effd700 1 -- 172.21.15.27:0/2660246348 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed40021020 con 0x7fed50103850 2024-05-21T03:34:45.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.213+0000 7fed5811d700 1 -- 172.21.15.27:0/2660246348 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed5019a3e0 con 0x7fed50103850 2024-05-21T03:34:45.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.213+0000 7fed4effd700 1 -- 172.21.15.27:0/2660246348 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed40004c50 con 0x7fed50103850 2024-05-21T03:34:45.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.213+0000 7fed4effd700 1 -- 172.21.15.27:0/2660246348 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed400052b0 con 0x7fed50103850 2024-05-21T03:34:45.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.213+0000 7fed5811d700 1 -- 172.21.15.27:0/2660246348 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed501a6700 con 0x7fed50103850 2024-05-21T03:34:45.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.214+0000 7fed5811d700 1 -- 172.21.15.27:0/2660246348 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed5006a230 con 0x7fed50103850 2024-05-21T03:34:45.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.216+0000 7fed4effd700 1 -- 172.21.15.27:0/2660246348 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7fed40024020 con 0x7fed50103850 2024-05-21T03:34:45.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.217+0000 7fed4effd700 1 --2- 172.21.15.27:0/2660246348 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fed380b29f0 0x7fed380b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:45.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.217+0000 7fed556b8700 1 --2- 172.21.15.27:0/2660246348 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fed380b29f0 0x7fed380b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:45.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.217+0000 7fed4effd700 1 -- 172.21.15.27:0/2660246348 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7fed4000f020 con 0x7fed50103850 2024-05-21T03:34:45.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.217+0000 7fed556b8700 1 --2- 172.21.15.27:0/2660246348 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fed380b29f0 0x7fed380b4eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fed501031b0 tx=0x7fed4400b410 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:34:45.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.220+0000 7fed4effd700 1 -- 172.21.15.27:0/2660246348 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fed400aba20 con 0x7fed50103850 2024-05-21T03:34:45.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.448+0000 7fed5811d700 1 -- 172.21.15.27:0/2660246348 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fed5002e560 con 0x7fed50103850 2024-05-21T03:34:45.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.449+0000 7fed4effd700 1 -- 172.21.15.27:0/2660246348 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v91) ==== 81+0+941 (secure 0 0 0) 0x7fed400ab320 con 0x7fed50103850 2024-05-21T03:34:45.449 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/744303480 2024-05-22T03:24:01.382214+0000 2024-05-21T03:34:45.449 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3254504266 2024-05-22T03:24:01.382214+0000 2024-05-21T03:34:45.449 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/4135246381 2024-05-22T03:24:01.382214+0000 2024-05-21T03:34:45.449 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/1185518061 2024-05-22T03:13:13.249220+0000 2024-05-21T03:34:45.449 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/4189166571 2024-05-22T03:24:01.382214+0000 2024-05-21T03:34:45.449 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/1185518061 2024-05-22T03:13:13.249220+0000 2024-05-21T03:34:45.450 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/4194431212 2024-05-22T03:13:13.249220+0000 2024-05-21T03:34:45.450 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/3683408243 2024-05-22T03:24:01.382214+0000 2024-05-21T03:34:45.450 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1890032101 2024-05-22T03:13:13.249220+0000 2024-05-21T03:34:45.450 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/175229478 2024-05-22T03:12:45.739195+0000 2024-05-21T03:34:45.450 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/999325706 2024-05-22T03:12:45.739195+0000 2024-05-21T03:34:45.450 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/175229478 2024-05-22T03:12:45.739195+0000 2024-05-21T03:34:45.450 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/3683408243 2024-05-22T03:24:01.382214+0000 2024-05-21T03:34:45.450 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/4184128736 2024-05-22T03:13:13.249220+0000 2024-05-21T03:34:45.450 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3000197506 2024-05-22T03:12:45.739195+0000 2024-05-21T03:34:45.450 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/972677319 2024-05-22T03:12:45.739195+0000 2024-05-21T03:34:45.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.451+0000 7fed5811d700 1 -- 172.21.15.27:0/2660246348 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fed380b29f0 msgr2=0x7fed380b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:45.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.451+0000 7fed5811d700 1 --2- 172.21.15.27:0/2660246348 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fed380b29f0 0x7fed380b4eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fed501031b0 tx=0x7fed4400b410 comp rx=0 tx=0).stop 2024-05-21T03:34:45.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.451+0000 7fed5811d700 1 -- 172.21.15.27:0/2660246348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed50103850 msgr2=0x7fed5019f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:45.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.451+0000 7fed5811d700 1 --2- 172.21.15.27:0/2660246348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed50103850 0x7fed5019f830 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fed40009fd0 tx=0x7fed40007a20 comp rx=0 tx=0).stop 2024-05-21T03:34:45.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.451+0000 7fed5811d700 1 -- 172.21.15.27:0/2660246348 shutdown_connections 2024-05-21T03:34:45.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.451+0000 7fed5811d700 1 --2- 172.21.15.27:0/2660246348 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fed380b29f0 0x7fed380b4eb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:45.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.451+0000 7fed5811d700 1 --2- 172.21.15.27:0/2660246348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed50108920 0x7fed501999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:45.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.451+0000 7fed5811d700 1 --2- 172.21.15.27:0/2660246348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed50103850 0x7fed5019f830 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:45.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.451+0000 7fed5811d700 1 --2- 172.21.15.27:0/2660246348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed50104200 0x7fed5019fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:45.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.451+0000 7fed5811d700 1 -- 172.21.15.27:0/2660246348 >> 172.21.15.27:0/2660246348 conn(0x7fed5007a5d0 msgr2=0x7fed501018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:34:45.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.451+0000 7fed5811d700 1 -- 172.21.15.27:0/2660246348 shutdown_connections 2024-05-21T03:34:45.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:45.451+0000 7fed5811d700 1 -- 172.21.15.27:0/2660246348 wait complete. 2024-05-21T03:34:45.453 INFO:teuthology.orchestra.run.smithi027.stderr:listed 16 entries 2024-05-21T03:34:45.915 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:45 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2660246348' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-21T03:34:46.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:45 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/2660246348' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-21T03:34:46.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:45 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/2660246348' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-21T03:34:46.342 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-21T03:34:46.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T03:34:46.342 DEBUG:teuthology.orchestra.run.smithi027:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-21T03:34:46.436 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2024-05-21T03:34:46.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T03:34:46.437 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-21T03:34:46.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T03:34:46.478 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-05-21T03:34:46.563 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-21T03:34:46.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T03:34:46.563 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-21T03:34:46.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T03:34:46.603 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2024-05-21T03:34:46.697 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-21T03:34:46.709 INFO:tasks.cephadm:Teardown begin 2024-05-21T03:34:46.709 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-21T03:34:46.709 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:34:46.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:46 smithi027 ceph-mon[32179]: pgmap v333: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T03:34:46.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:46 smithi027 ceph-mon[32179]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:34:46.959 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:34:47.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:46 smithi039 ceph-mon[36429]: pgmap v333: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T03:34:47.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:46 smithi039 ceph-mon[36429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:34:47.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:46 smithi165 ceph-mon[36269]: pgmap v333: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T03:34:47.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:46 smithi165 ceph-mon[36269]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T03:34:48.889 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:48 smithi027 ceph-mon[32179]: pgmap v334: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-21T03:34:49.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.005+0000 7fee4b7a7700 1 -- 172.21.15.27:0/1478353990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee441095a0 msgr2=0x7fee44109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:49.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.005+0000 7fee4b7a7700 1 --2- 172.21.15.27:0/1478353990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee441095a0 0x7fee44109980 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fee3400b0a0 tx=0x7fee3400b3b0 comp rx=0 tx=0).stop 2024-05-21T03:34:49.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.005+0000 7fee4b7a7700 1 -- 172.21.15.27:0/1478353990 shutdown_connections 2024-05-21T03:34:49.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.005+0000 7fee4b7a7700 1 --2- 172.21.15.27:0/1478353990 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee4406c470 0x7fee441103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:49.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.005+0000 7fee4b7a7700 1 --2- 172.21.15.27:0/1478353990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee441095a0 0x7fee44109980 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:49.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.005+0000 7fee4b7a7700 1 --2- 172.21.15.27:0/1478353990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee4406bab0 0x7fee4406bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:49.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.006+0000 7fee4b7a7700 1 -- 172.21.15.27:0/1478353990 >> 172.21.15.27:0/1478353990 conn(0x7fee440793b0 msgr2=0x7fee440797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:34:49.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.006+0000 7fee4b7a7700 1 -- 172.21.15.27:0/1478353990 shutdown_connections 2024-05-21T03:34:49.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.006+0000 7fee4b7a7700 1 -- 172.21.15.27:0/1478353990 wait complete. 2024-05-21T03:34:49.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.006+0000 7fee4b7a7700 1 Processor -- start 2024-05-21T03:34:49.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.006+0000 7fee4b7a7700 1 -- start start 2024-05-21T03:34:49.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.007+0000 7fee4b7a7700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee4406bab0 0x7fee4419f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:49.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.007+0000 7fee4b7a7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee4406c470 0x7fee4419fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:49.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.007+0000 7fee4b7a7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee441095a0 0x7fee44199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:49.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.007+0000 7fee4b7a7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fee441136f0 con 0x7fee441095a0 2024-05-21T03:34:49.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.007+0000 7fee4b7a7700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fee44113570 con 0x7fee4406c470 2024-05-21T03:34:49.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.007+0000 7fee4b7a7700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fee44113870 con 0x7fee4406bab0 2024-05-21T03:34:49.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.007+0000 7fee49d44700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee441095a0 0x7fee44199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:49.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.007+0000 7fee49d44700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee441095a0 0x7fee44199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52162/0 (socket says 172.21.15.27:52162) 2024-05-21T03:34:49.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.007+0000 7fee49d44700 1 -- 172.21.15.27:0/856053560 learned_addr learned my addr 172.21.15.27:0/856053560 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:34:49.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.007+0000 7fee49543700 1 --2- 172.21.15.27:0/856053560 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee4406bab0 0x7fee4419f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:49.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.007+0000 7fee48d42700 1 --2- 172.21.15.27:0/856053560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee4406c470 0x7fee4419fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:49.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.007+0000 7fee49d44700 1 -- 172.21.15.27:0/856053560 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee4406bab0 msgr2=0x7fee4419f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:49.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.007+0000 7fee49d44700 1 --2- 172.21.15.27:0/856053560 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee4406bab0 0x7fee4419f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:49.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.007+0000 7fee49d44700 1 -- 172.21.15.27:0/856053560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee4406c470 msgr2=0x7fee4419fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:49.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.007+0000 7fee49d44700 1 --2- 172.21.15.27:0/856053560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee4406c470 0x7fee4419fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:49.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.007+0000 7fee49d44700 1 -- 172.21.15.27:0/856053560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee4419a190 con 0x7fee441095a0 2024-05-21T03:34:49.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.007+0000 7fee49d44700 1 --2- 172.21.15.27:0/856053560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee441095a0 0x7fee44199a30 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00d980 tx=0x7fee3c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:34:49.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.007+0000 7fee427fc700 1 -- 172.21.15.27:0/856053560 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c017070 con 0x7fee441095a0 2024-05-21T03:34:49.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.008+0000 7fee427fc700 1 -- 172.21.15.27:0/856053560 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee3c005020 con 0x7fee441095a0 2024-05-21T03:34:49.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.008+0000 7fee4b7a7700 1 -- 172.21.15.27:0/856053560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee4419a480 con 0x7fee441095a0 2024-05-21T03:34:49.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.008+0000 7fee427fc700 1 -- 172.21.15.27:0/856053560 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c00fa10 con 0x7fee441095a0 2024-05-21T03:34:49.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.008+0000 7fee4b7a7700 1 -- 172.21.15.27:0/856053560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee441a66f0 con 0x7fee441095a0 2024-05-21T03:34:49.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.009+0000 7fee4b7a7700 1 -- 172.21.15.27:0/856053560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee4410ba80 con 0x7fee441095a0 2024-05-21T03:34:49.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.010+0000 7fee427fc700 1 -- 172.21.15.27:0/856053560 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7fee3c004a30 con 0x7fee441095a0 2024-05-21T03:34:49.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.011+0000 7fee427fc700 1 --2- 172.21.15.27:0/856053560 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fee2c0b2a70 0x7fee2c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:49.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.011+0000 7fee49543700 1 --2- 172.21.15.27:0/856053560 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fee2c0b2a70 0x7fee2c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:49.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.011+0000 7fee427fc700 1 -- 172.21.15.27:0/856053560 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7fee3c0e2790 con 0x7fee441095a0 2024-05-21T03:34:49.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.011+0000 7fee49543700 1 --2- 172.21.15.27:0/856053560 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fee2c0b2a70 0x7fee2c0b4f30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fee34009e60 tx=0x7fee3400b7b0 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:34:49.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.014+0000 7fee427fc700 1 -- 172.21.15.27:0/856053560 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fee3c020020 con 0x7fee441095a0 2024-05-21T03:34:49.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:48 smithi039 ceph-mon[36429]: pgmap v334: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-21T03:34:49.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:48 smithi165 ceph-mon[36269]: pgmap v334: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-21T03:34:49.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.277+0000 7fee4b7a7700 1 -- 172.21.15.27:0/856053560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee4406a230 con 0x7fee441095a0 2024-05-21T03:34:49.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.278+0000 7fee427fc700 1 -- 172.21.15.27:0/856053560 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+11924 (secure 0 0 0) 0x7fee3c0aa1c0 con 0x7fee441095a0 2024-05-21T03:34:49.279 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:34:49.279 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/148854869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":148854869},{"type":"v1","addr":"172.21.15.27:6835","nonce":148854869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/4145544730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":4145544730},{"type":"v1","addr":"172.21.15.27:6837","nonce":4145544730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":53,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T03:20:34.055985+0000","modified":"2024-05-21T03:34:44.921408+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24472,"mds_2":24457,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/745486764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":745486764},{"type":"v1","addr":"172.21.15.27:6839","nonce":745486764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6841/3982752351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3982752351},{"type":"v1","addr":"172.21.15.27:6841","nonce":3982752351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/3497521608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3497521608},{"type":"v1","addr":"172.21.15.39:6835","nonce":3497521608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.165:6835/3576910218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3576910218},{"type":"v1","addr":"172.21.15.165:6835","nonce":3576910218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/3673846852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3673846852},{"type":"v1","addr":"172.21.15.39:6837","nonce":3673846852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6837/691356139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":691356139},{"type":"v1","addr":"172.21.15.165:6837","nonce":691356139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6839/4158170814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4158170814},{"type":"v1","addr":"172.21.15.165:6839","nonce":4158170814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6839/287607819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":287607819},{"type":"v1","addr":"172.21.15.39:6839","nonce":287607819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24457,24452,24451,24472]},"id":1}]} 2024-05-21T03:34:49.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.280+0000 7fee4b7a7700 1 -- 172.21.15.27:0/856053560 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fee2c0b2a70 msgr2=0x7fee2c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:49.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.280+0000 7fee4b7a7700 1 --2- 172.21.15.27:0/856053560 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fee2c0b2a70 0x7fee2c0b4f30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fee34009e60 tx=0x7fee3400b7b0 comp rx=0 tx=0).stop 2024-05-21T03:34:49.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.280+0000 7fee4b7a7700 1 -- 172.21.15.27:0/856053560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee441095a0 msgr2=0x7fee44199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:49.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.280+0000 7fee4b7a7700 1 --2- 172.21.15.27:0/856053560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee441095a0 0x7fee44199a30 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00d980 tx=0x7fee3c00dc90 comp rx=0 tx=0).stop 2024-05-21T03:34:49.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.280+0000 7fee4b7a7700 1 -- 172.21.15.27:0/856053560 shutdown_connections 2024-05-21T03:34:49.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.280+0000 7fee4b7a7700 1 --2- 172.21.15.27:0/856053560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee441095a0 0x7fee44199a30 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:49.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.280+0000 7fee4b7a7700 1 --2- 172.21.15.27:0/856053560 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee4406bab0 0x7fee4419f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:49.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.280+0000 7fee4b7a7700 1 --2- 172.21.15.27:0/856053560 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7fee2c0b2a70 0x7fee2c0b4f30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:49.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.280+0000 7fee4b7a7700 1 --2- 172.21.15.27:0/856053560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee4406c470 0x7fee4419fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:49.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.280+0000 7fee4b7a7700 1 -- 172.21.15.27:0/856053560 >> 172.21.15.27:0/856053560 conn(0x7fee440793b0 msgr2=0x7fee44101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:34:49.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.281+0000 7fee4b7a7700 1 -- 172.21.15.27:0/856053560 shutdown_connections 2024-05-21T03:34:49.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:49.281+0000 7fee4b7a7700 1 -- 172.21.15.27:0/856053560 wait complete. 2024-05-21T03:34:49.282 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 53 2024-05-21T03:34:49.926 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:49 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/856053560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:34:50.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:49 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/856053560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:34:50.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:49 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/856053560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:34:50.031 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T03:34:50.283 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:34:50.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:50 smithi027 ceph-mon[32179]: pgmap v335: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-21T03:34:51.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:50 smithi039 ceph-mon[36429]: pgmap v335: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-21T03:34:51.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:50 smithi165 ceph-mon[36269]: pgmap v335: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-21T03:34:52.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.949+0000 7f523ffbc700 1 -- 172.21.15.27:0/3420779843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f523806b220 msgr2=0x7f523806b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:52.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.949+0000 7f523ffbc700 1 --2- 172.21.15.27:0/3420779843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f523806b220 0x7f523806b6a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5228009990 tx=0x7f5228009ca0 comp rx=0 tx=0).stop 2024-05-21T03:34:52.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.949+0000 7f523ffbc700 1 -- 172.21.15.27:0/3420779843 shutdown_connections 2024-05-21T03:34:52.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.949+0000 7f523ffbc700 1 --2- 172.21.15.27:0/3420779843 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f523806bbe0 0x7f52381103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:52.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.949+0000 7f523ffbc700 1 --2- 172.21.15.27:0/3420779843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52381095b0 0x7f5238109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:52.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.949+0000 7f523ffbc700 1 --2- 172.21.15.27:0/3420779843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f523806b220 0x7f523806b6a0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:52.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.949+0000 7f523ffbc700 1 -- 172.21.15.27:0/3420779843 >> 172.21.15.27:0/3420779843 conn(0x7f52380793f0 msgr2=0x7f5238079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:34:52.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.949+0000 7f523ffbc700 1 -- 172.21.15.27:0/3420779843 shutdown_connections 2024-05-21T03:34:52.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.949+0000 7f523ffbc700 1 -- 172.21.15.27:0/3420779843 wait complete. 2024-05-21T03:34:52.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.950+0000 7f523ffbc700 1 Processor -- start 2024-05-21T03:34:52.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.950+0000 7f523ffbc700 1 -- start start 2024-05-21T03:34:52.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.950+0000 7f523ffbc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f523806b220 0x7f5238102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:52.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.950+0000 7f523ffbc700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f523806bbe0 0x7f5238102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:52.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.950+0000 7f523ffbc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52381095b0 0x7f5238102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:52.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.950+0000 7f523ffbc700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f52381136c0 con 0x7f523806b220 2024-05-21T03:34:52.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.950+0000 7f523ffbc700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5238113540 con 0x7f52381095b0 2024-05-21T03:34:52.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.950+0000 7f523ffbc700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5238113840 con 0x7f523806bbe0 2024-05-21T03:34:52.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.951+0000 7f523dd58700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f523806b220 0x7f5238102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:52.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.951+0000 7f523dd58700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f523806b220 0x7f5238102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52178/0 (socket says 172.21.15.27:52178) 2024-05-21T03:34:52.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.951+0000 7f523dd58700 1 -- 172.21.15.27:0/1873114602 learned_addr learned my addr 172.21.15.27:0/1873114602 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:34:52.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.951+0000 7f523e559700 1 --2- 172.21.15.27:0/1873114602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52381095b0 0x7f5238102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:52.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.951+0000 7f523d557700 1 --2- 172.21.15.27:0/1873114602 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f523806bbe0 0x7f5238102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:52.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.951+0000 7f523dd58700 1 -- 172.21.15.27:0/1873114602 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f523806bbe0 msgr2=0x7f5238102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:52.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.951+0000 7f523dd58700 1 --2- 172.21.15.27:0/1873114602 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f523806bbe0 0x7f5238102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:52.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.951+0000 7f523dd58700 1 -- 172.21.15.27:0/1873114602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52381095b0 msgr2=0x7f5238102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:52.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.951+0000 7f523dd58700 1 --2- 172.21.15.27:0/1873114602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52381095b0 0x7f5238102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:52.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.951+0000 7f523dd58700 1 -- 172.21.15.27:0/1873114602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52381a8100 con 0x7f523806b220 2024-05-21T03:34:52.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.951+0000 7f523dd58700 1 --2- 172.21.15.27:0/1873114602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f523806b220 0x7f5238102510 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5234007e20 tx=0x7f523400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:34:52.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.951+0000 7f522effd700 1 -- 172.21.15.27:0/1873114602 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234016020 con 0x7f523806b220 2024-05-21T03:34:52.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.951+0000 7f522effd700 1 -- 172.21.15.27:0/1873114602 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f523400ac90 con 0x7f523806b220 2024-05-21T03:34:52.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.951+0000 7f523ffbc700 1 -- 172.21.15.27:0/1873114602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52381a83f0 con 0x7f523806b220 2024-05-21T03:34:52.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.951+0000 7f522effd700 1 -- 172.21.15.27:0/1873114602 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52340115e0 con 0x7f523806b220 2024-05-21T03:34:52.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.951+0000 7f523ffbc700 1 -- 172.21.15.27:0/1873114602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52381a8900 con 0x7f523806b220 2024-05-21T03:34:52.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.953+0000 7f523ffbc700 1 -- 172.21.15.27:0/1873114602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f523806a230 con 0x7f523806b220 2024-05-21T03:34:52.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.954+0000 7f522effd700 1 -- 172.21.15.27:0/1873114602 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7f5234019020 con 0x7f523806b220 2024-05-21T03:34:52.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.954+0000 7f522effd700 1 --2- 172.21.15.27:0/1873114602 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f52240b29f0 0x7f52240b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:52.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.955+0000 7f523d557700 1 --2- 172.21.15.27:0/1873114602 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f52240b29f0 0x7f52240b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:52.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.955+0000 7f522effd700 1 -- 172.21.15.27:0/1873114602 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7f52340e1e80 con 0x7f523806b220 2024-05-21T03:34:52.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.955+0000 7f523d557700 1 --2- 172.21.15.27:0/1873114602 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f52240b29f0 0x7f52240b4eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f522800b5c0 tx=0x7f5228005960 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:34:52.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:51.959+0000 7f522effd700 1 -- 172.21.15.27:0/1873114602 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5234020070 con 0x7f523806b220 2024-05-21T03:34:52.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:52.222+0000 7f523ffbc700 1 -- 172.21.15.27:0/1873114602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5238103a40 con 0x7f523806b220 2024-05-21T03:34:52.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:52.223+0000 7f522effd700 1 -- 172.21.15.27:0/1873114602 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+11924 (secure 0 0 0) 0x7f52340a9940 con 0x7f523806b220 2024-05-21T03:34:52.223 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:34:52.223 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/148854869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":148854869},{"type":"v1","addr":"172.21.15.27:6835","nonce":148854869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/4145544730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":4145544730},{"type":"v1","addr":"172.21.15.27:6837","nonce":4145544730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":53,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T03:20:34.055985+0000","modified":"2024-05-21T03:34:44.921408+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24472,"mds_2":24457,"mds_3":24452,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6839/745486764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":745486764},{"type":"v1","addr":"172.21.15.27:6839","nonce":745486764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6841/3982752351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":3982752351},{"type":"v1","addr":"172.21.15.27:6841","nonce":3982752351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6833/1102873763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1102873763},{"type":"v1","addr":"172.21.15.39:6833","nonce":1102873763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/4176762898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":4176762898},{"type":"v1","addr":"172.21.15.165:6833","nonce":4176762898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/3497521608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3497521608},{"type":"v1","addr":"172.21.15.39:6835","nonce":3497521608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.165:6835/3576910218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3576910218},{"type":"v1","addr":"172.21.15.165:6835","nonce":3576910218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/3673846852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3673846852},{"type":"v1","addr":"172.21.15.39:6837","nonce":3673846852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6837/691356139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":691356139},{"type":"v1","addr":"172.21.15.165:6837","nonce":691356139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6839/4158170814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":4158170814},{"type":"v1","addr":"172.21.15.165:6839","nonce":4158170814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6839/287607819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":287607819},{"type":"v1","addr":"172.21.15.39:6839","nonce":287607819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24457,24452,24451,24472]},"id":1}]} 2024-05-21T03:34:52.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:52.225+0000 7f523ffbc700 1 -- 172.21.15.27:0/1873114602 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f52240b29f0 msgr2=0x7f52240b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:52.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:52.225+0000 7f523ffbc700 1 --2- 172.21.15.27:0/1873114602 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f52240b29f0 0x7f52240b4eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f522800b5c0 tx=0x7f5228005960 comp rx=0 tx=0).stop 2024-05-21T03:34:52.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:52.225+0000 7f523ffbc700 1 -- 172.21.15.27:0/1873114602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f523806b220 msgr2=0x7f5238102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:52.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:52.225+0000 7f523ffbc700 1 --2- 172.21.15.27:0/1873114602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f523806b220 0x7f5238102510 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5234007e20 tx=0x7f523400a340 comp rx=0 tx=0).stop 2024-05-21T03:34:52.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:52.225+0000 7f523ffbc700 1 -- 172.21.15.27:0/1873114602 shutdown_connections 2024-05-21T03:34:52.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:52.225+0000 7f523ffbc700 1 --2- 172.21.15.27:0/1873114602 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f52240b29f0 0x7f52240b4eb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:52.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:52.225+0000 7f523ffbc700 1 --2- 172.21.15.27:0/1873114602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52381095b0 0x7f5238102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:52.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:52.225+0000 7f523ffbc700 1 --2- 172.21.15.27:0/1873114602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f523806b220 0x7f5238102510 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:52.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:52.225+0000 7f523ffbc700 1 --2- 172.21.15.27:0/1873114602 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f523806bbe0 0x7f5238102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:52.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:52.225+0000 7f523ffbc700 1 -- 172.21.15.27:0/1873114602 >> 172.21.15.27:0/1873114602 conn(0x7f52380793f0 msgr2=0x7f5238101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:34:52.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:52.225+0000 7f523ffbc700 1 -- 172.21.15.27:0/1873114602 shutdown_connections 2024-05-21T03:34:52.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:52.225+0000 7f523ffbc700 1 -- 172.21.15.27:0/1873114602 wait complete. 2024-05-21T03:34:52.227 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 53 2024-05-21T03:34:52.837 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-21T03:34:52.861 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:52 smithi027 ceph-mon[32179]: pgmap v336: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.3 KiB/s rd, 938 B/s wr, 12 op/s 2024-05-21T03:34:52.861 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:52 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/1873114602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:34:53.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:52 smithi039 ceph-mon[36429]: pgmap v336: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.3 KiB/s rd, 938 B/s wr, 12 op/s 2024-05-21T03:34:53.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:52 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/1873114602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:34:53.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:52 smithi165 ceph-mon[36269]: pgmap v336: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.3 KiB/s rd, 938 B/s wr, 12 op/s 2024-05-21T03:34:53.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:52 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/1873114602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T03:34:53.088 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:34:54.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.795+0000 7f6390287700 1 -- 172.21.15.27:0/3471097072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63881095a0 msgr2=0x7f6388109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:54.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.795+0000 7f6390287700 1 --2- 172.21.15.27:0/3471097072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63881095a0 0x7f6388109980 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f637800b0a0 tx=0x7f637800b3b0 comp rx=0 tx=0).stop 2024-05-21T03:34:54.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.795+0000 7f6390287700 1 -- 172.21.15.27:0/3471097072 shutdown_connections 2024-05-21T03:34:54.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.795+0000 7f6390287700 1 --2- 172.21.15.27:0/3471097072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f638806c470 0x7f63881103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:54.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.795+0000 7f6390287700 1 --2- 172.21.15.27:0/3471097072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63881095a0 0x7f6388109980 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:54.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.795+0000 7f6390287700 1 --2- 172.21.15.27:0/3471097072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f638806bab0 0x7f638806bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:54.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.795+0000 7f6390287700 1 -- 172.21.15.27:0/3471097072 >> 172.21.15.27:0/3471097072 conn(0x7f63880793b0 msgr2=0x7f63880797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:34:54.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.795+0000 7f6390287700 1 -- 172.21.15.27:0/3471097072 shutdown_connections 2024-05-21T03:34:54.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.795+0000 7f6390287700 1 -- 172.21.15.27:0/3471097072 wait complete. 2024-05-21T03:34:54.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.796+0000 7f6390287700 1 Processor -- start 2024-05-21T03:34:54.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.796+0000 7f6390287700 1 -- start start 2024-05-21T03:34:54.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.796+0000 7f6390287700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f638806bab0 0x7f6388102500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:54.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.796+0000 7f6390287700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f638806c470 0x7f6388102a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:54.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.796+0000 7f6390287700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63881095a0 0x7f6388102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:54.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.796+0000 7f6390287700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6388113730 con 0x7f63881095a0 2024-05-21T03:34:54.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.796+0000 7f6390287700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f63881135b0 con 0x7f638806bab0 2024-05-21T03:34:54.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.796+0000 7f6390287700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f63881138b0 con 0x7f638806c470 2024-05-21T03:34:54.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.796+0000 7f638e824700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63881095a0 0x7f6388102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:54.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.796+0000 7f638e824700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63881095a0 0x7f6388102f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36422/0 (socket says 172.21.15.27:36422) 2024-05-21T03:34:54.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.796+0000 7f638e824700 1 -- 172.21.15.27:0/2463459903 learned_addr learned my addr 172.21.15.27:0/2463459903 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:34:54.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.796+0000 7f638d822700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f638806c470 0x7f6388102a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:54.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.796+0000 7f638e023700 1 --2- 172.21.15.27:0/2463459903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f638806bab0 0x7f6388102500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:54.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.797+0000 7f638e824700 1 -- 172.21.15.27:0/2463459903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f638806c470 msgr2=0x7f6388102a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:54.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.797+0000 7f638e824700 1 --2- 172.21.15.27:0/2463459903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f638806c470 0x7f6388102a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:54.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.797+0000 7f638e824700 1 -- 172.21.15.27:0/2463459903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f638806bab0 msgr2=0x7f6388102500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:54.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.797+0000 7f638e824700 1 --2- 172.21.15.27:0/2463459903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f638806bab0 0x7f6388102500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:54.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.797+0000 7f638e824700 1 -- 172.21.15.27:0/2463459903 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63881a81d0 con 0x7f63881095a0 2024-05-21T03:34:54.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.797+0000 7f638e023700 1 --2- 172.21.15.27:0/2463459903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f638806bab0 0x7f6388102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T03:34:54.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.797+0000 7f638e824700 1 --2- 172.21.15.27:0/2463459903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63881095a0 0x7f6388102f80 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f638000d980 tx=0x7f638000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:34:54.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.797+0000 7f63877fe700 1 -- 172.21.15.27:0/2463459903 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6380016070 con 0x7f63881095a0 2024-05-21T03:34:54.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.797+0000 7f63877fe700 1 -- 172.21.15.27:0/2463459903 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6380004830 con 0x7f63881095a0 2024-05-21T03:34:54.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.797+0000 7f6390287700 1 -- 172.21.15.27:0/2463459903 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63881a84c0 con 0x7f63881095a0 2024-05-21T03:34:54.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.797+0000 7f63877fe700 1 -- 172.21.15.27:0/2463459903 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63800054b0 con 0x7f63881095a0 2024-05-21T03:34:54.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.797+0000 7f6390287700 1 -- 172.21.15.27:0/2463459903 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63881a89b0 con 0x7f63881095a0 2024-05-21T03:34:54.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.798+0000 7f6390287700 1 -- 172.21.15.27:0/2463459903 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f638810ba80 con 0x7f63881095a0 2024-05-21T03:34:54.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.799+0000 7f63877fe700 1 -- 172.21.15.27:0/2463459903 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7f63800049e0 con 0x7f63881095a0 2024-05-21T03:34:54.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.800+0000 7f63877fe700 1 --2- 172.21.15.27:0/2463459903 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f63700b29f0 0x7f63700b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:54.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.800+0000 7f638e023700 1 --2- 172.21.15.27:0/2463459903 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f63700b29f0 0x7f63700b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:54.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.800+0000 7f63877fe700 1 -- 172.21.15.27:0/2463459903 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7f63800e2a20 con 0x7f63881095a0 2024-05-21T03:34:54.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.801+0000 7f638e023700 1 --2- 172.21.15.27:0/2463459903 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f63700b29f0 0x7f63700b4eb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6378009e60 tx=0x7f6378006230 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:34:54.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:54.804+0000 7f63877fe700 1 -- 172.21.15.27:0/2463459903 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f638001f020 con 0x7f63881095a0 2024-05-21T03:34:54.803 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:54 smithi027 ceph-mon[32179]: pgmap v337: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-21T03:34:55.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:54 smithi039 ceph-mon[36429]: pgmap v337: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-21T03:34:55.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:54 smithi165 ceph-mon[36269]: pgmap v337: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-21T03:34:55.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:55.026+0000 7f6390287700 1 -- 172.21.15.27:0/2463459903 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f638810b460 con 0x7f63881095a0 2024-05-21T03:34:55.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:55.028+0000 7f63877fe700 1 -- 172.21.15.27:0/2463459903 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+23150 (secure 0 0 0) 0x7f63800aa3d0 con 0x7f63881095a0 2024-05-21T03:34:55.027 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-21T03:34:55.028 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":91,"fsid":"c9ca94be-171f-11ef-bc9a-c7b262605968","created":"2024-05-21T03:12:18.217236+0000","modified":"2024-05-21T03:34:18.448091+0000","last_up_change":"2024-05-21T03:19:30.382150+0000","last_in_change":"2024-05-21T03:19:09.058249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T03:15:41.340194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T03:20:27.507346+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T03:20:30.830169+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":90,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c745c09f-9a76-4d05-af75-c7554f6d5e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6803","nonce":1399996449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6805","nonce":1399996449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6809","nonce":1399996449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":1399996449},{"type":"v1","addr":"172.21.15.27:6807","nonce":1399996449}]},"public_addr":"172.21.15.27:6803/1399996449","cluster_addr":"172.21.15.27:6805/1399996449","heartbeat_back_addr":"172.21.15.27:6809/1399996449","heartbeat_front_addr":"172.21.15.27:6807/1399996449","state":["exists","up"]},{"osd":1,"uuid":"8b73aa00-4364-438f-9722-53caa4e9bec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6801","nonce":1776318847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6803","nonce":1776318847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6807","nonce":1776318847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1776318847},{"type":"v1","addr":"172.21.15.39:6805","nonce":1776318847}]},"public_addr":"172.21.15.39:6801/1776318847","cluster_addr":"172.21.15.39:6803/1776318847","heartbeat_back_addr":"172.21.15.39:6807/1776318847","heartbeat_front_addr":"172.21.15.39:6805/1776318847","state":["exists","up"]},{"osd":2,"uuid":"0822043e-7d3c-4e98-9dc0-94b98b4d562a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6801","nonce":703322625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6803","nonce":703322625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6807","nonce":703322625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":703322625},{"type":"v1","addr":"172.21.15.165:6805","nonce":703322625}]},"public_addr":"172.21.15.165:6801/703322625","cluster_addr":"172.21.15.165:6803/703322625","heartbeat_back_addr":"172.21.15.165:6807/703322625","heartbeat_front_addr":"172.21.15.165:6805/703322625","state":["exists","up"]},{"osd":3,"uuid":"c563b1b2-efcb-4f8c-8121-c1938f83131e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6811","nonce":1933403135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6813","nonce":1933403135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6817","nonce":1933403135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":1933403135},{"type":"v1","addr":"172.21.15.27:6815","nonce":1933403135}]},"public_addr":"172.21.15.27:6811/1933403135","cluster_addr":"172.21.15.27:6813/1933403135","heartbeat_back_addr":"172.21.15.27:6817/1933403135","heartbeat_front_addr":"172.21.15.27:6815/1933403135","state":["exists","up"]},{"osd":4,"uuid":"0c793b87-b045-4b4a-89e7-87ec36deeb0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6809","nonce":155579331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6811","nonce":155579331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6815","nonce":155579331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":155579331},{"type":"v1","addr":"172.21.15.39:6813","nonce":155579331}]},"public_addr":"172.21.15.39:6809/155579331","cluster_addr":"172.21.15.39:6811/155579331","heartbeat_back_addr":"172.21.15.39:6815/155579331","heartbeat_front_addr":"172.21.15.39:6813/155579331","state":["exists","up"]},{"osd":5,"uuid":"367eeb5a-9d75-4644-bf46-317c03f33d8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6809","nonce":386850244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6811","nonce":386850244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6815","nonce":386850244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":386850244},{"type":"v1","addr":"172.21.15.165:6813","nonce":386850244}]},"public_addr":"172.21.15.165:6809/386850244","cluster_addr":"172.21.15.165:6811/386850244","heartbeat_back_addr":"172.21.15.165:6815/386850244","heartbeat_front_addr":"172.21.15.165:6813/386850244","state":["exists","up"]},{"osd":6,"uuid":"e000167f-24fd-4115-83f0-8afbfa8d8991","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6819","nonce":3531376679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6821","nonce":3531376679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6825","nonce":3531376679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":3531376679},{"type":"v1","addr":"172.21.15.27:6823","nonce":3531376679}]},"public_addr":"172.21.15.27:6819/3531376679","cluster_addr":"172.21.15.27:6821/3531376679","heartbeat_back_addr":"172.21.15.27:6825/3531376679","heartbeat_front_addr":"172.21.15.27:6823/3531376679","state":["exists","up"]},{"osd":7,"uuid":"4b8d486b-868e-4f33-98a1-b99686adeaaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6817","nonce":1550035852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6819","nonce":1550035852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6823","nonce":1550035852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":1550035852},{"type":"v1","addr":"172.21.15.39:6821","nonce":1550035852}]},"public_addr":"172.21.15.39:6817/1550035852","cluster_addr":"172.21.15.39:6819/1550035852","heartbeat_back_addr":"172.21.15.39:6823/1550035852","heartbeat_front_addr":"172.21.15.39:6821/1550035852","state":["exists","up"]},{"osd":8,"uuid":"2259f35c-2edd-4331-9a63-8c0c1acbdc71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6817","nonce":3184704971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6819","nonce":3184704971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6823","nonce":3184704971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":3184704971},{"type":"v1","addr":"172.21.15.165:6821","nonce":3184704971}]},"public_addr":"172.21.15.165:6817/3184704971","cluster_addr":"172.21.15.165:6819/3184704971","heartbeat_back_addr":"172.21.15.165:6823/3184704971","heartbeat_front_addr":"172.21.15.165:6821/3184704971","state":["exists","up"]},{"osd":9,"uuid":"10307ecc-8270-4be2-b62b-9708e09bb207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6827","nonce":2800048793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6829","nonce":2800048793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6833","nonce":2800048793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":2800048793},{"type":"v1","addr":"172.21.15.27:6831","nonce":2800048793}]},"public_addr":"172.21.15.27:6827/2800048793","cluster_addr":"172.21.15.27:6829/2800048793","heartbeat_back_addr":"172.21.15.27:6833/2800048793","heartbeat_front_addr":"172.21.15.27:6831/2800048793","state":["exists","up"]},{"osd":10,"uuid":"dfe1b8ca-ff8a-4b7f-ade5-ad4f00cf60cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6825","nonce":363500811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6827","nonce":363500811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6831","nonce":363500811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":363500811},{"type":"v1","addr":"172.21.15.39:6829","nonce":363500811}]},"public_addr":"172.21.15.39:6825/363500811","cluster_addr":"172.21.15.39:6827/363500811","heartbeat_back_addr":"172.21.15.39:6831/363500811","heartbeat_front_addr":"172.21.15.39:6829/363500811","state":["exists","up"]},{"osd":11,"uuid":"f9531145-5d09-4dbd-a050-3159ee09db24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6825","nonce":2274332243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6827","nonce":2274332243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6831","nonce":2274332243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":2274332243},{"type":"v1","addr":"172.21.15.165:6829","nonce":2274332243}]},"public_addr":"172.21.15.165:6825/2274332243","cluster_addr":"172.21.15.165:6827/2274332243","heartbeat_back_addr":"172.21.15.165:6831/2274332243","heartbeat_front_addr":"172.21.15.165:6829/2274332243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:14:47.077860+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:15:13.272879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:15:38.046651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:03.376483+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:29.179280+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:16:53.391261+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:17:18.472829+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:17:44.349925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:18:08.652193+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:18:35.282669+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:19:01.637151+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T03:19:27.461566+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/744303480":"2024-05-22T03:24:01.382214+0000","172.21.15.27:0/3254504266":"2024-05-22T03:24:01.382214+0000","172.21.15.27:0/4135246381":"2024-05-22T03:24:01.382214+0000","172.21.15.27:6800/1185518061":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/4189166571":"2024-05-22T03:24:01.382214+0000","172.21.15.27:6801/1185518061":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/4194431212":"2024-05-22T03:13:13.249220+0000","172.21.15.27:6801/3683408243":"2024-05-22T03:24:01.382214+0000","172.21.15.27:0/1890032101":"2024-05-22T03:13:13.249220+0000","172.21.15.27:6800/175229478":"2024-05-22T03:12:45.739195+0000","172.21.15.27:0/999325706":"2024-05-22T03:12:45.739195+0000","172.21.15.27:6801/175229478":"2024-05-22T03:12:45.739195+0000","172.21.15.27:6800/3683408243":"2024-05-22T03:24:01.382214+0000","172.21.15.27:0/4184128736":"2024-05-22T03:13:13.249220+0000","172.21.15.27:0/3000197506":"2024-05-22T03:12:45.739195+0000","172.21.15.27:0/972677319":"2024-05-22T03:12:45.739195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T03:34:55.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:55.030+0000 7f6390287700 1 -- 172.21.15.27:0/2463459903 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f63700b29f0 msgr2=0x7f63700b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:55.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:55.030+0000 7f6390287700 1 --2- 172.21.15.27:0/2463459903 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f63700b29f0 0x7f63700b4eb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6378009e60 tx=0x7f6378006230 comp rx=0 tx=0).stop 2024-05-21T03:34:55.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:55.030+0000 7f6390287700 1 -- 172.21.15.27:0/2463459903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63881095a0 msgr2=0x7f6388102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:55.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:55.030+0000 7f6390287700 1 --2- 172.21.15.27:0/2463459903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63881095a0 0x7f6388102f80 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f638000d980 tx=0x7f638000dc90 comp rx=0 tx=0).stop 2024-05-21T03:34:55.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:55.030+0000 7f6390287700 1 -- 172.21.15.27:0/2463459903 shutdown_connections 2024-05-21T03:34:55.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:55.030+0000 7f6390287700 1 --2- 172.21.15.27:0/2463459903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63881095a0 0x7f6388102f80 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:55.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:55.030+0000 7f6390287700 1 --2- 172.21.15.27:0/2463459903 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f63700b29f0 0x7f63700b4eb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:55.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:55.030+0000 7f6390287700 1 --2- 172.21.15.27:0/2463459903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f638806bab0 0x7f6388102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:55.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:55.030+0000 7f6390287700 1 --2- 172.21.15.27:0/2463459903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f638806c470 0x7f6388102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:55.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:55.030+0000 7f6390287700 1 -- 172.21.15.27:0/2463459903 >> 172.21.15.27:0/2463459903 conn(0x7f63880793b0 msgr2=0x7f6388101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:34:55.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:55.030+0000 7f6390287700 1 -- 172.21.15.27:0/2463459903 shutdown_connections 2024-05-21T03:34:55.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:55.030+0000 7f6390287700 1 -- 172.21.15.27:0/2463459903 wait complete. 2024-05-21T03:34:55.718 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs fail cephfs 2024-05-21T03:34:56.008 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:34:56.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:55 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/2463459903' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:34:56.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:55 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/2463459903' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:34:56.131 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:55 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2463459903' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T03:34:57.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:56 smithi039 ceph-mon[36429]: pgmap v338: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 13 op/s 2024-05-21T03:34:57.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:56 smithi165 ceph-mon[36269]: pgmap v338: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 13 op/s 2024-05-21T03:34:57.135 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:56 smithi027 ceph-mon[32179]: pgmap v338: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 13 op/s 2024-05-21T03:34:57.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.697+0000 7feee01b7700 1 -- 172.21.15.27:0/2017035643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed81024f0 msgr2=0x7feed81103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:57.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.697+0000 7feee01b7700 1 --2- 172.21.15.27:0/2017035643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed81024f0 0x7feed81103d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7feed000b0d0 tx=0x7feed000b3e0 comp rx=0 tx=0).stop 2024-05-21T03:34:57.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.697+0000 7feee01b7700 1 -- 172.21.15.27:0/2017035643 shutdown_connections 2024-05-21T03:34:57.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.697+0000 7feee01b7700 1 --2- 172.21.15.27:0/2017035643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed81024f0 0x7feed81103d0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:57.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.697+0000 7feee01b7700 1 --2- 172.21.15.27:0/2017035643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feed8109520 0x7feed8109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:57.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.697+0000 7feee01b7700 1 --2- 172.21.15.27:0/2017035643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feed8101b30 0x7feed8101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:57.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.697+0000 7feee01b7700 1 -- 172.21.15.27:0/2017035643 >> 172.21.15.27:0/2017035643 conn(0x7feed8078ab0 msgr2=0x7feed80ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:34:57.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.697+0000 7feee01b7700 1 -- 172.21.15.27:0/2017035643 shutdown_connections 2024-05-21T03:34:57.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.697+0000 7feee01b7700 1 -- 172.21.15.27:0/2017035643 wait complete. 2024-05-21T03:34:57.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.698+0000 7feee01b7700 1 Processor -- start 2024-05-21T03:34:57.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.698+0000 7feee01b7700 1 -- start start 2024-05-21T03:34:57.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.698+0000 7feee01b7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed8101b30 0x7feed819bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:57.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.698+0000 7feee01b7700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feed81024f0 0x7feed819c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:57.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.698+0000 7feee01b7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feed8109520 0x7feed81a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:57.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.698+0000 7feee01b7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feed8113760 con 0x7feed8101b30 2024-05-21T03:34:57.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.698+0000 7feee01b7700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7feed81135e0 con 0x7feed8109520 2024-05-21T03:34:57.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.698+0000 7feee01b7700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7feed81138e0 con 0x7feed81024f0 2024-05-21T03:34:57.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.698+0000 7feeddf53700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed8101b30 0x7feed819bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:57.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.698+0000 7feeddf53700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed8101b30 0x7feed819bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36438/0 (socket says 172.21.15.27:36438) 2024-05-21T03:34:57.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.698+0000 7feeddf53700 1 -- 172.21.15.27:0/2854878747 learned_addr learned my addr 172.21.15.27:0/2854878747 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:34:57.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.698+0000 7feedd752700 1 --2- 172.21.15.27:0/2854878747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feed81024f0 0x7feed819c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:57.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.698+0000 7feede754700 1 --2- 172.21.15.27:0/2854878747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feed8109520 0x7feed81a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:57.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.698+0000 7feeddf53700 1 -- 172.21.15.27:0/2854878747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feed81024f0 msgr2=0x7feed819c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:57.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.698+0000 7feeddf53700 1 --2- 172.21.15.27:0/2854878747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feed81024f0 0x7feed819c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:57.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.698+0000 7feeddf53700 1 -- 172.21.15.27:0/2854878747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feed8109520 msgr2=0x7feed81a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:57.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.698+0000 7feeddf53700 1 --2- 172.21.15.27:0/2854878747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feed8109520 0x7feed81a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:57.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.698+0000 7feeddf53700 1 -- 172.21.15.27:0/2854878747 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed81a0db0 con 0x7feed8101b30 2024-05-21T03:34:57.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.699+0000 7feeddf53700 1 --2- 172.21.15.27:0/2854878747 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed8101b30 0x7feed819bda0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7feed400c970 tx=0x7feed400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:34:57.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.699+0000 7feeceffd700 1 -- 172.21.15.27:0/2854878747 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed40078d0 con 0x7feed8101b30 2024-05-21T03:34:57.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.699+0000 7feeceffd700 1 -- 172.21.15.27:0/2854878747 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed4018480 con 0x7feed8101b30 2024-05-21T03:34:57.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.699+0000 7feee01b7700 1 -- 172.21.15.27:0/2854878747 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed81a10a0 con 0x7feed8101b30 2024-05-21T03:34:57.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.699+0000 7feeceffd700 1 -- 172.21.15.27:0/2854878747 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed400f6b0 con 0x7feed8101b30 2024-05-21T03:34:57.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.699+0000 7feee01b7700 1 -- 172.21.15.27:0/2854878747 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed81a8920 con 0x7feed8101b30 2024-05-21T03:34:57.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.700+0000 7feee01b7700 1 -- 172.21.15.27:0/2854878747 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feed806a230 con 0x7feed8101b30 2024-05-21T03:34:57.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.701+0000 7feeceffd700 1 -- 172.21.15.27:0/2854878747 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7feed4018930 con 0x7feed8101b30 2024-05-21T03:34:57.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.702+0000 7feeceffd700 1 --2- 172.21.15.27:0/2854878747 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7feec40b29f0 0x7feec40b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:34:57.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.702+0000 7feedd752700 1 --2- 172.21.15.27:0/2854878747 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7feec40b29f0 0x7feec40b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:34:57.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.702+0000 7feeceffd700 1 -- 172.21.15.27:0/2854878747 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8571+0+0 (secure 0 0 0) 0x7feed40e2ac0 con 0x7feed8101b30 2024-05-21T03:34:57.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.703+0000 7feedd752700 1 --2- 172.21.15.27:0/2854878747 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7feec40b29f0 0x7feec40b4eb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feed81045c0 tx=0x7feec80073d0 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:34:57.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.705+0000 7feeceffd700 1 -- 172.21.15.27:0/2854878747 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feed40aa500 con 0x7feed8101b30 2024-05-21T03:34:57.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.968+0000 7feee01b7700 1 -- 172.21.15.27:0/2854878747 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7feed819d100 con 0x7feed8101b30 2024-05-21T03:34:57.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.970+0000 7feeceffd700 1 -- 172.21.15.27:0/2854878747 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v54) ==== 161+0+0 (secure 0 0 0) 0x7feed40a9e00 con 0x7feed8101b30 2024-05-21T03:34:57.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.974+0000 7feee01b7700 1 -- 172.21.15.27:0/2854878747 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7feec40b29f0 msgr2=0x7feec40b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:57.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.974+0000 7feee01b7700 1 --2- 172.21.15.27:0/2854878747 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7feec40b29f0 0x7feec40b4eb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feed81045c0 tx=0x7feec80073d0 comp rx=0 tx=0).stop 2024-05-21T03:34:57.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.974+0000 7feee01b7700 1 -- 172.21.15.27:0/2854878747 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed8101b30 msgr2=0x7feed819bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:34:57.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.974+0000 7feee01b7700 1 --2- 172.21.15.27:0/2854878747 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed8101b30 0x7feed819bda0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7feed400c970 tx=0x7feed400cc80 comp rx=0 tx=0).stop 2024-05-21T03:34:57.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.974+0000 7feee01b7700 1 -- 172.21.15.27:0/2854878747 shutdown_connections 2024-05-21T03:34:57.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.974+0000 7feee01b7700 1 --2- 172.21.15.27:0/2854878747 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7feec40b29f0 0x7feec40b4eb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:57.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.974+0000 7feee01b7700 1 --2- 172.21.15.27:0/2854878747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feed8109520 0x7feed81a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:57.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.974+0000 7feee01b7700 1 --2- 172.21.15.27:0/2854878747 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed8101b30 0x7feed819bda0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:57.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.974+0000 7feee01b7700 1 --2- 172.21.15.27:0/2854878747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feed81024f0 0x7feed819c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:34:57.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.974+0000 7feee01b7700 1 -- 172.21.15.27:0/2854878747 >> 172.21.15.27:0/2854878747 conn(0x7feed8078ab0 msgr2=0x7feed80fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:34:57.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.974+0000 7feee01b7700 1 -- 172.21.15.27:0/2854878747 shutdown_connections 2024-05-21T03:34:57.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:34:57.974+0000 7feee01b7700 1 -- 172.21.15.27:0/2854878747 wait complete. 2024-05-21T03:34:57.975 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-21T03:34:58.271 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 21 03:34:58 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-i[66895]: ignoring --setuser ceph since I am not root 2024-05-21T03:34:58.271 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 21 03:34:58 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-i[66895]: ignoring --setgroup ceph since I am not root 2024-05-21T03:34:58.272 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 21 03:34:58 smithi165 ceph-mds[66899]: -- 172.21.15.165:0/2595471048 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555b80965a40 con 0x555b80939000 2024-05-21T03:34:58.272 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 21 03:34:58 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-i[66895]: starting mds.i at 2024-05-21T03:34:58.272 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 21 03:34:58 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-c[66593]: ignoring --setuser ceph since I am not root 2024-05-21T03:34:58.272 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 21 03:34:58 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-c[66593]: ignoring --setgroup ceph since I am not root 2024-05-21T03:34:58.272 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 21 03:34:58 smithi165 ceph-mds[66597]: -- 172.21.15.165:0/3985284364 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b8d13aba40 con 0x55b8d137ec00 2024-05-21T03:34:58.272 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 21 03:34:58 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-c[66593]: starting mds.c at 2024-05-21T03:34:58.272 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 21 03:34:58 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-l[65978]: ignoring --setuser ceph since I am not root 2024-05-21T03:34:58.273 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 21 03:34:58 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-l[65978]: ignoring --setgroup ceph since I am not root 2024-05-21T03:34:58.273 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 21 03:34:58 smithi165 ceph-mds[65982]: -- 172.21.15.165:0/1058469220 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563d22f93a40 con 0x563d22f66c00 2024-05-21T03:34:58.273 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 21 03:34:58 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-l[65978]: starting mds.l at 2024-05-21T03:34:58.273 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 21 03:34:58 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-f[66292]: ignoring --setuser ceph since I am not root 2024-05-21T03:34:58.273 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 21 03:34:58 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-f[66292]: ignoring --setgroup ceph since I am not root 2024-05-21T03:34:58.273 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 21 03:34:58 smithi165 ceph-mds[66296]: -- 172.21.15.165:0/1565629761 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561d321c9a40 con 0x561d3219cc00 2024-05-21T03:34:58.273 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 21 03:34:58 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-f[66292]: starting mds.f at 2024-05-21T03:34:58.273 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 21 03:34:58 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-f[66292]: 2024-05-21T03:34:58.019+0000 7fc8ecb66700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-05-21T03:34:58.288 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:57 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mon-a[32174]: 2024-05-21T03:34:57.968+0000 7f8e6cc10700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-21T03:34:58.289 INFO:journalctl@ceph.mds.a.smithi027.stdout:May 21 03:34:58 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-a[72258]: ignoring --setuser ceph since I am not root 2024-05-21T03:34:58.289 INFO:journalctl@ceph.mds.a.smithi027.stdout:May 21 03:34:58 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-a[72258]: ignoring --setgroup ceph since I am not root 2024-05-21T03:34:58.289 INFO:journalctl@ceph.mds.a.smithi027.stdout:May 21 03:34:58 smithi027 ceph-mds[72262]: -- 172.21.15.27:0/13721508 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562798539a40 con 0x56279850cc00 2024-05-21T03:34:58.289 INFO:journalctl@ceph.mds.a.smithi027.stdout:May 21 03:34:58 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-a[72258]: starting mds.a at 2024-05-21T03:34:58.289 INFO:journalctl@ceph.mds.g.smithi027.stdout:May 21 03:34:58 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-g[71801]: ignoring --setuser ceph since I am not root 2024-05-21T03:34:58.289 INFO:journalctl@ceph.mds.g.smithi027.stdout:May 21 03:34:58 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-g[71801]: ignoring --setgroup ceph since I am not root 2024-05-21T03:34:58.289 INFO:journalctl@ceph.mds.g.smithi027.stdout:May 21 03:34:58 smithi027 ceph-mds[71805]: -- 172.21.15.27:0/4141629342 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e5f8153a40 con 0x55e5f8126800 2024-05-21T03:34:58.289 INFO:journalctl@ceph.mds.g.smithi027.stdout:May 21 03:34:58 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-g[71801]: starting mds.g at 2024-05-21T03:34:58.519 INFO:journalctl@ceph.mds.h.smithi039.stdout:May 21 03:34:58 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-h[66143]: ignoring --setuser ceph since I am not root 2024-05-21T03:34:58.519 INFO:journalctl@ceph.mds.h.smithi039.stdout:May 21 03:34:58 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-h[66143]: ignoring --setgroup ceph since I am not root 2024-05-21T03:34:58.519 INFO:journalctl@ceph.mds.h.smithi039.stdout:May 21 03:34:58 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-h[66143]: starting mds.h at 2024-05-21T03:34:58.520 INFO:journalctl@ceph.mds.b.smithi039.stdout:May 21 03:34:58 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-b[66446]: ignoring --setuser ceph since I am not root 2024-05-21T03:34:58.520 INFO:journalctl@ceph.mds.b.smithi039.stdout:May 21 03:34:58 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-b[66446]: ignoring --setgroup ceph since I am not root 2024-05-21T03:34:58.520 INFO:journalctl@ceph.mds.b.smithi039.stdout:May 21 03:34:58 smithi039 ceph-mds[66450]: -- 172.21.15.39:0/3755176593 <== mon.2 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e187de7a40 con 0x55e187dbb000 2024-05-21T03:34:58.520 INFO:journalctl@ceph.mds.b.smithi039.stdout:May 21 03:34:58 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-b[66446]: starting mds.b at 2024-05-21T03:34:58.520 INFO:journalctl@ceph.mds.e.smithi039.stdout:May 21 03:34:58 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-e[65840]: ignoring --setuser ceph since I am not root 2024-05-21T03:34:58.520 INFO:journalctl@ceph.mds.e.smithi039.stdout:May 21 03:34:58 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-e[65840]: ignoring --setgroup ceph since I am not root 2024-05-21T03:34:58.520 INFO:journalctl@ceph.mds.e.smithi039.stdout:May 21 03:34:58 smithi039 ceph-mds[65844]: -- 172.21.15.39:0/1425453658 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a5a56c7a40 con 0x55a5a569a800 2024-05-21T03:34:58.520 INFO:journalctl@ceph.mds.e.smithi039.stdout:May 21 03:34:58 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-e[65840]: starting mds.e at 2024-05-21T03:34:58.521 INFO:journalctl@ceph.mds.k.smithi039.stdout:May 21 03:34:58 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-k[65526]: ignoring --setuser ceph since I am not root 2024-05-21T03:34:58.521 INFO:journalctl@ceph.mds.k.smithi039.stdout:May 21 03:34:58 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-k[65526]: ignoring --setgroup ceph since I am not root 2024-05-21T03:34:58.521 INFO:journalctl@ceph.mds.k.smithi039.stdout:May 21 03:34:58 smithi039 ceph-mds[65530]: -- 172.21.15.39:0/907187019 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dce7909a40 con 0x55dce78dcc00 2024-05-21T03:34:58.521 INFO:journalctl@ceph.mds.k.smithi039.stdout:May 21 03:34:58 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-k[65526]: starting mds.k at 2024-05-21T03:34:58.594 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-21T03:34:58.847 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:34:58.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:58 smithi027 ceph-mon[32179]: pgmap v339: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-21T03:34:58.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:58 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2854878747' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-21T03:34:58.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:58 smithi027 ceph-mon[32179]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-21T03:34:58.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:58 smithi027 ceph-mon[32179]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-21T03:34:58.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:58 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2854878747' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-21T03:34:58.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:58 smithi027 ceph-mon[32179]: osdmap e92: 12 total, 12 up, 12 in 2024-05-21T03:34:58.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:58 smithi027 ceph-mon[32179]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-21T03:34:59.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:58 smithi039 ceph-mon[36429]: pgmap v339: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-21T03:34:59.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:58 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/2854878747' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-21T03:34:59.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:58 smithi039 ceph-mon[36429]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-21T03:34:59.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:58 smithi039 ceph-mon[36429]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-21T03:34:59.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:58 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/2854878747' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-21T03:34:59.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:58 smithi039 ceph-mon[36429]: osdmap e92: 12 total, 12 up, 12 in 2024-05-21T03:34:59.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:58 smithi039 ceph-mon[36429]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-21T03:34:59.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:58 smithi165 ceph-mon[36269]: pgmap v339: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-21T03:34:59.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:58 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/2854878747' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-21T03:34:59.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:58 smithi165 ceph-mon[36269]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-21T03:34:59.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:58 smithi165 ceph-mon[36269]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-21T03:34:59.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:58 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/2854878747' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-21T03:34:59.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:58 smithi165 ceph-mon[36269]: osdmap e92: 12 total, 12 up, 12 in 2024-05-21T03:34:59.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:58 smithi165 ceph-mon[36269]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-21T03:35:00.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:59 smithi027 ceph-mon[32179]: mds.? [v2:172.21.15.165:6832/3820641567,v1:172.21.15.165:6835/3820641567] up:boot 2024-05-21T03:35:00.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:59 smithi027 ceph-mon[32179]: mds.? [v2:172.21.15.165:6833/1571360825,v1:172.21.15.165:6834/1571360825] up:boot 2024-05-21T03:35:00.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:59 smithi027 ceph-mon[32179]: mds.? [v2:172.21.15.27:6838/3680111719,v1:172.21.15.27:6839/3680111719] up:boot 2024-05-21T03:35:00.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:59 smithi027 ceph-mon[32179]: mds.? [v2:172.21.15.39:6832/850497663,v1:172.21.15.39:6834/850497663] up:boot 2024-05-21T03:35:00.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:59 smithi027 ceph-mon[32179]: mds.? [v2:172.21.15.39:6833/780238798,v1:172.21.15.39:6835/780238798] up:boot 2024-05-21T03:35:00.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:59 smithi027 ceph-mon[32179]: mds.? [v2:172.21.15.27:6840/714058312,v1:172.21.15.27:6841/714058312] up:boot 2024-05-21T03:35:00.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:59 smithi027 ceph-mon[32179]: mds.? [v2:172.21.15.165:6836/3071563985,v1:172.21.15.165:6837/3071563985] up:boot 2024-05-21T03:35:00.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:59 smithi027 ceph-mon[32179]: mds.? [v2:172.21.15.165:6838/1918660774,v1:172.21.15.165:6839/1918660774] up:boot 2024-05-21T03:35:00.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:59 smithi027 ceph-mon[32179]: mds.? [v2:172.21.15.39:6836/414676672,v1:172.21.15.39:6838/414676672] up:boot 2024-05-21T03:35:00.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:59 smithi027 ceph-mon[32179]: mds.? [v2:172.21.15.39:6837/1606310750,v1:172.21.15.39:6839/1606310750] up:boot 2024-05-21T03:35:00.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:59 smithi027 ceph-mon[32179]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-21T03:35:00.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:59 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T03:35:00.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:59 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-21T03:35:00.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:59 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T03:35:00.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:59 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T03:35:00.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:59 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T03:35:00.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:59 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T03:35:00.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:59 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T03:35:00.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:59 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T03:35:00.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:59 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T03:35:00.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:59 smithi027 ceph-mon[32179]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T03:35:00.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:34:59 smithi027 ceph-mon[32179]: pgmap v341: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 716 B/s wr, 10 op/s 2024-05-21T03:35:00.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:59 smithi039 ceph-mon[36429]: mds.? [v2:172.21.15.165:6832/3820641567,v1:172.21.15.165:6835/3820641567] up:boot 2024-05-21T03:35:00.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:59 smithi039 ceph-mon[36429]: mds.? [v2:172.21.15.165:6833/1571360825,v1:172.21.15.165:6834/1571360825] up:boot 2024-05-21T03:35:00.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:59 smithi039 ceph-mon[36429]: mds.? [v2:172.21.15.27:6838/3680111719,v1:172.21.15.27:6839/3680111719] up:boot 2024-05-21T03:35:00.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:59 smithi039 ceph-mon[36429]: mds.? [v2:172.21.15.39:6832/850497663,v1:172.21.15.39:6834/850497663] up:boot 2024-05-21T03:35:00.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:59 smithi039 ceph-mon[36429]: mds.? [v2:172.21.15.39:6833/780238798,v1:172.21.15.39:6835/780238798] up:boot 2024-05-21T03:35:00.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:59 smithi039 ceph-mon[36429]: mds.? [v2:172.21.15.27:6840/714058312,v1:172.21.15.27:6841/714058312] up:boot 2024-05-21T03:35:00.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:59 smithi039 ceph-mon[36429]: mds.? [v2:172.21.15.165:6836/3071563985,v1:172.21.15.165:6837/3071563985] up:boot 2024-05-21T03:35:00.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:59 smithi039 ceph-mon[36429]: mds.? [v2:172.21.15.165:6838/1918660774,v1:172.21.15.165:6839/1918660774] up:boot 2024-05-21T03:35:00.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:59 smithi039 ceph-mon[36429]: mds.? [v2:172.21.15.39:6836/414676672,v1:172.21.15.39:6838/414676672] up:boot 2024-05-21T03:35:00.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:59 smithi039 ceph-mon[36429]: mds.? [v2:172.21.15.39:6837/1606310750,v1:172.21.15.39:6839/1606310750] up:boot 2024-05-21T03:35:00.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:59 smithi039 ceph-mon[36429]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-21T03:35:00.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:59 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T03:35:00.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:59 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-21T03:35:00.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:59 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T03:35:00.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:59 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T03:35:00.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:59 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T03:35:00.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:59 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T03:35:00.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:59 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T03:35:00.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:59 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T03:35:00.270 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:59 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T03:35:00.271 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:59 smithi039 ceph-mon[36429]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T03:35:00.271 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:34:59 smithi039 ceph-mon[36429]: pgmap v341: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 716 B/s wr, 10 op/s 2024-05-21T03:35:00.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:59 smithi165 ceph-mon[36269]: mds.? [v2:172.21.15.165:6832/3820641567,v1:172.21.15.165:6835/3820641567] up:boot 2024-05-21T03:35:00.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:59 smithi165 ceph-mon[36269]: mds.? [v2:172.21.15.165:6833/1571360825,v1:172.21.15.165:6834/1571360825] up:boot 2024-05-21T03:35:00.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:59 smithi165 ceph-mon[36269]: mds.? [v2:172.21.15.27:6838/3680111719,v1:172.21.15.27:6839/3680111719] up:boot 2024-05-21T03:35:00.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:59 smithi165 ceph-mon[36269]: mds.? [v2:172.21.15.39:6832/850497663,v1:172.21.15.39:6834/850497663] up:boot 2024-05-21T03:35:00.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:59 smithi165 ceph-mon[36269]: mds.? [v2:172.21.15.39:6833/780238798,v1:172.21.15.39:6835/780238798] up:boot 2024-05-21T03:35:00.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:59 smithi165 ceph-mon[36269]: mds.? [v2:172.21.15.27:6840/714058312,v1:172.21.15.27:6841/714058312] up:boot 2024-05-21T03:35:00.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:59 smithi165 ceph-mon[36269]: mds.? [v2:172.21.15.165:6836/3071563985,v1:172.21.15.165:6837/3071563985] up:boot 2024-05-21T03:35:00.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:59 smithi165 ceph-mon[36269]: mds.? [v2:172.21.15.165:6838/1918660774,v1:172.21.15.165:6839/1918660774] up:boot 2024-05-21T03:35:00.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:59 smithi165 ceph-mon[36269]: mds.? [v2:172.21.15.39:6836/414676672,v1:172.21.15.39:6838/414676672] up:boot 2024-05-21T03:35:00.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:59 smithi165 ceph-mon[36269]: mds.? [v2:172.21.15.39:6837/1606310750,v1:172.21.15.39:6839/1606310750] up:boot 2024-05-21T03:35:00.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:59 smithi165 ceph-mon[36269]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-21T03:35:00.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:59 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T03:35:00.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:59 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-21T03:35:00.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:59 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T03:35:00.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:59 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T03:35:00.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:59 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T03:35:00.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:59 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T03:35:00.273 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:59 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T03:35:00.273 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:59 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T03:35:00.273 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:59 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T03:35:00.273 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:59 smithi165 ceph-mon[36269]: from='mgr.24913 172.21.15.27:0/1065093445' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T03:35:00.273 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:34:59 smithi165 ceph-mon[36269]: pgmap v341: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 716 B/s wr, 10 op/s 2024-05-21T03:35:00.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.676+0000 7f383532a700 1 -- 172.21.15.27:0/430046343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38301088f0 msgr2=0x7f3830110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:35:00.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.676+0000 7f383532a700 1 --2- 172.21.15.27:0/430046343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38301088f0 0x7f3830110470 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f382800b0d0 tx=0x7f382800b3e0 comp rx=0 tx=0).stop 2024-05-21T03:35:00.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.676+0000 7f383532a700 1 -- 172.21.15.27:0/430046343 shutdown_connections 2024-05-21T03:35:00.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.676+0000 7f383532a700 1 --2- 172.21.15.27:0/430046343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38301088f0 0x7f3830110470 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:35:00.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.676+0000 7f383532a700 1 --2- 172.21.15.27:0/430046343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38301037f0 0x7f3830103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:35:00.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.676+0000 7f383532a700 1 --2- 172.21.15.27:0/430046343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f38301041a0 0x7f38301081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:35:00.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.676+0000 7f383532a700 1 -- 172.21.15.27:0/430046343 >> 172.21.15.27:0/430046343 conn(0x7f383007a6a0 msgr2=0x7f383007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:35:00.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.676+0000 7f383532a700 1 -- 172.21.15.27:0/430046343 shutdown_connections 2024-05-21T03:35:00.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.676+0000 7f383532a700 1 -- 172.21.15.27:0/430046343 wait complete. 2024-05-21T03:35:00.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.676+0000 7f383532a700 1 Processor -- start 2024-05-21T03:35:00.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.677+0000 7f383532a700 1 -- start start 2024-05-21T03:35:00.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.677+0000 7f383532a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38301037f0 0x7f383019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:35:00.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.677+0000 7f383532a700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f38301041a0 0x7f383019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:35:00.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.677+0000 7f383532a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38301088f0 0x7f38301999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:35:00.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.677+0000 7f383532a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3830113730 con 0x7f38301088f0 2024-05-21T03:35:00.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.677+0000 7f383532a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f38301135b0 con 0x7f38301037f0 2024-05-21T03:35:00.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.677+0000 7f383532a700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f38301138b0 con 0x7f38301041a0 2024-05-21T03:35:00.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.677+0000 7f382f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38301088f0 0x7f38301999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:35:00.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.677+0000 7f382effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38301037f0 0x7f383019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:35:00.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.677+0000 7f382f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38301088f0 0x7f38301999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36484/0 (socket says 172.21.15.27:36484) 2024-05-21T03:35:00.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.677+0000 7f382f7fe700 1 -- 172.21.15.27:0/2158774165 learned_addr learned my addr 172.21.15.27:0/2158774165 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:35:00.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.677+0000 7f382e7fc700 1 --2- 172.21.15.27:0/2158774165 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f38301041a0 0x7f383019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:35:00.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.678+0000 7f382f7fe700 1 -- 172.21.15.27:0/2158774165 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f38301041a0 msgr2=0x7f383019fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:35:00.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.678+0000 7f382f7fe700 1 --2- 172.21.15.27:0/2158774165 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f38301041a0 0x7f383019fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:35:00.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.678+0000 7f382f7fe700 1 -- 172.21.15.27:0/2158774165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38301037f0 msgr2=0x7f383019f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:35:00.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.678+0000 7f382f7fe700 1 --2- 172.21.15.27:0/2158774165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38301037f0 0x7f383019f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:35:00.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.678+0000 7f382f7fe700 1 -- 172.21.15.27:0/2158774165 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f383019a150 con 0x7f38301088f0 2024-05-21T03:35:00.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.678+0000 7f382effd700 1 --2- 172.21.15.27:0/2158774165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38301037f0 0x7f383019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T03:35:00.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.678+0000 7f382f7fe700 1 --2- 172.21.15.27:0/2158774165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38301088f0 0x7f38301999f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f38280095c0 tx=0x7f3828007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:35:00.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.678+0000 7f3827fff700 1 -- 172.21.15.27:0/2158774165 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382801f070 con 0x7f38301088f0 2024-05-21T03:35:00.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.678+0000 7f383532a700 1 -- 172.21.15.27:0/2158774165 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f383019a3e0 con 0x7f38301088f0 2024-05-21T03:35:00.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.678+0000 7f3827fff700 1 -- 172.21.15.27:0/2158774165 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3828003c00 con 0x7f38301088f0 2024-05-21T03:35:00.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.678+0000 7f3827fff700 1 -- 172.21.15.27:0/2158774165 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382801b6a0 con 0x7f38301088f0 2024-05-21T03:35:00.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.678+0000 7f383532a700 1 -- 172.21.15.27:0/2158774165 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38301a66b0 con 0x7f38301088f0 2024-05-21T03:35:00.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.679+0000 7f383532a700 1 -- 172.21.15.27:0/2158774165 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f383006a230 con 0x7f38301088f0 2024-05-21T03:35:00.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.680+0000 7f3827fff700 1 -- 172.21.15.27:0/2158774165 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7f3828022070 con 0x7f38301088f0 2024-05-21T03:35:00.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.681+0000 7f3827fff700 1 --2- 172.21.15.27:0/2158774165 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f381c0b29f0 0x7f381c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:35:00.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.681+0000 7f382effd700 1 --2- 172.21.15.27:0/2158774165 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f381c0b29f0 0x7f381c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:35:00.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.681+0000 7f3827fff700 1 -- 172.21.15.27:0/2158774165 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8957+0+0 (secure 0 0 0) 0x7f382800f020 con 0x7f38301088f0 2024-05-21T03:35:00.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.682+0000 7f382effd700 1 --2- 172.21.15.27:0/2158774165 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f381c0b29f0 0x7f381c0b4eb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f38301083f0 tx=0x7f381800b410 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:35:00.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.685+0000 7f3827fff700 1 -- 172.21.15.27:0/2158774165 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3828029020 con 0x7f38301088f0 2024-05-21T03:35:00.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.947+0000 7f383532a700 1 -- 172.21.15.27:0/2158774165 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f383019aed0 con 0x7f38301088f0 2024-05-21T03:35:00.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.984+0000 7f3827fff700 1 -- 172.21.15.27:0/2158774165 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v56) ==== 104+0+0 (secure 0 0 0) 0x7f38280ab6c0 con 0x7f38301088f0 2024-05-21T03:35:00.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.987+0000 7f383532a700 1 -- 172.21.15.27:0/2158774165 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f381c0b29f0 msgr2=0x7f381c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:35:00.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.987+0000 7f383532a700 1 --2- 172.21.15.27:0/2158774165 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f381c0b29f0 0x7f381c0b4eb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f38301083f0 tx=0x7f381800b410 comp rx=0 tx=0).stop 2024-05-21T03:35:00.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.987+0000 7f383532a700 1 -- 172.21.15.27:0/2158774165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38301088f0 msgr2=0x7f38301999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:35:00.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.987+0000 7f383532a700 1 --2- 172.21.15.27:0/2158774165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38301088f0 0x7f38301999f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f38280095c0 tx=0x7f3828007a20 comp rx=0 tx=0).stop 2024-05-21T03:35:00.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.987+0000 7f383532a700 1 -- 172.21.15.27:0/2158774165 shutdown_connections 2024-05-21T03:35:00.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.987+0000 7f383532a700 1 --2- 172.21.15.27:0/2158774165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38301088f0 0x7f38301999f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:35:00.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.987+0000 7f383532a700 1 --2- 172.21.15.27:0/2158774165 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f381c0b29f0 0x7f381c0b4eb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:35:00.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.987+0000 7f383532a700 1 --2- 172.21.15.27:0/2158774165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38301037f0 0x7f383019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:35:00.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.987+0000 7f383532a700 1 --2- 172.21.15.27:0/2158774165 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f38301041a0 0x7f383019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:35:00.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.987+0000 7f383532a700 1 -- 172.21.15.27:0/2158774165 >> 172.21.15.27:0/2158774165 conn(0x7f383007a6a0 msgr2=0x7f3830101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:35:00.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.988+0000 7f383532a700 1 -- 172.21.15.27:0/2158774165 shutdown_connections 2024-05-21T03:35:00.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:00.988+0000 7f383532a700 1 -- 172.21.15.27:0/2158774165 wait complete. 2024-05-21T03:35:01.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:35:00 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/2158774165' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-21T03:35:01.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:35:00 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/2158774165' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-21T03:35:01.426 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:35:00 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2158774165' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-21T03:35:01.775 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-21T03:35:02.028 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:35:02.146 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:35:01 smithi027 ceph-mon[32179]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-21T03:35:02.147 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:35:01 smithi027 ceph-mon[32179]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-21T03:35:02.147 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:35:01 smithi027 ceph-mon[32179]: Cluster is now healthy 2024-05-21T03:35:02.147 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:35:01 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2158774165' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-21T03:35:02.147 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:35:01 smithi027 ceph-mon[32179]: fsmap 12 up:standby 2024-05-21T03:35:02.147 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:35:01 smithi027 ceph-mon[32179]: pgmap v342: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T03:35:02.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:35:01 smithi039 ceph-mon[36429]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-21T03:35:02.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:35:01 smithi039 ceph-mon[36429]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-21T03:35:02.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:35:01 smithi039 ceph-mon[36429]: Cluster is now healthy 2024-05-21T03:35:02.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:35:01 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/2158774165' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-21T03:35:02.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:35:01 smithi039 ceph-mon[36429]: fsmap 12 up:standby 2024-05-21T03:35:02.269 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:35:01 smithi039 ceph-mon[36429]: pgmap v342: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T03:35:02.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:35:01 smithi165 ceph-mon[36269]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-21T03:35:02.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:35:01 smithi165 ceph-mon[36269]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-21T03:35:02.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:35:01 smithi165 ceph-mon[36269]: Cluster is now healthy 2024-05-21T03:35:02.271 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:35:01 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/2158774165' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-21T03:35:02.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:35:01 smithi165 ceph-mon[36269]: fsmap 12 up:standby 2024-05-21T03:35:02.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:35:01 smithi165 ceph-mon[36269]: pgmap v342: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T03:35:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.680+0000 7f160f34f700 1 -- 172.21.15.27:0/1626683113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1608103590 msgr2=0x7f1608103a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:35:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.680+0000 7f160f34f700 1 --2- 172.21.15.27:0/1626683113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1608103590 0x7f1608103a10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009990 tx=0x7f15fc009ca0 comp rx=0 tx=0).stop 2024-05-21T03:35:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.680+0000 7f160f34f700 1 -- 172.21.15.27:0/1626683113 shutdown_connections 2024-05-21T03:35:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.680+0000 7f160f34f700 1 --2- 172.21.15.27:0/1626683113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1608103f50 0x7f1608110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:35:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.680+0000 7f160f34f700 1 --2- 172.21.15.27:0/1626683113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16081096a0 0x7f1608109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:35:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.680+0000 7f160f34f700 1 --2- 172.21.15.27:0/1626683113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1608103590 0x7f1608103a10 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:35:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.680+0000 7f160f34f700 1 -- 172.21.15.27:0/1626683113 >> 172.21.15.27:0/1626683113 conn(0x7f160807a6f0 msgr2=0x7f160807ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:35:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.680+0000 7f160f34f700 1 -- 172.21.15.27:0/1626683113 shutdown_connections 2024-05-21T03:35:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.680+0000 7f160f34f700 1 -- 172.21.15.27:0/1626683113 wait complete. 2024-05-21T03:35:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.681+0000 7f160f34f700 1 Processor -- start 2024-05-21T03:35:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.681+0000 7f160f34f700 1 -- start start 2024-05-21T03:35:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.681+0000 7f160f34f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1608103590 0x7f1608075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:35:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.681+0000 7f160f34f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1608103f50 0x7f1608075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:35:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.681+0000 7f160f34f700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16081096a0 0x7f16080762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:35:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.681+0000 7f160f34f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1608113730 con 0x7f1608103f50 2024-05-21T03:35:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.681+0000 7f160f34f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f16081135b0 con 0x7f1608103590 2024-05-21T03:35:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.681+0000 7f160f34f700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f16081138b0 con 0x7f16081096a0 2024-05-21T03:35:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.681+0000 7f160c8ea700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1608103f50 0x7f1608075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:35:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.681+0000 7f160c8ea700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1608103f50 0x7f1608075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32860/0 (socket says 172.21.15.27:32860) 2024-05-21T03:35:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.681+0000 7f160c8ea700 1 -- 172.21.15.27:0/1996210688 learned_addr learned my addr 172.21.15.27:0/1996210688 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:35:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.681+0000 7f160d0eb700 1 --2- 172.21.15.27:0/1996210688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1608103590 0x7f1608075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:35:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.682+0000 7f160d8ec700 1 --2- 172.21.15.27:0/1996210688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16081096a0 0x7f16080762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:35:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.682+0000 7f160c8ea700 1 -- 172.21.15.27:0/1996210688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16081096a0 msgr2=0x7f16080762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:35:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.682+0000 7f160c8ea700 1 --2- 172.21.15.27:0/1996210688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16081096a0 0x7f16080762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:35:03.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.682+0000 7f160c8ea700 1 -- 172.21.15.27:0/1996210688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1608103590 msgr2=0x7f1608075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:35:03.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.682+0000 7f160c8ea700 1 --2- 172.21.15.27:0/1996210688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1608103590 0x7f1608075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:35:03.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.682+0000 7f160c8ea700 1 -- 172.21.15.27:0/1996210688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16081a8140 con 0x7f1608103f50 2024-05-21T03:35:03.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.682+0000 7f160c8ea700 1 --2- 172.21.15.27:0/1996210688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1608103f50 0x7f1608075d80 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f15fc000c00 tx=0x7f15fc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:35:03.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.682+0000 7f16067fc700 1 -- 172.21.15.27:0/1996210688 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc027070 con 0x7f1608103f50 2024-05-21T03:35:03.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.682+0000 7f16067fc700 1 -- 172.21.15.27:0/1996210688 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15fc003f20 con 0x7f1608103f50 2024-05-21T03:35:03.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.682+0000 7f160f34f700 1 -- 172.21.15.27:0/1996210688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16081a83d0 con 0x7f1608103f50 2024-05-21T03:35:03.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.682+0000 7f16067fc700 1 -- 172.21.15.27:0/1996210688 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc022ca0 con 0x7f1608103f50 2024-05-21T03:35:03.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.682+0000 7f160f34f700 1 -- 172.21.15.27:0/1996210688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16081a8880 con 0x7f1608103f50 2024-05-21T03:35:03.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.683+0000 7f160f34f700 1 -- 172.21.15.27:0/1996210688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f160806a230 con 0x7f1608103f50 2024-05-21T03:35:03.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.685+0000 7f16067fc700 1 -- 172.21.15.27:0/1996210688 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7f15fc02a070 con 0x7f1608103f50 2024-05-21T03:35:03.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.685+0000 7f16067fc700 1 --2- 172.21.15.27:0/1996210688 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f15f00b2a70 0x7f15f00b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:35:03.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.685+0000 7f160d0eb700 1 --2- 172.21.15.27:0/1996210688 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f15f00b2a70 0x7f15f00b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:35:03.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.686+0000 7f16067fc700 1 -- 172.21.15.27:0/1996210688 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8957+0+0 (secure 0 0 0) 0x7f15fc0e3c50 con 0x7f1608103f50 2024-05-21T03:35:03.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.686+0000 7f160d0eb700 1 --2- 172.21.15.27:0/1996210688 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f15f00b2a70 0x7f15f00b4f30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f15f8009550 tx=0x7f15f80091f0 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:35:03.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.689+0000 7f16067fc700 1 -- 172.21.15.27:0/1996210688 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f15fc031070 con 0x7f1608103f50 2024-05-21T03:35:03.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:03.901+0000 7f160f34f700 1 -- 172.21.15.27:0/1996210688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f160802e560 con 0x7f1608103f50 2024-05-21T03:35:04.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:04.630+0000 7f16067fc700 1 -- 172.21.15.27:0/1996210688 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v93) ==== 181+0+0 (secure 0 0 0) 0x7f15fc013070 con 0x7f1608103f50 2024-05-21T03:35:04.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:04.633+0000 7f160f34f700 1 -- 172.21.15.27:0/1996210688 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f15f00b2a70 msgr2=0x7f15f00b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:35:04.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:04.633+0000 7f160f34f700 1 --2- 172.21.15.27:0/1996210688 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f15f00b2a70 0x7f15f00b4f30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f15f8009550 tx=0x7f15f80091f0 comp rx=0 tx=0).stop 2024-05-21T03:35:04.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:04.633+0000 7f160f34f700 1 -- 172.21.15.27:0/1996210688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1608103f50 msgr2=0x7f1608075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:35:04.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:04.633+0000 7f160f34f700 1 --2- 172.21.15.27:0/1996210688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1608103f50 0x7f1608075d80 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f15fc000c00 tx=0x7f15fc0047b0 comp rx=0 tx=0).stop 2024-05-21T03:35:04.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:04.633+0000 7f160f34f700 1 -- 172.21.15.27:0/1996210688 shutdown_connections 2024-05-21T03:35:04.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:04.633+0000 7f160f34f700 1 --2- 172.21.15.27:0/1996210688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16081096a0 0x7f16080762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:35:04.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:04.633+0000 7f160f34f700 1 --2- 172.21.15.27:0/1996210688 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f15f00b2a70 0x7f15f00b4f30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:35:04.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:04.633+0000 7f160f34f700 1 --2- 172.21.15.27:0/1996210688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1608103590 0x7f1608075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:35:04.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:04.634+0000 7f160f34f700 1 --2- 172.21.15.27:0/1996210688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1608103f50 0x7f1608075d80 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:35:04.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:04.634+0000 7f160f34f700 1 -- 172.21.15.27:0/1996210688 >> 172.21.15.27:0/1996210688 conn(0x7f160807a6f0 msgr2=0x7f1608102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:35:04.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:04.634+0000 7f160f34f700 1 -- 172.21.15.27:0/1996210688 shutdown_connections 2024-05-21T03:35:04.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:04.634+0000 7f160f34f700 1 -- 172.21.15.27:0/1996210688 wait complete. 2024-05-21T03:35:04.635 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2024-05-21T03:35:04.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:35:04 smithi027 ceph-mon[32179]: pgmap v343: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T03:35:04.927 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:35:04 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/1996210688' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-21T03:35:05.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:35:04 smithi039 ceph-mon[36429]: pgmap v343: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T03:35:05.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:35:04 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/1996210688' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-21T03:35:05.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:35:04 smithi165 ceph-mon[36269]: pgmap v343: 97 pgs: 97 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T03:35:05.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:35:04 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/1996210688' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-21T03:35:05.245 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-21T03:35:05.495 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T03:35:05.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:35:05 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/1996210688' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-21T03:35:05.914 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:35:05 smithi027 ceph-mon[32179]: osdmap e93: 12 total, 12 up, 12 in 2024-05-21T03:35:06.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:35:05 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/1996210688' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-21T03:35:06.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:35:05 smithi039 ceph-mon[36429]: osdmap e93: 12 total, 12 up, 12 in 2024-05-21T03:35:06.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:35:05 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/1996210688' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-21T03:35:06.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:35:05 smithi165 ceph-mon[36269]: osdmap e93: 12 total, 12 up, 12 in 2024-05-21T03:35:06.910 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:35:06 smithi027 ceph-mon[32179]: pgmap v345: 65 pgs: 65 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail 2024-05-21T03:35:07.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:35:06 smithi039 ceph-mon[36429]: pgmap v345: 65 pgs: 65 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail 2024-05-21T03:35:07.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:35:06 smithi165 ceph-mon[36269]: pgmap v345: 65 pgs: 65 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail 2024-05-21T03:35:07.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.272+0000 7f9444244700 1 -- 172.21.15.27:0/1862246757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f943c06c8c0 msgr2=0x7f943c108180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:35:07.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.272+0000 7f9444244700 1 --2- 172.21.15.27:0/1862246757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f943c06c8c0 0x7f943c108180 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f94300099c0 tx=0x7f9430009cd0 comp rx=0 tx=0).stop 2024-05-21T03:35:07.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.272+0000 7f9444244700 1 -- 172.21.15.27:0/1862246757 shutdown_connections 2024-05-21T03:35:07.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.272+0000 7f9444244700 1 --2- 172.21.15.27:0/1862246757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f943c108860 0x7f943c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:35:07.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.272+0000 7f9444244700 1 --2- 172.21.15.27:0/1862246757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f943c06bf10 0x7f943c06c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:35:07.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.272+0000 7f9444244700 1 --2- 172.21.15.27:0/1862246757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f943c06c8c0 0x7f943c108180 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:35:07.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.272+0000 7f9444244700 1 -- 172.21.15.27:0/1862246757 >> 172.21.15.27:0/1862246757 conn(0x7f943c078490 msgr2=0x7f943c0788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:35:07.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.272+0000 7f9444244700 1 -- 172.21.15.27:0/1862246757 shutdown_connections 2024-05-21T03:35:07.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.273+0000 7f9444244700 1 -- 172.21.15.27:0/1862246757 wait complete. 2024-05-21T03:35:07.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.273+0000 7f9444244700 1 Processor -- start 2024-05-21T03:35:07.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.273+0000 7f9444244700 1 -- start start 2024-05-21T03:35:07.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.273+0000 7f9444244700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f943c06bf10 0x7f943c1025a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:35:07.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.273+0000 7f9444244700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f943c06c8c0 0x7f943c102ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:35:07.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.273+0000 7f9444244700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f943c108860 0x7f943c103020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:35:07.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.273+0000 7f9444244700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f943c113790 con 0x7f943c108860 2024-05-21T03:35:07.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.273+0000 7f9444244700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f943c113610 con 0x7f943c06c8c0 2024-05-21T03:35:07.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.273+0000 7f9444244700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f943c113910 con 0x7f943c06bf10 2024-05-21T03:35:07.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.274+0000 7f94427e1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f943c108860 0x7f943c103020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:35:07.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.274+0000 7f94427e1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f943c108860 0x7f943c103020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32876/0 (socket says 172.21.15.27:32876) 2024-05-21T03:35:07.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.274+0000 7f94427e1700 1 -- 172.21.15.27:0/2783157744 learned_addr learned my addr 172.21.15.27:0/2783157744 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-21T03:35:07.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.274+0000 7f94417df700 1 --2- 172.21.15.27:0/2783157744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f943c06c8c0 0x7f943c102ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:35:07.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.274+0000 7f9441fe0700 1 --2- 172.21.15.27:0/2783157744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f943c06bf10 0x7f943c1025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:35:07.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.274+0000 7f94427e1700 1 -- 172.21.15.27:0/2783157744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f943c06bf10 msgr2=0x7f943c1025a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:35:07.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.274+0000 7f94427e1700 1 --2- 172.21.15.27:0/2783157744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f943c06bf10 0x7f943c1025a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:35:07.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.274+0000 7f94427e1700 1 -- 172.21.15.27:0/2783157744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f943c06c8c0 msgr2=0x7f943c102ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:35:07.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.274+0000 7f94427e1700 1 --2- 172.21.15.27:0/2783157744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f943c06c8c0 0x7f943c102ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:35:07.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.274+0000 7f94427e1700 1 -- 172.21.15.27:0/2783157744 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f943c1a8190 con 0x7f943c108860 2024-05-21T03:35:07.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.274+0000 7f94427e1700 1 --2- 172.21.15.27:0/2783157744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f943c108860 0x7f943c103020 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f943400c970 tx=0x7f943400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:35:07.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.274+0000 7f943affd700 1 -- 172.21.15.27:0/2783157744 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434004a30 con 0x7f943c108860 2024-05-21T03:35:07.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.274+0000 7f943affd700 1 -- 172.21.15.27:0/2783157744 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94340054d0 con 0x7f943c108860 2024-05-21T03:35:07.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.274+0000 7f9444244700 1 -- 172.21.15.27:0/2783157744 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f943c1a8480 con 0x7f943c108860 2024-05-21T03:35:07.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.275+0000 7f943affd700 1 -- 172.21.15.27:0/2783157744 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943400ec20 con 0x7f943c108860 2024-05-21T03:35:07.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.275+0000 7f9444244700 1 -- 172.21.15.27:0/2783157744 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f943c1a8940 con 0x7f943c108860 2024-05-21T03:35:07.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.276+0000 7f9444244700 1 -- 172.21.15.27:0/2783157744 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f943c06a230 con 0x7f943c108860 2024-05-21T03:35:07.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.278+0000 7f943affd700 1 -- 172.21.15.27:0/2783157744 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) ==== 149783+0+0 (secure 0 0 0) 0x7f9434004fc0 con 0x7f943c108860 2024-05-21T03:35:07.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.279+0000 7f943affd700 1 --2- 172.21.15.27:0/2783157744 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f94240b29f0 0x7f94240b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T03:35:07.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.279+0000 7f9441fe0700 1 --2- 172.21.15.27:0/2783157744 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f94240b29f0 0x7f94240b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T03:35:07.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.279+0000 7f943affd700 1 -- 172.21.15.27:0/2783157744 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8432+0+0 (secure 0 0 0) 0x7f94340e2410 con 0x7f943c108860 2024-05-21T03:35:07.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.279+0000 7f9441fe0700 1 --2- 172.21.15.27:0/2783157744 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f94240b29f0 0x7f94240b4eb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f942c009550 tx=0x7f942c0091b0 comp rx=0 tx=0).ready entity=mgr.24913 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T03:35:07.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.282+0000 7f943affd700 1 -- 172.21.15.27:0/2783157744 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9434014030 con 0x7f943c108860 2024-05-21T03:35:07.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.498+0000 7f9444244700 1 -- 172.21.15.27:0/2783157744 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f943c02e7a0 con 0x7f943c108860 2024-05-21T03:35:07.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.653+0000 7f943affd700 1 -- 172.21.15.27:0/2783157744 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v94) ==== 169+0+0 (secure 0 0 0) 0x7f94340a9e50 con 0x7f943c108860 2024-05-21T03:35:07.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.656+0000 7f9444244700 1 -- 172.21.15.27:0/2783157744 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f94240b29f0 msgr2=0x7f94240b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:35:07.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.656+0000 7f9444244700 1 --2- 172.21.15.27:0/2783157744 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f94240b29f0 0x7f94240b4eb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f942c009550 tx=0x7f942c0091b0 comp rx=0 tx=0).stop 2024-05-21T03:35:07.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.656+0000 7f9444244700 1 -- 172.21.15.27:0/2783157744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f943c108860 msgr2=0x7f943c103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T03:35:07.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.656+0000 7f9444244700 1 --2- 172.21.15.27:0/2783157744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f943c108860 0x7f943c103020 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f943400c970 tx=0x7f943400cc80 comp rx=0 tx=0).stop 2024-05-21T03:35:07.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.657+0000 7f9444244700 1 -- 172.21.15.27:0/2783157744 shutdown_connections 2024-05-21T03:35:07.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.657+0000 7f9444244700 1 --2- 172.21.15.27:0/2783157744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f943c108860 0x7f943c103020 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:35:07.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.657+0000 7f9444244700 1 --2- 172.21.15.27:0/2783157744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f943c06bf10 0x7f943c1025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:35:07.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.657+0000 7f9444244700 1 --2- 172.21.15.27:0/2783157744 >> [v2:172.21.15.27:6800/1477999924,v1:172.21.15.27:6801/1477999924] conn(0x7f94240b29f0 0x7f94240b4eb0 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f942c009550 tx=0x7f942c0091b0 comp rx=0 tx=0).stop 2024-05-21T03:35:07.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.657+0000 7f9444244700 1 --2- 172.21.15.27:0/2783157744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f943c06c8c0 0x7f943c102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T03:35:07.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.657+0000 7f9444244700 1 -- 172.21.15.27:0/2783157744 >> 172.21.15.27:0/2783157744 conn(0x7f943c078490 msgr2=0x7f943c107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T03:35:07.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.660+0000 7f9444244700 1 -- 172.21.15.27:0/2783157744 shutdown_connections 2024-05-21T03:35:07.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-21T03:35:07.660+0000 7f9444244700 1 -- 172.21.15.27:0/2783157744 wait complete. 2024-05-21T03:35:07.663 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2024-05-21T03:35:07.920 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:35:07 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2783157744' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-21T03:35:08.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:35:07 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/2783157744' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-21T03:35:08.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:35:07 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/2783157744' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-21T03:35:08.530 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T03:35:08.574 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T03:35:08.602 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T03:35:08.628 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-21T03:35:08.628 DEBUG:teuthology.orchestra.run.smithi027:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-21T03:35:08.644 DEBUG:teuthology.orchestra.run.smithi039:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-21T03:35:08.659 DEBUG:teuthology.orchestra.run.smithi165:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-21T03:35:08.682 INFO:tasks.cephadm:Stopping all daemons... 2024-05-21T03:35:08.683 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-21T03:35:08.683 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mon.a 2024-05-21T03:35:08.707 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:35:08 smithi027 ceph-mon[32179]: pgmap v346: 65 pgs: 65 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail 2024-05-21T03:35:08.708 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:35:08 smithi027 ceph-mon[32179]: from='client.? 172.21.15.27:0/2783157744' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-21T03:35:08.708 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:35:08 smithi027 ceph-mon[32179]: osdmap e94: 12 total, 12 up, 12 in 2024-05-21T03:35:09.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:35:08 smithi039 ceph-mon[36429]: pgmap v346: 65 pgs: 65 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail 2024-05-21T03:35:09.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:35:08 smithi039 ceph-mon[36429]: from='client.? 172.21.15.27:0/2783157744' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-21T03:35:09.019 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:35:08 smithi039 ceph-mon[36429]: osdmap e94: 12 total, 12 up, 12 in 2024-05-21T03:35:09.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:35:08 smithi165 ceph-mon[36269]: pgmap v346: 65 pgs: 65 active+clean; 18 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail 2024-05-21T03:35:09.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:35:08 smithi165 ceph-mon[36269]: from='client.? 172.21.15.27:0/2783157744' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-21T03:35:09.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:35:08 smithi165 ceph-mon[36269]: osdmap e94: 12 total, 12 up, 12 in 2024-05-21T03:35:09.148 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:35:08 smithi027 systemd[1]: Stopping Ceph mon.a for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:35:09.426 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:35:09 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mon-a[32174]: 2024-05-21T03:35:09.147+0000 7f8e7241b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T03:35:09.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:35:09 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mon-a[32174]: 2024-05-21T03:35:09.147+0000 7f8e7241b700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-21T03:35:09.427 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:35:09 smithi027 podman[104534]: 2024-05-21 03:35:09.188977266 +0000 UTC m=+0.288948798 container died eb3a7e6bf9c301052ffeb751719914039918507b8941d4bdcfebc68f6689b284 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-21T03:35:09.822 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:35:09 smithi027 podman[104534]: 2024-05-21 03:35:09.555610892 +0000 UTC m=+0.655582416 container cleanup eb3a7e6bf9c301052ffeb751719914039918507b8941d4bdcfebc68f6689b284 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mon-a, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-21T03:35:09.823 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:35:09 smithi027 bash[104534]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mon-a 2024-05-21T03:35:09.993 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mon.a.service' 2024-05-21T03:35:10.176 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:35:09 smithi027 podman[104546]: 2024-05-21 03:35:09.823669433 +0000 UTC m=+0.628710684 container remove eb3a7e6bf9c301052ffeb751719914039918507b8941d4bdcfebc68f6689b284 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-21T03:35:10.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:35:09 smithi027 systemd[1]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-21T03:35:10.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:35:09 smithi027 systemd[1]: Stopped Ceph mon.a for c9ca94be-171f-11ef-bc9a-c7b262605968. 2024-05-21T03:35:10.177 INFO:journalctl@ceph.mon.a.smithi027.stdout:May 21 03:35:09 smithi027 systemd[1]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mon.a.service: Consumed 20.177s CPU time. 2024-05-21T03:35:10.205 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T03:35:10.206 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-21T03:35:10.206 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-21T03:35:10.206 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mon.b 2024-05-21T03:35:10.496 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:35:10 smithi039 systemd[1]: Stopping Ceph mon.b for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:35:10.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:35:10 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mon-b[36425]: 2024-05-21T03:35:10.496+0000 7f54ff0c8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T03:35:10.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:35:10 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mon-b[36425]: 2024-05-21T03:35:10.496+0000 7f54ff0c8700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-21T03:35:10.769 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:35:10 smithi039 podman[72962]: 2024-05-21 03:35:10.629241443 +0000 UTC m=+0.232778653 container died e7b68b78f66c430aabdf5943ce1597171fce4f5e4b63fe0889a7f791bb9d68c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mon-b, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5) 2024-05-21T03:35:11.238 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:35:10 smithi039 podman[72962]: 2024-05-21 03:35:10.987706503 +0000 UTC m=+0.591243711 container cleanup e7b68b78f66c430aabdf5943ce1597171fce4f5e4b63fe0889a7f791bb9d68c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mon-b, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, org.label-schema.license=GPLv2) 2024-05-21T03:35:11.238 INFO:journalctl@ceph.mon.b.smithi039.stdout:May 21 03:35:10 smithi039 bash[72962]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mon-b 2024-05-21T03:35:11.378 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mon.b.service' 2024-05-21T03:35:11.420 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T03:35:11.420 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-21T03:35:11.420 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-21T03:35:11.420 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mon.c 2024-05-21T03:35:11.726 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:35:11 smithi165 systemd[1]: Stopping Ceph mon.c for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:35:12.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:35:11 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mon-c[36263]: 2024-05-21T03:35:11.725+0000 7f97f30c0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T03:35:12.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:35:11 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mon-c[36263]: 2024-05-21T03:35:11.725+0000 7f97f30c0700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-21T03:35:12.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:35:11 smithi165 podman[73437]: 2024-05-21 03:35:11.899332167 +0000 UTC m=+0.286794892 container died 0f5237d01455593d06e743a345847cb6ad84cba15816fa0b5590c0c6fe13ae49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mon-c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-21T03:35:12.519 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:35:12 smithi165 podman[73437]: 2024-05-21 03:35:12.310418738 +0000 UTC m=+0.697881457 container cleanup 0f5237d01455593d06e743a345847cb6ad84cba15816fa0b5590c0c6fe13ae49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-21T03:35:12.519 INFO:journalctl@ceph.mon.c.smithi165.stdout:May 21 03:35:12 smithi165 bash[73437]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mon-c 2024-05-21T03:35:12.659 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mon.c.service' 2024-05-21T03:35:12.707 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T03:35:12.707 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-21T03:35:12.707 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-21T03:35:12.708 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mgr.x 2024-05-21T03:35:13.043 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:35:12 smithi027 systemd[1]: Stopping Ceph mgr.x for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:35:13.427 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:35:13 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:35:13.042+0000 7f04aecc8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T03:35:13.427 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:35:13 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x[32406]: 2024-05-21T03:35:13.042+0000 7f04aecc8700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-05-21T03:35:13.427 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:35:13 smithi027 podman[104661]: 2024-05-21 03:35:13.074352375 +0000 UTC m=+0.172637669 container died 41efc524711bbeda604cbbf4d11631ed110b75a977140138b00ad7924fc722c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux ) 2024-05-21T03:35:13.811 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:35:13 smithi027 podman[104661]: 2024-05-21 03:35:13.535332382 +0000 UTC m=+0.633617676 container cleanup 41efc524711bbeda604cbbf4d11631ed110b75a977140138b00ad7924fc722c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-21T03:35:13.811 INFO:journalctl@ceph.mgr.x.smithi027.stdout:May 21 03:35:13 smithi027 bash[104661]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-x 2024-05-21T03:35:13.941 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mgr.x.service' 2024-05-21T03:35:13.977 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T03:35:13.978 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-21T03:35:13.978 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-21T03:35:13.978 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mgr.y 2024-05-21T03:35:14.269 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:35:14 smithi039 systemd[1]: Stopping Ceph mgr.y for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:35:14.584 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:35:14 smithi039 podman[73085]: 2024-05-21 03:35:14.304676081 +0000 UTC m=+0.136036798 container died fac9a0fe8ebc20b23a541902ffd6821aed728df818d32da298155cc3dc1ba468 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-21T03:35:15.019 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:35:14 smithi039 podman[73085]: 2024-05-21 03:35:14.801733843 +0000 UTC m=+0.633094554 container cleanup fac9a0fe8ebc20b23a541902ffd6821aed728df818d32da298155cc3dc1ba468 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-21T03:35:15.019 INFO:journalctl@ceph.mgr.y.smithi039.stdout:May 21 03:35:14 smithi039 bash[73085]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-y 2024-05-21T03:35:15.266 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mgr.y.service' 2024-05-21T03:35:15.300 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T03:35:15.300 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-21T03:35:15.300 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-21T03:35:15.300 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mgr.z 2024-05-21T03:35:15.636 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:35:15 smithi165 systemd[1]: Stopping Ceph mgr.z for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:35:15.906 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:35:15 smithi165 podman[73561]: 2024-05-21 03:35:15.636050617 +0000 UTC m=+0.140072363 container died cd5326e5d3958a25c9ff5681bbcee2664423324810903e02e29bc51e8bd74949 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T03:35:16.271 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:35:15 smithi165 podman[73561]: 2024-05-21 03:35:15.990262783 +0000 UTC m=+0.494284522 container cleanup cd5326e5d3958a25c9ff5681bbcee2664423324810903e02e29bc51e8bd74949 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea) 2024-05-21T03:35:16.271 INFO:journalctl@ceph.mgr.z.smithi165.stdout:May 21 03:35:15 smithi165 bash[73561]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mgr-z 2024-05-21T03:35:16.440 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mgr.z.service' 2024-05-21T03:35:16.472 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T03:35:16.473 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-21T03:35:16.473 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-21T03:35:16.473 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.0 2024-05-21T03:35:16.764 INFO:journalctl@ceph.osd.0.smithi027.stdout:May 21 03:35:16 smithi027 systemd[1]: Stopping Ceph osd.0 for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:35:17.176 INFO:journalctl@ceph.osd.0.smithi027.stdout:May 21 03:35:16 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-0[44703]: 2024-05-21T03:35:16.763+0000 7f6c7bfa5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T03:35:17.177 INFO:journalctl@ceph.osd.0.smithi027.stdout:May 21 03:35:16 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-0[44703]: 2024-05-21T03:35:16.763+0000 7f6c7bfa5700 -1 osd.0 94 *** Got signal Terminated *** 2024-05-21T03:35:17.177 INFO:journalctl@ceph.osd.0.smithi027.stdout:May 21 03:35:16 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-0[44703]: 2024-05-21T03:35:16.763+0000 7f6c7bfa5700 -1 osd.0 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T03:35:47.096 INFO:journalctl@ceph.osd.0.smithi027.stdout:May 21 03:35:46 smithi027 bash[104783]: time="2024-05-21T03:35:46Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-0 in 30 seconds, resorting to SIGKILL" 2024-05-21T03:35:47.096 INFO:journalctl@ceph.osd.0.smithi027.stdout:May 21 03:35:46 smithi027 podman[104783]: 2024-05-21 03:35:46.916690235 +0000 UTC m=+30.252794102 container died 49e997ffe7ed02b2c283b63ba12c7157ddc783b45efb77ad3a8f2b22c6c143dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-21T03:35:47.677 INFO:journalctl@ceph.osd.0.smithi027.stdout:May 21 03:35:47 smithi027 podman[104783]: 2024-05-21 03:35:47.355029123 +0000 UTC m=+30.691132983 container cleanup 49e997ffe7ed02b2c283b63ba12c7157ddc783b45efb77ad3a8f2b22c6c143dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T03:35:47.677 INFO:journalctl@ceph.osd.0.smithi027.stdout:May 21 03:35:47 smithi027 bash[104783]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-0 2024-05-21T03:35:47.677 INFO:journalctl@ceph.osd.0.smithi027.stdout:May 21 03:35:47 smithi027 podman[104800]: 2024-05-21 03:35:47.53125137 +0000 UTC m=+0.609953666 container remove 49e997ffe7ed02b2c283b63ba12c7157ddc783b45efb77ad3a8f2b22c6c143dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-21T03:35:47.677 INFO:journalctl@ceph.osd.0.smithi027.stdout:May 21 03:35:47 smithi027 systemd[1]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.0.service: Main process exited, code=exited, status=137/n/a 2024-05-21T03:35:48.352 INFO:journalctl@ceph.osd.0.smithi027.stdout:May 21 03:35:47 smithi027 podman[104890]: 2024-05-21 03:35:47.885462394 +0000 UTC m=+0.018298506 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T03:35:48.353 INFO:journalctl@ceph.osd.0.smithi027.stdout:May 21 03:35:48 smithi027 podman[104890]: 2024-05-21 03:35:48.082297529 +0000 UTC m=+0.215133612 container create aca84c9c3c48629654d3410702e2f42c5f14ba2c9fffe190a892fc91d2f194a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-0-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-21T03:35:48.353 INFO:journalctl@ceph.osd.0.smithi027.stdout:May 21 03:35:48 smithi027 podman[104890]: 2024-05-21 03:35:48.35370876 +0000 UTC m=+0.486544858 container init aca84c9c3c48629654d3410702e2f42c5f14ba2c9fffe190a892fc91d2f194a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-21T03:35:48.677 INFO:journalctl@ceph.osd.0.smithi027.stdout:May 21 03:35:48 smithi027 podman[104890]: 2024-05-21 03:35:48.357582922 +0000 UTC m=+0.490419007 container start aca84c9c3c48629654d3410702e2f42c5f14ba2c9fffe190a892fc91d2f194a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_BRANCH=HEAD, ceph=True) 2024-05-21T03:35:48.677 INFO:journalctl@ceph.osd.0.smithi027.stdout:May 21 03:35:48 smithi027 podman[104890]: 2024-05-21 03:35:48.437027004 +0000 UTC m=+0.569863096 container attach aca84c9c3c48629654d3410702e2f42c5f14ba2c9fffe190a892fc91d2f194a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-0-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-21T03:35:49.147 INFO:journalctl@ceph.osd.0.smithi027.stdout:May 21 03:35:48 smithi027 podman[104890]: 2024-05-21 03:35:48.679063444 +0000 UTC m=+0.811899536 container died aca84c9c3c48629654d3410702e2f42c5f14ba2c9fffe190a892fc91d2f194a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True) 2024-05-21T03:35:49.398 INFO:journalctl@ceph.osd.0.smithi027.stdout:May 21 03:35:49 smithi027 podman[104917]: 2024-05-21 03:35:49.382368451 +0000 UTC m=+0.697261856 container remove aca84c9c3c48629654d3410702e2f42c5f14ba2c9fffe190a892fc91d2f194a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux ) 2024-05-21T03:35:49.402 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.0.service' 2024-05-21T03:35:49.444 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T03:35:49.444 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-21T03:35:49.444 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-21T03:35:49.444 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.1 2024-05-21T03:35:49.769 INFO:journalctl@ceph.osd.1.smithi039.stdout:May 21 03:35:49 smithi039 systemd[1]: Stopping Ceph osd.1 for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:35:50.064 INFO:journalctl@ceph.osd.1.smithi039.stdout:May 21 03:35:49 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-1[41013]: 2024-05-21T03:35:49.808+0000 7fae6a3bc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T03:35:50.064 INFO:journalctl@ceph.osd.1.smithi039.stdout:May 21 03:35:49 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-1[41013]: 2024-05-21T03:35:49.808+0000 7fae6a3bc700 -1 osd.1 94 *** Got signal Terminated *** 2024-05-21T03:35:50.065 INFO:journalctl@ceph.osd.1.smithi039.stdout:May 21 03:35:49 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-1[41013]: 2024-05-21T03:35:49.808+0000 7fae6a3bc700 -1 osd.1 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T03:36:20.269 INFO:journalctl@ceph.osd.1.smithi039.stdout:May 21 03:36:19 smithi039 bash[73208]: time="2024-05-21T03:36:19Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-1 in 30 seconds, resorting to SIGKILL" 2024-05-21T03:36:20.269 INFO:journalctl@ceph.osd.1.smithi039.stdout:May 21 03:36:19 smithi039 podman[73208]: 2024-05-21 03:36:19.957706787 +0000 UTC m=+30.278888345 container died a2441f307a2438b3cb162d79902068746c53c30f5028a37b96584fcd768d7118 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-1, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T03:36:20.583 INFO:journalctl@ceph.osd.1.smithi039.stdout:May 21 03:36:20 smithi039 podman[73208]: 2024-05-21 03:36:20.454629301 +0000 UTC m=+30.775810849 container cleanup a2441f307a2438b3cb162d79902068746c53c30f5028a37b96584fcd768d7118 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-1, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T03:36:20.583 INFO:journalctl@ceph.osd.1.smithi039.stdout:May 21 03:36:20 smithi039 bash[73208]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-1 2024-05-21T03:36:21.019 INFO:journalctl@ceph.osd.1.smithi039.stdout:May 21 03:36:20 smithi039 podman[73237]: 2024-05-21 03:36:20.682181257 +0000 UTC m=+0.726713622 container remove a2441f307a2438b3cb162d79902068746c53c30f5028a37b96584fcd768d7118 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-21T03:36:21.019 INFO:journalctl@ceph.osd.1.smithi039.stdout:May 21 03:36:20 smithi039 systemd[1]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.1.service: Main process exited, code=exited, status=137/n/a 2024-05-21T03:36:21.510 INFO:journalctl@ceph.osd.1.smithi039.stdout:May 21 03:36:21 smithi039 podman[73336]: 2024-05-21 03:36:20.975317735 +0000 UTC m=+0.021328022 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T03:36:21.510 INFO:journalctl@ceph.osd.1.smithi039.stdout:May 21 03:36:21 smithi039 podman[73336]: 2024-05-21 03:36:21.124926754 +0000 UTC m=+0.170937020 container create 6131e7f1ed5bc1f8c3e95439ac25b2a4197d333083df0794577a2d9f03973455 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_CLEAN=True) 2024-05-21T03:36:21.769 INFO:journalctl@ceph.osd.1.smithi039.stdout:May 21 03:36:21 smithi039 podman[73336]: 2024-05-21 03:36:21.511038754 +0000 UTC m=+0.557049023 container init 6131e7f1ed5bc1f8c3e95439ac25b2a4197d333083df0794577a2d9f03973455 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-1-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T03:36:21.769 INFO:journalctl@ceph.osd.1.smithi039.stdout:May 21 03:36:21 smithi039 podman[73336]: 2024-05-21 03:36:21.514860814 +0000 UTC m=+0.560871074 container start 6131e7f1ed5bc1f8c3e95439ac25b2a4197d333083df0794577a2d9f03973455 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T03:36:21.769 INFO:journalctl@ceph.osd.1.smithi039.stdout:May 21 03:36:21 smithi039 podman[73336]: 2024-05-21 03:36:21.593974437 +0000 UTC m=+0.639984728 container attach 6131e7f1ed5bc1f8c3e95439ac25b2a4197d333083df0794577a2d9f03973455 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-21T03:36:22.146 INFO:journalctl@ceph.osd.1.smithi039.stdout:May 21 03:36:21 smithi039 podman[73336]: 2024-05-21 03:36:21.849419313 +0000 UTC m=+0.895429573 container died 6131e7f1ed5bc1f8c3e95439ac25b2a4197d333083df0794577a2d9f03973455 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-21T03:36:22.452 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.1.service' 2024-05-21T03:36:22.491 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T03:36:22.491 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-21T03:36:22.492 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-21T03:36:22.492 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.2 2024-05-21T03:36:22.771 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 21 03:36:22 smithi165 systemd[1]: Stopping Ceph osd.2 for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:36:23.271 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 21 03:36:22 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-2[41409]: 2024-05-21T03:36:22.796+0000 7fc04d24c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T03:36:23.271 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 21 03:36:22 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-2[41409]: 2024-05-21T03:36:22.796+0000 7fc04d24c700 -1 osd.2 94 *** Got signal Terminated *** 2024-05-21T03:36:23.271 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 21 03:36:22 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-2[41409]: 2024-05-21T03:36:22.796+0000 7fc04d24c700 -1 osd.2 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T03:36:48.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:36:47 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:36:47.910+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:36:48.771 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:36:48 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:36:48.298+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:36:49.177 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:36:48 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:36:48.873+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:36:49.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:36:48 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:36:48.945+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:36:49.771 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:36:49 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:36:49.297+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:36:49.771 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:36:49 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:36:49.297+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:36:49.985 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:36:49 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:36:49.654+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:36:50.176 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:36:49 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:36:49.883+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:36:50.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:36:49 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:36:49.984+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:36:50.771 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:36:50 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:36:50.329+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:36:50.771 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:36:50 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:36:50.342+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:36:51.006 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:36:50 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:36:50.699+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:36:51.176 INFO:journalctl@ceph.osd.3.smithi027.stdout:May 21 03:36:50 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-3[51625]: 2024-05-21T03:36:50.885+0000 7f37db85a700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.754147+0000 front 2024-05-21T03:35:18.754132+0000 (oldest deadline 2024-05-21T03:36:50.357258+0000) 2024-05-21T03:36:51.177 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:36:50 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:36:50.889+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:36:51.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:36:51 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:36:51.006+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:36:51.676 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:36:51 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:36:51.308+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:36:51.771 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:36:51 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:36:51.319+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:36:51.771 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:36:51 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:36:51.295+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:36:52.019 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:36:51 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:36:51.960+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:36:52.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:36:51 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:36:51.745+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:36:52.176 INFO:journalctl@ceph.osd.3.smithi027.stdout:May 21 03:36:51 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-3[51625]: 2024-05-21T03:36:51.903+0000 7f37db85a700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.754147+0000 front 2024-05-21T03:35:18.754132+0000 (oldest deadline 2024-05-21T03:36:50.357258+0000) 2024-05-21T03:36:52.177 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:36:51 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:36:51.918+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:36:52.672 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:36:52 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:36:52.313+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:36:52.672 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:36:52 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:36:52.307+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:36:52.676 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:36:52 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:36:52.346+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:36:52.952 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 21 03:36:52 smithi165 bash[73688]: time="2024-05-21T03:36:52Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-2 in 30 seconds, resorting to SIGKILL" 2024-05-21T03:36:53.018 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:36:52 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:36:52.990+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:36:53.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:36:52 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:36:52.753+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:36:53.176 INFO:journalctl@ceph.osd.3.smithi027.stdout:May 21 03:36:52 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-3[51625]: 2024-05-21T03:36:52.856+0000 7f37db85a700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.754147+0000 front 2024-05-21T03:35:18.754132+0000 (oldest deadline 2024-05-21T03:36:50.357258+0000) 2024-05-21T03:36:53.177 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:36:52 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:36:52.911+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:36:53.268 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 21 03:36:52 smithi165 podman[73688]: 2024-05-21 03:36:52.953232109 +0000 UTC m=+30.273279618 container died 39b841c0483430e3a1bdf21d80a89c0c836c16d422e0f3dcce403aeb1b2331ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-2, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-21T03:36:53.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:36:53 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:36:53.268+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:36:53.522 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:36:53 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:36:53.275+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:36:53.522 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 21 03:36:53 smithi165 podman[73688]: 2024-05-21 03:36:53.486289462 +0000 UTC m=+30.806336962 container cleanup 39b841c0483430e3a1bdf21d80a89c0c836c16d422e0f3dcce403aeb1b2331ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-2, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T03:36:53.522 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 21 03:36:53 smithi165 bash[73688]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-2 2024-05-21T03:36:53.676 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:36:53 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:36:53.331+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:36:54.021 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 21 03:36:53 smithi165 podman[73715]: 2024-05-21 03:36:53.679667922 +0000 UTC m=+0.728804396 container remove 39b841c0483430e3a1bdf21d80a89c0c836c16d422e0f3dcce403aeb1b2331ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514) 2024-05-21T03:36:54.021 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 21 03:36:53 smithi165 systemd[1]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.2.service: Main process exited, code=exited, status=137/n/a 2024-05-21T03:36:54.176 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:36:53 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:36:53.958+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:36:54.177 INFO:journalctl@ceph.osd.3.smithi027.stdout:May 21 03:36:53 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-3[51625]: 2024-05-21T03:36:53.891+0000 7f37db85a700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.754147+0000 front 2024-05-21T03:35:18.754132+0000 (oldest deadline 2024-05-21T03:36:50.357258+0000) 2024-05-21T03:36:54.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:36:53 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:36:53.782+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:36:54.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:36:53 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:36:53.973+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:36:54.383 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 21 03:36:54 smithi165 podman[73802]: 2024-05-21 03:36:54.008139762 +0000 UTC m=+0.018348741 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T03:36:54.384 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 21 03:36:54 smithi165 podman[73802]: 2024-05-21 03:36:54.122347941 +0000 UTC m=+0.132556927 container create 38dd77ba61cea41f4e29fc6d43eb3e0af596db188cd87bc7d7833b2e87e247a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea) 2024-05-21T03:36:54.384 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 21 03:36:54 smithi165 podman[73802]: 2024-05-21 03:36:54.384365534 +0000 UTC m=+0.394574504 container init 38dd77ba61cea41f4e29fc6d43eb3e0af596db188cd87bc7d7833b2e87e247a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T03:36:54.384 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:36:54 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:36:54.301+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:36:54.384 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:36:54 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:36:54.283+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:36:54.676 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:36:54 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:36:54.360+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:36:54.696 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 21 03:36:54 smithi165 podman[73802]: 2024-05-21 03:36:54.388168468 +0000 UTC m=+0.398377436 container start 38dd77ba61cea41f4e29fc6d43eb3e0af596db188cd87bc7d7833b2e87e247a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T03:36:54.696 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 21 03:36:54 smithi165 podman[73802]: 2024-05-21 03:36:54.467025326 +0000 UTC m=+0.477234299 container attach 38dd77ba61cea41f4e29fc6d43eb3e0af596db188cd87bc7d7833b2e87e247a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514) 2024-05-21T03:36:55.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:36:54 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:36:54.734+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:36:55.019 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:36:54 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:36:54.980+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:36:55.021 INFO:journalctl@ceph.osd.2.smithi165.stdout:May 21 03:36:54 smithi165 podman[73802]: 2024-05-21 03:36:54.697901759 +0000 UTC m=+0.708110728 container died 38dd77ba61cea41f4e29fc6d43eb3e0af596db188cd87bc7d7833b2e87e247a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-21T03:36:55.176 INFO:journalctl@ceph.osd.3.smithi027.stdout:May 21 03:36:54 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-3[51625]: 2024-05-21T03:36:54.892+0000 7f37db85a700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.754147+0000 front 2024-05-21T03:35:18.754132+0000 (oldest deadline 2024-05-21T03:36:50.357258+0000) 2024-05-21T03:36:55.177 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:36:54 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:36:54.911+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:36:55.323 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.2.service' 2024-05-21T03:36:55.362 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T03:36:55.362 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-21T03:36:55.362 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-21T03:36:55.363 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.3 2024-05-21T03:36:55.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:36:55 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:36:55.247+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:36:55.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:36:55 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:36:55.284+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:36:55.673 INFO:journalctl@ceph.osd.3.smithi027.stdout:May 21 03:36:55 smithi027 systemd[1]: Stopping Ceph osd.3 for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:36:55.673 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:36:55 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:36:55.374+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:36:55.926 INFO:journalctl@ceph.osd.3.smithi027.stdout:May 21 03:36:55 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-3[51625]: 2024-05-21T03:36:55.674+0000 7f37df266700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T03:36:55.927 INFO:journalctl@ceph.osd.3.smithi027.stdout:May 21 03:36:55 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-3[51625]: 2024-05-21T03:36:55.674+0000 7f37df266700 -1 osd.3 94 *** Got signal Terminated *** 2024-05-21T03:36:55.927 INFO:journalctl@ceph.osd.3.smithi027.stdout:May 21 03:36:55 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-3[51625]: 2024-05-21T03:36:55.674+0000 7f37df266700 -1 osd.3 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T03:36:55.927 INFO:journalctl@ceph.osd.3.smithi027.stdout:May 21 03:36:55 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-3[51625]: 2024-05-21T03:36:55.890+0000 7f37db85a700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.754147+0000 front 2024-05-21T03:35:18.754132+0000 (oldest deadline 2024-05-21T03:36:50.357258+0000) 2024-05-21T03:36:55.976 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:36:55 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:36:55.724+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:36:56.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:36:55 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:36:55.976+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:36:56.407 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:36:55 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:36:55.946+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:36:56.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:36:56 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:36:56.260+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:36:56.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:36:56 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:36:56.327+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:36:56.676 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:36:56 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:36:56.408+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:36:57.019 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:36:56 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:36:56.964+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:36:57.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:36:56 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:36:56.747+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:36:57.177 INFO:journalctl@ceph.osd.3.smithi027.stdout:May 21 03:36:56 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-3[51625]: 2024-05-21T03:36:56.909+0000 7f37db85a700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.754147+0000 front 2024-05-21T03:35:18.754132+0000 (oldest deadline 2024-05-21T03:36:50.357258+0000) 2024-05-21T03:36:57.177 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:36:56 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:36:56.961+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:36:57.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:36:57 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:36:57.217+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:36:57.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:36:57 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:36:57.333+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:36:57.884 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:36:57 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:36:57.450+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:36:58.019 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:36:58 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:36:58.007+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:36:58.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:36:57 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:36:57.761+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:36:58.176 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:36:57 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:36:57.974+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:36:58.177 INFO:journalctl@ceph.osd.3.smithi027.stdout:May 21 03:36:57 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-3[51625]: 2024-05-21T03:36:57.884+0000 7f37db85a700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.754147+0000 front 2024-05-21T03:35:18.754132+0000 (oldest deadline 2024-05-21T03:36:50.357258+0000) 2024-05-21T03:36:58.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:36:58 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:36:58.304+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:36:58.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:36:58 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:36:58.255+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:36:58.855 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:36:58 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:36:58.488+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:36:59.042 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:36:58 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:36:58.785+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:36:59.176 INFO:journalctl@ceph.osd.3.smithi027.stdout:May 21 03:36:58 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-3[51625]: 2024-05-21T03:36:58.856+0000 7f37db85a700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.754147+0000 front 2024-05-21T03:35:18.754132+0000 (oldest deadline 2024-05-21T03:36:50.357258+0000) 2024-05-21T03:36:59.177 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:36:58 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:36:58.994+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:36:59.519 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:36:59 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:36:59.042+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:36:59.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:36:59 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:36:59.323+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:36:59.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:36:59 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:36:59.224+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:36:59.813 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:36:59 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:36:59.458+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:00.176 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:36:59 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:36:59.959+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:00.177 INFO:journalctl@ceph.osd.3.smithi027.stdout:May 21 03:36:59 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-3[51625]: 2024-05-21T03:36:59.814+0000 7f37db85a700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.754147+0000 front 2024-05-21T03:35:18.754132+0000 (oldest deadline 2024-05-21T03:36:50.357258+0000) 2024-05-21T03:37:00.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:00 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:00.021+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:00.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:36:59 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:36:59.821+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:00.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:00 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:00.346+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:00.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:00 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:00.196+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:00.671 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:00 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:00.410+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:01.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:01 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:01.007+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:01.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:00 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:00.854+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:01.273 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:00 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:00.998+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:01.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:01 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:01.343+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:01.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:01 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:01.210+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:01.676 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:01 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:01.426+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:02.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:02 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:02.028+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:02.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:01 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:01.904+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:02.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:02 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:02.001+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:02.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:02 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:02.373+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:02.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:02 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:02.176+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:02.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:02 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:02.435+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:03.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:03 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:03.043+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:03.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:02 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:02.935+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:03.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:02 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:02.990+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:03.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:03 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:03.195+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:03.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:03 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:03.363+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:03.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:03 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:03.467+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:04.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:04 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:04.046+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:04.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:03 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:03.932+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:04.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:04 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:04.026+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:04.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:04 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:04.173+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:04.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:04 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:04.320+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:04.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:04 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:04.494+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:05.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:05 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:05.066+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:05.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:04 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:04.956+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:05.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:05 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:05.056+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:05.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:05 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:05.176+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:05.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:05 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:05.366+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:05.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:05 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:05.530+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:06.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:06 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:06.037+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:06.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:05 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:05.983+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:06.385 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:06 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:06.210+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:06.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:06 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:06.040+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:06.771 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:06 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:06.385+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:06.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:06 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:06.516+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:07.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:07 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:07.034+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:07.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:07 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:07.006+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:07.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:06 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:06.998+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:07.522 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:07 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:07.402+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:07.522 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:07 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:07.217+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:07.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:07 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:07.563+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:08.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:08 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:08.060+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:08.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:07 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:07.984+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:08.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:08 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:08.034+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:08.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:08 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:08.441+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:08.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:08 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:08.254+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:08.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:08 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:08.570+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:09.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:09 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:09.108+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:09.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:08 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:08.976+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:09.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:09 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:09.048+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:09.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:09 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:09.432+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:09.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:09 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:09.221+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:09.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:09 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:09.561+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:10.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:10 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:10.083+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:10.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:09 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:09.943+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:10.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:10 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:10.069+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:10.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:10 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:10.466+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:10.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:10 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:10.259+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:10.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:10 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:10.577+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:11.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:11 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:11.107+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:11.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:10 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:10.913+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:11.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:11 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:11.066+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:11.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:11 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:11.252+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:11.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:11 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:11.447+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:11.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:11 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:11.590+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:12.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:11 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:11.869+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:12.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:12 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:12.061+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:12.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:12 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:12.103+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:12.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:12 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:12.429+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:12.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:12 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:12.236+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:12.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:12 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:12.572+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:13.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:13 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:13.013+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:13.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:12 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:12.874+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:13.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:13 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:13.079+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:13.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:13 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:13.458+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:13.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:13 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:13.267+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:13.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:13 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:13.548+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:14.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:13 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:13.971+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:14.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:13 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:13.869+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:14.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:14 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:14.057+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:14.771 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:14 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:14.278+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:14.771 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:14 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:14.446+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:14.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:14 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:14.531+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:15.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:15 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:15.017+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:15.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:14 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:14.873+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:15.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:15 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:15.089+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:15.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:15 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:15.467+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:15.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:15 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:15.247+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:15.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:15 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:15.494+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:16.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:16 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:16.010+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:16.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:15 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:15.911+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:16.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:16 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:16.076+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:16.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:16 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:16.452+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:16.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:16 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:16.265+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:16.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:16 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:16.485+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:17.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:16 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:16.964+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:17.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:16 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:16.865+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:17.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:17 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:17.112+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:17.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:17 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:17.427+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:17.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:17 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:17.262+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:17.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:17 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:17.453+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:18.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:17 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:17.852+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:18.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:17 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:17.966+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:18.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:18 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:18.113+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:18.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:18 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:18.457+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:18.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:18 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:18.254+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:18.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:18 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:18.491+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:19.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:18 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:18.885+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:19.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:18 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:18.980+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:19.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:19 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:19.091+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:19.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:19 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:19.268+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:19.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:19 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:19.492+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:19.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:19 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:19.513+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:20.170 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:19 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:19.931+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:20.170 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:19 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:19.908+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:20.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:20 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:20.092+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:20.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:20 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:20.219+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:20.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:20 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:20.456+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:20.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:20 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:20.456+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.219819+0000 front 2024-05-21T03:35:51.219960+0000 (oldest deadline 2024-05-21T03:37:20.323244+0000) 2024-05-21T03:37:20.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:20 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:20.520+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:21.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:20 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:20.901+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:21.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:20 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:20.941+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:21.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:21 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:21.113+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:21.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:21 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:21.113+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:21.463 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:21 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:21.209+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:21.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:21 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:21.483+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:21.771 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:21 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:21.462+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:21.771 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:21 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:21.462+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.219819+0000 front 2024-05-21T03:35:51.219960+0000 (oldest deadline 2024-05-21T03:37:20.323244+0000) 2024-05-21T03:37:21.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:21 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:21.525+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:21.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:21 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:21.525+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:22.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:21 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:21.931+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:22.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:21 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:21.925+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:22.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:22 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:22.126+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:22.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:22 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:22.126+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:22.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:22 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:22.200+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:22.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:22 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:22.438+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:22.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:22 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:22.438+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.219819+0000 front 2024-05-21T03:35:51.219960+0000 (oldest deadline 2024-05-21T03:37:20.323244+0000) 2024-05-21T03:37:22.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:22 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:22.473+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:22.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:22 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:22.516+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:22.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:22 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:22.516+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:23.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:22 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:22.905+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:23.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:22 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:22.879+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:23.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:23 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:23.120+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:23.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:23 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:23.120+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:23.477 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:23 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:23.278+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:23.477 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:23 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:23.153+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:23.477 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:23 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:23.153+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:23.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:23 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:23.468+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:23.771 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:23 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:23.476+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:23.771 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:23 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:23.476+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.219819+0000 front 2024-05-21T03:35:51.219960+0000 (oldest deadline 2024-05-21T03:37:20.323244+0000) 2024-05-21T03:37:23.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:23 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:23.485+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:23.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:23 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:23.485+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:24.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:23 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:23.931+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:24.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:23 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:23.856+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:24.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:24 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:24.091+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:24.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:24 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:24.091+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:24.483 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:24 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:24.243+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:24.484 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:24 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:24.144+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:24.484 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:24 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:24.144+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:24.768 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:24 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:24.423+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:24.771 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:24 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:24.483+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:24.771 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:24 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:24.483+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.219819+0000 front 2024-05-21T03:35:51.219960+0000 (oldest deadline 2024-05-21T03:37:20.323244+0000) 2024-05-21T03:37:24.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:24 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:24.495+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:24.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:24 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:24.495+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:25.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:24 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:24.924+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:25.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:24 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:24.850+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:25.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:24 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:24.850+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:25.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:25 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:25.063+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:25.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:25 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:25.063+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:25.511 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:25 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:25.260+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:25.511 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:25 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:25.107+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:25.511 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:25 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:25.107+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:25.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:25 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:25.396+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:25.771 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:25 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:25.510+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:25.771 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:25 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:25.510+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.219819+0000 front 2024-05-21T03:35:51.219960+0000 (oldest deadline 2024-05-21T03:37:20.323244+0000) 2024-05-21T03:37:25.820 INFO:journalctl@ceph.osd.3.smithi027.stdout:May 21 03:37:25 smithi027 bash[105020]: time="2024-05-21T03:37:25Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-3 in 30 seconds, resorting to SIGKILL" 2024-05-21T03:37:25.820 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:25 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:25.459+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:25.820 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:25 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:25.459+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:26.082 INFO:journalctl@ceph.osd.3.smithi027.stdout:May 21 03:37:25 smithi027 podman[105020]: 2024-05-21 03:37:25.821354975 +0000 UTC m=+30.253855301 container died d7ffb6f2a711106a9deee9ff0df0056aadd761f2b1e46505ce9d309aa1a2ab9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-21T03:37:26.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:25 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:25.951+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:26.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:25 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:25.894+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:26.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:25 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:25.894+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:26.379 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:26 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:26.081+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:26.379 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:26 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:26.081+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:26.521 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:26 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:26.293+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:26.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:26 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:26.116+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:26.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:26 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:26.116+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:26.665 INFO:journalctl@ceph.osd.3.smithi027.stdout:May 21 03:37:26 smithi027 podman[105020]: 2024-05-21 03:37:26.380722652 +0000 UTC m=+30.813222968 container cleanup d7ffb6f2a711106a9deee9ff0df0056aadd761f2b1e46505ce9d309aa1a2ab9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514) 2024-05-21T03:37:26.665 INFO:journalctl@ceph.osd.3.smithi027.stdout:May 21 03:37:26 smithi027 bash[105020]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-3 2024-05-21T03:37:26.665 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:26 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:26.442+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:26.665 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:26 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:26.442+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:26.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:26 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:26.433+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:26.926 INFO:journalctl@ceph.osd.3.smithi027.stdout:May 21 03:37:26 smithi027 podman[105036]: 2024-05-21 03:37:26.666054502 +0000 UTC m=+0.847604262 container remove d7ffb6f2a711106a9deee9ff0df0056aadd761f2b1e46505ce9d309aa1a2ab9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-3, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea) 2024-05-21T03:37:26.927 INFO:journalctl@ceph.osd.3.smithi027.stdout:May 21 03:37:26 smithi027 systemd[1]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.3.service: Main process exited, code=exited, status=137/n/a 2024-05-21T03:37:27.021 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:26 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:26.544+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:27.022 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:26 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:26.544+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.219819+0000 front 2024-05-21T03:35:51.219960+0000 (oldest deadline 2024-05-21T03:37:20.323244+0000) 2024-05-21T03:37:27.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:26 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:26.917+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:27.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:26 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:26.891+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:27.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:26 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:26.891+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:27.285 INFO:journalctl@ceph.osd.3.smithi027.stdout:May 21 03:37:26 smithi027 podman[105121]: 2024-05-21 03:37:26.902466916 +0000 UTC m=+0.018296120 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T03:37:27.285 INFO:journalctl@ceph.osd.3.smithi027.stdout:May 21 03:37:27 smithi027 podman[105121]: 2024-05-21 03:37:27.033574447 +0000 UTC m=+0.149403636 container create 5041dfb48b0db621449b14c1c5a6743526e9b916406f3c6893a4484218a7daf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-21T03:37:27.285 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:27 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:27.110+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:27.285 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:27 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:27.110+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:27.521 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:27 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:27.296+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:27.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:27 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:27.146+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:27.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:27 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:27.146+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:27.603 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:27 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:27.486+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:27.603 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:27 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:27.486+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:27.603 INFO:journalctl@ceph.osd.3.smithi027.stdout:May 21 03:37:27 smithi027 podman[105121]: 2024-05-21 03:37:27.286202439 +0000 UTC m=+0.402031641 container init 5041dfb48b0db621449b14c1c5a6743526e9b916406f3c6893a4484218a7daf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T03:37:27.603 INFO:journalctl@ceph.osd.3.smithi027.stdout:May 21 03:37:27 smithi027 podman[105121]: 2024-05-21 03:37:27.289925037 +0000 UTC m=+0.405754226 container start 5041dfb48b0db621449b14c1c5a6743526e9b916406f3c6893a4484218a7daf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T03:37:27.603 INFO:journalctl@ceph.osd.3.smithi027.stdout:May 21 03:37:27 smithi027 podman[105121]: 2024-05-21 03:37:27.36932802 +0000 UTC m=+0.485157221 container attach 5041dfb48b0db621449b14c1c5a6743526e9b916406f3c6893a4484218a7daf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T03:37:27.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:27 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:27.478+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:27.927 INFO:journalctl@ceph.osd.3.smithi027.stdout:May 21 03:37:27 smithi027 podman[105121]: 2024-05-21 03:37:27.603952572 +0000 UTC m=+0.719781780 container died 5041dfb48b0db621449b14c1c5a6743526e9b916406f3c6893a4484218a7daf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-21T03:37:28.021 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:27 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:27.567+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:28.021 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:27 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:27.567+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.219819+0000 front 2024-05-21T03:35:51.219960+0000 (oldest deadline 2024-05-21T03:37:20.323244+0000) 2024-05-21T03:37:28.198 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.3.service' 2024-05-21T03:37:28.238 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T03:37:28.239 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-21T03:37:28.239 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-21T03:37:28.239 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.4 2024-05-21T03:37:28.261 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:27 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:27.910+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:28.261 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:27 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:27.937+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:28.261 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:27 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:27.937+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:28.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:28 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:28.088+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:28.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:28 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:28.088+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:28.519 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:28 smithi039 systemd[1]: Stopping Ceph osd.4 for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:37:28.519 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:28 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:28.485+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:28.521 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:28 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:28.268+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:28.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:28 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:28.192+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:28.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:28 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:28.192+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:28.895 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:28 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:28.521+0000 7fa0902c4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T03:37:28.895 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:28 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:28.521+0000 7fa0902c4700 -1 osd.4 94 *** Got signal Terminated *** 2024-05-21T03:37:28.895 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:28 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:28.521+0000 7fa0902c4700 -1 osd.4 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T03:37:28.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:28 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:28.527+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:28.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:28 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:28.527+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:29.021 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:28 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:28.559+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:29.021 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:28 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:28.559+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.219819+0000 front 2024-05-21T03:35:51.219960+0000 (oldest deadline 2024-05-21T03:37:20.323244+0000) 2024-05-21T03:37:29.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:28 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:28.926+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:29.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:28 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:28.895+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:29.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:28 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:28.895+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:29.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:29 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:29.042+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:29.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:29 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:29.042+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:29.521 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:29 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:29.252+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:29.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:29 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:29.232+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:29.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:29 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:29.232+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:29.886 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:29 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:29.520+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:29.886 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:29 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:29.885+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:29.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:29 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:29.499+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:29.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:29 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:29.499+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:30.021 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:29 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:29.523+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:30.021 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:29 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:29.523+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.219819+0000 front 2024-05-21T03:35:51.219960+0000 (oldest deadline 2024-05-21T03:37:20.323244+0000) 2024-05-21T03:37:30.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:29 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:29.910+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:30.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:29 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:29.910+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:30.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:30 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:30.078+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:30.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:30 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:30.078+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:30.521 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:30 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:30.258+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:30.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:30 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:30.489+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:30.522 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:30 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:30.489+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.219819+0000 front 2024-05-21T03:35:51.219960+0000 (oldest deadline 2024-05-21T03:37:20.323244+0000) 2024-05-21T03:37:30.522 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:30 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:30.245+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:30.522 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:30 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:30.245+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:30.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:30 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:30.483+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:30.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:30 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:30.463+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:30.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:30 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:30.464+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:31.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:30 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:30.858+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:31.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:30 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:30.870+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:31.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:30 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:30.870+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:31.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:31 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:31.047+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:31.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:31 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:31.047+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:31.521 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:31 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:31.244+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:31.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:31 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:31.260+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:31.522 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:31 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:31.260+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:31.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:31 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:31.449+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:31.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:31 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:31.487+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:31.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:31 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:31.487+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:32.021 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:31 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:31.520+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:32.021 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:31 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:31.520+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.219819+0000 front 2024-05-21T03:35:51.219960+0000 (oldest deadline 2024-05-21T03:37:20.323244+0000) 2024-05-21T03:37:32.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:31 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:31.818+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:32.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:31 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:31.905+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:32.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:31 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:31.905+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:32.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:32 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:32.012+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:32.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:32 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:32.012+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:32.521 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:32 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:32.219+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:32.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:32 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:32.275+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:32.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:32 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:32.275+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:32.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:32 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:32.421+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:32.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:32 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:32.462+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:32.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:32 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:32.462+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:33.021 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:32 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:32.558+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:33.021 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:32 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:32.558+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.219819+0000 front 2024-05-21T03:35:51.219960+0000 (oldest deadline 2024-05-21T03:37:20.323244+0000) 2024-05-21T03:37:33.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:32 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4[47921]: 2024-05-21T03:37:32.856+0000 7fa0888b0700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:17.455411+0000 front 2024-05-21T03:35:17.455294+0000 (oldest deadline 2024-05-21T03:36:47.059611+0000) 2024-05-21T03:37:33.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:32 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:32.933+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:33.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:32 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:32.933+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:33.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:33 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:33.013+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:33.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:33 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:33.013+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:33.521 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:33 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:33.225+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:33.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:33 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:33.261+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:33.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:33 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:33.261+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:33.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:33 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:33.425+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:33.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:33 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:33.469+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:33.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:33 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:33.469+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:34.021 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:33 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:33.531+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:34.021 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:33 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:33.531+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.219819+0000 front 2024-05-21T03:35:51.219960+0000 (oldest deadline 2024-05-21T03:37:20.323244+0000) 2024-05-21T03:37:34.269 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:34 smithi039 podman[73445]: 2024-05-21 03:37:34.225734145 +0000 UTC m=+5.797225360 container died 5bf104643617e73e2134d243bb1dc0857ce89d45d95933162a76c6c9e8b2ff31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, org.label-schema.build-date=20240514) 2024-05-21T03:37:34.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:33 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:33.976+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:34.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:33 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:33.976+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:34.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:34 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:34.045+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:34.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:34 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:34.045+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:34.514 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:34 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:34.251+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:34.514 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:34 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:34.251+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:34.515 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:34 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:34.208+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:34.704 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:34 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:34.413+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:34.771 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:34 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:34.513+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:34.771 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:34 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:34.513+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.219819+0000 front 2024-05-21T03:35:51.219960+0000 (oldest deadline 2024-05-21T03:37:20.323244+0000) 2024-05-21T03:37:34.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:34 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:34.508+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:34.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:34 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:34.508+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:34.967 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:34 smithi039 podman[73445]: 2024-05-21 03:37:34.705706467 +0000 UTC m=+6.277197681 container cleanup 5bf104643617e73e2134d243bb1dc0857ce89d45d95933162a76c6c9e8b2ff31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.license=GPLv2) 2024-05-21T03:37:34.967 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:34 smithi039 bash[73445]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4 2024-05-21T03:37:34.967 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:34 smithi039 podman[73461]: 2024-05-21 03:37:34.948464149 +0000 UTC m=+0.723183937 container remove 5bf104643617e73e2134d243bb1dc0857ce89d45d95933162a76c6c9e8b2ff31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-21T03:37:35.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:34 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:34.966+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:35.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:34 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:34.966+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:35.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:35 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:35.074+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:35.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:35 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:35.074+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:35.513 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:35 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:35.225+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:35.513 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:35 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:35.228+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:35.513 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:35 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:35.228+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:35.567 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:35 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:35.371+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:35.567 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:35 smithi039 podman[73554]: 2024-05-21 03:37:35.174821828 +0000 UTC m=+0.018112545 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T03:37:35.567 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:35 smithi039 podman[73554]: 2024-05-21 03:37:35.274323848 +0000 UTC m=+0.117614556 container create d4a50317f2cd0b247cadf3928e2b97b87ed5d418cc1a69f490ee05b1bda4f2ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T03:37:35.568 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:35 smithi039 podman[73554]: 2024-05-21 03:37:35.484942139 +0000 UTC m=+0.328232857 container init d4a50317f2cd0b247cadf3928e2b97b87ed5d418cc1a69f490ee05b1bda4f2ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, ceph=True) 2024-05-21T03:37:35.568 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:35 smithi039 podman[73554]: 2024-05-21 03:37:35.488143211 +0000 UTC m=+0.331433928 container start d4a50317f2cd0b247cadf3928e2b97b87ed5d418cc1a69f490ee05b1bda4f2ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-21T03:37:35.771 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:35 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:35.513+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:35.771 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:35 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:35.513+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.219819+0000 front 2024-05-21T03:35:51.219960+0000 (oldest deadline 2024-05-21T03:37:20.323244+0000) 2024-05-21T03:37:35.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:35 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:35.525+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:35.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:35 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:35.525+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:35.929 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:35 smithi039 podman[73554]: 2024-05-21 03:37:35.568243006 +0000 UTC m=+0.411533733 container attach d4a50317f2cd0b247cadf3928e2b97b87ed5d418cc1a69f490ee05b1bda4f2ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-21T03:37:35.929 INFO:journalctl@ceph.osd.4.smithi039.stdout:May 21 03:37:35 smithi039 podman[73554]: 2024-05-21 03:37:35.798881988 +0000 UTC m=+0.642172712 container died d4a50317f2cd0b247cadf3928e2b97b87ed5d418cc1a69f490ee05b1bda4f2ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-4-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514) 2024-05-21T03:37:36.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:35 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:35.928+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:36.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:35 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:35.928+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:36.383 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.4.service' 2024-05-21T03:37:36.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:36 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:36.057+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:36.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:36 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:36.057+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:36.464 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T03:37:36.465 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-21T03:37:36.465 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-21T03:37:36.465 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.5 2024-05-21T03:37:36.487 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:36 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:36.200+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:36.487 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:36 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:36.190+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:36.487 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:36 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:36.190+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:36.651 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:36 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:36.370+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:36.771 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:36 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:36.502+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:36.771 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:36 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:36.502+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.219819+0000 front 2024-05-21T03:35:51.219960+0000 (oldest deadline 2024-05-21T03:37:20.323244+0000) 2024-05-21T03:37:36.771 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:36 smithi165 systemd[1]: Stopping Ceph osd.5 for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:37:36.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:36 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:36.564+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:36.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:36 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:36.565+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:37.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:36 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:36.895+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:37.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:36 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:36.895+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:37.171 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:36 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:36.785+0000 7f49bc79e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T03:37:37.172 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:36 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:36.785+0000 7f49bc79e700 -1 osd.5 94 *** Got signal Terminated *** 2024-05-21T03:37:37.172 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:36 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:36.785+0000 7f49bc79e700 -1 osd.5 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T03:37:37.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:37 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:37.073+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:37.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:37 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:37.073+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:37.483 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:37 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:37.192+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:37.483 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:37 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:37.172+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:37.483 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:37 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:37.172+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:37.626 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:37 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:37.363+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:37.771 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:37 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:37.483+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:37.771 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:37 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:37.483+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.219819+0000 front 2024-05-21T03:35:51.219960+0000 (oldest deadline 2024-05-21T03:37:20.323244+0000) 2024-05-21T03:37:37.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:37 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:37.573+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:37.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:37 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:37.573+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:38.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:37 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:37.881+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:38.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:37 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:37.881+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:38.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:38 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:38.105+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:38.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:38 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:38.105+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:38.488 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:38 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:38.220+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:38.488 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:38 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:38.220+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:38.488 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:38 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:38.220+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:38.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:38 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:38.397+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:38.771 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:38 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:38.488+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:38.771 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:38 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:38.488+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.219819+0000 front 2024-05-21T03:35:51.219960+0000 (oldest deadline 2024-05-21T03:37:20.323244+0000) 2024-05-21T03:37:38.920 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:38 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:38.525+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:38.920 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:38 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:38.525+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:39.176 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:39 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:39.143+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:39.177 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:39 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:39.143+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:39.268 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:38 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:38.897+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:39.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:38 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:38.897+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:39.521 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:39 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:39.201+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:39.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:39 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:39.190+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:39.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:39 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:39.190+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:39.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:39 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:39.395+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:39.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:39 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:39.498+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:39.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:39 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:39.498+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:40.021 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:39 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:39.531+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:40.021 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:39 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:39.531+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.219819+0000 front 2024-05-21T03:35:51.219960+0000 (oldest deadline 2024-05-21T03:37:20.323244+0000) 2024-05-21T03:37:40.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:39 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:39.891+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:40.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:39 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:39.891+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:40.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:40 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:40.104+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:40.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:40 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:40.104+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:40.521 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:40 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:40.221+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:40.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:40 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:40.159+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:40.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:40 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:40.159+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:40.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:40 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:40.404+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:40.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:40 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:40.512+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:40.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:40 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:40.512+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:41.021 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:40 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:40.535+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:41.021 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:40 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:40.535+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.219819+0000 front 2024-05-21T03:35:51.219960+0000 (oldest deadline 2024-05-21T03:37:20.323244+0000) 2024-05-21T03:37:41.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:40 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:40.928+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:41.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:40 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:40.928+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:41.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:41 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:41.060+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:41.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:41 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:41.060+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:41.511 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:41 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:41.174+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:41.511 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:41 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:41.110+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:41.511 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:41 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:41.110+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:41.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:41 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:41.394+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:41.771 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:41 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:41.511+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.416421+0000 front 2024-05-21T03:35:18.416456+0000 (oldest deadline 2024-05-21T03:36:47.718732+0000) 2024-05-21T03:37:41.771 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:37:41 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5[47903]: 2024-05-21T03:37:41.511+0000 7f49b4d8a700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.219819+0000 front 2024-05-21T03:35:51.219960+0000 (oldest deadline 2024-05-21T03:37:20.323244+0000) 2024-05-21T03:37:41.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:41 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:41.513+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:41.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:41 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:41.513+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:42.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:41 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:41.933+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:42.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:41 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:41.933+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:42.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:42 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:42.043+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:42.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:42 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:42.043+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:42.521 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:42 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:42.221+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:42.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:42 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:42.105+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:42.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:42 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:42.105+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:42.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:42 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:42.370+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:42.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:42 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:42.499+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:42.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:42 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:42.499+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:43.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:42 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:42.894+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:43.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:42 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:42.894+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:43.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:43 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:43.051+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:43.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:43 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:43.051+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:43.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:43 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:43.062+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:43.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:43 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:43.062+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:43.521 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:43 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:43.228+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:43.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:43 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:43.407+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:43.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:43 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:43.454+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:43.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:43 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:43.454+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:44.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:43 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:43.915+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:44.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:43 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:43.915+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:44.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:44 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:44.076+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:44.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:44 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:44.076+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:44.521 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:44 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:44.191+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:44.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:44 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:44.064+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:44.522 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:44 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:44.064+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:44.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:44 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:44.422+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:44.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:44 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:44.458+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:44.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:44 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:44.458+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:45.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:44 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:44.881+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:45.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:44 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:44.881+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:45.413 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:45 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:45.108+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:45.414 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:45 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:45.108+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:45.521 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:45 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:45.191+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:45.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:45 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:45.068+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:45.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:45 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:45.068+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:45.677 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:45 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:45.413+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:45.677 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:45 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:45.413+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:45.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:45 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:45.441+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:46.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:45 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:45.900+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:46.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:45 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:45.900+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:46.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:46 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:46.138+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:46.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:46 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:46.138+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:46.521 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:46 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:46.190+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:46.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:46 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:46.038+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:46.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:46 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:46.038+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:46.770 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:46 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:46.415+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:46.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:46 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:46.430+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:46.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:46 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:46.430+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:47.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:46 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:46.872+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:47.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:46 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:46.872+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:47.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:47 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:47.180+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:47.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:46 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:46.992+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:47.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:46 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:46.992+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:47.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:47 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:47.139+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:47.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:47 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:47.139+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:47.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:47 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:47.459+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:47.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:47 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:47.455+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:47.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:47 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:47.455+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:48.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:47 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:47.895+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:48.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:47 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:47.895+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:48.419 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:48 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:48.169+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:48.419 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:48 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:48.169+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:48.521 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:48 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:48.217+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:48.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:48 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:48.028+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:48.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:48 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:48.028+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:48.676 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:48 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:48.419+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:48.676 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:48 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:48.419+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:48.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:48 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:48.493+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:49.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:48 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:48.901+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:49.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:48 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:48.901+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:49.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:49 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:49.194+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:49.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:48 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:48.989+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:49.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:48 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:48.989+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:49.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:49 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:49.146+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:49.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:49 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:49.146+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:49.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:49 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:49.459+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:49.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:49 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:49.427+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:49.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:49 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:49.427+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:50.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:49 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:49.923+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:50.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:49 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:49.923+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:50.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:50 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:50.165+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:50.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:50 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:50.006+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:50.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:50 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:50.006+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:50.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:50 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:50.140+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:50.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:50 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:50.140+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:50.562 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:50 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:50.496+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:50.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:50 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:50.443+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:50.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:50 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:50.443+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:51.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:50 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:50.901+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:51.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:50 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:50.901+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:51.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:51 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:51.108+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:51.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:51 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:51.108+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:51.521 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:51 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:51.139+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:51.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:51 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:51.055+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:51.522 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:51 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:51.055+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:51.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:51 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:51.518+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:51.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:51 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:51.484+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:51.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:51 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:51.484+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:52.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:51 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:51.871+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:52.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:51 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:51.871+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:52.422 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:52 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:52.092+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:52.423 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:52 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:52.045+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:52.423 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:52 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:52.045+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:52.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:52 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:52.115+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:52.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:52 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:52.115+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:52.841 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:52 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:52.568+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:52.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:52 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:52.452+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:52.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:52 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:52.452+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:53.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:52 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:52.841+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:53.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:52 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:52.841+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:53.423 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:53 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:53.129+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:53.423 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:53 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:53.129+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:53.521 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:53 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:53.107+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:53.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:53 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:53.070+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:53.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:53 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:53.070+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:53.676 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:53 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:53.422+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:53.676 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:53 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:53.422+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:53.874 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:53 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:53.556+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:54.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:53 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:53.875+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:54.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:53 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:53.875+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:54.392 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:54 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:54.101+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:54.392 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:54 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:54.101+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:54.521 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:54 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:54.102+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:54.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:54 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:54.084+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:54.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:54 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:54.084+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:54.676 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:54 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:54.392+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:54.677 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:54 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:54.392+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:54.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:54 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:54.515+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:54.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:54 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:54.515+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.087560+0000 front 2024-05-21T03:36:27.087536+0000 (oldest deadline 2024-05-21T03:37:54.490886+0000) 2024-05-21T03:37:55.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:54 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:54.841+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:55.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:54 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:54.841+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:55.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:54 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:54.841+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:37:55.346 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:55 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:55.088+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:55.346 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:55 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:55.088+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:55.346 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:55 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:55.088+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:37:55.521 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:55 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:55.104+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:55.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:55 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:55.099+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:55.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:55 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:55.099+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:55.676 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:55 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:55.346+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:55.676 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:55 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:55.346+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:55.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:55 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:55.491+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:55.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:55 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:55.491+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.087560+0000 front 2024-05-21T03:36:27.087536+0000 (oldest deadline 2024-05-21T03:37:54.490886+0000) 2024-05-21T03:37:56.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:55 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:55.874+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:56.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:55 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:55.874+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:56.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:55 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:55.874+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:37:56.331 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:56 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:56.078+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:56.331 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:56 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:56.078+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:56.331 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:56 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:56.078+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:37:56.521 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:56 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:56.057+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:56.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:56 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:56.077+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:56.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:56 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:56.077+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:56.676 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:56 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:56.331+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:56.677 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:56 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:56.331+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:56.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:56 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:56.451+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:56.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:56 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:56.451+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.087560+0000 front 2024-05-21T03:36:27.087536+0000 (oldest deadline 2024-05-21T03:37:54.490886+0000) 2024-05-21T03:37:57.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:56 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:56.846+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:57.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:56 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:56.846+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:57.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:56 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:56.846+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:37:57.358 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:57 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:57.104+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:57.358 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:57 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:57.104+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:57.358 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:57 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:57.104+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:37:57.521 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:57 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:57.073+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:57.521 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:57 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:57.073+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:37:57.522 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:57 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:57.078+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:57.522 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:57 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:57.078+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:57.522 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:57 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:57.078+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:37:57.676 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:57 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:57.357+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:57.677 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:57 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:57.357+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:57.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:57 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:57.453+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:57.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:57 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:57.453+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.087560+0000 front 2024-05-21T03:36:27.087536+0000 (oldest deadline 2024-05-21T03:37:54.490886+0000) 2024-05-21T03:37:58.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:57 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:57.803+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:58.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:57 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:57.803+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:58.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:57 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:57.803+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:37:58.385 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:58 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:58.117+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:58.385 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:58 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:58.117+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:58.386 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:58 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:58.117+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:37:58.521 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:58 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:58.057+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:58.521 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:58 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:58.057+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:37:58.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:58 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:58.065+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:58.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:58 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:58.065+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:58.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:58 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:58.065+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:37:58.676 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:58 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:58.385+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:58.676 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:58 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:58.385+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:58.677 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:58 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:58.385+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.625870+0000 front 2024-05-21T03:36:27.625797+0000 (oldest deadline 2024-05-21T03:37:57.929376+0000) 2024-05-21T03:37:58.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:58 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:58.419+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:58.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:58 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:58.419+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.087560+0000 front 2024-05-21T03:36:27.087536+0000 (oldest deadline 2024-05-21T03:37:54.490886+0000) 2024-05-21T03:37:59.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:58 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:58.820+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:37:59.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:58 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:58.820+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:37:59.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:58 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:58.820+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:37:59.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:59 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:59.012+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:37:59.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:37:59 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:37:59.012+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:37:59.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:59 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:59.021+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:37:59.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:59 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:59.021+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:37:59.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:59 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:59.021+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:37:59.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:59 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:59.111+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:37:59.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:59 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:59.111+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:37:59.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:37:59 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:37:59.111+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:37:59.427 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:59 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:59.359+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:37:59.427 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:59 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:59.359+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:37:59.427 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:37:59 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:37:59.359+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.625870+0000 front 2024-05-21T03:36:27.625797+0000 (oldest deadline 2024-05-21T03:37:57.929376+0000) 2024-05-21T03:37:59.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:59 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:59.378+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:37:59.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:37:59 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:37:59.378+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.087560+0000 front 2024-05-21T03:36:27.087536+0000 (oldest deadline 2024-05-21T03:37:54.490886+0000) 2024-05-21T03:38:00.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:59 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:59.792+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:00.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:59 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:59.792+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:00.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:37:59 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:37:59.792+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:00.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:00 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:00.034+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:00.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:00 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:00.034+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:00.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:59 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:59.977+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:00.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:59 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:59.977+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:00.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:37:59 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:37:59.977+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:00.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:00 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:00.150+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:00.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:00 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:00.150+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:00.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:00 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:00.150+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:00.427 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:00 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:00.359+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:38:00.427 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:00 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:00.359+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:38:00.427 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:00 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:00.359+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.625870+0000 front 2024-05-21T03:36:27.625797+0000 (oldest deadline 2024-05-21T03:37:57.929376+0000) 2024-05-21T03:38:00.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:00 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:00.362+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:38:00.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:00 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:00.362+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.087560+0000 front 2024-05-21T03:36:27.087536+0000 (oldest deadline 2024-05-21T03:37:54.490886+0000) 2024-05-21T03:38:01.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:00 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:00.837+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:01.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:00 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:00.837+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:01.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:00 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:00.837+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:01.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:01 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:01.048+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:01.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:01 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:01.048+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:01.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:00 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:00.963+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:01.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:00 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:00.963+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:01.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:00 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:00.963+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:01.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:01 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:01.154+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:01.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:01 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:01.154+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:01.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:01 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:01.154+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:01.427 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:01 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:01.373+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:38:01.427 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:01 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:01.373+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:38:01.427 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:01 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:01.373+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.625870+0000 front 2024-05-21T03:36:27.625797+0000 (oldest deadline 2024-05-21T03:37:57.929376+0000) 2024-05-21T03:38:01.610 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:01 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:01.317+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:38:01.610 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:01 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:01.317+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.087560+0000 front 2024-05-21T03:36:27.087536+0000 (oldest deadline 2024-05-21T03:37:54.490886+0000) 2024-05-21T03:38:02.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:01 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:01.842+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:02.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:01 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:01.842+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:02.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:01 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:01.842+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:02.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:02 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:02.040+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:02.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:02 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:02.040+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:02.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:01 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:01.988+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:02.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:01 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:01.988+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:02.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:01 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:01.988+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:02.676 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:02 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:02.178+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:02.677 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:02 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:02.178+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:02.677 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:02 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:02.178+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:02.677 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:02 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:02.421+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:38:02.677 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:02 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:02.421+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:38:02.677 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:02 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:02.421+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.625870+0000 front 2024-05-21T03:36:27.625797+0000 (oldest deadline 2024-05-21T03:37:57.929376+0000) 2024-05-21T03:38:02.768 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:02 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:02.307+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:38:02.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:02 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:02.307+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.087560+0000 front 2024-05-21T03:36:27.087536+0000 (oldest deadline 2024-05-21T03:37:54.490886+0000) 2024-05-21T03:38:03.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:02 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:02.801+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:03.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:02 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:02.801+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:03.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:02 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:02.801+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:03.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:03 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:03.075+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:03.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:03 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:03.076+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:03.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:02 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:02.991+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:03.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:02 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:02.991+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:03.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:02 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:02.991+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:03.676 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:03 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:03.375+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:38:03.677 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:03 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:03.375+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:38:03.677 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:03 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:03.375+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.625870+0000 front 2024-05-21T03:36:27.625797+0000 (oldest deadline 2024-05-21T03:37:57.929376+0000) 2024-05-21T03:38:03.677 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:03 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:03.193+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:03.677 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:03 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:03.193+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:03.677 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:03 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:03.193+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:03.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:03 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:03.334+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:38:03.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:03 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:03.334+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.087560+0000 front 2024-05-21T03:36:27.087536+0000 (oldest deadline 2024-05-21T03:37:54.490886+0000) 2024-05-21T03:38:04.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:03 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:03.786+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:04.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:03 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:03.786+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:04.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:03 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:03.786+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:04.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:04 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:04.105+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:04.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:04 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:04.105+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:04.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:03 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:03.973+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:04.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:03 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:03.973+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:04.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:03 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:03.973+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:04.676 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:04 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:04.401+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:38:04.677 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:04 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:04.401+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:38:04.677 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:04 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:04.401+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.625870+0000 front 2024-05-21T03:36:27.625797+0000 (oldest deadline 2024-05-21T03:37:57.929376+0000) 2024-05-21T03:38:04.677 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:04 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:04.204+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:04.677 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:04 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:04.204+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:04.677 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:04 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:04.204+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:04.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:04 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:04.339+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:38:04.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:04 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:04.339+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.087560+0000 front 2024-05-21T03:36:27.087536+0000 (oldest deadline 2024-05-21T03:37:54.490886+0000) 2024-05-21T03:38:05.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:04 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:04.794+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:05.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:04 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:04.794+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:05.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:04 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:04.794+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:05.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:05 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:05.091+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:05.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:05 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:05.091+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:05.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:04 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:04.977+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:05.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:04 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:04.977+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:05.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:04 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:04.977+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:05.677 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:05 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:05.243+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:05.677 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:05 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:05.243+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:05.677 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:05 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:05.243+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:05.677 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:05 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:05.440+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:38:05.677 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:05 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:05.440+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:38:05.677 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:05 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:05.440+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.625870+0000 front 2024-05-21T03:36:27.625797+0000 (oldest deadline 2024-05-21T03:37:57.929376+0000) 2024-05-21T03:38:05.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:05 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:05.294+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:38:05.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:05 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:05.294+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.087560+0000 front 2024-05-21T03:36:27.087536+0000 (oldest deadline 2024-05-21T03:37:54.490886+0000) 2024-05-21T03:38:06.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:05 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:05.820+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:06.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:05 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:05.820+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:06.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:05 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:05.820+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:06.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:06 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:06.063+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:06.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:06 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:06.063+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:06.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:05 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:05.993+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:06.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:05 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:05.993+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:06.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:05 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:05.993+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:06.475 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:06 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:06.208+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:06.475 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:06 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:06.208+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:06.475 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:06 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:06.208+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:06.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:06 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:06.315+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:38:06.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:06 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:06.315+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.087560+0000 front 2024-05-21T03:36:27.087536+0000 (oldest deadline 2024-05-21T03:37:54.490886+0000) 2024-05-21T03:38:06.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:06 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:06.474+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:38:06.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:06 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:06.474+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:38:06.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:06 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:06.474+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.625870+0000 front 2024-05-21T03:36:27.625797+0000 (oldest deadline 2024-05-21T03:37:57.929376+0000) 2024-05-21T03:38:07.219 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:38:06 smithi165 bash[73919]: time="2024-05-21T03:38:06Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5 in 30 seconds, resorting to SIGKILL" 2024-05-21T03:38:07.219 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:38:06 smithi165 podman[73919]: 2024-05-21 03:38:06.938999715 +0000 UTC m=+30.286737933 container died 57e27c2c1b1eba6cdcd3fbf67b9fe0ccfb1fc3c2cbb0d697ff9832892834bb21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T03:38:07.219 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:07 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:07.036+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:07.220 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:07 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:07.036+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:07.220 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:06 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:06.956+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:07.220 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:06 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:06.956+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:07.220 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:06 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:06.956+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:07.268 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:06 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:06.869+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:07.268 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:06 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:06.869+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:07.268 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:06 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:06.869+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:07.433 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:07 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:07.177+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:07.433 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:07 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:07.177+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:07.433 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:07 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:07.177+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:07.519 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:07 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:07.268+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:38:07.519 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:07 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:07.268+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.087560+0000 front 2024-05-21T03:36:27.087536+0000 (oldest deadline 2024-05-21T03:37:54.490886+0000) 2024-05-21T03:38:07.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:38:07 smithi165 podman[73919]: 2024-05-21 03:38:07.29376942 +0000 UTC m=+30.641507640 container cleanup 57e27c2c1b1eba6cdcd3fbf67b9fe0ccfb1fc3c2cbb0d697ff9832892834bb21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-21T03:38:07.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:38:07 smithi165 bash[73919]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5 2024-05-21T03:38:07.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:07 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:07.433+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:38:07.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:07 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:07.433+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:38:07.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:07 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:07.433+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.625870+0000 front 2024-05-21T03:36:27.625797+0000 (oldest deadline 2024-05-21T03:37:57.929376+0000) 2024-05-21T03:38:07.949 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:38:07 smithi165 podman[73936]: 2024-05-21 03:38:07.645684042 +0000 UTC m=+0.703698131 container remove 57e27c2c1b1eba6cdcd3fbf67b9fe0ccfb1fc3c2cbb0d697ff9832892834bb21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-21T03:38:07.949 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:38:07 smithi165 systemd[1]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.5.service: Main process exited, code=exited, status=137/n/a 2024-05-21T03:38:08.209 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:08 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:08.043+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:08.209 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:08 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:08.043+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:08.209 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:38:08 smithi165 podman[74024]: 2024-05-21 03:38:07.969157454 +0000 UTC m=+0.021065310 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T03:38:08.209 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:38:08 smithi165 podman[74024]: 2024-05-21 03:38:08.096847701 +0000 UTC m=+0.148755603 container create e38c98c772ed06f2ed3e7422d9c66ac6a36819077e987d2e0ab50c1ec2abac86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T03:38:08.209 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:07 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:07.949+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:08.209 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:07 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:07.949+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:08.210 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:07 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:07.949+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:08.225 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:07 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:07.833+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:08.225 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:07 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:07.833+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:08.225 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:07 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:07.833+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:08.519 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:08 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:08.225+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:38:08.519 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:08 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:08.225+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.087560+0000 front 2024-05-21T03:36:27.087536+0000 (oldest deadline 2024-05-21T03:37:54.490886+0000) 2024-05-21T03:38:08.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:38:08 smithi165 podman[74024]: 2024-05-21 03:38:08.349429238 +0000 UTC m=+0.401337077 container init e38c98c772ed06f2ed3e7422d9c66ac6a36819077e987d2e0ab50c1ec2abac86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-21T03:38:08.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:38:08 smithi165 podman[74024]: 2024-05-21 03:38:08.352442605 +0000 UTC m=+0.404350453 container start e38c98c772ed06f2ed3e7422d9c66ac6a36819077e987d2e0ab50c1ec2abac86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5-deactivate, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514) 2024-05-21T03:38:08.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:38:08 smithi165 podman[74024]: 2024-05-21 03:38:08.432944112 +0000 UTC m=+0.484851971 container attach e38c98c772ed06f2ed3e7422d9c66ac6a36819077e987d2e0ab50c1ec2abac86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-21T03:38:08.677 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:08 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:08.415+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:38:08.677 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:08 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:08.415+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:38:08.677 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:08 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:08.415+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.625870+0000 front 2024-05-21T03:36:27.625797+0000 (oldest deadline 2024-05-21T03:37:57.929376+0000) 2024-05-21T03:38:08.677 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:08 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:08.180+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:08.677 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:08 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:08.180+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:08.677 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:08 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:08.180+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:08.935 INFO:journalctl@ceph.osd.5.smithi165.stdout:May 21 03:38:08 smithi165 podman[74024]: 2024-05-21 03:38:08.651240597 +0000 UTC m=+0.703148438 container died e38c98c772ed06f2ed3e7422d9c66ac6a36819077e987d2e0ab50c1ec2abac86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-5-deactivate, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-21T03:38:09.202 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:08 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:08.938+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:09.202 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:08 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:08.938+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:09.203 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:08 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:08.938+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:09.203 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:09 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:09.024+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:09.203 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:09 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:09.024+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:09.218 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.5.service' 2024-05-21T03:38:09.268 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:08 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:08.826+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:09.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:08 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:08.826+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:09.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:08 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:08.826+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:09.294 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T03:38:09.294 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-21T03:38:09.294 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-21T03:38:09.294 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.6 2024-05-21T03:38:09.415 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:09 smithi027 systemd[1]: Stopping Ceph osd.6 for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:38:09.415 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:09 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:09.155+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:09.415 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:09 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:09.155+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:09.416 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:09 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:09.155+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:09.676 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:09 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:09.415+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:38:09.677 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:09 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:09.415+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:38:09.677 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:09 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:09.415+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.625870+0000 front 2024-05-21T03:36:27.625797+0000 (oldest deadline 2024-05-21T03:37:57.929376+0000) 2024-05-21T03:38:09.677 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:09 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:09.592+0000 7f95bf2b4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T03:38:09.677 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:09 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:09.592+0000 7f95bf2b4700 -1 osd.6 94 *** Got signal Terminated *** 2024-05-21T03:38:09.677 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:09 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:09.592+0000 7f95bf2b4700 -1 osd.6 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T03:38:09.768 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:09 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:09.276+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:38:09.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:09 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:09.276+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.087560+0000 front 2024-05-21T03:36:27.087536+0000 (oldest deadline 2024-05-21T03:37:54.490886+0000) 2024-05-21T03:38:10.243 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:09 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:09.840+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:10.244 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:09 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:09.840+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:10.244 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:09 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:09.840+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:10.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:10 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:10.010+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:10.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:10 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:10.010+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:10.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:09 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:09.980+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:10.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:09 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:09.980+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:10.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:09 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:09.980+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:10.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:10 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:10.126+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:10.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:10 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:10.126+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:10.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:10 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:10.127+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:10.519 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:10 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:10.244+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:38:10.519 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:10 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:10.244+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.087560+0000 front 2024-05-21T03:36:27.087536+0000 (oldest deadline 2024-05-21T03:37:54.490886+0000) 2024-05-21T03:38:10.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:10 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:10.433+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:38:10.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:10 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:10.433+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:38:10.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:10 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:10.433+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.625870+0000 front 2024-05-21T03:36:27.625797+0000 (oldest deadline 2024-05-21T03:37:57.929376+0000) 2024-05-21T03:38:11.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:10 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:10.841+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:11.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:10 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:10.841+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:11.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:10 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:10.841+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:11.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:11 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:11.056+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:11.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:11 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:11.056+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:11.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:10 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:10.957+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:11.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:10 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:10.957+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:11.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:10 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:10.957+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:11.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:11 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:11.098+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:11.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:11 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:11.098+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:11.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:11 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:11.098+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:11.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:11 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:11.274+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:38:11.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:11 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:11.274+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.087560+0000 front 2024-05-21T03:36:27.087536+0000 (oldest deadline 2024-05-21T03:37:54.490886+0000) 2024-05-21T03:38:11.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:11 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:11.458+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:38:11.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:11 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:11.458+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:38:11.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:11 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:11.458+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.625870+0000 front 2024-05-21T03:36:27.625797+0000 (oldest deadline 2024-05-21T03:37:57.929376+0000) 2024-05-21T03:38:12.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:11 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:11.813+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:12.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:11 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:11.813+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:12.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:11 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:11.813+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:12.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:12 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:12.024+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:12.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:12 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:12.024+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:12.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:11 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:11.915+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:12.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:11 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:11.915+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:12.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:11 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:11.915+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:12.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:12 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:12.127+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:12.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:12 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:12.127+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:12.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:12 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:12.127+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:12.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:12 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:12.308+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:38:12.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:12 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:12.308+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.087560+0000 front 2024-05-21T03:36:27.087536+0000 (oldest deadline 2024-05-21T03:37:54.490886+0000) 2024-05-21T03:38:12.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:12 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:12.475+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:38:12.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:12 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:12.475+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:38:12.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:12 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:12.475+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.625870+0000 front 2024-05-21T03:36:27.625797+0000 (oldest deadline 2024-05-21T03:37:57.929376+0000) 2024-05-21T03:38:13.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:12 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:12.818+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:13.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:12 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:12.818+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:13.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:12 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:12.818+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:13.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:13 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:13.023+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:13.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:13 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:13.023+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:13.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:12 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:12.921+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:13.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:12 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:12.921+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:13.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:12 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:12.921+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:13.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:13 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:13.110+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:13.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:13 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:13.110+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:13.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:13 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:13.110+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:13.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:13 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:13.355+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:38:13.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:13 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:13.355+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.087560+0000 front 2024-05-21T03:36:27.087536+0000 (oldest deadline 2024-05-21T03:37:54.490886+0000) 2024-05-21T03:38:13.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:13 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:13.515+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:38:13.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:13 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:13.515+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:38:13.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:13 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:13.515+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.625870+0000 front 2024-05-21T03:36:27.625797+0000 (oldest deadline 2024-05-21T03:37:57.929376+0000) 2024-05-21T03:38:14.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:13 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:13.838+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:14.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:13 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:13.838+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:14.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:13 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:13.838+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:14.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:14 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:14.068+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:14.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:14 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:14.068+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:14.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:13 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:13.879+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:14.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:13 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:13.879+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:14.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:13 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:13.879+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:14.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:14 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:14.083+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:14.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:14 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:14.083+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:14.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:14 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:14.083+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:14.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:14 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:14.334+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:38:14.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:14 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:14.334+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.087560+0000 front 2024-05-21T03:36:27.087536+0000 (oldest deadline 2024-05-21T03:37:54.490886+0000) 2024-05-21T03:38:14.926 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:14 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:14.515+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.415401+0000 front 2024-05-21T03:35:21.415441+0000 (oldest deadline 2024-05-21T03:36:50.519390+0000) 2024-05-21T03:38:14.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:14 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:14.515+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.020315+0000 front 2024-05-21T03:35:54.020339+0000 (oldest deadline 2024-05-21T03:37:20.823659+0000) 2024-05-21T03:38:14.927 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:14 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6[58451]: 2024-05-21T03:38:14.515+0000 7f95bb8a8700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.625870+0000 front 2024-05-21T03:36:27.625797+0000 (oldest deadline 2024-05-21T03:37:57.929376+0000) 2024-05-21T03:38:15.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:14 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:14.807+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:15.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:14 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:14.807+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:15.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:14 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:14.807+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:15.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:15 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:15.048+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:15.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:15 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:15.048+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:15.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:14 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:14.857+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:15.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:14 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:14.857+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:15.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:14 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:14.857+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:15.427 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:15 smithi027 podman[105240]: 2024-05-21 03:38:15.292828178 +0000 UTC m=+5.804753723 container died 4a0b825924fdc32bced7d5a21c039fbd86d47f5a1e3458122f14b58db8aea9ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-21T03:38:15.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:15 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:15.072+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:15.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:15 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:15.072+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:15.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:15 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:15.072+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:15.758 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:15 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:15.324+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:38:15.758 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:15 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:15.324+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.087560+0000 front 2024-05-21T03:36:27.087536+0000 (oldest deadline 2024-05-21T03:37:54.490886+0000) 2024-05-21T03:38:15.833 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:15 smithi027 podman[105240]: 2024-05-21 03:38:15.599596514 +0000 UTC m=+6.111522051 container cleanup 4a0b825924fdc32bced7d5a21c039fbd86d47f5a1e3458122f14b58db8aea9ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-21T03:38:15.833 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:15 smithi027 bash[105240]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6 2024-05-21T03:38:16.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:15 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:15.758+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:16.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:15 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:15.758+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:16.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:15 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:15.758+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:16.175 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:15 smithi027 podman[105260]: 2024-05-21 03:38:15.834306257 +0000 UTC m=+0.539722952 container remove 4a0b825924fdc32bced7d5a21c039fbd86d47f5a1e3458122f14b58db8aea9ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-21T03:38:16.175 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:16 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:16.047+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:16.175 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:16 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:16.047+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:16.176 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:16 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:16.047+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:16.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:16 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:16.076+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:16.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:16 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:16.076+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:16.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:15 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:15.899+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:16.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:15 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:15.899+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:16.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:15 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:15.899+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:16.427 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:16 smithi027 podman[105349]: 2024-05-21 03:38:16.079009938 +0000 UTC m=+0.018072638 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T03:38:16.427 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:16 smithi027 podman[105349]: 2024-05-21 03:38:16.193810024 +0000 UTC m=+0.132872759 container create b69336c8613d9a5f72e1400c9f89dba69c904b4c3a7d22c4fe7a899c6be1a159 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514) 2024-05-21T03:38:16.745 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:16 smithi027 podman[105349]: 2024-05-21 03:38:16.437842107 +0000 UTC m=+0.376904800 container init b69336c8613d9a5f72e1400c9f89dba69c904b4c3a7d22c4fe7a899c6be1a159 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6-deactivate, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-21T03:38:16.746 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:16 smithi027 podman[105349]: 2024-05-21 03:38:16.441468642 +0000 UTC m=+0.380531336 container start b69336c8613d9a5f72e1400c9f89dba69c904b4c3a7d22c4fe7a899c6be1a159 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-21T03:38:16.746 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:16 smithi027 podman[105349]: 2024-05-21 03:38:16.504403658 +0000 UTC m=+0.443466353 container attach b69336c8613d9a5f72e1400c9f89dba69c904b4c3a7d22c4fe7a899c6be1a159 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T03:38:16.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:16 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:16.319+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:38:16.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:16 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:16.319+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.087560+0000 front 2024-05-21T03:36:27.087536+0000 (oldest deadline 2024-05-21T03:37:54.490886+0000) 2024-05-21T03:38:17.023 INFO:journalctl@ceph.osd.6.smithi027.stdout:May 21 03:38:16 smithi027 podman[105377]: 2024-05-21 03:38:16.775269715 +0000 UTC m=+0.019208034 container died b69336c8613d9a5f72e1400c9f89dba69c904b4c3a7d22c4fe7a899c6be1a159 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-21T03:38:17.243 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.6.service' 2024-05-21T03:38:17.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:16 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:16.776+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:17.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:16 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:16.776+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:17.269 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:16 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:16.776+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:17.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:17 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:17.045+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:17.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:17 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:17.045+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:17.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:16 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:16.912+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:17.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:16 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:16.912+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:17.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:16 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:16.912+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:17.283 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:17 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:17.024+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:17.283 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:17 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:17.024+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:17.283 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:17 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:17.024+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:17.285 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T03:38:17.285 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-21T03:38:17.285 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-21T03:38:17.285 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.7 2024-05-21T03:38:17.589 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:17 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:17.292+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:38:17.590 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:17 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:17.292+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.087560+0000 front 2024-05-21T03:36:27.087536+0000 (oldest deadline 2024-05-21T03:37:54.490886+0000) 2024-05-21T03:38:17.590 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:17 smithi039 systemd[1]: Stopping Ceph osd.7 for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:38:17.590 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:17 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:17.589+0000 7f55714b3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T03:38:18.019 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:17 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:17.589+0000 7f55714b3700 -1 osd.7 94 *** Got signal Terminated *** 2024-05-21T03:38:18.019 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:17 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:17.589+0000 7f55714b3700 -1 osd.7 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T03:38:18.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:17 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:17.777+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:18.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:17 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:17.777+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:18.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:17 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:17.777+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:18.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:18 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:18.020+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:18.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:18 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:18.020+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:18.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:17 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:17.893+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:18.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:17 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:17.893+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:18.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:17 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:17.893+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:18.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:18 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:18.072+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:18.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:18 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:18.072+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:18.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:18 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:18.072+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:18.757 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:18 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:18.278+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:38:18.757 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:18 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:18.278+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.087560+0000 front 2024-05-21T03:36:27.087536+0000 (oldest deadline 2024-05-21T03:37:54.490886+0000) 2024-05-21T03:38:19.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:18 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:18.757+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:19.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:18 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:18.757+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:19.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:18 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:18.757+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:19.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:18 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:18.978+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:19.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:18 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:18.978+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:19.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:18 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:18.868+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:19.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:18 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:18.868+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:19.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:18 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:18.868+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:19.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:19 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:19.122+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:19.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:19 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:19.122+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:19.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:19 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:19.122+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:19.728 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:19 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:19.281+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:38:19.728 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:19 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:19.281+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.087560+0000 front 2024-05-21T03:36:27.087536+0000 (oldest deadline 2024-05-21T03:37:54.490886+0000) 2024-05-21T03:38:20.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:19 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:19.728+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:20.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:19 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:19.728+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:20.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:19 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:19.728+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:20.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:19 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:19.929+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:20.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:19 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:19.929+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:20.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:19 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:19.845+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:20.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:19 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:19.845+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:20.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:19 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:19.845+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:20.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:20 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:20.100+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:20.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:20 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:20.100+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:20.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:20 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:20.100+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:20.518 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:20 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:20.249+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:38:20.519 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:20 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:20.249+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.087560+0000 front 2024-05-21T03:36:27.087536+0000 (oldest deadline 2024-05-21T03:37:54.490886+0000) 2024-05-21T03:38:21.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:20 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:20.743+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:21.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:20 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:20.743+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:21.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:20 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:20.743+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:21.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:20 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:20.961+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:21.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:20 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:20.961+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:21.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:20 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:20.814+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:21.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:20 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:20.814+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:21.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:20 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:20.814+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:21.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:21 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:21.146+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:21.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:21 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:21.146+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:21.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:21 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:21.146+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:21.519 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:21 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:21.234+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:38:21.519 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:21 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:21.234+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.087560+0000 front 2024-05-21T03:36:27.087536+0000 (oldest deadline 2024-05-21T03:37:54.490886+0000) 2024-05-21T03:38:22.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:21 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:21.697+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:22.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:21 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:21.697+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:22.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:21 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:21.697+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:22.172 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:21 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:21.926+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:22.172 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:21 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:21.926+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:22.172 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:21 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:21.827+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:22.172 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:21 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:21.827+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:22.172 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:21 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:21.827+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:22.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:22 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:22.123+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:22.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:22 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:22.123+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:22.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:22 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:22.123+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:22.519 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:22 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:22.218+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:52.781953+0000 front 2024-05-21T03:35:52.781973+0000 (oldest deadline 2024-05-21T03:37:20.686063+0000) 2024-05-21T03:38:22.519 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:22 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7[54488]: 2024-05-21T03:38:22.218+0000 7f5569a9f700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.087560+0000 front 2024-05-21T03:36:27.087536+0000 (oldest deadline 2024-05-21T03:37:54.490886+0000) 2024-05-21T03:38:23.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:22 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:22.665+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:23.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:22 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:22.666+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:23.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:22 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:22.666+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:23.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:22 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:22.908+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:23.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:22 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:22.908+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:23.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:22 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:22.809+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:23.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:22 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:22.809+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:23.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:22 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:22.809+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:23.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:23 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:23.117+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:23.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:23 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:23.117+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:23.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:23 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:23.117+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:23.518 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:23 smithi039 podman[73684]: 2024-05-21 03:38:23.114400715 +0000 UTC m=+5.622483170 container died ebc2f4cd30b683a5b67d4d95a365d43bf140255902eb776386f742514a8f087f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0) 2024-05-21T03:38:23.815 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:23 smithi039 podman[73684]: 2024-05-21 03:38:23.623110916 +0000 UTC m=+6.131193371 container cleanup ebc2f4cd30b683a5b67d4d95a365d43bf140255902eb776386f742514a8f087f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-21T03:38:23.815 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:23 smithi039 bash[73684]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7 2024-05-21T03:38:23.815 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:23 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:23.673+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:23.815 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:23 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:23.673+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:23.816 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:23 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:23.674+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:24.243 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:23 smithi039 podman[73702]: 2024-05-21 03:38:23.816133519 +0000 UTC m=+0.698940742 container remove ebc2f4cd30b683a5b67d4d95a365d43bf140255902eb776386f742514a8f087f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-21T03:38:24.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:23 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:23.870+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:24.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:23 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:23.870+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:24.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:23 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:23.799+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:24.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:23 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:23.799+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:24.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:23 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:23.799+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:24.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:24 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:24.137+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:24.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:24 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:24.137+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:24.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:24 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:24.137+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:24.493 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:24 smithi039 podman[73789]: 2024-05-21 03:38:24.146704188 +0000 UTC m=+0.017814429 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T03:38:24.493 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:24 smithi039 podman[73789]: 2024-05-21 03:38:24.267071678 +0000 UTC m=+0.138181904 container create 05ab6bd5ced97c0e800d5812ded67651a7d0ea2d56cfd7d64267ff3af819f9c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T03:38:24.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:24 smithi039 podman[73789]: 2024-05-21 03:38:24.494402498 +0000 UTC m=+0.365512717 container init 05ab6bd5ced97c0e800d5812ded67651a7d0ea2d56cfd7d64267ff3af819f9c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-21T03:38:24.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:24 smithi039 podman[73789]: 2024-05-21 03:38:24.498001259 +0000 UTC m=+0.369111488 container start 05ab6bd5ced97c0e800d5812ded67651a7d0ea2d56cfd7d64267ff3af819f9c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7-deactivate, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-21T03:38:24.769 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:24 smithi039 podman[73789]: 2024-05-21 03:38:24.577366473 +0000 UTC m=+0.448476703 container attach 05ab6bd5ced97c0e800d5812ded67651a7d0ea2d56cfd7d64267ff3af819f9c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-21T03:38:24.769 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:24 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:24.683+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:24.770 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:24 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:24.683+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:24.770 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:24 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:24.683+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:25.097 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:24 smithi039 podman[73789]: 2024-05-21 03:38:24.813898675 +0000 UTC m=+0.685008918 container died 05ab6bd5ced97c0e800d5812ded67651a7d0ea2d56cfd7d64267ff3af819f9c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-21T03:38:25.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:24 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:24.876+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:25.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:24 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:24.876+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:25.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:24 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:24.821+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:25.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:24 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:24.821+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:25.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:24 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:24.821+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:25.348 INFO:journalctl@ceph.osd.7.smithi039.stdout:May 21 03:38:25 smithi039 podman[73815]: 2024-05-21 03:38:25.347332618 +0000 UTC m=+0.527478294 container remove 05ab6bd5ced97c0e800d5812ded67651a7d0ea2d56cfd7d64267ff3af819f9c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T03:38:25.370 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.7.service' 2024-05-21T03:38:25.409 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T03:38:25.410 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-21T03:38:25.410 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-21T03:38:25.410 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.8 2024-05-21T03:38:25.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:25 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:25.108+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:25.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:25 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:25.108+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:25.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:25 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:25.108+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:25.730 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:25 smithi165 systemd[1]: Stopping Ceph osd.8 for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:38:25.730 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:25 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:25.729+0000 7f9798e1e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T03:38:26.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:25 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:25.642+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:26.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:25 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:25.642+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:26.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:25 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:25.642+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:26.021 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:25 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:25.816+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:26.021 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:25 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:25.816+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:26.021 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:25 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:25.816+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:26.021 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:25 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:25.729+0000 7f9798e1e700 -1 osd.8 94 *** Got signal Terminated *** 2024-05-21T03:38:26.021 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:25 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:25.729+0000 7f9798e1e700 -1 osd.8 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T03:38:26.021 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:25 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:25.830+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:26.021 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:25 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:25.830+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:26.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:26 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:26.062+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:26.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:26 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:26.062+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:26.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:26 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:26.062+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:27.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:26 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:26.655+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:27.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:26 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:26.655+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:27.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:26 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:26.655+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:27.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:26 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:26.801+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:27.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:26 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:26.801+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:27.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:26 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:26.801+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:59.070793+0000 front 2024-05-21T03:36:59.070845+0000 (oldest deadline 2024-05-21T03:38:26.474228+0000) 2024-05-21T03:38:27.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:26 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:26.838+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:27.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:26 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:26.838+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:27.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:26 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:26.838+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:27.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:27 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:27.077+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:27.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:27 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:27.077+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:27.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:27 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:27.077+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:27.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:27 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:27.077+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.413717+0000 front 2024-05-21T03:36:56.413780+0000 (oldest deadline 2024-05-21T03:38:26.117579+0000) 2024-05-21T03:38:28.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:27 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:27.643+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:28.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:27 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:27.643+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:28.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:27 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:27.643+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:28.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:27 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:27.810+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:28.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:27 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:27.810+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:28.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:27 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:27.810+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:59.070793+0000 front 2024-05-21T03:36:59.070845+0000 (oldest deadline 2024-05-21T03:38:26.474228+0000) 2024-05-21T03:38:28.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:27 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:27.864+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:28.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:27 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:27.864+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:28.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:27 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:27.864+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:28.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:28 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:28.087+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:28.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:28 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:28.087+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:28.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:28 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:28.087+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:28.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:28 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:28.087+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.413717+0000 front 2024-05-21T03:36:56.413780+0000 (oldest deadline 2024-05-21T03:38:26.117579+0000) 2024-05-21T03:38:29.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:28 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:28.651+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:29.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:28 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:28.651+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:29.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:28 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:28.651+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:29.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:28 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:28.651+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:29.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:28 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:28.844+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:29.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:28 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:28.844+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:29.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:28 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:28.844+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:59.070793+0000 front 2024-05-21T03:36:59.070845+0000 (oldest deadline 2024-05-21T03:38:26.474228+0000) 2024-05-21T03:38:29.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:28 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:28.856+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:29.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:28 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:28.856+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:29.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:28 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:28.856+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:29.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:29 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:29.126+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:29.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:29 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:29.126+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:29.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:29 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:29.126+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:29.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:29 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:29.126+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.413717+0000 front 2024-05-21T03:36:56.413780+0000 (oldest deadline 2024-05-21T03:38:26.117579+0000) 2024-05-21T03:38:30.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:29 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:29.611+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:30.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:29 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:29.611+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:30.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:29 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:29.611+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:30.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:29 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:29.611+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:30.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:29 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:29.841+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:51.961520+0000 front 2024-05-21T03:35:51.961328+0000 (oldest deadline 2024-05-21T03:37:22.364787+0000) 2024-05-21T03:38:30.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:29 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:29.841+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.166212+0000 front 2024-05-21T03:36:25.166342+0000 (oldest deadline 2024-05-21T03:37:56.169837+0000) 2024-05-21T03:38:30.271 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:29 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8[54440]: 2024-05-21T03:38:29.841+0000 7f9795412700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:59.070793+0000 front 2024-05-21T03:36:59.070845+0000 (oldest deadline 2024-05-21T03:38:26.474228+0000) 2024-05-21T03:38:30.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:29 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:29.890+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:30.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:29 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:29.890+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:30.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:29 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:29.891+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:30.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:30 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:30.105+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:30.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:30 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:30.105+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:30.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:30 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:30.105+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:30.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:30 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:30.105+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.413717+0000 front 2024-05-21T03:36:56.413780+0000 (oldest deadline 2024-05-21T03:38:26.117579+0000) 2024-05-21T03:38:31.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:30 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:30.567+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:31.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:30 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:30.567+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:31.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:30 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:30.567+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:31.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:30 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:30.567+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:31.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:30 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:30.869+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:31.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:30 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:30.869+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:31.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:30 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:30.869+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:31.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:31 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:31.141+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:31.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:31 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:31.141+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:31.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:31 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:31.141+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:31.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:31 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:31.141+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.413717+0000 front 2024-05-21T03:36:56.413780+0000 (oldest deadline 2024-05-21T03:38:26.117579+0000) 2024-05-21T03:38:32.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:31 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:31.563+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:32.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:31 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:31.563+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:32.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:31 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:31.563+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:32.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:31 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:31.563+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:32.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:31 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:31.905+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:32.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:31 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:31.905+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:32.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:31 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:31.905+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:32.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:32 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:32.104+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:32.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:32 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:32.104+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:32.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:32 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:32.104+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:32.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:32 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:32.104+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.413717+0000 front 2024-05-21T03:36:56.413780+0000 (oldest deadline 2024-05-21T03:38:26.117579+0000) 2024-05-21T03:38:33.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:32 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:32.554+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:33.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:32 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:32.554+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:33.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:32 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:32.554+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:33.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:32 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:32.554+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:33.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:32 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:32.879+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:33.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:32 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:32.879+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:33.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:32 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:32.879+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:33.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:33 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:33.107+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:33.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:33 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:33.107+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:33.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:33 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:33.107+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:33.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:33 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:33.107+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.413717+0000 front 2024-05-21T03:36:56.413780+0000 (oldest deadline 2024-05-21T03:38:26.117579+0000) 2024-05-21T03:38:34.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:33 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:33.563+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:34.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:33 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:33.563+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:34.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:33 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:33.563+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:34.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:33 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:33.563+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:34.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:33 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:33.874+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:34.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:33 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:33.874+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:34.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:33 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:33.874+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:34.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:34 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:34.096+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:34.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:34 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:34.096+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:34.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:34 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:34.096+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:34.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:34 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:34.096+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.413717+0000 front 2024-05-21T03:36:56.413780+0000 (oldest deadline 2024-05-21T03:38:26.117579+0000) 2024-05-21T03:38:35.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:34 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:34.609+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:35.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:34 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:34.609+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:35.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:34 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:34.609+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:35.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:34 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:34.609+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:35.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:34 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:34.853+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:35.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:34 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:34.853+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:35.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:34 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:34.853+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:35.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:35 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:35.047+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:35.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:35 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:35.047+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:35.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:35 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:35.047+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:35.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:35 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:35.047+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.413717+0000 front 2024-05-21T03:36:56.413780+0000 (oldest deadline 2024-05-21T03:38:26.117579+0000) 2024-05-21T03:38:36.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:35 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:35.642+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:36.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:35 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:35.642+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:36.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:35 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:35.642+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:36.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:35 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:35.642+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:36.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:35 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:35.870+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:36.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:35 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:35.870+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:36.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:35 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:35.870+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:36.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:35 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:35.870+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:38:36.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:36 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:36.032+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:36.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:36 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:36.032+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:36.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:36 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:36.032+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:36.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:36 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:36.032+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.413717+0000 front 2024-05-21T03:36:56.413780+0000 (oldest deadline 2024-05-21T03:38:26.117579+0000) 2024-05-21T03:38:37.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:36 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:36.671+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:37.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:36 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:36.671+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:37.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:36 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:36.671+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:37.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:36 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:36.671+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:37.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:36 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:36.671+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.744632+0000 front 2024-05-21T03:37:30.744615+0000 (oldest deadline 2024-05-21T03:38:36.644810+0000) 2024-05-21T03:38:37.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:36 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:36.840+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:37.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:36 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:36.840+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:37.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:36 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:36.840+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:37.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:36 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:36.840+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:38:37.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:37 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:37.022+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:37.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:37 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:37.022+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:37.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:37 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:37.022+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:37.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:37 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:37.022+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.413717+0000 front 2024-05-21T03:36:56.413780+0000 (oldest deadline 2024-05-21T03:38:26.117579+0000) 2024-05-21T03:38:37.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:37 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:37.022+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.218459+0000 front 2024-05-21T03:37:30.218348+0000 (oldest deadline 2024-05-21T03:38:36.118226+0000) 2024-05-21T03:38:38.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:37 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:37.627+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:38.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:37 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:37.627+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:38.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:37 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:37.627+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:38.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:37 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:37.627+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:38.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:37 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:37.627+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.744632+0000 front 2024-05-21T03:37:30.744615+0000 (oldest deadline 2024-05-21T03:38:36.644810+0000) 2024-05-21T03:38:38.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:37 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:37.831+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:38.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:37 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:37.831+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:38.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:37 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:37.831+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:38.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:37 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:37.831+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:38:38.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:37 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:37.989+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:38.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:37 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:37.989+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:38.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:37 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:37.989+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:38.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:37 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:37.989+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.413717+0000 front 2024-05-21T03:36:56.413780+0000 (oldest deadline 2024-05-21T03:38:26.117579+0000) 2024-05-21T03:38:38.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:37 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:37.989+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.218459+0000 front 2024-05-21T03:37:30.218348+0000 (oldest deadline 2024-05-21T03:38:36.118226+0000) 2024-05-21T03:38:39.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:38 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:38.647+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:39.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:38 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:38.647+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:39.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:38 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:38.647+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:39.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:38 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:38.647+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:39.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:38 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:38.647+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.744632+0000 front 2024-05-21T03:37:30.744615+0000 (oldest deadline 2024-05-21T03:38:36.644810+0000) 2024-05-21T03:38:39.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:38 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:38.869+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:39.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:38 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:38.869+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:39.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:38 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:38.869+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:39.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:38 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:38.869+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:38:39.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:38 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:38.998+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:39.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:38 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:38.998+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:39.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:38 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:38.998+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:39.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:38 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:38.998+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.413717+0000 front 2024-05-21T03:36:56.413780+0000 (oldest deadline 2024-05-21T03:38:26.117579+0000) 2024-05-21T03:38:39.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:38 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:38.998+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.218459+0000 front 2024-05-21T03:37:30.218348+0000 (oldest deadline 2024-05-21T03:38:36.118226+0000) 2024-05-21T03:38:40.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:39 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:39.661+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:40.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:39 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:39.661+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:40.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:39 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:39.661+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:40.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:39 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:39.661+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:40.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:39 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:39.661+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.744632+0000 front 2024-05-21T03:37:30.744615+0000 (oldest deadline 2024-05-21T03:38:36.644810+0000) 2024-05-21T03:38:40.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:39 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:39.829+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:40.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:39 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:39.829+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:40.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:39 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:39.829+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:40.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:39 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:39.829+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:38:40.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:39 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:39.976+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:40.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:39 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:39.976+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:40.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:39 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:39.976+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:40.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:39 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:39.976+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.413717+0000 front 2024-05-21T03:36:56.413780+0000 (oldest deadline 2024-05-21T03:38:26.117579+0000) 2024-05-21T03:38:40.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:39 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:39.976+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.218459+0000 front 2024-05-21T03:37:30.218348+0000 (oldest deadline 2024-05-21T03:38:36.118226+0000) 2024-05-21T03:38:41.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:40 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:40.645+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:41.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:40 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:40.645+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:41.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:40 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:40.645+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:41.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:40 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:40.645+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:41.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:40 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:40.645+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.744632+0000 front 2024-05-21T03:37:30.744615+0000 (oldest deadline 2024-05-21T03:38:36.644810+0000) 2024-05-21T03:38:41.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:40 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:40.794+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:41.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:40 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:40.794+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:41.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:40 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:40.794+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:41.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:40 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:40.794+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:38:41.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:40 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:40.938+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:41.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:40 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:40.938+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:41.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:40 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:40.938+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:41.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:40 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:40.938+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.413717+0000 front 2024-05-21T03:36:56.413780+0000 (oldest deadline 2024-05-21T03:38:26.117579+0000) 2024-05-21T03:38:41.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:40 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:40.938+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.218459+0000 front 2024-05-21T03:37:30.218348+0000 (oldest deadline 2024-05-21T03:38:36.118226+0000) 2024-05-21T03:38:42.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:41 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:41.639+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:42.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:41 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:41.639+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:42.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:41 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:41.639+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:42.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:41 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:41.639+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:42.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:41 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:41.639+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.744632+0000 front 2024-05-21T03:37:30.744615+0000 (oldest deadline 2024-05-21T03:38:36.644810+0000) 2024-05-21T03:38:42.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:41 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:41.802+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:42.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:41 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:41.802+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:42.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:41 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:41.802+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:42.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:41 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:41.802+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:38:42.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:41 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:41.967+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:42.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:41 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:41.967+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:42.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:41 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:41.967+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:42.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:41 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:41.967+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.413717+0000 front 2024-05-21T03:36:56.413780+0000 (oldest deadline 2024-05-21T03:38:26.117579+0000) 2024-05-21T03:38:42.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:41 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:41.967+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.218459+0000 front 2024-05-21T03:37:30.218348+0000 (oldest deadline 2024-05-21T03:38:36.118226+0000) 2024-05-21T03:38:43.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:42 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:42.682+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:43.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:42 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:42.682+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:43.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:42 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:42.682+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:43.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:42 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:42.682+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:43.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:42 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:42.682+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.744632+0000 front 2024-05-21T03:37:30.744615+0000 (oldest deadline 2024-05-21T03:38:36.644810+0000) 2024-05-21T03:38:43.177 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:42 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:42.923+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:43.177 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:42 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:42.923+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:43.177 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:42 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:42.923+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:43.177 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:42 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:42.923+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.413717+0000 front 2024-05-21T03:36:56.413780+0000 (oldest deadline 2024-05-21T03:38:26.117579+0000) 2024-05-21T03:38:43.177 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:42 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:42.923+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.218459+0000 front 2024-05-21T03:37:30.218348+0000 (oldest deadline 2024-05-21T03:38:36.118226+0000) 2024-05-21T03:38:43.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:42 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:42.785+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:43.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:42 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:42.785+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:43.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:42 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:42.785+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:43.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:42 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:42.785+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:38:44.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:43 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:43.653+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:44.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:43 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:43.653+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:44.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:43 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:43.653+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:44.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:43 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:43.653+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:44.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:43 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:43.653+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.744632+0000 front 2024-05-21T03:37:30.744615+0000 (oldest deadline 2024-05-21T03:38:36.644810+0000) 2024-05-21T03:38:44.176 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:43 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:43.881+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:44.177 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:43 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:43.881+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:44.177 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:43 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:43.881+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:44.177 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:43 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:43.881+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.413717+0000 front 2024-05-21T03:36:56.413780+0000 (oldest deadline 2024-05-21T03:38:26.117579+0000) 2024-05-21T03:38:44.177 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:43 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:43.881+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.218459+0000 front 2024-05-21T03:37:30.218348+0000 (oldest deadline 2024-05-21T03:38:36.118226+0000) 2024-05-21T03:38:44.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:43 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:43.788+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:44.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:43 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:43.788+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:44.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:43 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:43.788+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:44.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:43 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:43.788+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:38:45.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:44 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:44.636+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:45.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:44 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:44.636+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:45.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:44 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:44.636+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:45.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:44 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:44.636+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:45.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:44 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:44.636+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.744632+0000 front 2024-05-21T03:37:30.744615+0000 (oldest deadline 2024-05-21T03:38:36.644810+0000) 2024-05-21T03:38:45.177 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:44 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:44.904+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:45.177 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:44 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:44.904+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:45.177 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:44 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:44.904+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:45.177 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:44 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:44.904+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.413717+0000 front 2024-05-21T03:36:56.413780+0000 (oldest deadline 2024-05-21T03:38:26.117579+0000) 2024-05-21T03:38:45.177 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:44 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:44.904+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.218459+0000 front 2024-05-21T03:37:30.218348+0000 (oldest deadline 2024-05-21T03:38:36.118226+0000) 2024-05-21T03:38:45.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:44 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:44.835+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:45.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:44 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:44.835+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:45.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:44 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:44.835+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:45.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:44 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:44.835+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:38:46.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:45 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:45.588+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:46.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:45 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:45.588+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:46.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:45 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:45.588+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:46.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:45 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:45.588+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:46.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:45 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:45.588+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.744632+0000 front 2024-05-21T03:37:30.744615+0000 (oldest deadline 2024-05-21T03:38:36.644810+0000) 2024-05-21T03:38:46.097 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:45 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:45.808+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:46.097 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:45 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:45.808+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:46.097 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:45 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:45.808+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:46.097 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:45 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:45.808+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:38:46.176 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:45 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:45.906+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:46.176 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:45 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:45.906+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:46.177 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:45 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:45.906+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:46.177 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:45 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:45.906+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.413717+0000 front 2024-05-21T03:36:56.413780+0000 (oldest deadline 2024-05-21T03:38:26.117579+0000) 2024-05-21T03:38:46.177 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:45 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:45.906+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.218459+0000 front 2024-05-21T03:37:30.218348+0000 (oldest deadline 2024-05-21T03:38:36.118226+0000) 2024-05-21T03:38:46.378 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:46 smithi165 podman[74133]: 2024-05-21 03:38:46.098096762 +0000 UTC m=+20.494288775 container died 2fc5bb0e1f5831a8fbcead87d626ad0b116fe64961b681d28be2e10df44c737c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-21T03:38:46.713 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:46 smithi165 podman[74133]: 2024-05-21 03:38:46.470621122 +0000 UTC m=+20.866813136 container cleanup 2fc5bb0e1f5831a8fbcead87d626ad0b116fe64961b681d28be2e10df44c737c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T03:38:46.713 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:46 smithi165 bash[74133]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8 2024-05-21T03:38:47.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:46 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:46.634+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:47.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:46 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:46.634+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:47.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:46 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:46.634+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:47.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:46 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:46.634+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:47.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:46 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:46.634+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.744632+0000 front 2024-05-21T03:37:30.744615+0000 (oldest deadline 2024-05-21T03:38:36.644810+0000) 2024-05-21T03:38:47.021 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:46 smithi165 podman[74149]: 2024-05-21 03:38:46.713674531 +0000 UTC m=+0.614434092 container remove 2fc5bb0e1f5831a8fbcead87d626ad0b116fe64961b681d28be2e10df44c737c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-21T03:38:47.021 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:46 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:46.844+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:47.021 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:46 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:46.844+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:47.021 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:46 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:46.844+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:47.021 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:46 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:46.844+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:38:47.350 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:47 smithi165 podman[74239]: 2024-05-21 03:38:47.039563739 +0000 UTC m=+0.113551401 container create 047dc33d4e2f6db0c7bc2894dd8d54badaea354f60589823c49a0f17311978b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514) 2024-05-21T03:38:47.350 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:47 smithi165 podman[74239]: 2024-05-21 03:38:46.943277311 +0000 UTC m=+0.017264991 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T03:38:47.350 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:47 smithi165 podman[74239]: 2024-05-21 03:38:47.267278248 +0000 UTC m=+0.341265910 container init 047dc33d4e2f6db0c7bc2894dd8d54badaea354f60589823c49a0f17311978b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-21T03:38:47.350 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:47 smithi165 podman[74239]: 2024-05-21 03:38:47.270470071 +0000 UTC m=+0.344457755 container start 047dc33d4e2f6db0c7bc2894dd8d54badaea354f60589823c49a0f17311978b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea) 2024-05-21T03:38:47.350 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:47 smithi165 podman[74239]: 2024-05-21 03:38:47.350610911 +0000 UTC m=+0.424598590 container attach 047dc33d4e2f6db0c7bc2894dd8d54badaea354f60589823c49a0f17311978b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T03:38:47.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:46 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:46.929+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:47.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:46 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:46.929+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:47.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:46 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:46.929+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:47.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:46 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:46.929+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.413717+0000 front 2024-05-21T03:36:56.413780+0000 (oldest deadline 2024-05-21T03:38:26.117579+0000) 2024-05-21T03:38:47.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:46 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:46.929+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.218459+0000 front 2024-05-21T03:37:30.218348+0000 (oldest deadline 2024-05-21T03:38:36.118226+0000) 2024-05-21T03:38:47.771 INFO:journalctl@ceph.osd.8.smithi165.stdout:May 21 03:38:47 smithi165 podman[74239]: 2024-05-21 03:38:47.572058304 +0000 UTC m=+0.646045980 container died 047dc33d4e2f6db0c7bc2894dd8d54badaea354f60589823c49a0f17311978b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5) 2024-05-21T03:38:48.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:47 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:47.681+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:48.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:47 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:47.681+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:48.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:47 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:47.681+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:48.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:47 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:47.681+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:48.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:47 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:47.681+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.744632+0000 front 2024-05-21T03:37:30.744615+0000 (oldest deadline 2024-05-21T03:38:36.644810+0000) 2024-05-21T03:38:48.136 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:47 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:47.827+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:48.136 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:47 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:47.827+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:48.137 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:47 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:47.827+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:48.137 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:47 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:47.827+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:38:48.157 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.8.service' 2024-05-21T03:38:48.194 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T03:38:48.195 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-21T03:38:48.195 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-21T03:38:48.195 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.9 2024-05-21T03:38:48.225 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:47 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:47.952+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:48.225 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:47 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:47.952+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:48.225 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:47 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:47.952+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:48.225 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:47 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:47.952+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.413717+0000 front 2024-05-21T03:36:56.413780+0000 (oldest deadline 2024-05-21T03:38:26.117579+0000) 2024-05-21T03:38:48.225 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:47 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:47.952+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.218459+0000 front 2024-05-21T03:37:30.218348+0000 (oldest deadline 2024-05-21T03:38:36.118226+0000) 2024-05-21T03:38:48.501 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:48 smithi027 systemd[1]: Stopping Ceph osd.9 for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:38:48.927 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:48 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:48.501+0000 7f4309870700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T03:38:48.927 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:48 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:48.501+0000 7f4309870700 -1 osd.9 94 *** Got signal Terminated *** 2024-05-21T03:38:48.927 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:48 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:48.501+0000 7f4309870700 -1 osd.9 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T03:38:49.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:48 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:48.679+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:49.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:48 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:48.679+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:49.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:48 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:48.679+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:49.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:48 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:48.680+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:49.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:48 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:48.680+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.744632+0000 front 2024-05-21T03:37:30.744615+0000 (oldest deadline 2024-05-21T03:38:36.644810+0000) 2024-05-21T03:38:49.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:48 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:48.785+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:49.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:48 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:48.785+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:49.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:48 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:48.785+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:49.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:48 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:48.785+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:38:49.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:48 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:48.983+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:49.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:48 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:48.983+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:49.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:48 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:48.983+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:49.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:48 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:48.983+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.413717+0000 front 2024-05-21T03:36:56.413780+0000 (oldest deadline 2024-05-21T03:38:26.117579+0000) 2024-05-21T03:38:49.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:48 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:48.983+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.218459+0000 front 2024-05-21T03:37:30.218348+0000 (oldest deadline 2024-05-21T03:38:36.118226+0000) 2024-05-21T03:38:50.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:49 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:49.675+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:50.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:49 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:49.675+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:50.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:49 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:49.675+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:50.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:49 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:49.675+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:50.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:49 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:49.675+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.744632+0000 front 2024-05-21T03:37:30.744615+0000 (oldest deadline 2024-05-21T03:38:36.644810+0000) 2024-05-21T03:38:50.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:49 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:49.790+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:50.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:49 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:49.790+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:50.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:49 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:49.790+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:50.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:49 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:49.790+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:38:50.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:49 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:49.938+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:50.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:49 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:49.938+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:50.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:49 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:49.938+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:50.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:49 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:49.938+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.413717+0000 front 2024-05-21T03:36:56.413780+0000 (oldest deadline 2024-05-21T03:38:26.117579+0000) 2024-05-21T03:38:50.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:49 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:49.938+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.218459+0000 front 2024-05-21T03:37:30.218348+0000 (oldest deadline 2024-05-21T03:38:36.118226+0000) 2024-05-21T03:38:51.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:50 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:50.703+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:51.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:50 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:50.703+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:51.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:50 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:50.703+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:51.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:50 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:50.703+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:51.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:50 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:50.703+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.744632+0000 front 2024-05-21T03:37:30.744615+0000 (oldest deadline 2024-05-21T03:38:36.644810+0000) 2024-05-21T03:38:51.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:50 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:50.832+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:51.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:50 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:50.832+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:51.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:50 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:50.832+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:51.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:50 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:50.832+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:38:51.426 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:50 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:50.960+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:51.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:50 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:50.960+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:51.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:50 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:50.960+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:51.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:50 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:50.960+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.413717+0000 front 2024-05-21T03:36:56.413780+0000 (oldest deadline 2024-05-21T03:38:26.117579+0000) 2024-05-21T03:38:51.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:50 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:50.960+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.218459+0000 front 2024-05-21T03:37:30.218348+0000 (oldest deadline 2024-05-21T03:38:36.118226+0000) 2024-05-21T03:38:52.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:51 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:51.698+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:52.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:51 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:51.698+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:52.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:51 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:51.698+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:52.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:51 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:51.698+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:52.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:51 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:51.698+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.744632+0000 front 2024-05-21T03:37:30.744615+0000 (oldest deadline 2024-05-21T03:38:36.644810+0000) 2024-05-21T03:38:52.172 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:51 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:51.858+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:52.172 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:51 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:51.858+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:52.172 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:51 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:51.858+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:52.173 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:51 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:51.858+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:38:52.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:51 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:51.931+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:52.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:51 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:51.931+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:52.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:51 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:51.931+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:52.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:51 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:51.931+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.413717+0000 front 2024-05-21T03:36:56.413780+0000 (oldest deadline 2024-05-21T03:38:26.117579+0000) 2024-05-21T03:38:52.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:51 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:51.931+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.218459+0000 front 2024-05-21T03:37:30.218348+0000 (oldest deadline 2024-05-21T03:38:36.118226+0000) 2024-05-21T03:38:53.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:52 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:52.651+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:53.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:52 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:52.651+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:53.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:52 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:52.651+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:53.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:52 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:52.651+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:53.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:52 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:52.651+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.744632+0000 front 2024-05-21T03:37:30.744615+0000 (oldest deadline 2024-05-21T03:38:36.644810+0000) 2024-05-21T03:38:53.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:52 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:52.848+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:53.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:52 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:52.848+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:53.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:52 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:52.848+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:53.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:52 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:52.848+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:38:53.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:52 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:52.962+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:20.500818+0000 front 2024-05-21T03:35:20.500881+0000 (oldest deadline 2024-05-21T03:36:48.404472+0000) 2024-05-21T03:38:53.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:52 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:52.962+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.205543+0000 front 2024-05-21T03:35:54.205648+0000 (oldest deadline 2024-05-21T03:37:20.408119+0000) 2024-05-21T03:38:53.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:52 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:52.962+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:25.109084+0000 front 2024-05-21T03:36:25.109149+0000 (oldest deadline 2024-05-21T03:37:54.212793+0000) 2024-05-21T03:38:53.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:52 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:52.962+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.413717+0000 front 2024-05-21T03:36:56.413780+0000 (oldest deadline 2024-05-21T03:38:26.117579+0000) 2024-05-21T03:38:53.427 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:52 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9[65283]: 2024-05-21T03:38:52.962+0000 7f4305e64700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.218459+0000 front 2024-05-21T03:37:30.218348+0000 (oldest deadline 2024-05-21T03:38:36.118226+0000) 2024-05-21T03:38:54.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:53 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:53.680+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:54.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:53 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:53.680+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:54.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:53 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:53.680+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:54.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:53 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:53.681+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:54.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:53 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:53.681+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.744632+0000 front 2024-05-21T03:37:30.744615+0000 (oldest deadline 2024-05-21T03:38:36.644810+0000) 2024-05-21T03:38:54.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:53 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:53.813+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:54.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:53 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:53.813+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:54.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:53 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:53.813+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:54.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:53 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:53.813+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:38:54.301 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:54 smithi027 podman[105468]: 2024-05-21 03:38:54.019752935 +0000 UTC m=+5.620129690 container died 0c30aa73ebb3dc024d98cf6eb73427c3759837866772c757fb497b2c149ff146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T03:38:54.667 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:54 smithi027 podman[105468]: 2024-05-21 03:38:54.400787385 +0000 UTC m=+6.001164148 container cleanup 0c30aa73ebb3dc024d98cf6eb73427c3759837866772c757fb497b2c149ff146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-21T03:38:54.668 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:54 smithi027 bash[105468]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9 2024-05-21T03:38:54.926 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:54 smithi027 podman[105483]: 2024-05-21 03:38:54.668760515 +0000 UTC m=+0.643703823 container remove 0c30aa73ebb3dc024d98cf6eb73427c3759837866772c757fb497b2c149ff146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-21T03:38:55.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:54 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:54.649+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:55.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:54 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:54.649+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:55.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:54 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:54.649+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:55.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:54 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:54.649+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:55.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:54 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:54.649+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.744632+0000 front 2024-05-21T03:37:30.744615+0000 (oldest deadline 2024-05-21T03:38:36.644810+0000) 2024-05-21T03:38:55.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:54 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:54.799+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:55.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:54 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:54.799+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:55.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:54 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:54.799+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:55.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:54 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:54.799+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:38:55.312 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:55 smithi027 podman[105577]: 2024-05-21 03:38:55.03636075 +0000 UTC m=+0.106237554 container create ccde8b37bfc80e9c7cb07530fa7d86993906a84131cb1f1c9e593e45e4903185 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-21T03:38:55.313 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:55 smithi027 podman[105577]: 2024-05-21 03:38:54.94773641 +0000 UTC m=+0.017613231 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T03:38:55.313 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:55 smithi027 podman[105577]: 2024-05-21 03:38:55.23040295 +0000 UTC m=+0.300279767 container init ccde8b37bfc80e9c7cb07530fa7d86993906a84131cb1f1c9e593e45e4903185 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9-deactivate, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-21T03:38:55.313 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:55 smithi027 podman[105577]: 2024-05-21 03:38:55.233803355 +0000 UTC m=+0.303680161 container start ccde8b37bfc80e9c7cb07530fa7d86993906a84131cb1f1c9e593e45e4903185 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T03:38:55.313 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:55 smithi027 podman[105577]: 2024-05-21 03:38:55.313835326 +0000 UTC m=+0.383712145 container attach ccde8b37bfc80e9c7cb07530fa7d86993906a84131cb1f1c9e593e45e4903185 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-21T03:38:55.677 INFO:journalctl@ceph.osd.9.smithi027.stdout:May 21 03:38:55 smithi027 podman[105577]: 2024-05-21 03:38:55.543917789 +0000 UTC m=+0.613794608 container died ccde8b37bfc80e9c7cb07530fa7d86993906a84131cb1f1c9e593e45e4903185 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-21T03:38:56.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:55 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:55.694+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:56.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:55 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:55.694+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:56.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:55 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:55.694+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:56.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:55 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:55.694+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:56.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:55 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:55.694+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.744632+0000 front 2024-05-21T03:37:30.744615+0000 (oldest deadline 2024-05-21T03:38:36.644810+0000) 2024-05-21T03:38:56.235 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.9.service' 2024-05-21T03:38:56.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:55 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:55.815+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:56.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:55 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:55.815+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:56.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:55 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:55.816+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:56.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:55 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:55.816+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:38:56.319 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T03:38:56.319 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-21T03:38:56.320 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-21T03:38:56.320 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.10 2024-05-21T03:38:56.624 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:56 smithi039 systemd[1]: Stopping Ceph osd.10 for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:38:57.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:56 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:56.624+0000 7fb4b75d9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T03:38:57.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:56 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:56.624+0000 7fb4b75d9700 -1 osd.10 94 *** Got signal Terminated *** 2024-05-21T03:38:57.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:56 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:56.624+0000 7fb4b75d9700 -1 osd.10 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T03:38:57.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:56 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:56.680+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:57.020 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:56 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:56.680+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:57.020 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:56 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:56.680+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:57.020 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:56 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:56.680+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:57.020 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:56 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:56.680+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.744632+0000 front 2024-05-21T03:37:30.744615+0000 (oldest deadline 2024-05-21T03:38:36.644810+0000) 2024-05-21T03:38:57.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:56 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:56.786+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:57.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:56 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:56.786+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:57.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:56 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:56.786+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:57.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:56 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:56.786+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:38:58.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:57 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:57.723+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:58.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:57 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:57.723+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:58.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:57 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:57.723+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:58.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:57 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:57.723+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:58.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:57 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:57.723+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.744632+0000 front 2024-05-21T03:37:30.744615+0000 (oldest deadline 2024-05-21T03:38:36.644810+0000) 2024-05-21T03:38:58.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:57 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:57.793+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:58.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:57 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:57.793+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:58.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:57 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:57.793+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:58.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:57 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:57.793+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:38:59.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:58 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:58.676+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:38:59.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:58 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:58.676+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:38:59.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:58 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:58.676+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:38:59.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:58 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:58.676+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:38:59.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:58 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:58.676+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.744632+0000 front 2024-05-21T03:37:30.744615+0000 (oldest deadline 2024-05-21T03:38:36.644810+0000) 2024-05-21T03:38:59.021 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:58 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:58.765+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:38:59.021 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:58 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:58.765+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:38:59.021 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:58 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:58.765+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:38:59.021 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:58 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:58.765+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:39:00.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:59 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:59.639+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:39:00.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:59 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:59.639+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:39:00.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:59 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:59.639+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:39:00.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:59 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:59.639+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:39:00.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:38:59 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:38:59.639+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.744632+0000 front 2024-05-21T03:37:30.744615+0000 (oldest deadline 2024-05-21T03:38:36.644810+0000) 2024-05-21T03:39:00.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:59 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:59.812+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:39:00.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:59 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:59.812+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:39:00.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:59 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:59.812+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:39:00.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:38:59 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:38:59.812+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:39:01.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:39:00 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:39:00.631+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:39:01.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:39:00 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:39:00.631+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:39:01.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:39:00 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:39:00.631+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:39:01.020 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:39:00 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:39:00.631+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:39:01.020 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:39:00 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:39:00.631+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.744632+0000 front 2024-05-21T03:37:30.744615+0000 (oldest deadline 2024-05-21T03:38:36.644810+0000) 2024-05-21T03:39:01.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:00 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:00.807+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:39:01.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:00 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:00.807+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:39:01.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:00 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:00.807+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:39:01.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:00 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:00.807+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:39:02.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:39:01 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:39:01.587+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:18.025613+0000 front 2024-05-21T03:35:18.025624+0000 (oldest deadline 2024-05-21T03:36:48.929523+0000) 2024-05-21T03:39:02.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:39:01 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:39:01.587+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:54.730886+0000 front 2024-05-21T03:35:54.730824+0000 (oldest deadline 2024-05-21T03:37:24.836450+0000) 2024-05-21T03:39:02.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:39:01 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:39:01.587+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:24.836762+0000 front 2024-05-21T03:36:24.836793+0000 (oldest deadline 2024-05-21T03:37:54.540102+0000) 2024-05-21T03:39:02.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:39:01 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:39:01.587+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-05-21T03:36:56.840783+0000 front 2024-05-21T03:36:56.840794+0000 (oldest deadline 2024-05-21T03:38:28.443656+0000) 2024-05-21T03:39:02.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:39:01 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10[61114]: 2024-05-21T03:39:01.587+0000 7fb4b3bcd700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:30.744632+0000 front 2024-05-21T03:37:30.744615+0000 (oldest deadline 2024-05-21T03:38:36.644810+0000) 2024-05-21T03:39:02.212 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:01 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:01.804+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:39:02.213 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:01 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:01.804+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:39:02.213 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:01 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:01.804+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:39:02.213 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:01 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:01.804+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:39:02.519 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:39:02 smithi039 podman[73907]: 2024-05-21 03:39:02.273321073 +0000 UTC m=+5.749023842 container died b10241f55e4ac727722736fac548b109b13c61a2be79c2caf37b01c6502318ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-05-21T03:39:03.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:39:02 smithi039 podman[73907]: 2024-05-21 03:39:02.632175737 +0000 UTC m=+6.107878505 container cleanup b10241f55e4ac727722736fac548b109b13c61a2be79c2caf37b01c6502318ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.license=GPLv2) 2024-05-21T03:39:03.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:39:02 smithi039 bash[73907]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10 2024-05-21T03:39:03.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:02 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:02.848+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:39:03.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:02 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:02.848+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:39:03.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:02 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:02.848+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:39:03.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:02 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:02.848+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:39:03.414 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:39:03 smithi039 podman[73922]: 2024-05-21 03:39:03.024705988 +0000 UTC m=+0.749440894 container remove b10241f55e4ac727722736fac548b109b13c61a2be79c2caf37b01c6502318ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-21T03:39:03.735 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:39:03 smithi039 podman[74016]: 2024-05-21 03:39:03.316697084 +0000 UTC m=+0.017493781 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T03:39:03.736 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:39:03 smithi039 podman[74016]: 2024-05-21 03:39:03.417078569 +0000 UTC m=+0.117875256 container create 69151583fa89d64a40143aa60832df08e7732d77eab8bc44a006bb1b0ab98b8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-21T03:39:03.736 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:39:03 smithi039 podman[74016]: 2024-05-21 03:39:03.653687594 +0000 UTC m=+0.354484295 container init 69151583fa89d64a40143aa60832df08e7732d77eab8bc44a006bb1b0ab98b8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T03:39:03.736 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:39:03 smithi039 podman[74016]: 2024-05-21 03:39:03.657053716 +0000 UTC m=+0.357850409 container start 69151583fa89d64a40143aa60832df08e7732d77eab8bc44a006bb1b0ab98b8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T03:39:03.736 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:39:03 smithi039 podman[74016]: 2024-05-21 03:39:03.736320204 +0000 UTC m=+0.437116914 container attach 69151583fa89d64a40143aa60832df08e7732d77eab8bc44a006bb1b0ab98b8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-21T03:39:04.019 INFO:journalctl@ceph.osd.10.smithi039.stdout:May 21 03:39:03 smithi039 podman[74016]: 2024-05-21 03:39:03.96123074 +0000 UTC m=+0.662027435 container died 69151583fa89d64a40143aa60832df08e7732d77eab8bc44a006bb1b0ab98b8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-21T03:39:04.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:03 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:03.823+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:39:04.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:03 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:03.823+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:39:04.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:03 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:03.823+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:39:04.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:03 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:03.823+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:39:04.607 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.10.service' 2024-05-21T03:39:04.650 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T03:39:04.651 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-21T03:39:04.651 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-21T03:39:04.651 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.11 2024-05-21T03:39:04.947 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:04 smithi165 systemd[1]: Stopping Ceph osd.11 for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:39:04.947 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:04 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:04.850+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:39:04.947 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:04 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:04.850+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:39:04.947 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:04 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:04.850+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:39:04.947 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:04 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:04.850+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:39:04.947 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:04 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:04.947+0000 7f7109074700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T03:39:04.947 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:04 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:04.947+0000 7f7109074700 -1 osd.11 94 *** Got signal Terminated *** 2024-05-21T03:39:04.948 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:04 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:04.947+0000 7f7109074700 -1 osd.11 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T03:39:06.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:05 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:05.866+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:39:06.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:05 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:05.866+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:39:06.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:05 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:05.866+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:39:06.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:05 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:05.866+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:39:07.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:06 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:06.893+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:39:07.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:06 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:06.893+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:39:07.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:06 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:06.893+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:39:07.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:06 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:06.893+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:39:08.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:07 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:07.863+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:39:08.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:07 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:07.863+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:39:08.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:07 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:07.863+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:39:08.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:07 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:07.863+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:39:09.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:08 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:08.817+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:39:09.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:08 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:08.817+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:39:09.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:08 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:08.817+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:39:09.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:08 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:08.817+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:39:09.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:08 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:08.817+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-21T03:37:40.253468+0000 front 2024-05-21T03:37:40.253390+0000 (oldest deadline 2024-05-21T03:39:08.256114+0000) 2024-05-21T03:39:10.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:09 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:09.835+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-05-21T03:35:21.433437+0000 front 2024-05-21T03:35:21.433456+0000 (oldest deadline 2024-05-21T03:36:48.936010+0000) 2024-05-21T03:39:10.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:09 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:09.835+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6804 osd.1 since back 2024-05-21T03:35:50.036742+0000 front 2024-05-21T03:35:50.036815+0000 (oldest deadline 2024-05-21T03:37:23.141437+0000) 2024-05-21T03:39:10.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:09 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:09.835+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-05-21T03:36:27.242455+0000 front 2024-05-21T03:36:27.242358+0000 (oldest deadline 2024-05-21T03:37:56.346201+0000) 2024-05-21T03:39:10.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:09 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:09.835+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.39:6812 osd.4 since back 2024-05-21T03:37:31.552283+0000 front 2024-05-21T03:37:31.552316+0000 (oldest deadline 2024-05-21T03:38:35.652150+0000) 2024-05-21T03:39:10.272 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:09 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11[61272]: 2024-05-21T03:39:09.835+0000 7f7101660700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-05-21T03:37:40.253468+0000 front 2024-05-21T03:37:40.253390+0000 (oldest deadline 2024-05-21T03:39:08.256114+0000) 2024-05-21T03:39:10.897 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:10 smithi165 podman[74358]: 2024-05-21 03:39:10.628951406 +0000 UTC m=+5.791509242 container died a063c7f2b774225ba7610d1d1c031658e21db20e434505b4dd58760b9edf589e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-21T03:39:11.256 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:10 smithi165 podman[74358]: 2024-05-21 03:39:10.997425514 +0000 UTC m=+6.159983349 container cleanup a063c7f2b774225ba7610d1d1c031658e21db20e434505b4dd58760b9edf589e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-05-21T03:39:11.256 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:10 smithi165 bash[74358]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11 2024-05-21T03:39:11.521 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:11 smithi165 podman[74373]: 2024-05-21 03:39:11.257190146 +0000 UTC m=+0.630571645 container remove a063c7f2b774225ba7610d1d1c031658e21db20e434505b4dd58760b9edf589e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-21T03:39:11.859 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:11 smithi165 podman[74462]: 2024-05-21 03:39:11.473249716 +0000 UTC m=+0.017645957 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T03:39:11.860 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:11 smithi165 podman[74462]: 2024-05-21 03:39:11.574865674 +0000 UTC m=+0.119261901 container create 26d1ab80637ae388cef798239c27df4c4c1e1df46a445d30c69c4e903a6ba0bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T03:39:11.860 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:11 smithi165 podman[74462]: 2024-05-21 03:39:11.785869494 +0000 UTC m=+0.330265712 container init 26d1ab80637ae388cef798239c27df4c4c1e1df46a445d30c69c4e903a6ba0bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-21T03:39:11.860 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:11 smithi165 podman[74462]: 2024-05-21 03:39:11.788877312 +0000 UTC m=+0.333273543 container start 26d1ab80637ae388cef798239c27df4c4c1e1df46a445d30c69c4e903a6ba0bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T03:39:11.860 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:11 smithi165 podman[74462]: 2024-05-21 03:39:11.860868106 +0000 UTC m=+0.405264335 container attach 26d1ab80637ae388cef798239c27df4c4c1e1df46a445d30c69c4e903a6ba0bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11-deactivate, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-05-21T03:39:12.271 INFO:journalctl@ceph.osd.11.smithi165.stdout:May 21 03:39:12 smithi165 podman[74462]: 2024-05-21 03:39:12.081436292 +0000 UTC m=+0.625832524 container died 26d1ab80637ae388cef798239c27df4c4c1e1df46a445d30c69c4e903a6ba0bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T03:39:12.694 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@osd.11.service' 2024-05-21T03:39:12.731 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T03:39:12.731 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-21T03:39:12.732 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-21T03:39:12.732 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.a 2024-05-21T03:39:13.035 INFO:journalctl@ceph.mds.a.smithi027.stdout:May 21 03:39:12 smithi027 systemd[1]: Stopping Ceph mds.a for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:39:13.427 INFO:journalctl@ceph.mds.a.smithi027.stdout:May 21 03:39:13 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-a[72258]: 2024-05-21T03:39:13.035+0000 7f40fe92d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T03:39:13.427 INFO:journalctl@ceph.mds.a.smithi027.stdout:May 21 03:39:13 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-a[72258]: 2024-05-21T03:39:13.035+0000 7f40fe92d700 -1 mds.a *** got signal Terminated *** 2024-05-21T03:39:14.409 INFO:journalctl@ceph.mds.a.smithi027.stdout:May 21 03:39:14 smithi027 podman[105682]: 2024-05-21 03:39:14.040835892 +0000 UTC m=+1.109760123 container died 8230c3edc1621e10369cc1392f409c5070d11c6b48f99be11244b3b2fabd414a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-a, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-21T03:39:14.409 INFO:journalctl@ceph.mds.a.smithi027.stdout:May 21 03:39:14 smithi027 podman[105682]: 2024-05-21 03:39:14.410539484 +0000 UTC m=+1.479463711 container cleanup 8230c3edc1621e10369cc1392f409c5070d11c6b48f99be11244b3b2fabd414a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-a, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-21T03:39:14.676 INFO:journalctl@ceph.mds.a.smithi027.stdout:May 21 03:39:14 smithi027 bash[105682]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-a 2024-05-21T03:39:14.677 INFO:journalctl@ceph.mds.a.smithi027.stdout:May 21 03:39:14 smithi027 podman[105697]: 2024-05-21 03:39:14.612027194 +0000 UTC m=+0.564273288 container remove 8230c3edc1621e10369cc1392f409c5070d11c6b48f99be11244b3b2fabd414a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-a, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-21T03:39:14.859 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.a.service' 2024-05-21T03:39:14.895 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T03:39:14.895 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-21T03:39:14.895 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-21T03:39:14.895 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.d 2024-05-21T03:39:15.176 INFO:journalctl@ceph.mds.d.smithi027.stdout:May 21 03:39:14 smithi027 systemd[1]: Stopping Ceph mds.d for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:39:15.676 INFO:journalctl@ceph.mds.d.smithi027.stdout:May 21 03:39:15 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-d[70970]: 2024-05-21T03:39:15.213+0000 7fd01b536700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T03:39:15.677 INFO:journalctl@ceph.mds.d.smithi027.stdout:May 21 03:39:15 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-d[70970]: 2024-05-21T03:39:15.213+0000 7fd01b536700 -1 mds.d *** got signal Terminated *** 2024-05-21T03:39:19.489 INFO:journalctl@ceph.mds.d.smithi027.stdout:May 21 03:39:19 smithi027 podman[105805]: 2024-05-21 03:39:19.224734369 +0000 UTC m=+4.106905330 container died b69acbcdc74ef24509044679211d9e4d29ba5610057eca83b9df095ce246bd11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, ceph=True) 2024-05-21T03:39:19.865 INFO:journalctl@ceph.mds.d.smithi027.stdout:May 21 03:39:19 smithi027 podman[105805]: 2024-05-21 03:39:19.67328605 +0000 UTC m=+4.555456998 container cleanup b69acbcdc74ef24509044679211d9e4d29ba5610057eca83b9df095ce246bd11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-d, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514) 2024-05-21T03:39:19.865 INFO:journalctl@ceph.mds.d.smithi027.stdout:May 21 03:39:19 smithi027 bash[105805]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-d 2024-05-21T03:39:20.113 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.d.service' 2024-05-21T03:39:20.136 INFO:journalctl@ceph.mds.d.smithi027.stdout:May 21 03:39:19 smithi027 podman[105817]: 2024-05-21 03:39:19.866477126 +0000 UTC m=+0.636923628 container remove b69acbcdc74ef24509044679211d9e4d29ba5610057eca83b9df095ce246bd11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-21T03:39:20.136 INFO:journalctl@ceph.mds.d.smithi027.stdout:May 21 03:39:20 smithi027 systemd[1]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.d.service: Deactivated successfully. 2024-05-21T03:39:20.136 INFO:journalctl@ceph.mds.d.smithi027.stdout:May 21 03:39:20 smithi027 systemd[1]: Stopped Ceph mds.d for c9ca94be-171f-11ef-bc9a-c7b262605968. 2024-05-21T03:39:20.136 INFO:journalctl@ceph.mds.d.smithi027.stdout:May 21 03:39:20 smithi027 systemd[1]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.d.service: Consumed 1.164s CPU time. 2024-05-21T03:39:20.149 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T03:39:20.150 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-21T03:39:20.150 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-21T03:39:20.150 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.g 2024-05-21T03:39:20.426 INFO:journalctl@ceph.mds.g.smithi027.stdout:May 21 03:39:20 smithi027 systemd[1]: Stopping Ceph mds.g for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:39:20.927 INFO:journalctl@ceph.mds.g.smithi027.stdout:May 21 03:39:20 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-g[71801]: 2024-05-21T03:39:20.486+0000 7f5e1f924700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T03:39:20.927 INFO:journalctl@ceph.mds.g.smithi027.stdout:May 21 03:39:20 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-g[71801]: 2024-05-21T03:39:20.486+0000 7f5e1f924700 -1 mds.g *** got signal Terminated *** 2024-05-21T03:39:22.336 INFO:journalctl@ceph.mds.g.smithi027.stdout:May 21 03:39:22 smithi027 podman[105936]: 2024-05-21 03:39:22.042797783 +0000 UTC m=+1.663653429 container died 1fe1fed96868b998b051e877ff95aca649b808c2266acf5b605e105024fa44d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-g, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-21T03:39:22.677 INFO:journalctl@ceph.mds.g.smithi027.stdout:May 21 03:39:22 smithi027 podman[105936]: 2024-05-21 03:39:22.436499795 +0000 UTC m=+2.057355430 container cleanup 1fe1fed96868b998b051e877ff95aca649b808c2266acf5b605e105024fa44d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-g, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T03:39:22.677 INFO:journalctl@ceph.mds.g.smithi027.stdout:May 21 03:39:22 smithi027 bash[105936]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-g 2024-05-21T03:39:22.836 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.g.service' 2024-05-21T03:39:22.872 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T03:39:22.872 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-21T03:39:22.872 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-21T03:39:22.872 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.j 2024-05-21T03:39:23.206 INFO:journalctl@ceph.mds.j.smithi027.stdout:May 21 03:39:22 smithi027 systemd[1]: Stopping Ceph mds.j for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:39:23.207 INFO:journalctl@ceph.mds.j.smithi027.stdout:May 21 03:39:23 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-j[70550]: 2024-05-21T03:39:23.206+0000 7f61040fb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T03:39:23.207 INFO:journalctl@ceph.mds.j.smithi027.stdout:May 21 03:39:23 smithi027 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-j[70550]: 2024-05-21T03:39:23.206+0000 7f61040fb700 -1 mds.j *** got signal Terminated *** 2024-05-21T03:39:27.489 INFO:journalctl@ceph.mds.j.smithi027.stdout:May 21 03:39:27 smithi027 podman[106059]: 2024-05-21 03:39:27.21780141 +0000 UTC m=+4.115655011 container died f9ede6e050edc8d10e24e5a84003cd2e431fc04cea41f4af245c863e00cd792b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-j, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.vendor=CentOS) 2024-05-21T03:39:27.840 INFO:journalctl@ceph.mds.j.smithi027.stdout:May 21 03:39:27 smithi027 podman[106059]: 2024-05-21 03:39:27.573285 +0000 UTC m=+4.471138603 container cleanup f9ede6e050edc8d10e24e5a84003cd2e431fc04cea41f4af245c863e00cd792b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD) 2024-05-21T03:39:27.840 INFO:journalctl@ceph.mds.j.smithi027.stdout:May 21 03:39:27 smithi027 bash[106059]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-j 2024-05-21T03:39:28.005 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.j.service' 2024-05-21T03:39:28.039 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T03:39:28.039 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-21T03:39:28.039 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-21T03:39:28.039 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.b 2024-05-21T03:39:28.340 INFO:journalctl@ceph.mds.b.smithi039.stdout:May 21 03:39:28 smithi039 systemd[1]: Stopping Ceph mds.b for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:39:28.618 INFO:journalctl@ceph.mds.b.smithi039.stdout:May 21 03:39:28 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-b[66446]: 2024-05-21T03:39:28.339+0000 7f94d4855700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T03:39:28.619 INFO:journalctl@ceph.mds.b.smithi039.stdout:May 21 03:39:28 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-b[66446]: 2024-05-21T03:39:28.339+0000 7f94d4855700 -1 mds.b *** got signal Terminated *** 2024-05-21T03:39:30.324 INFO:journalctl@ceph.mds.b.smithi039.stdout:May 21 03:39:30 smithi039 podman[74134]: 2024-05-21 03:39:30.050928375 +0000 UTC m=+1.811006553 container died 1f3a14b16cf9f44eb54e8dadb9525bcdba1315938f03d8bfabd9ed6bdcf81faf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-21T03:39:30.758 INFO:journalctl@ceph.mds.b.smithi039.stdout:May 21 03:39:30 smithi039 podman[74134]: 2024-05-21 03:39:30.499345322 +0000 UTC m=+2.259423500 container cleanup 1f3a14b16cf9f44eb54e8dadb9525bcdba1315938f03d8bfabd9ed6bdcf81faf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-b, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T03:39:30.758 INFO:journalctl@ceph.mds.b.smithi039.stdout:May 21 03:39:30 smithi039 bash[74134]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-b 2024-05-21T03:39:30.940 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.b.service' 2024-05-21T03:39:30.976 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T03:39:30.976 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-21T03:39:30.976 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-21T03:39:30.977 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.e 2024-05-21T03:39:31.311 INFO:journalctl@ceph.mds.e.smithi039.stdout:May 21 03:39:31 smithi039 systemd[1]: Stopping Ceph mds.e for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:39:31.769 INFO:journalctl@ceph.mds.e.smithi039.stdout:May 21 03:39:31 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-e[65840]: 2024-05-21T03:39:31.310+0000 7fe6adf83700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T03:39:31.769 INFO:journalctl@ceph.mds.e.smithi039.stdout:May 21 03:39:31 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-e[65840]: 2024-05-21T03:39:31.310+0000 7fe6adf83700 -1 mds.e *** got signal Terminated *** 2024-05-21T03:39:34.311 INFO:journalctl@ceph.mds.e.smithi039.stdout:May 21 03:39:34 smithi039 podman[74268]: 2024-05-21 03:39:34.056059234 +0000 UTC m=+2.845634223 container died 67ae61586fc6f90be192e7fed616f34c03ab3e557b49659a63b592d2fd093478 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-21T03:39:34.578 INFO:journalctl@ceph.mds.e.smithi039.stdout:May 21 03:39:34 smithi039 podman[74268]: 2024-05-21 03:39:34.402853605 +0000 UTC m=+3.192428585 container cleanup 67ae61586fc6f90be192e7fed616f34c03ab3e557b49659a63b592d2fd093478 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T03:39:34.578 INFO:journalctl@ceph.mds.e.smithi039.stdout:May 21 03:39:34 smithi039 bash[74268]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-e 2024-05-21T03:39:34.833 INFO:journalctl@ceph.mds.e.smithi039.stdout:May 21 03:39:34 smithi039 podman[74280]: 2024-05-21 03:39:34.579084213 +0000 UTC m=+0.518220420 container remove 67ae61586fc6f90be192e7fed616f34c03ab3e557b49659a63b592d2fd093478 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5) 2024-05-21T03:39:34.837 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.e.service' 2024-05-21T03:39:34.871 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T03:39:34.871 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-21T03:39:34.871 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-21T03:39:34.872 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.h 2024-05-21T03:39:35.189 INFO:journalctl@ceph.mds.h.smithi039.stdout:May 21 03:39:34 smithi039 systemd[1]: Stopping Ceph mds.h for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:39:35.518 INFO:journalctl@ceph.mds.h.smithi039.stdout:May 21 03:39:35 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-h[66143]: 2024-05-21T03:39:35.188+0000 7fe1b11d9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T03:39:35.519 INFO:journalctl@ceph.mds.h.smithi039.stdout:May 21 03:39:35 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-h[66143]: 2024-05-21T03:39:35.188+0000 7fe1b11d9700 -1 mds.h *** got signal Terminated *** 2024-05-21T03:39:38.332 INFO:journalctl@ceph.mds.h.smithi039.stdout:May 21 03:39:38 smithi039 podman[74390]: 2024-05-21 03:39:38.057886316 +0000 UTC m=+2.960385571 container died 4156b82453d026441ad60d265fe7b6f21bf2a9e9c8a9d2812c9dd9eee457960f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-h, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514) 2024-05-21T03:39:38.689 INFO:journalctl@ceph.mds.h.smithi039.stdout:May 21 03:39:38 smithi039 podman[74390]: 2024-05-21 03:39:38.414866014 +0000 UTC m=+3.317365274 container cleanup 4156b82453d026441ad60d265fe7b6f21bf2a9e9c8a9d2812c9dd9eee457960f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-h, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0) 2024-05-21T03:39:38.690 INFO:journalctl@ceph.mds.h.smithi039.stdout:May 21 03:39:38 smithi039 bash[74390]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-h 2024-05-21T03:39:38.858 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.h.service' 2024-05-21T03:39:38.891 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T03:39:38.891 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-21T03:39:38.891 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-21T03:39:38.891 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.k 2024-05-21T03:39:39.226 INFO:journalctl@ceph.mds.k.smithi039.stdout:May 21 03:39:38 smithi039 systemd[1]: Stopping Ceph mds.k for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:39:39.519 INFO:journalctl@ceph.mds.k.smithi039.stdout:May 21 03:39:39 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-k[65526]: 2024-05-21T03:39:39.225+0000 7f577a48e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T03:39:39.519 INFO:journalctl@ceph.mds.k.smithi039.stdout:May 21 03:39:39 smithi039 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-k[65526]: 2024-05-21T03:39:39.225+0000 7f577a48e700 -1 mds.k *** got signal Terminated *** 2024-05-21T03:39:42.359 INFO:journalctl@ceph.mds.k.smithi039.stdout:May 21 03:39:42 smithi039 podman[74513]: 2024-05-21 03:39:42.047992595 +0000 UTC m=+2.916252447 container died 8ad1a58936f916c735955ffffd45f65a57c02138f9d1c8bf83ca0af704b06f8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-k, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.vendor=CentOS) 2024-05-21T03:39:42.686 INFO:journalctl@ceph.mds.k.smithi039.stdout:May 21 03:39:42 smithi039 podman[74513]: 2024-05-21 03:39:42.360066956 +0000 UTC m=+3.228326799 container cleanup 8ad1a58936f916c735955ffffd45f65a57c02138f9d1c8bf83ca0af704b06f8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-k, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-21T03:39:42.687 INFO:journalctl@ceph.mds.k.smithi039.stdout:May 21 03:39:42 smithi039 bash[74513]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-k 2024-05-21T03:39:42.687 INFO:journalctl@ceph.mds.k.smithi039.stdout:May 21 03:39:42 smithi039 podman[74527]: 2024-05-21 03:39:42.577774474 +0000 UTC m=+0.522345974 container remove 8ad1a58936f916c735955ffffd45f65a57c02138f9d1c8bf83ca0af704b06f8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-k, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, io.buildah.version=1.33.5) 2024-05-21T03:39:42.710 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.k.service' 2024-05-21T03:39:42.741 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T03:39:42.741 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-21T03:39:42.742 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-21T03:39:42.742 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.c 2024-05-21T03:39:43.021 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 21 03:39:42 smithi165 systemd[1]: Stopping Ceph mds.c for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:39:43.294 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 21 03:39:43 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-c[66593]: 2024-05-21T03:39:43.044+0000 7f550a3aa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T03:39:43.294 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 21 03:39:43 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-c[66593]: 2024-05-21T03:39:43.044+0000 7f550a3aa700 -1 mds.c *** got signal Terminated *** 2024-05-21T03:39:46.306 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 21 03:39:46 smithi165 podman[74566]: 2024-05-21 03:39:46.042493276 +0000 UTC m=+3.101711257 container died f306b4b0d890f9a4be97dafd9b2a73f3f4a43f0c8740802a183e8ef2d41afc37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T03:39:46.674 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 21 03:39:46 smithi165 podman[74566]: 2024-05-21 03:39:46.473509793 +0000 UTC m=+3.532727767 container cleanup f306b4b0d890f9a4be97dafd9b2a73f3f4a43f0c8740802a183e8ef2d41afc37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-c, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T03:39:46.674 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 21 03:39:46 smithi165 bash[74566]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-c 2024-05-21T03:39:46.907 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.c.service' 2024-05-21T03:39:46.928 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 21 03:39:46 smithi165 podman[74590]: 2024-05-21 03:39:46.675004236 +0000 UTC m=+0.626942589 container remove f306b4b0d890f9a4be97dafd9b2a73f3f4a43f0c8740802a183e8ef2d41afc37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514) 2024-05-21T03:39:46.928 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 21 03:39:46 smithi165 systemd[1]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.c.service: Deactivated successfully. 2024-05-21T03:39:46.928 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 21 03:39:46 smithi165 systemd[1]: Stopped Ceph mds.c for c9ca94be-171f-11ef-bc9a-c7b262605968. 2024-05-21T03:39:46.928 INFO:journalctl@ceph.mds.c.smithi165.stdout:May 21 03:39:46 smithi165 systemd[1]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.c.service: Consumed 6.876s CPU time. 2024-05-21T03:39:46.942 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T03:39:46.942 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-21T03:39:46.942 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-21T03:39:46.942 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.f 2024-05-21T03:39:47.269 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 21 03:39:47 smithi165 systemd[1]: Stopping Ceph mds.f for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:39:47.521 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 21 03:39:47 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-f[66292]: 2024-05-21T03:39:47.269+0000 7fc8eab62700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T03:39:47.521 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 21 03:39:47 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-f[66292]: 2024-05-21T03:39:47.269+0000 7fc8eab62700 -1 mds.f *** got signal Terminated *** 2024-05-21T03:39:50.310 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 21 03:39:50 smithi165 podman[74701]: 2024-05-21 03:39:50.044398438 +0000 UTC m=+2.884994946 container died 5588f1b9598aa03b19200bfcf9293c037ba916fd450585db3a17a99ed87df461 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_CLEAN=True) 2024-05-21T03:39:50.748 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 21 03:39:50 smithi165 podman[74701]: 2024-05-21 03:39:50.385975334 +0000 UTC m=+3.226571836 container cleanup 5588f1b9598aa03b19200bfcf9293c037ba916fd450585db3a17a99ed87df461 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-f, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-21T03:39:50.748 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 21 03:39:50 smithi165 bash[74701]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-f 2024-05-21T03:39:50.748 INFO:journalctl@ceph.mds.f.smithi165.stdout:May 21 03:39:50 smithi165 podman[74714]: 2024-05-21 03:39:50.554599889 +0000 UTC m=+0.511705997 container remove 5588f1b9598aa03b19200bfcf9293c037ba916fd450585db3a17a99ed87df461 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-f, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T03:39:50.768 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.f.service' 2024-05-21T03:39:50.800 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T03:39:50.801 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-21T03:39:50.801 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-21T03:39:50.801 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.i 2024-05-21T03:39:51.021 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 21 03:39:50 smithi165 systemd[1]: Stopping Ceph mds.i for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:39:51.521 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 21 03:39:51 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-i[66895]: 2024-05-21T03:39:51.123+0000 7f6f88104700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T03:39:51.521 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 21 03:39:51 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-i[66895]: 2024-05-21T03:39:51.123+0000 7f6f88104700 -1 mds.i *** got signal Terminated *** 2024-05-21T03:39:54.332 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 21 03:39:54 smithi165 podman[74821]: 2024-05-21 03:39:54.056238017 +0000 UTC m=+3.040739904 container died 9929b3291d14931ef3dca492fa2245988a9c893f8cb5f8cb82529d9c6ab3db22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T03:39:54.674 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 21 03:39:54 smithi165 podman[74821]: 2024-05-21 03:39:54.440685036 +0000 UTC m=+3.425186920 container cleanup 9929b3291d14931ef3dca492fa2245988a9c893f8cb5f8cb82529d9c6ab3db22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-i, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_CLEAN=True) 2024-05-21T03:39:54.674 INFO:journalctl@ceph.mds.i.smithi165.stdout:May 21 03:39:54 smithi165 bash[74821]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-i 2024-05-21T03:39:54.820 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.i.service' 2024-05-21T03:39:54.851 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T03:39:54.851 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-21T03:39:54.852 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-21T03:39:54.852 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.l 2024-05-21T03:39:55.271 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 21 03:39:54 smithi165 systemd[1]: Stopping Ceph mds.l for c9ca94be-171f-11ef-bc9a-c7b262605968... 2024-05-21T03:39:55.271 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 21 03:39:55 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-l[65978]: 2024-05-21T03:39:55.161+0000 7fe5c5b25700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T03:39:55.271 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 21 03:39:55 smithi165 ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-l[65978]: 2024-05-21T03:39:55.161+0000 7fe5c5b25700 -1 mds.l *** got signal Terminated *** 2024-05-21T03:39:58.319 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 21 03:39:58 smithi165 podman[74943]: 2024-05-21 03:39:58.050537497 +0000 UTC m=+2.986876300 container died 3b6d506082ca28485d84139410a31df5360ea437f138d620060ab75907f4ebb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, org.label-schema.build-date=20240514) 2024-05-21T03:39:58.670 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 21 03:39:58 smithi165 podman[74943]: 2024-05-21 03:39:58.428298354 +0000 UTC m=+3.364637149 container cleanup 3b6d506082ca28485d84139410a31df5360ea437f138d620060ab75907f4ebb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-l, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-21T03:39:58.670 INFO:journalctl@ceph.mds.l.smithi165.stdout:May 21 03:39:58 smithi165 bash[74943]: ceph-c9ca94be-171f-11ef-bc9a-c7b262605968-mds-l 2024-05-21T03:39:58.814 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9ca94be-171f-11ef-bc9a-c7b262605968@mds.l.service' 2024-05-21T03:39:58.846 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T03:39:58.846 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-21T03:39:58.846 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-21T03:39:59.048 INFO:teuthology.orchestra.run.smithi027.stdout:Deleting cluster with fsid: c9ca94be-171f-11ef-bc9a-c7b262605968 2024-05-21T03:40:01.664 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-21T03:40:01.865 INFO:teuthology.orchestra.run.smithi039.stdout:Deleting cluster with fsid: c9ca94be-171f-11ef-bc9a-c7b262605968 2024-05-21T03:40:04.436 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-21T03:40:04.629 INFO:teuthology.orchestra.run.smithi165.stdout:Deleting cluster with fsid: c9ca94be-171f-11ef-bc9a-c7b262605968 2024-05-21T03:40:06.890 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T03:40:06.924 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T03:40:06.956 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T03:40:06.982 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-21T03:40:06.984 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-21_01:33:07-fs:workload-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7717852/remote/smithi027/crash 2024-05-21T03:40:06.985 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/crash -- . 2024-05-21T03:40:07.027 INFO:teuthology.orchestra.run.smithi027.stderr:tar: /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-21T03:40:07.027 INFO:teuthology.orchestra.run.smithi027.stderr:tar: Error is not recoverable: exiting now 2024-05-21T03:40:07.029 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-21_01:33:07-fs:workload-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7717852/remote/smithi039/crash 2024-05-21T03:40:07.030 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/crash -- . 2024-05-21T03:40:07.074 INFO:teuthology.orchestra.run.smithi039.stderr:tar: /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-21T03:40:07.074 INFO:teuthology.orchestra.run.smithi039.stderr:tar: Error is not recoverable: exiting now 2024-05-21T03:40:07.076 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-21_01:33:07-fs:workload-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7717852/remote/smithi165/crash 2024-05-21T03:40:07.077 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/crash -- . 2024-05-21T03:40:07.109 INFO:teuthology.orchestra.run.smithi165.stderr:tar: /var/lib/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-21T03:40:07.109 INFO:teuthology.orchestra.run.smithi165.stderr:tar: Error is not recoverable: exiting now 2024-05-21T03:40:07.110 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-21T03:40:07.110 DEBUG:teuthology.orchestra.run.smithi027:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-21T03:40:07.151 INFO:tasks.cephadm:Compressing logs... 2024-05-21T03:40:07.151 DEBUG:teuthology.orchestra.run.smithi027:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-21T03:40:07.193 DEBUG:teuthology.orchestra.run.smithi039:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-21T03:40:07.196 DEBUG:teuthology.orchestra.run.smithi165:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-21T03:40:07.219 INFO:teuthology.orchestra.run.smithi165.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose: No such file or directory -- 2024-05-21T03:40:07.219 INFO:teuthology.orchestra.run.smithi165.stderr: /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph.log 2024-05-21T03:40:07.219 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.2.log 2024-05-21T03:40:07.220 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose --/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph.log: /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.f.log 2024-05-21T03:40:07.220 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-21T03:40:07.225 INFO:teuthology.orchestra.run.smithi027.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph.log 2024-05-21T03:40:07.225 INFO:teuthology.orchestra.run.smithi027.stderr:: No such file or directory 2024-05-21T03:40:07.225 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-21T03:40:07.226 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.g.log 2024-05-21T03:40:07.226 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph.log: /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.j.log 2024-05-21T03:40:07.226 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.3.log 2024-05-21T03:40:07.226 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.0.log 2024-05-21T03:40:07.226 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.j.log: /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mgr.x.log 2024-05-21T03:40:07.226 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mon.a.log 2024-05-21T03:40:07.226 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.6.log 2024-05-21T03:40:07.226 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mon.a.log: 87.2% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-21T03:40:07.226 INFO:teuthology.orchestra.run.smithi027.stderr: 90.3% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-21T03:40:07.227 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.5.log 2024-05-21T03:40:07.227 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.c.log 2024-05-21T03:40:07.227 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mon.c.log 2024-05-21T03:40:07.227 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.f.log: /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph.audit.log: /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mgr.z.log 2024-05-21T03:40:07.227 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.i.log 2024-05-21T03:40:07.227 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-21T03:40:07.227 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.i.log: 87.2% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-21T03:40:07.227 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.11.log 2024-05-21T03:40:07.227 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph.cephadm.log: 84.1% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-21T03:40:07.228 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.8.log 2024-05-21T03:40:07.228 INFO:teuthology.orchestra.run.smithi165.stderr: 90.5% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-21T03:40:07.232 INFO:teuthology.orchestra.run.smithi027.stderr: 94.9% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.j.log.gz 2024-05-21T03:40:07.232 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-21T03:40:07.232 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.a.log 2024-05-21T03:40:07.232 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-21T03:40:07.232 INFO:teuthology.orchestra.run.smithi027.stderr: 84.4% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-21T03:40:07.235 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.5.log: /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.11.log: 92.0% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mgr.z.log.gz 2024-05-21T03:40:07.235 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-21T03:40:07.237 INFO:teuthology.orchestra.run.smithi039.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph.log 2024-05-21T03:40:07.237 INFO:teuthology.orchestra.run.smithi039.stderr:: No such file or directory 2024-05-21T03:40:07.238 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-21T03:40:07.238 INFO:teuthology.orchestra.run.smithi039.stderr:gzip/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph.log: -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.7.log 2024-05-21T03:40:07.238 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.b.log 2024-05-21T03:40:07.238 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.4.log 2024-05-21T03:40:07.238 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.1.log 2024-05-21T03:40:07.238 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.b.log: /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mgr.y.log 2024-05-21T03:40:07.238 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.h.log 2024-05-21T03:40:07.238 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.10.log 2024-05-21T03:40:07.238 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% 90.5% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-21T03:40:07.238 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph.log.gz/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.h.log: 2024-05-21T03:40:07.238 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-21T03:40:07.238 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.e.log 2024-05-21T03:40:07.238 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph.cephadm.log: 84.1% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-21T03:40:07.238 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-21T03:40:07.239 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.e.log: 91.9% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mgr.y.log.gz 2024-05-21T03:40:07.239 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mon.b.log 2024-05-21T03:40:07.241 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.l.log 2024-05-21T03:40:07.243 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.d.log 2024-05-21T03:40:07.249 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.k.log 2024-05-21T03:40:07.250 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mon.b.log: 91.1% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-21T03:40:07.256 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.9.log 2024-05-21T03:40:07.256 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-21T03:40:07.257 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-21T03:40:07.258 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.d.log: 91.1% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-21T03:40:07.258 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-21T03:40:07.263 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.9.log: 94.9% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.d.log.gz 2024-05-21T03:40:07.271 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-21T03:40:07.272 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-21T03:40:07.276 INFO:teuthology.orchestra.run.smithi165.stderr: 91.0% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-21T03:40:07.292 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/cephadm.log: 94.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-21T03:40:07.382 INFO:teuthology.orchestra.run.smithi027.stderr: 94.9% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.g.log.gz 2024-05-21T03:40:07.399 INFO:teuthology.orchestra.run.smithi027.stderr: 95.0% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.a.log.gz 2024-05-21T03:40:07.535 INFO:teuthology.orchestra.run.smithi039.stderr: 95.1% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.k.log.gz 2024-05-21T03:40:07.589 INFO:teuthology.orchestra.run.smithi165.stderr: 94.4% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.l.log.gz 2024-05-21T03:40:07.604 INFO:teuthology.orchestra.run.smithi027.stderr: 90.1% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mgr.x.log.gz 2024-05-21T03:40:07.647 INFO:teuthology.orchestra.run.smithi165.stderr: 91.9% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mon.c.log.gz 2024-05-21T03:40:07.655 INFO:teuthology.orchestra.run.smithi039.stderr: 92.0% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mon.b.log.gz 2024-05-21T03:40:07.892 INFO:teuthology.orchestra.run.smithi039.stderr: 95.5% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.e.log.gz 2024-05-21T03:40:08.129 INFO:teuthology.orchestra.run.smithi027.stderr: 90.4% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mon.a.log.gz 2024-05-21T03:40:08.346 INFO:teuthology.orchestra.run.smithi165.stderr: 95.0% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.f.log.gz 2024-05-21T03:40:08.459 INFO:teuthology.orchestra.run.smithi039.stderr: 95.0% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.h.log.gz 2024-05-21T03:40:08.866 INFO:teuthology.orchestra.run.smithi165.stderr: 94.4% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.i.log.gz 2024-05-21T03:40:08.886 INFO:teuthology.orchestra.run.smithi165.stderr: 95.1% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.c.log.gz 2024-05-21T03:40:14.591 INFO:teuthology.orchestra.run.smithi039.stderr: 93.4% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.4.log.gz 2024-05-21T03:40:14.870 INFO:teuthology.orchestra.run.smithi039.stderr: 94.3% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-mds.b.log.gz 2024-05-21T03:40:15.174 INFO:teuthology.orchestra.run.smithi027.stderr: 93.2% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.9.log.gz 2024-05-21T03:40:16.544 INFO:teuthology.orchestra.run.smithi165.stderr: 93.2% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.8.log.gz 2024-05-21T03:40:17.675 INFO:teuthology.orchestra.run.smithi165.stderr: 93.2% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.5.log.gz 2024-05-21T03:40:17.748 INFO:teuthology.orchestra.run.smithi165.stderr: 93.2% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.2.log.gz 2024-05-21T03:40:18.515 INFO:teuthology.orchestra.run.smithi027.stderr: 93.1% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.6.log.gz 2024-05-21T03:40:18.759 INFO:teuthology.orchestra.run.smithi039.stderr: 93.2% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.10.log.gz 2024-05-21T03:40:20.405 INFO:teuthology.orchestra.run.smithi165.stderr: 93.2% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.11.log.gz 2024-05-21T03:40:20.406 INFO:teuthology.orchestra.run.smithi165.stderr: 2024-05-21T03:40:20.407 INFO:teuthology.orchestra.run.smithi165.stderr:real 0m13.197s 2024-05-21T03:40:20.407 INFO:teuthology.orchestra.run.smithi165.stderr:user 0m46.611s 2024-05-21T03:40:20.407 INFO:teuthology.orchestra.run.smithi165.stderr:sys 0m1.670s 2024-05-21T03:40:20.496 INFO:teuthology.orchestra.run.smithi039.stderr: 93.2% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.7.log.gz 2024-05-21T03:40:21.627 INFO:teuthology.orchestra.run.smithi039.stderr: 93.2% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.1.log.gz 2024-05-21T03:40:21.629 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-05-21T03:40:21.629 INFO:teuthology.orchestra.run.smithi039.stderr:real 0m14.419s 2024-05-21T03:40:21.629 INFO:teuthology.orchestra.run.smithi039.stderr:user 0m54.727s 2024-05-21T03:40:21.630 INFO:teuthology.orchestra.run.smithi039.stderr:sys 0m1.746s 2024-05-21T03:40:21.836 INFO:teuthology.orchestra.run.smithi027.stderr: 93.1% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.3.log.gz 2024-05-21T03:40:22.178 INFO:teuthology.orchestra.run.smithi027.stderr: 93.1% -- replaced with /var/log/ceph/c9ca94be-171f-11ef-bc9a-c7b262605968/ceph-osd.0.log.gz 2024-05-21T03:40:22.180 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-05-21T03:40:22.180 INFO:teuthology.orchestra.run.smithi027.stderr:real 0m14.972s 2024-05-21T03:40:22.180 INFO:teuthology.orchestra.run.smithi027.stderr:user 0m48.783s 2024-05-21T03:40:22.180 INFO:teuthology.orchestra.run.smithi027.stderr:sys 0m1.575s 2024-05-21T03:40:22.181 INFO:tasks.cephadm:Archiving logs... 2024-05-21T03:40:22.181 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-21_01:33:07-fs:workload-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7717852/remote/smithi027/log 2024-05-21T03:40:22.182 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-21T03:40:29.380 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-21_01:33:07-fs:workload-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7717852/remote/smithi039/log 2024-05-21T03:40:29.382 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-21T03:40:36.252 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-21_01:33:07-fs:workload-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7717852/remote/smithi165/log 2024-05-21T03:40:36.253 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-21T03:40:42.771 INFO:tasks.cephadm:Removing cluster... 2024-05-21T03:40:42.771 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 --force 2024-05-21T03:40:42.976 INFO:teuthology.orchestra.run.smithi027.stdout:Deleting cluster with fsid: c9ca94be-171f-11ef-bc9a-c7b262605968 2024-05-21T03:40:43.314 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 --force 2024-05-21T03:40:43.516 INFO:teuthology.orchestra.run.smithi039.stdout:Deleting cluster with fsid: c9ca94be-171f-11ef-bc9a-c7b262605968 2024-05-21T03:40:43.854 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c9ca94be-171f-11ef-bc9a-c7b262605968 --force 2024-05-21T03:40:44.047 INFO:teuthology.orchestra.run.smithi165.stdout:Deleting cluster with fsid: c9ca94be-171f-11ef-bc9a-c7b262605968 2024-05-21T03:40:44.362 INFO:tasks.cephadm:Removing cephadm ... 2024-05-21T03:40:44.362 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-21T03:40:44.381 DEBUG:teuthology.orchestra.run.smithi039:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-21T03:40:44.398 DEBUG:teuthology.orchestra.run.smithi165:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-21T03:40:44.413 INFO:tasks.cephadm:Teardown complete 2024-05-21T03:40:44.414 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-21T03:40:44.428 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-21T03:40:44.428 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-21T03:40:44.430 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-21T03:40:44.440 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-21T03:40:44.545 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-21T03:40:44.555 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-21T03:40:44.555 DEBUG:teuthology.orchestra.run.smithi027:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T03:40:44.558 DEBUG:teuthology.orchestra.run.smithi039:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T03:40:44.560 DEBUG:teuthology.orchestra.run.smithi165:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T03:40:44.572 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2024-05-21T03:40:44.572 INFO:teuthology.orchestra.run.smithi039.stderr:bash: line 1: ntpq: command not found 2024-05-21T03:40:44.574 INFO:teuthology.orchestra.run.smithi165.stderr:bash: line 1: ntpq: command not found 2024-05-21T03:40:44.576 INFO:teuthology.orchestra.run.smithi039.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T03:40:44.576 INFO:teuthology.orchestra.run.smithi039.stdout:=============================================================================== 2024-05-21T03:40:44.577 INFO:teuthology.orchestra.run.smithi039.stdout:^* hv01.front.sepia.ceph.com 3 6 377 14 +1137us[+1127us] +/- 60ms 2024-05-21T03:40:44.577 INFO:teuthology.orchestra.run.smithi039.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 16 -1417us[-1426us] +/- 59ms 2024-05-21T03:40:44.577 INFO:teuthology.orchestra.run.smithi039.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 16 +402us[ +393us] +/- 71ms 2024-05-21T03:40:44.577 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T03:40:44.578 INFO:teuthology.orchestra.run.smithi165.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T03:40:44.578 INFO:teuthology.orchestra.run.smithi165.stdout:=============================================================================== 2024-05-21T03:40:44.578 INFO:teuthology.orchestra.run.smithi165.stdout:^* hv01.front.sepia.ceph.com 3 6 377 13 +1098us[+1099us] +/- 60ms 2024-05-21T03:40:44.578 INFO:teuthology.orchestra.run.smithi165.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 15 -1463us[-1461us] +/- 59ms 2024-05-21T03:40:44.579 INFO:teuthology.orchestra.run.smithi165.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 +366us[ +368us] +/- 71ms 2024-05-21T03:40:44.579 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T03:40:44.579 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T03:40:44.579 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2024-05-21T03:40:44.579 INFO:teuthology.orchestra.run.smithi027.stdout:^* hv01.front.sepia.ceph.com 3 6 377 14 +1174us[+1174us] +/- 60ms 2024-05-21T03:40:44.579 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 13 -1382us[-1382us] +/- 59ms 2024-05-21T03:40:44.579 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 +438us[ +438us] +/- 71ms 2024-05-21T03:40:44.579 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T03:40:44.579 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-21T03:40:44.595 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-21T03:40:44.596 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-21T03:40:44.648 DEBUG:teuthology.orchestra.run.smithi027:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-21T03:40:44.734 DEBUG:teuthology.orchestra.run.smithi039:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-21T03:40:44.803 DEBUG:teuthology.orchestra.run.smithi165:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-21T03:40:44.869 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T03:40:44.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T03:40:44.901 DEBUG:teuthology.orchestra.run.smithi039:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T03:40:44.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T03:40:44.929 DEBUG:teuthology.orchestra.run.smithi165:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T03:40:44.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T03:40:44.955 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-21T03:40:44.966 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-21T03:40:44.996 INFO:teuthology.task.internal:Duration was 2214.177859 seconds 2024-05-21T03:40:44.997 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-21T03:40:45.080 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-21T03:40:45.080 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-21T03:40:45.083 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-21T03:40:45.085 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-21T03:40:45.121 INFO:teuthology.orchestra.run.smithi039.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T03:40:45.122 INFO:teuthology.orchestra.run.smithi165.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T03:40:45.123 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T03:40:45.575 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-21T03:40:45.575 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi027.front.sepia.ceph.com 2024-05-21T03:40:45.575 DEBUG:teuthology.orchestra.run.smithi027:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-21T03:40:45.597 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi039.front.sepia.ceph.com 2024-05-21T03:40:45.597 DEBUG:teuthology.orchestra.run.smithi039:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-21T03:40:45.640 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi165.front.sepia.ceph.com 2024-05-21T03:40:45.641 DEBUG:teuthology.orchestra.run.smithi165:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-21T03:40:45.675 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-21T03:40:45.676 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-21T03:40:45.678 DEBUG:teuthology.orchestra.run.smithi039:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-21T03:40:45.683 DEBUG:teuthology.orchestra.run.smithi165:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-21T03:40:45.741 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-21T03:40:45.742 DEBUG:teuthology.orchestra.run.smithi027:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-21T03:40:45.747 DEBUG:teuthology.orchestra.run.smithi039:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-21T03:40:45.750 DEBUG:teuthology.orchestra.run.smithi165:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-21T03:40:46.507 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-21T03:40:46.519 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-21T03:40:46.519 DEBUG:teuthology.orchestra.run.smithi027:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-21T03:40:46.545 DEBUG:teuthology.orchestra.run.smithi039:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-21T03:40:46.571 DEBUG:teuthology.orchestra.run.smithi165:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-21T03:40:46.596 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-21T03:40:46.607 DEBUG:teuthology.orchestra.run.smithi027:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-21T03:40:46.610 DEBUG:teuthology.orchestra.run.smithi039:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-21T03:40:46.614 DEBUG:teuthology.orchestra.run.smithi165:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-21T03:40:46.631 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = core 2024-05-21T03:40:46.635 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern = core 2024-05-21T03:40:46.659 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = core 2024-05-21T03:40:46.683 DEBUG:teuthology.orchestra.run.smithi027:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-21T03:40:46.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T03:40:46.700 DEBUG:teuthology.orchestra.run.smithi039:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-21T03:40:46.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T03:40:46.724 DEBUG:teuthology.orchestra.run.smithi165:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-21T03:40:46.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T03:40:46.738 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-21T03:40:46.749 INFO:teuthology.task.internal:Transferring archived files... 2024-05-21T03:40:46.750 DEBUG:teuthology.misc:Transferring archived files from smithi027:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-21_01:33:07-fs:workload-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7717852/remote/smithi027 2024-05-21T03:40:46.750 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-21T03:40:46.811 DEBUG:teuthology.misc:Transferring archived files from smithi039:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-21_01:33:07-fs:workload-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7717852/remote/smithi039 2024-05-21T03:40:46.811 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-21T03:40:46.848 DEBUG:teuthology.misc:Transferring archived files from smithi165:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-21_01:33:07-fs:workload-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7717852/remote/smithi165 2024-05-21T03:40:46.848 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-21T03:40:46.883 INFO:teuthology.task.internal:Removing archive directory... 2024-05-21T03:40:46.884 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-21T03:40:46.887 DEBUG:teuthology.orchestra.run.smithi039:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-21T03:40:46.889 DEBUG:teuthology.orchestra.run.smithi165:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-21T03:40:46.932 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-21T03:40:46.947 INFO:teuthology.task.internal:Not uploading archives. 2024-05-21T03:40:46.948 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-21T03:40:46.961 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-21T03:40:46.961 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-21T03:40:46.964 DEBUG:teuthology.orchestra.run.smithi039:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-21T03:40:46.966 DEBUG:teuthology.orchestra.run.smithi165:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-21T03:40:46.976 INFO:teuthology.orchestra.run.smithi027.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 21 03:40 /home/ubuntu/cephtest 2024-05-21T03:40:46.978 INFO:teuthology.orchestra.run.smithi039.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 21 03:40 /home/ubuntu/cephtest 2024-05-21T03:40:46.986 INFO:teuthology.orchestra.run.smithi165.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 21 03:40 /home/ubuntu/cephtest 2024-05-21T03:40:46.987 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-21T03:40:46.999 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-21T03:40:47.047 INFO:teuthology.run:Summary data: description: fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/ffsb}} duration: 2214.1778593063354 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-05-21T03:40:47.047 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-21T03:40:47.140 INFO:teuthology.run:pass